1.4 CMS Computing Concepts: Programming, GitHub, etc
Complete:
Detailed Review status
Goals of this page
You will learn what programming language(s) you are expected to know, what software packages you should be familiar with, and how to access software from the
code repository.
Note that in 2013, the CMS software repository was moved from CVS to git. CMS no longer uses CVS. The new git software repository is hosted at a website called
GitHub
, which provides extra features beyond git's file version control.
Contents
Operating Systems
Familiarity with
Linux is necessary. Current versions of the CMS Offline software, CMSSW, run on Scientific Linux CERN 6 (SLC6) (which
is built on Scientific Linux 6 (SL6), which is in turn built on Red Hat Enterprise Linux 6 (RHEL6)). Run I (2010-2012) was based on SL*5. (The upgrading process, from
versions SL*5 to SL*6, started in 2014, and the 7_0_0 release).
Programming Languages
CMSSW code is written in C++. You will need to be able to write compatible code in C++. There are many good text books available. See Walter Brown's (of
FNAL)
book recommendations for C++ Programmers
.
Analysis Packages
You will need to know how to use
ROOT. It is an object-oriented data analysis framework.
GitHub
CMS software, including CMSSW, is now maintained in a GitHub software repository.
You'll need to use
GitHub
to check out code and configuration files to your area so that you can modify and use them. There
are lots of working examples in
GitHub
for you to try; various workbook tutorials point you to specific directories. Since
CMSSW is such a large project we use the sparse check-out features of git. Because of this it is highly recommended that you look at the CMS git
tutorials
to understand how this is done.
Here we give
instructions on checking out code from GitHub. When you are working in the CMS environment, you can run any of the CMS code
in the release without checking it out. You only need to "check out" code that you need to modify, code that has been modified by someone else, or code that
depends on modified code. Then that code can be rebuilt.
A "release" is a set of software which has a version number and has been thoroughly tested to all work together. The CMS code evolves with stable, working
releases numbered and advertised from time to time as a recommended set of code to use.
Git Documentation
The primary documentation for usage of git in CMS is at
the following locations:
The information located there is much more detailed and complete that
the information that follows.
(Warning: Early in the introduction of git into CMS, CMS created a thing called the Topic Collector which some documentation still references. The Topic
Collector is no longer used for any purpose. GitHub directly offers features that replace that functionality and now we use them. If you see references or
links to the Topic Collector, you should understand you are looking at obsolete documentation that needs to be updated)
Obtaining a personal GitHub account:
If you want to modify code and then have your modifications merged
into the official repository of CMSSW code, you will need a personal
GitHub account. You would first edit the code and commit it to a git
repository on the machine where you work (maybe the machine on your
desk or lxplus). Then you would push the changes to the repository in
your personal GitHub account. Then you would issue a pull request to
have the modifications pulled from your personal GitHub repository
to the official CMSSW GitHub repository by a release manager.
A personal GitHub account has other uses. It is also a useful
way to send your code modifications to other people or groups. It can
also be used as a way to backup code modifications that you make
and maintain version control of your own personal code.
But a personal GitHub account is only mandatory if you want to merge code
into the official CMSSW code repository. You can download code,
build it, and run it locally without a personal GitHub account.
First you have to obtain a GitHub account. Go to
github.com
and follow the instructions to open a new account if you do not already
have one.
Then fork the official CMSSW repository. That creates a copy of it in your personal
GitHub account. Fork by going to
https://github.com/cms-sw/cmssw
and
click on the fork button and follow the instructions. After the forking is done you see you have a new repository cmssw,
https://github.com/<your-username>/cmssw.
Setup personal information
- Login on lxplus at CERN or wherever you plan to do your code development work.
- Run the following commands:
- git config --global user.name "First name Last name"
- git config --global user.email <Your-Email-Address>
- git config --global user.github <Your-Just-Created-GitHub-Account-Username>
Easy access of GitHub (optional)
yourKey.pub
- To use your key you have to execute following commands after each login:
-
eval $(ssh-agent); ssh-add /path/to/your/key
- Test your configuration:
ssh -T git@github.com
- You should see: Hi XYZ! You've successfully authenticated, but GitHub does not provide shell access.
Checkout a CMSSW package from Git
- Setup a CMSSW environment
cmsrel CMSSW_7_4_15
cd CMSSW_7_4_15/src
cmsenv
- Checkout a package, e.g.
PhysicsTools/PatExamples
git cms-addpkg PhysicsTools/PatExamples
No release tags specified, using default CMSSW_7_4_15.
...
You are on branch from-CMSSW_7_4_15
Checking out packages
PhysicsTools/PatExamples
* Create your own development branch e.g.
mydev
git checkout -b mydev
Switched to a new branch 'mydev'
- Check which branches are available
git branch
CMSSW_7_4_X
from-CMSSW_7_4_15
* mydev
echo '#modfication' >> PhysicsTools/PatExamples/BuildFile.xml #this adds a line containing #modification at the end of BuildFile.xml
- Check if modification is recognized
git status
# On branch mydev
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: PhysicsTools/PatExamples/BuildFile.xml
#
no changes added to commit (use "git add" and/or "git commit -a")
- Add and commit your changes
git add PhysicsTools/PatExamples/BuildFile.xml
git commit -m "test message"
[mydev 6812826] test message
1 files changed, 1 insertions(+), 0 deletions(-)
Your modifications were commited to your
local git repository.
You can push these changes to your personal GitHub
repository with the following command.
git push my-cmssw mydev
Before you do this you will have to have already created
your personal GitHub account and forked the cmssw repository.
(Creating and forking are one time actions. They do not need
to be done with every push, see above for more information).
If obtain this message:
ERROR: Repository not found.
fatal: The remote end hung up unexpectedly
git wasn't able to find the my-cmssw repository in you GitHub account. Check if you forked the
https://github.com/cms-sw/cmssw
repository.
If the push is successful you see:
Counting objects: 9, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 436 bytes, done.
Total 5 (delta 4), reused 0 (delta 0)
To git@github.com:fhoehle/cmssw.git
* [new branch] mydev -> mydev
You visit your account at GitHub and click on the branches drop-down menu you see your new branch mydev. Inspect the PhysicsTools folder and you see that
PatExamples were changed recently with a message
test message.
It showed you are checkout done of specific packages and how you are able to save your local changes at your private GitHub account. Git has a lot more feature
which help and support you in developing and keeping track of your code. Please visit GitHub.com for more detailed information on branches, merging them, tags,
...
Git Tutorials and Info
Tutorials
CMSSW releases
To find out which CMSSW releases are available, do:
scram list CMSSW
You'll see a list with names like CMSSW_x_y_z, some with other suffixes. The names point to directories, e.g.,
CMSSW CMSSW_7_4_15 --> /cvmfs/cms.cern.ch/slc6_amd64_gcc491/cms/cmssw/CMSSW_7_4_15
For each listed directory, subsystems are under its
src
directory, i.e., under
/cvmfs/cms.cern.ch/slc6_amd64_gcc491/cms/cmssw/CMSSW_7_4_15/src
. As an
example, let's list the subsystems under this directory:
ls /cvmfs/cms.cern.ch/slc6_amd64_gcc491/cms/cmssw/CMSSW_7_4_15/src
Alignment DQM IOMC RecoEcal SimCalorimetry
AnalysisAlgos DQMOffline IOPool RecoEgamma SimDataFormats
AnalysisDataFormats DQMServices IORawData RecoHI SimG4CMS
CalibCalorimetry EgammaAnalysis JetMETAnalysis RecoJets SimG4Core
CalibFormats ElectroWeakAnalysis JetMETCorrections RecoLocalCalo SimGeneral
CalibMuon EventFilter L1Trigger RecoLocalMuon SimMuon
Calibration FastSimDataFormats L1TriggerConfig RecoLocalTracker SimRomanPot
CalibTracker FastSimulation L1TriggerOffline RecoLuminosity SimTracker
CaloOnlineTools Fireworks MagneticField RecoMET SimTransport
CommonTools FWCore Mixing RecoMuon SUSYBSMAnalysis
CondCore GeneratorInterface MuonAnalysis RecoParticleFlow TauAnalysis
CondFormats Geometry OnlineDB RecoPixelVertexing TBDataFormats
CondTools GeometryReaders PackageList.cmssw RecoPixelVZero TopQuarkAnalysis
Configuration HeavyFlavorAnalysis PerfTools RecoRomanPot TrackingTools
DataFormats HeavyIonsAnalysis PhysicsTools RecoTauTag TrackPropagation
DetectorDescription HiggsAnalysis QCDAnalysis RecoTBCalo Utilities
DiffractiveForwardAnalysis HLTrigger RecoBTag RecoTracker Validation
Documentation HLTriggerOffline RecoBTau RecoVertex
DPGAnalysis IgTools RecoCaloTools RecoVZero
Each subsystem contains several packages, for example
ls /cvmfs/cms.cern.ch/slc6_amd64_gcc491/cms/cmssw/CMSSW_7_4_15/src/Alignment/
CocoaAnalysis CocoaUtilities KalmanAlignmentAlgorithm OfflineValidation
CocoaApplication CommonAlignment LaserAlignment ReferenceTrajectories
CocoaDaq CommonAlignmentAlgorithm LaserAlignmentSimulation SurveyAnalysis
CocoaDDLObjects CommonAlignmentMonitor LaserDQM TrackerAlignment
CocoaFit CommonAlignmentParametrization MillePedeAlignmentAlgorithm TwoBodyDecay
CocoaModel CommonAlignmentProducer MuonAlignment
CocoaToDDL HIPAlignmentAlgorithm MuonAlignmentAlgorithms
with directories such as interface (containing the header files), src (containing the *.cc files), plugins (defining the modules to be used), python (containing
the configuration files), doc (containing files needed for reference manual documenation), test (containing test setup) and data (obsolete).
%RESPONSIBLE%
SudhirMalik
%REVIEW%
JuanEduardoRamirez - Dec 2015