The OpenTitan source tree is maintained on GitHub in a monolithic repository called opentitan.
This file provides some notes on using GitHub for developing in the monolithic repository based on notes taken by a relatively inexperienced git user. There is much more to using git, a possible next step is to reference Resources to learn Git.
To develop in the repo you will need to get a copy on your local machine. To allow contributions to be made back to the main repo (through a process called a Pull Request) you need to first make your own copy of the repo on GitHub then transfer that to your local machine.
You will need to log in to GitHub, go to the opentitan repository and click on “Fork”. This will generate a copy in your GitHub area that you can use.
Then setup git on your local machine and set some standard parameters to ensure your name and email are correctly inserted. These commands set them globally for all your use of git on the local machine so you may have done this step already, there is no need to repeat it. (See below if you want to use a different email address for this repo.)
Check the parameters:
$ git config -l
And if they do not exist then set them:
$ git config --global user.name "My Name" $ git config --global user.email "my_name@email.com"
git will take care of prompting for your GitHub user name and password when it is required, but it can be useful to allow it to cache these credentials (set here to an hour using the timeout in seconds) so you don't have to enter every time:
$ git config --global credential.helper 'cache --timeout=3600'
Now make a local copy of your GitHub copy of the repo and let git know that it is derived from the upstream lowRISC repo:
$ cd <where the repo should go> $ git clone https://github.com/$GITHUB_USER/opentitan.git $ cd opentitan $ git remote add upstream https://github.com/lowRISC/opentitan.git $ git remote set-url --push upstream disabled $ git remote -v
The git remote -v should give your GitHub copy as origin and the lowRISC one as upstream. Making this link will allow you to keep your local and GitHub repos up to date with the lowRISC one.
If you want a different email address (or name) for the lowRISC repo then you can set it locally in the repo (similar to above but without the --global flag). This command must be executed from a directory inside the local copy of the repo. (There is no need for the first cd if you are following the previous step.)
$ cd opentitan $ git config user.email "my_name@lowrisc.org"
The repo that you have created locally will initially be on the master branch. In general you should not make changes on this branch, just use it to track your GitHub repo and synchronize with the lowRISC master repo.
The typical workflow is to make your own branch which it is conventional to name based on the change you are making:
$ git checkout -b forchange $ git status
The status will initially indicate there are no changes, but as you add, delete or edit files it will let you know the state of things.
Once you are happy with your changes, commit them to the local repo by adding the files to the changes (if things are clean you can add using git commit -a instead of a number of add commands):
$ git add... $ git commit
The commit will make you add a message. The first line of this is a short summary of the change. It should be prefixed with a word in square brackets indicating the area being changed, typically the IP or Tool name. For example:
[doc/um] Add notes on using GitHub and the repo
After this there should be a blank line and the main description of the change. If you are fixing an issue then add a line at the end of the message Fixes #nn where nn is the issue number. This will link the fix and close out the issue when it is added to the lowRISC repo.
When you have finished everything locally (it is good practice to do a status check to ensure things are clean) you can push your branch (eg forchange) to your GitHub repo (the origin):
$ git status
$ git push origin forchange
Then you need to go to your repo in Github and either select branch from the pulldown or often there is a status message that you can click on, review the changes and make a Pull Request. You can add reviewers and get your change reviewed.
If you need to make changes to satisfy the reviews then you do that in your local repo on the same branch. You will need to add files and commit again. It is normally best to squash your changes into a single commit by doing it with --amend which will give you a chance to edit the message. If you do this you need to force -f the push back to your repo.
$ git add... $ git commit --amend $ git status $ git push -f origin forchange
Once the reviewers are happy you can “Squash and merge” the Pull Request on GitHub, delete the branch there (it offers to do this when you do the merge). You can delete the branch in your local repo with:
$ git checkout master $ git branch -D forchange
There is a little work to do to keep everything in sync. Normally you want to first get your local repo master branch up to date with the lowRISC repo (upstream) and then you use that to update your GitHub copy (origin).
$ git checkout master
$ git pull upstream master
$ git push origin
If you do this while you have changes on some other branch then before a Pull Request will work you need to be sure your branch merges cleanly into the new lowRISC repo. Assuming you got the local master branch up to date with the procedure above you can now rebase your changes on the new master. Assuming you have your changes on the local branch forchange:
$ git checkout forchange
$ git rebase master
If you are lucky this will just work, it unwinds your changes, gets the updated master and replays your changes. If there are conflicts then you need a big pot of coffee and patience (see next section).
Once everything has rebased properly you can do
$ git log
And see that the changes you commited on the branch are at the top of the log followed by the latest changes on the master branch.
If a rebase fails because of conflicts between your changes and the code you are rebasing to then git will leave your working directories in a bit of a mess and expect you to fix it. Often the conflict is simple (eg you and someone else added a new routine at the same place in the file) and resolution is simple (have both in the new output). Sometimes there is more to untangle if different changes were done to the same routine. In either case git has marked that you are in a conflict state and work is needed before you can go back to using your local git tree as usual.
The git output actually describes what to do (once you are used to how to read it). For example:
$ git rebase master First, rewinding head to replay your work on top of it... Applying: [util][pystyle] Clean python style in single file tools Using index info to reconstruct a base tree... M util/diff_generated_util_output.py M util/build_docs.py Falling back to patching base and 3-way merge... Auto-merging util/build_docs.py CONFLICT (content): Merge conflict in util/build_docs.py Auto-merging util/diff_generated_util_output.py error: Failed to merge in the changes. Patch failed at 0001 [util][pystyle] Clean python style in single file tools Use 'git am --show-current-patch' to see the failed patch Resolve all conflicts manually, mark them as resolved with "git add/rm <conflicted_files>", then run "git rebase --continue". You can instead skip this commit: run "git rebase --skip". To abort and get back to the state before "git rebase", run "git rebase --abort".
The last line of this gives the ultimate out. You can abort the rebase and figure some other way to proceed. As it says, this is done with:
$ git rebase --abort
After executing this command you are back to a clean tree with your changes intact, but they are still based on whatever the earlier state of the repo was. Normally you will have to resolve the conflict sometime, but the escape hatch can be useful if you don't have time immediately!
In the normal case, read the output to find the file with the problem. In this case Merge conflict in util/build_docs.py (The merge of util/diff_generated_util_output.py was successful even though it is mentioned in the middle of what looks like error output.)
If the file is opened with an editor the points at which there are conflicts will have diff-style change information embedded in to them. For example:
<<<<<<< HEAD import livereload import docgen.generate ======= import docgen import livereload >>>>>>> [util][pystyle] Clean python style in single file tools
In this case the master tree (between <<<<<<< HEAD and =======) was modified to import docgen.generate rather than just docgen and the local tree (between ======= and >>>>>>> followed by the first line of the commit message) had been changed to re-order the imports. These lines have to be edited to get the correct merged result and the diff markers removed. There may be multiple points in the file where fixes are needed. Once all conflicts have been addressed the file can be git added and once all files addressed the rebase continued.
After the fix a status report will remind you where you are.
$ git status rebase in progress; onto cb85dc4 You are currently rebasing branch 'sastyle' on 'cb85dc4'. (all conflicts fixed: run "git rebase --continue") Changes to be committed: (use "git reset HEAD <file>..." to unstage) modified: diff_generated_util_output.py modified: build_docs.py Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: build_docs.py
This gives the same instructions as the original merge failure message and gives the comfort that all conflicts were fixed. To finish up you need to follow the instructions.
$ git add build_docs.py $ git rebase --continue Applying: [util][pystyle] Clean python style in single file tools
If there were more than one patch outstanding (which isn't usual if you use the commit --amend flow) then you may get subsequent conflicts following the rebase --continue as other patches are replayed.
You can check the rebase worked as expected by looking at the log to see your branch is one commit (or more if there were more) ahead of the master branch.
$ git log commit dd8721d2b1529c575c4aef988219fbf2ecd3fd1b (HEAD -> sastyle) Author: Mark Hayter <mark.hayter@gmail.com> Date: Thu Jan 10 09:41:20 2019 +0000 [util][pystyle] Clean python style in single file tools Result of lintpy.py --fix on the diff and build_docs tools Tested with ./diff_generated_util_output.py master commit cb85dc42199e925ad09c45d33f6483a14764b93e (upstream/master, origin/master, origin/HEAD, master)
This shows the new commit (HEAD of the branch sastyle) and the preceding commit is at the master branch (and at the same point as master on both origin and upstream so everything is in sync at master).
At this point the conflicts have been cleared and the local repo can be used as expected.
You may find it useful to change the default for the way git reports conflicts in a file. See Take the pain out of git conflict resolution: use diff3