nf-core list: Handle exception when HEAD is detached

[Zethson]

(base) [zeth@master course]$ nf-core list

                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

Traceback (most recent call last):
  File "/home/zeth/anaconda3/bin/nf-core", line 239, in <module>
    nf_core_cli()
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/zeth/anaconda3/bin/nf-core", line 72, in list
    nf_core.list.list_workflows(sort, json, keywords)
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/nf_core/list.py", line 36, in list_workflows
    wfs.get_local_nf_workflows()
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/nf_core/list.py", line 112, in get_local_nf_workflows
    wf.get_local_nf_workflow_details()
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/nf_core/list.py", line 298, in get_local_nf_workflow_details
    self.branch = str(repo.active_branch)
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/git/repo/base.py", line 696, in active_branch
    return self.head.reference
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/git/refs/symbolic.py", line 275, in _get_reference
    raise TypeError("%s is a detached symbolic reference as it points to %r" % (self, sha))
TypeError: HEAD is a detached symbolic reference as it points to '124cbbe5a070e1ee1dec25da1f65c9baea179b91'

I guess there's an issue with a git branch?

[Zethson]

(base) [zeth@master course]$ pip list | grep nf-core
nf-core                            1.5 

[Zethson]

Ups, sorry. I should have opened this issue at nf-core tools. I'll just leave this open here for now, but if you want me to I can reopen this at tools.

[ewels]

No worries - GitHub has a new feature which allows you to transfer issues, so I've just moved this to tools.

Relevant code: (problem is line 298)

tools/nf_core/list.py

Lines 293 to 301 in 1855d34

	# Pull information from the local git repository
	if self.local_path is not None:
	repo = git.Repo(self.local_path)
	self.commit_sha = str(repo.head.commit.hexsha)
	self.remote_url = str(repo.remotes.origin.url)
	self.branch = str(repo.active_branch)
	self.last_pull = os.stat(os.path.join(self.local_path, '.git', 'FETCH_HEAD')).st_mtime
	self.last_pull_date = datetime.datetime.fromtimestamp(self.last_pull).strftime("%Y-%m-%d %H:%M:%S")
	self.last_pull_pretty = pretty_date(self.last_pull)

Can you rerun with verbose logging please? nf-core -v list

Also, if you can go through all of your nf-core repos that have been pulled by nextflow and check their git statuses:

cd ~/.nextflow/assets/nf-core/
cd rnaseq/
git status
# ..etc

My guess is that one of these will give the same detached status. I'm not sure how this could have happened - we should add some code to the listing command to handle this exception type though.

If you want to solve this, just reset the repo that has the detached head:

git reset --hard HEAD

Then I think everything should work fine again.

Phil

[Zethson]

                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'


DEBUG: Fetching list of nf-core workflows

DEBUG: Guessed nextflow assets directory - pulling nf-core dirnames

DEBUG: Fetching extra info about 1 local workflows

DEBUG: Guessed nextflow assets workflow directory

DEBUG: Popen(['git', 'cat-file', '--batch-check'], cwd=/home/zeth/.nextflow/assets/nf-core/mhcquant, universal_newlines=False, shell=None)
Traceback (most recent call last):
  File "/home/zeth/anaconda3/bin/nf-core", line 239, in <module>
    nf_core_cli()
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/zeth/anaconda3/bin/nf-core", line 72, in list
    nf_core.list.list_workflows(sort, json, keywords)
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/nf_core/list.py", line 36, in list_workflows
    wfs.get_local_nf_workflows()
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/nf_core/list.py", line 112, in get_local_nf_workflows
    wf.get_local_nf_workflow_details()
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/nf_core/list.py", line 298, in get_local_nf_workflow_details
    self.branch = str(repo.active_branch)
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/git/repo/base.py", line 696, in active_branch
    return self.head.reference
  File "/home/zeth/anaconda3/lib/python3.7/site-packages/git/refs/symbolic.py", line 275, in _get_reference
    raise TypeError("%s is a detached symbolic reference as it points to %r" % (self, sha))
TypeError: HEAD is a detached symbolic reference as it points to '124cbbe5a070e1ee1dec25da1f65c9baea179b91'

[Zethson]

I only have MHCQuant on my machine.
There is no detached head in this repository. But certainly it's related to this.

The debug log shows:

cwd=/home/zeth/.nextflow/assets/nf-core/mhcquant

[Zethson]

Removing

.nextflow/assets/nf-core/mhcquant

solved the issue.

[ewels]

Is there anything to fix here? I’m a bit lost as to what the issue is that needs solving..

[Zethson]

I accidentally reproduced it a month later or so, but it might just be an issue with my setup. No idea.
You can close this I guess.

[ewels]

Thinking about it more... I would guess that this happens when nextflow pulls a workflow to your home directory cache (nextflow run nf-core/pipeline or nextflow pull nf-core/pipeline ), and then you rebase and force-push commits to that repo. This would result in the nextflow version being detached.

This has happened before and broken the website, which pulls the tools repo and others. That's why that update script now forces a hard git reset every time: https://github.com/nf-core/nf-co.re/blob/e000fde00471a483715cba0ac80b37f8ef26821b/public_html/deploy.php#L14-L15

git fetch && git reset --hard origin/master

This shouldn't really happen, as master should always be super stable. But now I can see why it might happen, I figure we should probably be defensive and handle the exception. Should be easy enough to do..