Add the vulnerability report

[git-hyagi]

closes: #6773

[bmbouter]

I think you should write a new section in the dev docs, e.g. here for a plugin writer showing example code of say the python vuln reporting as an example.

[jobselko]

@git-hyagi @bmbouter I would like to work on implementing this functionality in pulp python once this PR is done.

[jobselko]

I would appreciate it if someone could take a look at the queue and threading parts. The rest (aside from the things I already mentioned) looks good to me.

[gerrod3]

Maybe I'm missing how this is used in pulp-services, but it feels like this is currently missing the features needed to be useful for a user. e.g. How do I see what python/npm/rpm content is vulnerable? How do I see what vulnerabilities are in my repo-version?

[gerrod3]

I think this is going to create a lot of CreatedResources, but I guess this is the only way to see what Vulnerabilites are in a repo-version?

[gerrod3]

Final nits. No more refactors after this, I promise 😅

[gerrod3]

Either we continue using pulp_python as an example or we change the get_content_from_repo_version to be more generic.

[gerrod3]

Let's do one last change to make this cleaner. Rename scan_packages to scan_package. Then create a new scan_packages method inside VulnerabilityReportScanner and move this async for logic + the progress report updating inside of the new method. i.e

class VulnerabilityReportScanner:
    def __init__(self, semaphore, generator, session=None):
        self.semaphore = semaphore
        self.generator = generator
        ...
    async def scan_packages(self):
         tasks = []
         async for content in self.generator:
              tasks.append(asyncio.create_task(self.scan_package(content))
         await asyncio.gather(*tasks)
         await self.session.close()  # This should remove that warning message we saw in the logs
         if self.created.done > 0:
             ...
             
# now in check_content
def check_content(func, args):
    ...
    semaphore = asyncio.Semaphore(settings.VULN_REPORT_TASK_LIMITER)
    vuln_report_scanner = VulnerabilityReportScanner(semaphore, func(*args))
    await vuln_report_scanner.scan_packages()

[gerrod3]

Don't think we are using this anywhere.

Suggested change

VULNERABILITY_TASK_THREAD_TIMEOUT = 60

[gerrod3]

This looks good. 🎉 Thanks for all the hard work on this!

[jobselko]

🎉