Skip to content

Add library bz2 to compress and uncompress Ubuntu ovals#4834

Merged
bah07 merged 17 commits into3.12from
dev-bzip2-library
Apr 6, 2020
Merged

Add library bz2 to compress and uncompress Ubuntu ovals#4834
bah07 merged 17 commits into3.12from
dev-bzip2-library

Conversation

@bah07
Copy link
Copy Markdown
Contributor

@bah07 bah07 commented Apr 2, 2020

Related issue
4808

Description

Since a few days the Ubuntu feed is obtained from a compressed file in bz2 format.
With this PR the corresponding library is added to be able to compress and decompress this file format.

Configuration options

Not aplicable.

Logs/Alerts example

Check the Debian and Ubuntu feeds are in database

select target, count(*) from vulnerabilities group by target

FEED CVEs
BIONIC 26718
BUSTER 18617
JESSIE 23353
STRETCH 21597
TRUSTY 10019
WHEEZY 835
XENIAL 32544

Tests

  • Compilation without warnings in every supported platform
    • Linux
  • Source installation
  • Review logs syntax and correct language
  • Memory tests for Linux
    • Scan-build report
    • Valgrind (memcheck and descriptor leaks check)
  • Added unit tests (for new features)

Comment thread src/shared/bzip2_op.c Outdated

input = fopen(filebz2, "rb");
if (!input) {
merror(FOPEN_ERROR, file, errno, strerror(errno));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error shows file instead of filebz2

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solved, thanks @chemamartinez

Comment thread src/shared/bzip2_op.c Outdated

output = fopen(file, "wb" );
if (!output) {
merror(FOPEN_ERROR, filebz2, errno, strerror(errno));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error shows filebz2 instead of file

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solved, thanks @chemamartinez

#define WM_VULNDETECTOR_DOWN_ATTEMPTS 5
#define VU_DEF_IGNORE_TIME 21600 // 6 hours
#define VU_TEMP_FILE "tmp/vuln-temp"
#define VU_TEMP_FILE_BZ2 "tmp/vuln-temp.bz2"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be replaced by `#define VU_TEMP_FILE_BZ2 VU_TEMP_FILE ".bz2"

@chemamartinez
Copy link
Copy Markdown
Contributor

The uncompression process has been tested over valgrind to check any memory error. Here the report:

==102141== LEAK SUMMARY:
==102141==    definitely lost: 0 bytes in 0 blocks
==102141==    indirectly lost: 0 bytes in 0 blocks
==102141==      possibly lost: 5,736 bytes in 7 blocks
==102141==    still reachable: 362,673 bytes in 181 blocks
==102141==                       of which reachable via heuristic:
==102141==                         length64           : 223,880 bytes in 122 blocks
==102141==         suppressed: 0 bytes in 0 blocks
==102141== Reachable blocks (those to which a pointer was found) are not shown.
==102141== To see them, rerun with: --leak-check=full --show-leak-kinds=all

@bah07 bah07 requested a review from chemamartinez April 6, 2020 10:21
@vikman90 vikman90 changed the title Add library bz2 to compress an uncompress Ubuntu ovals Add library bz2 to compress and uncompress Ubuntu OVAL feeds Apr 6, 2020
@bah07 bah07 changed the title Add library bz2 to compress and uncompress Ubuntu OVAL feeds Add library bz2 to compress and uncompress Ubuntu ovals Apr 6, 2020
Comment thread src/shared/bzip2_op.c Outdated
Comment thread src/shared/bzip2_op.c Outdated
Comment thread src/shared/bzip2_op.c Outdated
Comment thread src/headers/bzip2_op.h Outdated
Comment thread src/headers/bzip2_op.h Outdated
Borja and others added 5 commits April 6, 2020 12:51
Co-Authored-By: Vikman Fdez-Castro <victor@wazuh.com>
Co-Authored-By: Vikman Fdez-Castro <victor@wazuh.com>
Co-Authored-By: Vikman Fdez-Castro <victor@wazuh.com>
Co-Authored-By: Vikman Fdez-Castro <victor@wazuh.com>
Co-Authored-By: Vikman Fdez-Castro <victor@wazuh.com>
@bah07 bah07 requested a review from vikman90 April 6, 2020 11:32
Copy link
Copy Markdown
Member

@vikman90 vikman90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Issue with Ubuntu Feed

3 participants