|
| 1 | +=============================================== |
| 2 | +Add more data sources and improve data quality |
| 3 | +=============================================== |
| 4 | + |
| 5 | + |
| 6 | +**Organization:** `AboutCode <https://aboutcode.org>`_ |
| 7 | + |
| 8 | +**Project:** `Vulnerablecode <https://github.com/aboutcode-org/vulnerablecode>`_ |
| 9 | + |
| 10 | +**Mentee:** `Ambuj Kulshreshtha (ambuj-1211) <https://github.com/ambuj-1211>`_ |
| 11 | + |
| 12 | +**Mentors:** |
| 13 | + |
| 14 | +- `Philippe Ombredanne <https://github.com/pombredanne>`_ |
| 15 | +- `Ayan Sinha Mahapatra <https://github.com/AyanSinhaMahapatra>`_ |
| 16 | +- `Ziad Hany <https://github.com/ziadhany>`_ |
| 17 | +- `Tushar Goel <https://github.com/TG1999>`_ |
| 18 | +- `Keshav Priyadarshi <https://github.com/keshav-space>`_ |
| 19 | + |
| 20 | + |
| 21 | +Overview |
| 22 | +-------- |
| 23 | + |
| 24 | +There is a large number of pending tickets for datasources. This project focuses on adding more vulnerability data sources and consume them. I have considered following issues to solve `Collect advisories for AlmaLinux #1201 <https://github.com/aboutcode-org/vulnerablecode/issues/1201>`_, `Collect vulnerabilities from Amazon Linux #72 <https://github.com/aboutcode-org/vulnerablecode/issues/72>`_ , `Collect Oracle Linux #75 <https://github.com/aboutcode-org/vulnerablecode/issues/75>`_ , `Add data in CSAF format #1315 <https://github.com/aboutcode-org/vulnerablecode/issues/1315>`_, `VCIO does not collect some Severity (cvssv3.1) scores for a CVE #1238 <https://github.com/aboutcode-org/vulnerablecode/issues/1238>`_, `Add CWE support in all importers #1093 <https://github.com/aboutcode-org/vulnerablecode/issues/1093>`_ and `Collect rockylinux advisories #753 <https://github.com/aboutcode-org/vulnerablecode/issues/753>`_. Consuming these datasources will help to create a large database for vulnerabilities. |
| 25 | + |
| 26 | + |
| 27 | +Implementation |
| 28 | +-------------- |
| 29 | + |
| 30 | +- **Created Importers to add more advisory data from different data sources:** |
| 31 | + |
| 32 | + - I have added a few new importer modules to the VulnerableCode project to incorporate advisory data from different data sources. Some of the importers I created include the `Curl Importer`, `RockyLinux Importer`, `AlmaLinux Importer`, and `Amazon Linux Importer`. I also worked on creating an importer to retrieve data in CSAF format from the `cisagov repo <https://github.com/cisagov/CSAF/tree/develop/csaf_files>`_. |
| 33 | + |
| 34 | +- **Added CWE support in multiple importers:** |
| 35 | + |
| 36 | + - Many importers did not include CWE information, this was mentioned here: `Add CWE support in all importers #1093 <https://github.com/aboutcode-org/vulnerablecode/issues/1093>`_, so I solved this issue to add cwe data in multiple importers. There are still many importers that do not have CWE data available in their root data sources. I will add CWE data for them in the future if their data sources are updated. |
| 37 | + |
| 38 | +- **Found bugs in some Vulnerablities** |
| 39 | + |
| 40 | + - There is an issue `VCIO does not collect some Severity (cvssv3.1) scores for a CVE #1238 <https://github.com/aboutcode-org/vulnerablecode/issues/1238>`_, that I need to resolve regarding specific CVE data, specifically addressing the inconsistency in the severity information. |
| 41 | +- **Testing:** |
| 42 | + |
| 43 | + - I have built proper doctests for each importer, describing each |
| 44 | + function in the module in terms of its parameters and return values. |
| 45 | + |
| 46 | + - Proper unit tests have been created for each module I built |
| 47 | + to ensure the proper functioning of these modules. |
| 48 | + |
| 49 | +Linked Pull Requests |
| 50 | +-------------------- |
| 51 | + |
| 52 | +.. list-table:: |
| 53 | + :widths: 10 60 30 10 |
| 54 | + :header-rows: 1 |
| 55 | + |
| 56 | + * - Sr. no |
| 57 | + - Name |
| 58 | + - Link |
| 59 | + - Status |
| 60 | + * - 1 |
| 61 | + - Added Curl Advisories |
| 62 | + - `aboutcode.org/vulnerablecode#1439 <https://github.com/aboutcode-org/vulnerablecode/pull/1439>`_ |
| 63 | + - Open |
| 64 | + * - 2 |
| 65 | + - Added AlmaLinux Advisories |
| 66 | + - `aboutcode.org/vulnerablecode#1491 <https://github.com/aboutcode-org/vulnerablecode/pull/1491>`_ |
| 67 | + - Open |
| 68 | + * - 3 |
| 69 | + - Added CWE support in multiple importers |
| 70 | + - `aboutcode.org/vulnerablecode#1526 <https://github.com/aboutcode-org/vulnerablecode/pull/1526>`_ |
| 71 | + - Open |
| 72 | + * - 4 |
| 73 | + - Added RockyLinux advisories |
| 74 | + - `aboutcode.org/vulnerablecode#1535 <https://github.com/aboutcode-org/vulnerablecode/pull/1535>`_ |
| 75 | + - Open |
| 76 | + * - 5 |
| 77 | + - Added Amazon Linux advisories |
| 78 | + - `aboutcode.org/vulnerablecode#1569 <https://github.com/aboutcode-org/vulnerablecode/pull/1569>`_ |
| 79 | + - Open |
| 80 | + |
| 81 | +Related Issues |
| 82 | +-------------- |
| 83 | + |
| 84 | +.. list-table:: |
| 85 | + :widths: 10 60 30 |
| 86 | + :header-rows: 1 |
| 87 | + |
| 88 | + * - Sr. no |
| 89 | + - Name |
| 90 | + - Link |
| 91 | + * - 1 |
| 92 | + - Add CURL advisories data source |
| 93 | + - `#1166 <https://github.com/aboutcode-org/vulnerablecode/issues/1166>`_ |
| 94 | + * - 2 |
| 95 | + - Collect advisories for AlmaLinux |
| 96 | + - `#1201 <https://github.com/aboutcode-org/vulnerablecode/issues/1201>`_ |
| 97 | + * - 3 |
| 98 | + - Add CWE support in all importers |
| 99 | + - `#1093 <https://github.com/aboutcode-org/vulnerablecode/issues/1093>`_ |
| 100 | + * - 4 |
| 101 | + - Collect rockylinux advisories |
| 102 | + - `#753 <https://github.com/aboutcode-org/vulnerablecode/issues/753>`_ |
| 103 | + * - 5 |
| 104 | + - Collect vulnerabilities from Amazon Linux |
| 105 | + - `#72 <https://github.com/aboutcode-org/vulnerablecode/issues/72>`_ |
| 106 | + * - 6 |
| 107 | + - Add data in CSAF format |
| 108 | + - `#1315 <https://github.com/aboutcode-org/vulnerablecode/issues/1315>`_ |
| 109 | + * - 7 |
| 110 | + - Collect Oracle Linux |
| 111 | + - `#75 <https://github.com/aboutcode-org/vulnerablecode/issues/75>`_ |
| 112 | + * - 8 |
| 113 | + - VCIO does not collect some Severity (cvssv3.1) scores for a CVE |
| 114 | + - `#1238 <https://github.com/aboutcode-org/vulnerablecode/issues/1238>`_ |
| 115 | + |
| 116 | + |
| 117 | +Pre GSoC Work |
| 118 | +--------------- |
| 119 | + |
| 120 | +I started my contributions to AboutCode by the `Add Curl Advisories issue <https://github.com/aboutcode-org/scancode.io>`_, I added the curl advisories datasources to vulnerablecode database. This issue helped me to: |
| 121 | + |
| 122 | +- Understand the importers. |
| 123 | + |
| 124 | +- Understand the database models of VulnerableCode. |
| 125 | + |
| 126 | +- Understand the structure of `AdvisoryData`. |
| 127 | + |
| 128 | +- I also explored many components, such as `PackageURL`, `AffectedPackage`, `Severities`, etc. |
| 129 | + |
| 130 | +Post GSoC |
| 131 | +---------- |
| 132 | + |
| 133 | +I am committed to working on the pull request to ensure it is merged |
| 134 | +successfully, addressing any reviews and feedback from the mentors. I will prioritize |
| 135 | +completing any remaining tasks related to my GSoC work. This includes fixing issues |
| 136 | +such as bugs for specific CVEs that lack severity CVSSv3 scores and references |
| 137 | +from NVD (as there are a few of these CVEs). Once these tasks are completed, |
| 138 | +I plan to explore and contribute to more projects within AboutCode. |
| 139 | + |
| 140 | +Links |
| 141 | +------ |
| 142 | + |
| 143 | +* `Project Idea <https://github.com/aboutcode-org/aboutcode/wiki/GSOC-2024-Project-Ideas#vulnerablecode-add-more-data-sources-and-mine-the-graph-to-find-correlations-between-vulnerabilities-category-a>`_ |
| 144 | + |
| 145 | +* `Official GSoC project page <https://summerofcode.withgoogle.com/programs/2024/projects/O745WFKh>`_ |
| 146 | + |
| 147 | +* `GSoC Proposal <https://docs.google.com/document/d/1u7JlLL8ru133O3p4MCe7yYNo1ESsf5dheio5EBSiCFw/edit?usp=sharing>`_ |
| 148 | + |
| 149 | +* `Project Board <https://github.com/orgs/nexB/projects/62/views/6>`_ |
| 150 | + |
| 151 | +Acknowledgements |
| 152 | +---------------- |
| 153 | + |
| 154 | +I would like to thank my mentors: |
| 155 | + |
| 156 | +- `Ziad Hany`_ |
| 157 | +- `Tushar Goel`_ |
| 158 | +- `Philippe Ombredanne`_ |
| 159 | +- `Ayan Sinha Mahapatra`_ |
| 160 | +- `Keshav Priyadarshi`_ |
| 161 | + |
| 162 | +This summer was full of new challenges and learning. I got to learn a lot from everyone on the team. |
| 163 | +The weekly status calls were incredibly helpful in solving all my doubts. It was fun building for |
| 164 | +AboutCode, and I will continue to contribute to the codebase of VulnerableCode and |
| 165 | +other projects as well. I plan to explore more projects in AboutCode and contribute to |
| 166 | +them because I would love to be a part of this wonderful project. |
| 167 | + |
| 168 | +Thank you, everyone, for your continuous support and belief in me. |
| 169 | +Your guidance and encouragement have been invaluable, and I am truly grateful |
| 170 | +for all the help and trust you've shown me throughout this journey. |
| 171 | + |
| 172 | + |
| 173 | +.. _Ziad Hany: https://github.com/ziadhany |
| 174 | +.. _Tushar Goel: https://github.com/TG1999 |
| 175 | +.. _Philippe Ombredanne: https://github.com/pombredanne |
| 176 | +.. _Ayan Sinha Mahapatra: https://github.com/AyanSinhaMahapatra |
| 177 | +.. _Keshav Priyadarshi: https://github.com/keshav-space |
0 commit comments