This repository contains three Python scripts used for bulk triaging file using Assemblyline.
- Submitter (
al-incident-submitter): pushes files from a directory to an Assemblyline instance for analysis. - Result analyzer (
al-incident-analyzer): pulls the submissions from the Assemblyline instance and reports on if the submissions are safe/unsafe. - Downloader (
al-incident-downloader): downloads files submitted to Assemblyline that are under a certain score threshold, matching the folder structure of the files as they were submitted.
- You will need the URL of an Assemblyline instance that you have an account on, for best results make its loaded with your best Yara rules, Sandboxes etc.
- Want to create your own Assemblyline instance? HOW-TO
- You will need two API keys generated by Assemblyline, ideally one with read access and another with write access.
- The Write-only key will be used for the "Submitter" and the Read-only key will be used for the "Result Analysis" and the "Downloader".
- This helps in the context of incident response to reduce the exposure of your Assemblyline instance.
- Install the following packages:
libffi-dev,libssl-dev- (APT)
sudo apt-get install libffi-dev libssl-dev python3 - (YUM)
sudo yum install libffi-dev libssl-dev python3
- (APT)
- Upgrade PIP:
python3 -m pip install --upgrade pip python3 -m pip install assemblyline-incident-manager
- Download and install the most recent Python .msi installer from https://www.python.org/downloads/release.
- Upgrade PIP:
python -m pip install --upgrade pip python -m pip install assemblyline-incident-manager
al-incident-submitter --help
Usage: al-incident-submitter [OPTIONS] COMMAND [ARGS]...
Example: al-incident-submitter --url="https://<domain-of-Assemblyline-
instance>" --username="<user-name>"
--apikey="/path/to/file/containing/apikey"
--classification="<classification>" --service_selection="<service-
name>,<service-name>" --path="/path/to/scan"
--incident_num=123
Options:
--url TEXT The target URL that hosts Assemblyline.
[required]
--username TEXT Your Assemblyline account username. [required]
--apikey PATH A path to a file that contains only your
Assemblyline account API key. NOTE that this API
key requires write access. [required]
--ttl INTEGER The amount of time that you want your Assemblyline
submissions to live on the Assemblyline system (in
days).
--classification TEXT The classification level for each file submitted
to Assemblyline. [required]
--service_selection TEXT A comma-separated list (no spaces!) of service
names (case-sensitive) to send files to. If not provided, all
services will be selected.
-t, --is_test A flag that indicates that you're running a test.
--path PATH The directory path containing files that you want
to submit to Assemblyline. [required]
-f, --fresh Restart ingestion from the beginning.
--incident_num TEXT The incident number for each file to be associated
with. [required]
--resubmit-dynamic All files that score higher than 500 will be
resubmitted for dynamic analysis.
--alert Generate alerts for this submission.
--threads INTEGER Number of threads that will ingest files to
Assemblyline.
--dedup_hashes Only submit files with unique hashes. If you want
100% file coverage in a given path, do not use
this flag
--priority INTEGER Provide a priority number which will cause the
ingestion to go to a specific priority queue.
--do_not_verify_ssl Ignore SSL errors (insecure!)
--help Show this message and exit.
al-incident-analyzer --help
Usage: al-incident-analyzer [OPTIONS] COMMAND [ARGS]...
Example: al-incident-analyzer --url="https://<domain-of-
Assemblyline-instance>" --username="<user-name>"
--apikey="/path/to/file/containing/apikey" --incident_num=123
Options:
--url TEXT The target URL that hosts Assemblyline. [required]
-u, --username TEXT Your Assemblyline account username. [required]
--apikey PATH A path to a file that contains only your Assemblyline
account API key. NOTE that this API key requires write
access. [required]
--min_score INTEGER The minimum score for files that we want to query from
Assemblyline.
--incident_num TEXT The incident number for each file to be associated
with. [required]
-t, --is_test A flag that indicates that you're running a test.
--help Show this message and exit.
Now check the report.csv file that was created. This file will contain what files are safe/unsafe.
al-incident-downloader --help
Usage: al-incident-downloader [OPTIONS] COMMAND [ARGS]...
Example: al-incident-downloader --url="https://<domain-of-
Assemblyline-instance>" --username="<user-name>"
--apikey="/path/to/file/containing/apikey" --incident_num=123
--max_score=100 --download_path=/path/to/where/you/want/downloads
--upload_path=/path/from/where/files/were/uploaded/from
Options:
--url TEXT The target URL that hosts Assemblyline.
[required]
-u, --username TEXT Your Assemblyline account username.
[required]
--apikey PATH A path to a file that contains only your
Assemblyline account API key. NOTE that this
API key requires read access. [required]
--max_score INTEGER The maximum score for files that we want to
download from Assemblyline. [required]
--incident_num TEXT The incident number that each file is
associated with. [required]
--download_path PATH The path to the folder that we will download
files to. [required]
--upload_path PATH The base path from which the files were
ingested from.
[required]
-t, --is_test A flag that indicates that you're running a
test.
--num_of_downloaders INTEGER The number of threads that will be created to
facilitate downloading the files.
--do_not_verify_ssl Verify SSL when creating and using the
Assemblyline Client.
--help Show this message and exit.
If you check the download path you supplied, you should have all files downloaded there.
Ce répertoire contient trois scripts Python pour assisté le triage de grande quantité de fichiers avec Assemblyline.
- Soumission (
al-incident-submitter): envoi les fichiers contenu dans un dossier vers une instance Assemblyline pour l'analyze. - Résultats d'analyse (
al-incident-analyzer): analyse les soumissions et génère un rapport. - Téléchargeur (
al-incident-downloader): télécharge les fichiers sous un certain pointage en préservant la structure original.
- Vous aurez besoin d'un instance d'Assemblyline à jour et avec vos meilleurs règles Yara, "Sandboxes" etc.
- Voici comment crée vôtre propre instance: LIEN
- Nous vous recommandons d'utilisé deux clé d'api, un
write onlyet uneread only- La clé
Write-onlysera utilisé pour soumettre vos fichier avec le script "Submitter" et la cléRead-onlysera pour "Result Analysis" et le "Downloader". - Cette séparation aidera a securisé vôtre instance Assemblyline dans un context de réponse aux incidents
- La clé
- Installé les packages suivants:
libffi-dev,libssl-dev- (APT)
sudo apt-get install libffi-dev libssl-dev python3 - (YUM)
sudo yum install libffi-dev libssl-dev python3
- (APT)
- Mise à jour de PIP:
python3 -m pip install --upgrade pip python3 -m pip install assemblyline-incident-manager
- Installé Python 3: https://www.python.org/downloads/release.
- Mise à jour de PIP:
python -m pip install --upgrade pip python -m pip install assemblyline-incident-manager
al-incident-submitter --help
Usage: al-incident-submitter [OPTIONS] COMMAND [ARGS]...
Example: al-incident-submitter --url="https://<domain-of-Assemblyline-
instance>" --username="<user-name>"
--apikey="/path/to/file/containing/apikey"
--classification="<classification>" --service_selection="<service-
name>,<service-name>" --path="/path/to/scan"
--incident_num=123
Options:
--url TEXT The target URL that hosts Assemblyline.
[required]
--username TEXT Your Assemblyline account username. [required]
--apikey PATH A path to a file that contains only your
Assemblyline account API key. NOTE that this API
key requires write access. [required]
--ttl INTEGER The amount of time that you want your Assemblyline
submissions to live on the Assemblyline system (in
days).
--classification TEXT The classification level for each file submitted
to Assemblyline. [required]
--service_selection TEXT A comma-separated list (no spaces!) of service
names (case-sensitive) to send files to. If not provided, all
services will be selected.
-t, --is_test A flag that indicates that you're running a test.
--path PATH The directory path containing files that you want
to submit to Assemblyline. [required]
-f, --fresh Restart ingestion from the beginning.
--incident_num TEXT The incident number for each file to be associated
with. [required]
--resubmit-dynamic All files that score higher than 500 will be
resubmitted for dynamic analysis.
--alert Generate alerts for this submission.
--threads INTEGER Number of threads that will ingest files to
Assemblyline.
--dedup_hashes Only submit files with unique hashes. If you want
100% file coverage in a given path, do not use
this flag
--priority INTEGER Provide a priority number which will cause the
ingestion to go to a specific priority queue.
--do_not_verify_ssl Ignore SSL errors (insecure!)
--help Show this message and exit.
al-incident-analyzer --help
Usage: al-incident-analyzer [OPTIONS] COMMAND [ARGS]...
Example: al-incident-analyzer --url="https://<domain-of-
Assemblyline-instance>" --username="<user-name>"
--apikey="/path/to/file/containing/apikey" --incident_num=123
Options:
--url TEXT The target URL that hosts Assemblyline. [required]
-u, --username TEXT Your Assemblyline account username. [required]
--apikey PATH A path to a file that contains only your Assemblyline
account API key. NOTE that this API key requires write
access. [required]
--min_score INTEGER The minimum score for files that we want to query from
Assemblyline.
--incident_num TEXT The incident number for each file to be associated
with. [required]
-t, --is_test A flag that indicates that you're running a test.
--help Show this message and exit.
Regardez le rapport dans report.csv. Ce fichier contient un rapport des détections.
al-incident-downloader --help
Usage: al-incident-downloader [OPTIONS] COMMAND [ARGS]...
Example: al-incident-downloader --url="https://<domain-of-
Assemblyline-instance>" --username="<user-name>"
--apikey="/path/to/file/containing/apikey" --incident_num=123
--max_score=100 --download_path=/path/to/where/you/want/downloads
--upload_path=/path/from/where/files/were/uploaded/from
Options:
--url TEXT The target URL that hosts Assemblyline.
[required]
-u, --username TEXT Your Assemblyline account username.
[required]
--apikey PATH A path to a file that contains only your
Assemblyline account API key. NOTE that this
API key requires read access. [required]
--max_score INTEGER The maximum score for files that we want to
download from Assemblyline. [required]
--incident_num TEXT The incident number that each file is
associated with. [required]
--download_path PATH The path to the folder that we will download
files to. [required]
--upload_path PATH The base path from which the files were
ingested from.
[required]
-t, --is_test A flag that indicates that you're running a
test.
--num_of_downloaders INTEGER The number of threads that will be created to
facilitate downloading the files.
--do_not_verify_ssl Verify SSL when creating and using the
Assemblyline Client.
--help Show this message and exit.
Tous les fichiers sans détections seront téléchargé dans le dossier choisi.