Migrating events from non-MISP tool into MISP

[GDAdamSiwek]

I am new to MISP, am curious if I can migrate events from a legacy tool into MISP, and can the attributes from the legacy events become custom attributes on a MISP event?

Any help appreciated!

[Rafiot]

Can you give us a sample? I'm not sure to understand what you mean.

[GDAdamSiwek]

Here is a subset of the columns of an event from my legacy:

1. id
2. eval_date
3. date_created
4. date_updated
5. indicator_type

Can I create an event via PyMISP, and pin these attributes to it?

[Rafiot]

I'd go for an object with each of these attributes, this is probably the easiest.

[GDAdamSiwek]

So, the procedure for this is:

1. Create Object template
2. Register Template with MISP instance
3. Create event, object and load legacy attributes into object
4. Attach object to event

?

[Rafiot]

Pretty much yes. You can also just create the template in the subdirectory where all the other templates are in PyMISP and use it directly from there.

The question you need to answer is how you want to add them into event(s): one event with all the data in objects? One event/month? One event/day? It will depend how many objects you end-up having, and what you want to do with the data.

[GDAdamSiwek]

Much THANKS for that info, but I need further clarity. The five fields I mentioned before, all five go into a single object or there are five objects, one per field?

…

________________________________ From: Raphaël Vinot ***@***.***> Sent: Wednesday, September 15, 2021 4:23 PM To: MISP/PyMISP ***@***.***> Cc: Siwek, Adam ***@***.***>; Author ***@***.***> Subject: Re: [MISP/PyMISP] Migrating events from non-MISP tool into MISP (Discussion #785) [External: Use caution with links & attachments] Pretty much yes. You can also just create the template in the subdirectory where all the other templates are in PyMISP and use it directly from there. The question you need to answer is how you want to add them into event(s): one event with all the data in objects? One event/month? One event/day? It will depend how many objects you end-up having, and what you want to do with the data. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<https://urldefense.us/v3/__https://github.com/MISP/PyMISP/discussions/785*discussioncomment-1335420__;Iw!!JRQnnSFuzw7wjAKq6ti6!isy1-0RH6bnzkHf7HaaMmr17Ps6jDGsQVSejNIXshwnTPpdFEIddpgVOTlx5OAY$>, or unsubscribe<https://urldefense.us/v3/__https://github.com/notifications/unsubscribe-auth/AVUUVKSMWU6NUY6SOCCNLKTUCD6DLANCNFSM5ECZYGYQ__;!!JRQnnSFuzw7wjAKq6ti6!isy1-0RH6bnzkHf7HaaMmr17Ps6jDGsQVSejNIXshwnTPpdFEIddpgVOOXlVMhI$>. Triage notifications on the go with GitHub Mobile for iOS<https://urldefense.us/v3/__https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!JRQnnSFuzw7wjAKq6ti6!isy1-0RH6bnzkHf7HaaMmr17Ps6jDGsQVSejNIXshwnTPpdFEIddpgVOY19JBBE$> or Android<https://urldefense.us/v3/__https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign*3Dnotification-email*26utm_medium*3Demail*26utm_source*3Dgithub__;JSUlJSU!!JRQnnSFuzw7wjAKq6ti6!isy1-0RH6bnzkHf7HaaMmr17Ps6jDGsQVSejNIXshwnTPpdFEIddpgVOYJtPhjE$>.

[Rafiot]

an object contains attributes, so you will have N attributes in your object. One other thing you might want to do is to use the first_seen/last_seen field in the object itself to represent created and updated, if it makes sense.

The other thing I'm not clear about is where the actual indicator is? You're talking of indicator type, but not the indicator itself.

[GDAdamSiwek]

FYI, I've inherited this task( migration to MISP), so I might not have the vocabulary right about indicator types and more. But thanks for the feedback!

…

________________________________ From: Raphaël Vinot ***@***.***> Sent: Wednesday, September 15, 2021 5:10 PM To: MISP/PyMISP ***@***.***> Cc: Siwek, Adam ***@***.***>; Author ***@***.***> Subject: Re: [MISP/PyMISP] Migrating events from non-MISP tool into MISP (Discussion #785) [External: Use caution with links & attachments] an object contains attributes, so you will have N attributes in your object. One other thing you might want to do is to use the first_seen/last_seen field in the object itself to represent created and updated, if it makes sense. The other thing I'm not clear about is where the actual indicator is? You're talking of indicator type, but not the indicator itself. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<https://urldefense.us/v3/__https://github.com/MISP/PyMISP/discussions/785*discussioncomment-1335561__;Iw!!JRQnnSFuzw7wjAKq6ti6!ktcmEQETv29ojgBl46gryxKFLuoWkxcgb_Nf3M785wfTZ258xBWZ61UAqBKIrjs$>, or unsubscribe<https://urldefense.us/v3/__https://github.com/notifications/unsubscribe-auth/AVUUVKUEZDLZTXMQEE2MOUTUCEDUXANCNFSM5ECZYGYQ__;!!JRQnnSFuzw7wjAKq6ti6!ktcmEQETv29ojgBl46gryxKFLuoWkxcgb_Nf3M785wfTZ258xBWZ61UAMNwE-Hs$>. Triage notifications on the go with GitHub Mobile for iOS<https://urldefense.us/v3/__https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!JRQnnSFuzw7wjAKq6ti6!ktcmEQETv29ojgBl46gryxKFLuoWkxcgb_Nf3M785wfTZ258xBWZ61UAxx1w51k$> or Android<https://urldefense.us/v3/__https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign*3Dnotification-email*26utm_medium*3Demail*26utm_source*3Dgithub__;JSUlJSU!!JRQnnSFuzw7wjAKq6ti6!ktcmEQETv29ojgBl46gryxKFLuoWkxcgb_Nf3M785wfTZ258xBWZ61UAzEM_8Z8$>.

[Rafiot]

You're welcome! If you need further help, don't hesitate. And if you can share an actual sample, it might make things easier.

[GDAdamSiwek]

I'm not able to share sample data anytime soon, but , shifting lanes, I have a question about searches of MISP object. Can the web UI perform queries on the custom object attributes?

…

________________________________ From: Raphaël Vinot ***@***.***> Sent: Thursday, September 16, 2021 3:22 AM To: MISP/PyMISP ***@***.***> Cc: Siwek, Adam ***@***.***>; Author ***@***.***> Subject: Re: [MISP/PyMISP] Migrating events from non-MISP tool into MISP (Discussion #785) [External: Use caution with links & attachments] You're welcome! If you need further help, don't hesitate. And if you can share an actual sample, it might make things easier. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<https://urldefense.us/v3/__https://github.com/MISP/PyMISP/discussions/785*discussioncomment-1336960__;Iw!!JRQnnSFuzw7wjAKq6ti6!iwPrapD3HR3PEZZhm767_64sObuibmWh1Z8lQlEwbQx7lOl-71a2btFfY8CceW0$>, or unsubscribe<https://urldefense.us/v3/__https://github.com/notifications/unsubscribe-auth/AVUUVKQUBTBFC2I4UBNFNTTUCGLJ7ANCNFSM5ECZYGYQ__;!!JRQnnSFuzw7wjAKq6ti6!iwPrapD3HR3PEZZhm767_64sObuibmWh1Z8lQlEwbQx7lOl-71a2btFfNzgOjrM$>. Triage notifications on the go with GitHub Mobile for iOS<https://urldefense.us/v3/__https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!JRQnnSFuzw7wjAKq6ti6!iwPrapD3HR3PEZZhm767_64sObuibmWh1Z8lQlEwbQx7lOl-71a2btFfMnBKssI$> or Android<https://urldefense.us/v3/__https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign*3Dnotification-email*26utm_medium*3Demail*26utm_source*3Dgithub__;JSUlJSU!!JRQnnSFuzw7wjAKq6ti6!iwPrapD3HR3PEZZhm767_64sObuibmWh1Z8lQlEwbQx7lOl-71a2btFfXohQII8$>.

[Rafiot]

The attributes aren't custom, only the objects containing standard attributes are, so answer is yes.

[GDAdamSiwek]

Raphaël, The template file to create a custom object exists and resides in the objects/templates directory. But how do enforce the template during MISP object creation?

…

________________________________ From: Raphaël Vinot ***@***.***> Sent: Thursday, September 16, 2021 11:22 AM To: MISP/PyMISP ***@***.***> Cc: Siwek, Adam ***@***.***>; Author ***@***.***> Subject: Re: [MISP/PyMISP] Migrating events from non-MISP tool into MISP (Discussion #785) [External: Use caution with links & attachments] The attributes aren't custom, only the objects containing standard attributes are, so answer is yes. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<https://urldefense.us/v3/__https://github.com/MISP/PyMISP/discussions/785*discussioncomment-1342926__;Iw!!JRQnnSFuzw7wjAKq6ti6!iZuiVbqvT3qsHYtyJ_K5o7tv6yGqA9UzTN7okoKhP008C93NRaXDeMh6VGacbcw$>, or unsubscribe<https://urldefense.us/v3/__https://github.com/notifications/unsubscribe-auth/AVUUVKS6PTG5S6JNWJKTGQLUCIDU7ANCNFSM5ECZYGYQ__;!!JRQnnSFuzw7wjAKq6ti6!iZuiVbqvT3qsHYtyJ_K5o7tv6yGqA9UzTN7okoKhP008C93NRaXDeMh6rPJeSJM$>. Triage notifications on the go with GitHub Mobile for iOS<https://urldefense.us/v3/__https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!JRQnnSFuzw7wjAKq6ti6!iZuiVbqvT3qsHYtyJ_K5o7tv6yGqA9UzTN7okoKhP008C93NRaXDeMh65dOftj8$> or Android<https://urldefense.us/v3/__https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign*3Dnotification-email*26utm_medium*3Demail*26utm_source*3Dgithub__;JSUlJSU!!JRQnnSFuzw7wjAKq6ti6!iZuiVbqvT3qsHYtyJ_K5o7tv6yGqA9UzTN7okoKhP008C93NRaXDeMh6SC0dcvQ$>.

[GDAdamSiwek]

Raphaël, OK, I created the Object template file, and located it in the MISP misp_object template directory. But how do I enforce that template during object creation?

[Rafiot]

There are a few moving parts, so it's a bit hard to answer. As you have a custom template, I'm assuming that you installed PyMISP from the repository.

You can look at this tutorial: https://github.com/MISP/PyMISP/blob/main/docs/tutorial/FullOverview.ipynb (search for Use locally defined object templates).