How do I use/troubleshoot BrowserHistory artifact collection filter? #4752
Comments
|
joachimmetz
changed the title
$ pinfo.py hist.db
************************** Plaso Storage Information ***************************
Filename : hist.db
Format version : 20230327
Serialization format : json
|
what do the debug logs tell you or the user accounts in the hist.db database ? Also see: https://plaso.readthedocs.io/en/latest/sources/Troubleshooting.html |
|
First at all, thank you very much for your help.
The hist.db doesnt have any events: $ pinfo.py hist.db
************************** Plaso Storage Information ***************************
Filename : hist.db
Format version : 20230327
Serialization format : json
--------------------------------------------------------------------------------
*********************************** Sessions ***********************************
d962f0e8-627f-44dc-9436-9803034faf74 : 2023-11-04T10:37:28.546754+00:00
--------------------------------------------------------------------------------
******************************** Event sources *********************************
Total : 0
--------------------------------------------------------------------------------
No events stored.
No events labels stored.
No warnings stored.
No analysis reports stored.I see in the logs lines like these: 2023-11-04 10:37:46,357 [DEBUG] (MainProcess) PID:24890 <artifact_filters> building find spec from path: C:\Users\nromanoff\AppData\Local\Microsoft\Windows\WebCache\WebCacheV*.dat
2023-11-04 10:37:46,357 [WARNING] (MainProcess) PID:24890 <artifact_filters> The path filter must be defined as an absolute path: "C:\Users\nromanoff\AppData\Local\Microsoft\Windows\WebCache\WebCacheV*.dat"
2023-11-04 10:37:46,357 [DEBUG] (MainProcess) PID:24890 <artifact_filters> building find spec from path: C:\Users\rsydow-a\AppData\Local\Microsoft\Windows\WebCache\WebCacheV*.dat
2023-11-04 10:37:46,357 [WARNING] (MainProcess) PID:24890 <artifact_filters> The path filter must be defined as an absolute path: "C:\Users\rsydow-a\AppData\Local\Microsoft\Windows\WebCache\WebCacheV*.dat"I have been testing and if I change in the artifact definitions the %%users.*%% variables to the corresponding \Users\*\AppData.... , it works. Snippet: ....
- '%%users.localappdata%%\Microsoft\Windows\WebCache\WebCacheV*.dat'
To:
- '\Users\*\AppData\Local\Microsoft\Windows\WebCache\WebCacheV*.dat'Then I ran log2timeline again with the BrowserHistory artifact $ log2timeline.py -d --parsers webhist --artifact_filters BrowserHistory --vss_store=none --storage_file history.db /mnt/hgfs/imgs/base-rd-04-cdrive.E01This way the warnings disappear and I get the Internet Explorer history. $ pinfo.py history.db
************************** Plaso Storage Information ***************************
Filename : history.db
Format version : 20230327
Serialization format : json
--------------------------------------------------------------------------------
*********************************** Sessions ***********************************
f87fe762-2b36-47a3-824e-f40ee8cc2d32 : 2023-11-04T19:01:55.014016+00:00
--------------------------------------------------------------------------------
******************************** Event sources *********************************
Total : 4
--------------------------------------------------------------------------------
************************* Events generated per parser **************************
Parser (plugin) name : Number of events
--------------------------------------------------------------------------------
msie_webcache : 2995
Total : 2995
--------------------------------------------------------------------------------
No events labels stored.
******************* Extraction warnings generated per parser *******************
Parser (plugin) name : Number of warnings
--------------------------------------------------------------------------------
esedb/msie_webcache : 6
--------------------------------------------------------------------------------
************** Path specifications with most extraction warnings ***************
Number of warnings : Pathspec
--------------------------------------------------------------------------------
3 : type: OS, location: /mnt/hgfs/imgs/base-rd-04-cdrive.E01
: type: EWF
: type: NTFS, location:
\Users\administrator.shieldbase\AppData\Local\Microsoft\Windows\WebCache\WebCacheV01.dat,
MFT attribute: 2, MFT entry: 6867
3 : type: OS, location: /mnt/hgfs/imgs/base-rd-04-cdrive.E01
: type: EWF
: type: NTFS, location:
\Users\nromanoff\AppData\Local\Microsoft\Windows\WebCache\WebCacheV01.dat,
MFT attribute: 2, MFT entry: 12178
--------------------------------------------------------------------------------
No analysis reports stored.Should I change all %%users.*%% variables in the artifact yamls and set absolute paths avoiding these user variables? Thank you very much again. |
joachimmetz
added
the
needs closer look
|
I'll have a closer look when time permits. |
|
Hi again! I have been debugging in the artifact_filters.py file and the _BuildFindSpecsFromFileSourcePath function. with two artifact filters that are a clear example of the observed behavior.
The function is: def _BuildFindSpecsFromFileSourcePath(
self, source_path, path_separator, environment_variables, user_accounts):
"""Builds find specifications from a file source type.
Args:
source_path (str): file system path defined by the source.
path_separator (str): file system path segment separator.
environment_variables (list[EnvironmentVariableArtifact]):
environment variables.
user_accounts (list[UserAccountArtifact]): user accounts.
Returns:
list[dfvfs.FindSpec]: find specifications for the file source type.
"""
find_specs = []
for path_glob in path_helper.PathHelper.ExpandGlobStars(
source_path, path_separator):
logger.debug('building find spec from path glob: {0:s}'.format(
path_glob))
for path in path_helper.PathHelper.ExpandUsersVariablePath(
path_glob, path_separator, user_accounts):
logger.debug('building find spec from path: {0:s}'.format(path))
if '%' in path:
path = path_helper.PathHelper.ExpandWindowsPath(
path, environment_variables)
logger.debug('building find spec from expanded path: {0:s}'.format(
path))
if not path.startswith(path_separator):
logger.warning((
'The path filter must be defined as an absolute path: '
'"{0:s}"').format(path))
continue
try:
find_spec = dfvfs_file_system_searcher.FindSpec(
case_sensitive=False, location_glob=path,
location_separator=path_separator)
except ValueError as exception:
logger.error((
'Unable to build find specification for path: "{0:s}" with '
'error: {1!s}').format(path, exception))
continue
find_specs.append(find_spec)
return find_specsThe WindowsSystemRegistryFiles filter does everything as expected. 2024-02-11 12:40:43,680 [DEBUG] (MainProcess) PID:84266 <artifact_filters> building find spec from path glob: %%environ_systemroot%%\System32\config\SYSTEM
2024-02-11 12:40:43,681 [DEBUG] (MainProcess) PID:84266 <artifact_filters> building find spec from path: %%environ_systemroot%%\System32\config\SYSTEM
2024-02-11 12:40:43,681 [DEBUG] (MainProcess) PID:84266 <artifact_filters> building find spec from expanded path: \Windows\System32\config\SYSTEMThe returned path starts with "\" so it continues the execution and gets the right path. With WindowsUserRegistryFiles and the user variables %%user.*%% the returned path starts with "C:" and not with the path_separator "\" so it enters this if: if not path.startswith(path_separator):
logger.warning((
'The path filter must be defined as an absolute path: '
'"{0:s}"').format(path))
continueAnd returns the warning we can see in the logs: 2024-02-11 12:44:50,602 [DEBUG] (MainProcess) PID:84295 <artifact_filters> building find spec from path: C:\Users\nromanoff\NTUSER.DAT
2024-02-11 12:44:50,602 [WARNING] (MainProcess) PID:84295 <artifact_filters> The path filter must be defined as an absolute path: "C:\Users\nromanoff\NTUSER.DAT"And is never saved in the variable find_spec because of the continue command in the previous if. try:
find_spec = dfvfs_file_system_searcher.FindSpec(
case_sensitive=False, location_glob=path,
location_separator=path_separator)Now, we add a new if to remove "C:" string from the returned path if path.startswith("C:"):
path = path[2:]
logger.debug(('ROBI REMOVES STRING C: '
'"{0:s}"').format(path))This time the path starts with the expected path_separator "\" and the execution continues: 2024-02-11 13:02:28,607 [DEBUG] (MainProcess) PID:84369 <artifact_filters> building find spec from path: C:\Users\nromanoff\NTUSER.DAT
2024-02-11 13:02:28,607 [DEBUG] (MainProcess) PID:84369 <artifact_filters> ROBI REMOVES STRING C: "\Users\nromanoff\NTUSER.DAT"And it works! Let's try the original command that I opened this issue with. If I run the BrowserHistory again it works perfectly: log2timeline.py -d --parsers webhist --artifact_filters BrowserHistory --vss_store=none --storage_file hist.db /mnt/hgfs/imgs/base-rd-04-cdrive.E01
psort.py -o l2tcsv -w hist.csv hist.dbI get back what I expected: .....
12/05/2018,14:50:11,UTC,....,WEBHIST,MSIE WebCache container record,Expiration Time,-,BASE-RD-04,URL: Visited: spsql@https://login.live.com/oauth20_authorize.srf?client_id=00...,URL: ........
12/10/2018,01:54:52,UTC,....,WEBHIST,MSIE WebCache container record,Expiration Time,-,BASE-RD-04,URL: Visited: nromanoff@https://login.live.com/login.srf?wa=wsignin1.0&rpsnv=...,URL: Visit..
.....I don't know if this change may have an impact on other Plaso capabilities. I am not a Python expert. |
Describe the problem:
When I run log2timeline with BrowserHistory artifact I dont get any results nor do I see any error.
To Reproduce:
Plaso Version: Latest: 20230717
OS Version; Ubuntu 22.04.3 LTS (Fresh install)
Source data: base-rd-04-cdrive.E01 from FOR508
Installation Method: I installed plaso with the recommendations of the official documentation (add universe; add ppa; and apt install plaso-tools).
Steps to reproduce:
Run log2timeline with BrowserHistory artifact on base-rd-04 image from FOR508 with this command:
Command output:
Log Output:
Then I ran psort in this way and no results found:
The same happens if I run with docker.
Expected Behavior
Get browser history from an image
The text was updated successfully, but these errors were encountered: