Going beyond log files, accepting memory as its own format. đ
Although, logging is a powerful way to troubleshoot and investigate events itâs often limited by the initial input format of the logs during the collection process.
As the complexity of attacks increase, itâs almost natural for defensive capabilities to also evolveâââparticularly in the logging capabilities area. Seasoned attackers focus their operational efforts on advanced infiltration techniques but also on anti-forensics techniques to erase evidences of their visits as we have seen in the some of the files leaked by TheShadowBrokers last year.
The evolution of backdoors, such as in-memory kernel-mode backdoors like DOUBLEPULSAR, makes it even harder for traditional log capture and analysis softwares to understand when an attacker was in your network, and for how long.
âYour perimeter is not the boundary of your network itâs the boundary of your telemetryââââthe grugq
Workflow đ
As you can see above, we designed a four-step compressive process for organizations willing to increase the health-check process of their critical assets by integrating memory logging/forensics to their capabilities and processes.
Identify đ
Lately, more and more incidents against financial institutions have been publicly reported in the media. As part of defining your perimeter of defense, youâll identify what âcritical assetsâ such as servers you want to increase your visibility on. Those critical assets can be production servers or even honeypots, that youâd tightly control.
Capture đ
Memory acquisition of a running system is a very simple & straightforward process which can be achieved quickly through our standalone utility, Comae DumpIt, downloadable for free on our platform at https://my.comae.io.
Comae DumpIt generates full memory Microsoft crash dumps, rather that legacy raw memory dumps, on the fly for interoperability and efficiency purposes. Furthermore, Comae DumpIt is standalone allowing the user more flexibility (PsExec, PsRemote etc.) in how to deploy the utility without having to deal with the friction of installing an agent or endpoint.
There are three main scenarios that we identified where memory acquisition is valuable:
-
Manually, after an incident. This scenario is the most common for memory forensics.
-
As part of orchestration. Many of our users leverage memory forensics to have a more comprehensive understanding of the alerts returned by their endpoints. This could be automated through playbooks to automatically perform a memory acquisition after a suspicious activity has been detected by a third-party endpoint.
Configuring a scheduled task for DumpIt.
-
Periodically through scheduled jobs. This scenario is the most interesting for increasing logging capabilities, as acquisition is triggered by periodical event (once a month, once a week, etcâŚ) before archiving the memory copies locally (e.g. a file share) as youâd normally do with logs. Configurable directly through PowerShell or Windowsâ Task Scheduler, this allows the user to keep periodical copies of their machines states to later enable possibilities such as retro-threat-hunting.
$Action = New-ScheduledTaskAction -Execute C:\Comae-Toolkit-3.0.20180207.1\x64\DumpIt.exe -Argument â/Q /Râ -WorkingDirectory âC:\Logsâ $Trigger = New-ScheduledTaskTrigger -Weekly -At â9amâ -DaysOfWeek âMondayâ $Settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -StartWhenAvailable -Hidden Register-ScheduledTask -Action $Action -Trigger $Trigger -Settings $Settings -TaskName âDumpItâ -User System -RunLevel Highest
Performance and scalability are vital metrics when it comes to memory acquisition of large servers.
One of the main pain point reported from our users, before they adopting Comae DumpIt, was that several vendors have issues during memory acquisition for servers with more than 64GBs of RAM which prevented them to investigate large servers.
Attached are the results of our acquisition benchmark on traditional desktops and servers with up to half a terabyte of memory. As you can see, the acquisition time is significantly faster (less than 10 minutes) if our built-in compression flag is enabled on such servers.
Archive đ
As large organizations with several terabytes of memory images would usually opt to store and keep memory images locally in a corporate file share. Those locally stored images can be sent later to our analysis platform Comae Stardust, or be preprocessed to only send the extracted metadata returned by our preprocessing utility Comae Dmp2Json.
During the acquisition, users can also chose to compress the output file on the fly using the /R or /COMPRESS options in order to create a .zdmp (Comae compressed crash dump) file. In addition of file size optimization, this also allows the acquisition to be significantly faster by reducing the number of I/O operations to diskâââespecially when acquiring the memory from servers with a gargantuan amount of physical memory.
Analyze đ
Memory images management can easily be done locally using our PowerShell command line interface available on GitHub. The below command is a sample command that allows the users to pre-process several acquired images from a given folder into their Comae metadata snapshot format before uploading them to our analysis platform.
Get-ChildItem -Path C:\Logs -File | ForEach-Object { Convert-DumpFileToSnapshot -FilePath $_.FullName -Directory "C:\Snapshots" } Get-ChildItem -Path C:\Snapshots -File | ForEach-Object { Send-ComaeSnapshot -Key "<APIKEY>" -Path $_.FullName -ItemType "File" }
Rethinking Logging đ
Donât hesitate to share your comments or feedback on our platform and use cases you encountered where memory forensics was a very valuable asset for your company.