{"id":16527,"date":"2020-01-28T16:05:00","date_gmt":"2020-01-28T15:05:00","guid":{"rendered":"https:\/\/herolab-usd.formwandler.rocks\/news-tool-zur-forensischen-datenanalyse-auf-dfrws-usa-2019\/"},"modified":"2021-08-04T14:51:12","modified_gmt":"2021-08-04T12:51:12","slug":"news-tool-for-forensic-data-analysis-at-dfrws-usa-2019","status":"publish","type":"post","link":"https:\/\/herolab.usd.de\/en\/news-tool-for-forensic-data-analysis-at-dfrws-usa-2019\/","title":{"rendered":"Tool for Forensic Data Analysis at DFRWS USA 2019"},"content":{"rendered":"\n<p>Christian Meng, usd HeroLab Consultant, developed an open source tool for forensic data analysis and recovery of deleted SQLite data sets, \u201c<a href=\"https:\/\/github.com\/bring2lite\/bring2lite\" target=\"_blank\" rel=\"noopener\">bring2lite<\/a>\u201d as part of his final thesis at the University of Applied Science Darmstadt together with Prof. Dr. Harald Baier. In July, Prof. Dr. Harald Baier presented the tool at&nbsp;<a href=\"http:\/\/www.dfrws.org\/conferences\/dfrws-usa-2019\" target=\"_blank\" rel=\"noopener\">DFRWS USA<\/a>, one of the leading conferences on digital forensics.<\/p>\n\n\n\n<p>\u201cResearch in this area is essential for adapting methodological and technical basics for forensic analyses to the ever-growing problems,\u201d explains Christian Meng. \u201cbring2lite is a tool that helps overcoming certain challenges in digital forensics. I was very happy to be able to present our tool to the expert audience at DFRWS USA,\u201d adds Prof. Dr. Harald Baier.<\/p>\n\n\n\n<p><strong>Christian Meng gives an introduction to the topic:<\/strong><\/p>\n\n\n\n<p>Applications, such as WhatsApp, Skype or Firefox, all use the database SQLite. From an IT forensics point of view, being able to recover deleted data sets is a sensible and useful feature. My thesis had the following objectives:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Analyze under which circumstances deleted data sets can be restored in the database<\/li><li>\u2022 Develop an open source tool based on this analysis (\u201c<a href=\"https:\/\/github.com\/bring2lite\/bring2lite\" target=\"_blank\" rel=\"noopener\">bring2lite<\/a>\u201c)<\/li><li>Compare \u201c<a href=\"https:\/\/github.com\/bring2lite\/bring2lite\" target=\"_blank\" rel=\"noopener\">bring2lite<\/a>\u201d with other, already established, tools<\/li><\/ul>\n\n\n\n<p>Analysis:<\/p>\n\n\n\n<p>Within the scope of the analysis, 6 scenarios were defined that represent the insert and delete operations of an SQLite database. The scenarios cover various complexity levels. One example is inserting a new record that creates exactly one new page and a new record within the binary tree. Each of the methods used to insert data was then replicated with different database settings. In consequence, 72 new distinct databases were created. Examining these databases, one could note database settings such as&nbsp;<em>secure_delete=0<\/em>,&nbsp;<em>auto_vacuum=0<\/em>&nbsp;or&nbsp;<em>journal_mode=WAL&nbsp;<\/em>benefited data extraction.<\/p>\n\n\n\n<p>Functional Principle of \u201c<a class=\"rank-math-link\" href=\"https:\/\/github.com\/bring2lite\/bring2lite\" target=\"_blank\" rel=\"noopener\">bring2lite<\/a>\u201d:<\/p>\n\n\n\n<p>The developed tool processes data structures as specified on the official website of the SQLite maintainers. The data processing consists of 7 individual steps, which can be seen in Figure 7. The last two steps of the procedure include creating additional SQLite-specific files, which will be ignored in this research. These files could potentially contain deleted data, however, the research focuses on restoring data in the actual database.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"699\" height=\"181\" src=\"https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-1.png\" alt=\"\" class=\"wp-image-15575\" srcset=\"https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-1.png 699w, https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-1-480x124.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 699px, 100vw\" \/><\/figure>\n\n\n\n<p>In the first step, the SQLite master table is processed. This table contains the scheme of the database and the corresponding root node of the binary tree, in which the actual data resides. By doing this, one can get an understanding of how the formats and data types of the individual tables, and therefore the deleted data, must look like. Steps 3 to 5 represent the actual processing of the deleted data. Free blocks (step 3) are individual, deleted data records within normal SQLite files. These free blocks are structured as linked lists. In consequence, only the first 4 bytes of a record will be overwritten. Due to the missing 4 bytes, SQLite cannot process this data anymore. Additionally, the first 2 bytes of the database represent the address of the next free block, which can be used to find and extract all free blocks in the database.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"768\" height=\"530\" src=\"https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-2.png\" alt=\"\" class=\"wp-image-15574\" srcset=\"https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-2.png 768w, https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-2-480x331.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 768px, 100vw\" \/><\/figure>\n\n\n\n<p>The unallocated area, which is processed in step 4, is located between the header of a binary tree page (node) and the beginning of the data area, as shown in the figure above. When data at the beginning of the data section is deleted, only a pointer is changed to the new start of active data. Therefore, the data that was supposed to be deleted remains intact. In order to find this data, the unallocated area must simply be extracted and shrinked to the size of the actual data.<\/p>\n\n\n\n<p>All the free lists will be extracted and processed in step 5. A free list is generated by SQLite every time a whole binary tree page is removed. When this occurs, the content of the pages will not be deleted, but the pointer pointing to this page will be removed from the regular binary tree and converted into a free list. This structure can still contain a forensic analyst might find interesting.<\/p>\n\n\n\n<p>Example:<\/p>\n\n\n\n<p>Once installed, the software \u201e<a class=\"rank-math-link\" href=\"https:\/\/github.com\/bring2lite\/bring2lite\" target=\"_blank\" rel=\"noopener\">bring2lite<\/a>\u201c can be executed like this:&nbsp;<em>python3 bring2lite\/main.py<\/em>&nbsp;<\/p>\n\n\n\n<p>Then, the tool will print the following help page:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"713\" height=\"441\" src=\"https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-3.png\" alt=\"\" class=\"wp-image-15573\" srcset=\"https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-3.png 713w, https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-3-480x297.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 713px, 100vw\" \/><\/figure>\n\n\n\n<p>If a user specifies a path to an SQLite database or a directory, respectively, the processing of the files looks like this:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"199\" src=\"https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-4-1024x199.png\" alt=\"\" class=\"wp-image-15572\" srcset=\"https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-4-980x190.png 980w, https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-4-480x93.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) and (max-width: 980px) 980px, (min-width: 981px) 1024px, 100vw\" \/><\/figure>\n\n\n\n<p>For each file generated using \u201c<a class=\"rank-math-link\" href=\"https:\/\/github.com\/bring2lite\/bring2lite\" target=\"_blank\" rel=\"noopener\">bring2lite<\/a>\u201d, a hash is calculated to prevent unwanted changes to the files. \u201c<a class=\"rank-math-link\" href=\"https:\/\/github.com\/bring2lite\/bring2lite\" target=\"_blank\" rel=\"noopener\">bring2lite<\/a>\u201dcreates a main directory for each database and also creates a directory for each structure that contains deleted records. For example, the database&nbsp;<em>0A-01.db<\/em>&nbsp;looks like this:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"953\" height=\"482\" src=\"https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-5.png\" alt=\"\" class=\"wp-image-15571\" srcset=\"https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-5.png 953w, https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-5-480x243.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 953px, 100vw\" \/><\/figure>\n\n\n\n<p>As illustrated in the figure, deleted database records could be extracted from the database using free lists. Also the database scheme is extracted into a separate directory. This is helpful if the database contains more than one table, and thus, more schemes are of importance.<\/p>\n\n\n\n<p>The database file which represents an example database is located in the directory&nbsp;<em>\/0A-01.db\/freelists\/freelist_trunk_pages<\/em>&nbsp;and is named&nbsp;<em>2-page.log<\/em>, where the name describes the page in which the records are found. All the extracted data is stored in the CSV file format:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"641\" height=\"487\" src=\"https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-6.png\" alt=\"\" class=\"wp-image-15570\" srcset=\"https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-6.png 641w, https:\/\/herolab.usd.de\/wp-content\/uploads\/sites\/9\/2021\/07\/usd-herolab-news-bring2lite-6-480x365.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 641px, 100vw\" \/><\/figure>\n\n\n\n<p>Comparison with different tools:<\/p>\n\n\n\n<p>In order to evaluate the capacity of \u201c<a class=\"rank-math-link\" href=\"https:\/\/github.com\/bring2lite\/bring2lite\" target=\"_blank\" rel=\"noopener\">bring2lite<\/a>\u201d properly, 27 standardized databases were processed and the amount of restored data records was reviewed. \u201c<a href=\"https:\/\/github.com\/bring2lite\/bring2lite\" target=\"_blank\" rel=\"noopener\">bring2lite<\/a>\u201d achieved a restoration rate of 52.9%. This result shows that \u201c<a class=\"rank-math-link\" href=\"https:\/\/github.com\/bring2lite\/bring2lite\" target=\"_blank\" rel=\"noopener\">bring2lite<\/a>\u201d is more effective than all of the 8 tools it was compared to.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Christian Meng, usd HeroLab Consultant, developed an open source tool for forensic data analysis and recovery of deleted SQLite data sets, \u201cbring2lite\u201d as part of his final thesis at the University of Applied Science Darmstadt together with Prof. Dr. Harald Baier. In July, Prof. Dr. Harald Baier presented the tool at&nbsp;DFRWS USA, one of the [&hellip;]<\/p>\n","protected":false},"author":96,"featured_media":16528,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"off","_et_pb_old_content":"","_et_gb_content_width":"","inline_featured_image":false,"footnotes":""},"categories":[76],"tags":[130,131,86],"class_list":["post-16527","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news","tag-dfrws-usa-en","tag-digitale-forensik-en","tag-security-research-en"],"_links":{"self":[{"href":"https:\/\/herolab.usd.de\/en\/wp-json\/wp\/v2\/posts\/16527","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/herolab.usd.de\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/herolab.usd.de\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/herolab.usd.de\/en\/wp-json\/wp\/v2\/users\/96"}],"replies":[{"embeddable":true,"href":"https:\/\/herolab.usd.de\/en\/wp-json\/wp\/v2\/comments?post=16527"}],"version-history":[{"count":0,"href":"https:\/\/herolab.usd.de\/en\/wp-json\/wp\/v2\/posts\/16527\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/herolab.usd.de\/en\/wp-json\/wp\/v2\/media\/16528"}],"wp:attachment":[{"href":"https:\/\/herolab.usd.de\/en\/wp-json\/wp\/v2\/media?parent=16527"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/herolab.usd.de\/en\/wp-json\/wp\/v2\/categories?post=16527"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/herolab.usd.de\/en\/wp-json\/wp\/v2\/tags?post=16527"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}