LogViewer: viewing and searching large text files

Posted: 02/01/2018 in free stuff
Tags:

LogViewer is designed to work with any large text files so that even very large files can be opened, viewed and searched.

Its original use case is for DFIR cases that involve log analysis. Whilst I use grep (well actually I use sift to extract data from logs, it is handy to be able to view log files, search for terms, hide lines whilst you get an idea what the log file contains, what actions are being performed.

The use of the custom control would make debugging any future issues a lot harder, so after a bit of thought, I used the ObjectListView library. The ObjectListView library is a custom list view control for use with .Net projects, I have used it extensively as it is easy to use and works with large datasets.

The core operation of LogViewer works in the same way as Highlighter e.g. parse the file, find the line offsets and line lengths, then when a line is needed for display, an existing file stream is used to seek to the offset, and then read X bytes.

I tested the v0.0.1 release of LogViewer against v1.1.3 of Mandiant Highlighter. My test log file was 1.2 GB and had 4.4 million rows. The following shows the operation and duration of the operation to compare:

  • Load (LogViewer): 15s
  • Load (Highlighter): 42s
  • Search (LogViewer): 1m 5s
  • Search (Highlighter): 2m 15s
  • Show Only Highlighted (LogViewer): 2s (+ the search operation above 1m 5s) Total: 1m 7s
  • Show Only Highlighted (Highlighter): Killed after 35m

The main reasons for this being faster is that it has removed some functionality and I have optimised the file load code so that there is less memory allocation and unnecessary checks/logic, plus Highlighter does some Md5 calcs etc.

Features

  • Very fast
  • Supports huge files
  • Cumulative search
  • Can disable/enable search terms that are cumulative and the results are displayed instantly
  • Export current view
  • Show/Hide matched lines
  • Four search modes (SubString Case Insensitive, SubString Case Sensitive, Regex Case Insensitive, Regex Case Sensitive)

General

  • To stop an action such as load, search, export, you double click on the progress bar, located in the status bar
  • The context menu holds the majority of actions
  • Lots of stuff to be fixed/added!

Download

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s