template_log_parser package

Submodules

template_log_parser.log_functions module

template_log_parser.log_functions.filter_line(line: str, match: str | list[str] | None = None, eliminate: str | list[str] | None = None, match_type: Literal['any', 'all'] = 'any', eliminate_type: Literal['any', 'all'] = 'any') → bool[source]

Return True if log file line adheres to filter criteria

Eliminate applied second, and therefore supersedes any words in match should conflicts exist.

Parameters:

line (str) – A single log file line
match (str, List[str], None) – (optional) A single word or list of words must be present within the line otherwise dropped.
eliminate (str, List[str], None) – (optional) A single word or a list of words if present within line will result in it being dropped
match_type (Literal["any", "all"]) – (optional) criteria to determine if any words must be present to match, or all words
eliminate_type (Literal["any", "all"]) – (optional) criteria to determine if any words must be present to eliminate, or all words

Returns:

True if string contains the match criteria and does not contain the eliminate criteria, else False

Return type:

bool

template_log_parser.log_functions.get_lines_from_file(f: str | Path | BytesIO | StringIO | TextIOBase) → list[str][source]

Return a list of strings from a flat file

Parameters:: f (str, Path, BytesIO, StringIO, TextIOBase) – Path to file or filelike object, most commonly in the format of some_log_process.log
Returns:: list of string
Return type:: list[str]
Raises:: ValueError – If wrong file type is provided

Return a Pandas DataFrame with named columns as specified by templates

Parameters:

file (str, Path, BytesIO, StringIO, TextIOBase) – Path to file or filelike object, most commonly in the format of some_log_process.log
templates (list[SimpleTemplate]) – formatted as a list of namedtuple (SimpleTemplate) [(compiled_template, event_type, search_string), …]
match (str, list[str], None) – (optional) A single word or list of words must be present within the line otherwise dropped.
eliminate (str, list[str], None) – (optional) A single word or a list of words if present within line will result in it being dropped
match_type (Literal["any", "all"]) – (optional) criteria to determine if any words must be present to match, or all words
eliminate_type (Literal["any", "all"]) – (optional) criteria to determine if any words must be present to eliminate, or all words

Returns:

DataFrame with columns found in matching templates

Return type:

Pandas.DataFrame

Raises:

ValueError – If wrong file type is provided

Note:

eliminate applied second, and therefore supersedes any words in match should duplicate criteria exist.

template_log_parser.log_functions.parse_function(event: str, templates: list[SimpleTemplate]) → dict[str, str][source]

Return a dictionary of information parsed from a log file string based on matching template.

Parameters:

event (str) – String data, should ideally match a repeated format throughout a text file
templates (list[SimpleTemplate]) – formatted as a list of namedtuple (SimpleTemplate) [(compiled_template, event_type, search_string), …]

Returns:

dictionary containing: - event_type along parsed values if successful. Otherwise, {“Unparsed_text”: original_text, “event_type”: “Other”}

Return type:

dict[str, str]

Return a single Pandas Dataframe or dictionary of DataFrames whose keys are the log file event types, utilizing templates.

Parameters:

file (str, Path, BytesIO, StringIO, TextIOBase) – Path to file or filelike object, most commonly in the format of some_log_process.log
templates (list[SimpleTemplate]) – formatted as a list of namedtuple (SimpleTemplate) [(compiled_template, event_type, search_string), …]
dict_format ((optional) bool) – Return a dictionary of DataFrames when True, one large DataFrame when False, True by default
datetime_columns (List[str]) – (optional) Columns to be converted using Pandas.to_datetime()
match (str, List[str], None) – (optional) A single word or list of words must be present within the line otherwise dropped.
eliminate (str, List[str], None) – (optional) A single word or a list of words if present within line will result in it being dropped
match_type (Literal["any", "all"]) – (optional) criteria to determine if any words must be present to match, or all words
eliminate_type (Literal["any", "all"]) – (optional) criteria to determine if any words must be present to eliminate, or all words

Returns:

dict formatted as {‘event_type_1’: df_1, ‘event_type_2’: df_2, …}, Pandas Dataframe will include all event types and all columns

Return type:

Dict[str, Pandas.DataFrame], Pandas Dataframe

template_log_parser.definitions module

class template_log_parser.definitions.SimpleTemplate(template: Parser, event_type: str, search_string: str)[source]

Bases: NamedTuple

Compiled template, event_type string, and search_string

event_type: str: Alias for field number 1

search_string: str: Alias for field number 2

template: Parser: Alias for field number 0

template_log_parser.template_functions module

template_log_parser.template_functions.compile_templates(templates: list[list[str]], search_string_criteria: Literal['find', 'copy'] = 'find') → list[SimpleTemplate][source]

Return a list of namedtuple (simple_template) after compiling templates and identifying search_string

Parameters:

templates (list[list[str]]) – list containing list of strings whose length is 2 or 3 [[template, event_type, search_string (optional)], …]
search_string_criteria (Literal["find", "copy"]) – decision to either find a search_string using regex or copy the event_type as the search_string if none provided

Returns:

list of SimpleTemplate namedtuple

Return type:

list[SimpleTemplate]

Note:

expected format: [[template, event_type, search_string], [template, event_type], …]

template_log_parser package

Submodules

template_log_parser.log_functions module

template_log_parser.definitions module

template_log_parser.template_functions module

Module contents