Normalization Manage Parsers
In brief
The parser-management documentation updates custom unifying-function examples to pass `disabled=disabled` where supported, including AgentEvent and DNS parser examples, and adds or updates parser entries.
What Defender admins need to know
Administrators creating custom normalization parsers can use the updated parameter lists to verify compatibility between unifying and source-specific parsers.
Summaries are generated from the documentation change itself.
Documentation change
The comparison below shows only the changed extract. Use the full-page view for complete context.
The syntax of the line to add is different for each schema. The following table uses the parameters defined in ParserParams in the corresponding Parsers/ASim<SchemaName>/Parsers/im<SchemaName>.yaml file in the Microsoft Sentinel repository. Replace _parser_name_ with your filtering source-specific parser's function name. Use named arguments to avoid binding values to the wrong parameters when declaration order differs.
Before adding a line, verify that your custom unifying function declares the referenced parameters and that your source-specific parser accepts them. Forward pack=pack only if the source-specific parser supports it. If your custom unifying function declares disabled, also forward disabled=disabled to source-specific parsers that support it, even when it isn't included in the schema's row below.it.
| Schema | Parser | Line to add |
|---|---|---|
| AgentEvent | Im_AgentEventCustom |
_parser_name_ (starttime=starttime, endtime=endtime, agentid_has_any=agentid_has_any, agentname_has_any=agentname_has_any, username_has_any=username_has_any, disabled=disabled, pack=pack) |
| AlertEvent | Im_AlertEventCustom |
_parser_name_ (starttime=starttime, endtime=endtime, ipaddr_has_any_prefix=ipaddr_has_any_prefix, hostname_has_any=hostname_has_any, username_has_any=username_has_any, attacktactics_has_any=attacktactics_has_any, attacktechniques_has_any=attacktechniques_has_any, threatcategory_has_any=threatcategory_has_any, alertverdict_has_any=alertverdict_has_any, eventseverity_has_any=eventseverity_has_any, disabled=disabled, pack=pack) |
| AssetEntity | Im_AssetEntityCustom |
_parser_name_ (starttime=starttime, endtime=endtime, entityid_has_any=entityid_has_any, entityname_has_any=entityname_has_any, assettype_in=assettype_in, path_has_any=path_has_any, assetowner_has_any=assetowner_has_any, entitysource_has_any=entitysource_has_any, disabled=disabled, pack=pack) |
| AuditEvent | Im_AuditEventCustom |
_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, actorusername_has_any=actorusername_has_any, operation_has_any=operation_has_any, eventtype_in=eventtype_in, eventresult=eventresult, object_has_any=object_has_any, newvalue_has_any=newvalue_has_any, disabled=disabled, pack=pack) |
| Authentication | Im_AuthenticationCustom |
_parser_name_ (starttime=starttime, endtime=endtime, username_has_any=username_has_any, targetappname_has_any=targetappname_has_any, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, srchostname_has_any=srchostname_has_any, eventtype_in=eventtype_in, eventresultdetails_in=eventresultdetails_in, eventresult=eventresult, disabled=disabled, pack=pack) |
| DhcpEvent | Im_DhcpEventCustom |
_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, srchostname_has_any=srchostname_has_any, srcusername_has_any=srcusername_has_any, eventresult=eventresult, disabled=disabled, pack=pack) |
| Dns | Im_DnsCustom |
_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr=srcipaddr, domain_has_any=domain_has_any, responsecodename=responsecodename, response_has_ipv4=response_has_ipv4, response_has_any_prefix=response_has_any_prefix, eventtype=eventtype, disabled=disabled, pack=pack) |
| EmailEvent | Im_EmailEventCustom | _parser_name_ (starttime=starttime, endtime=endtime, ipaddr_has_any_prefix=ipaddr_has_any_prefix, emailfromaddress_has_any=emailfromaddress_has_any, emailrecipient_has_any=emailrecipient_has_any, emailsubject_has_any=emailsubject_has_any, emaildirection_in=emaildirection_in, emaildeliverylocation_in=emaildeliverylocation_in, eventresult=eventresult, disabled=disabled, pack=pack) |
| FileEvent | Im_FileEventCustom |
_parser_name_ (starttime=starttime, endtime=endtime, eventtype_in=eventtype_in, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, actorusername_has_any=actorusername_has_any, targetfilepath_has_any=targetfilepath_has_any, srcfilepath_has_any=srcfilepath_has_any, hashes_has_any=hashes_has_any, dvchostname_has_any=dvchostname_has_any, disabled=disabled, pack=pack) |
| NetworkSession | Im_NetworkSessionCustom |
_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, dstipaddr_has_any_prefix=dstipaddr_has_any_prefix, ipaddr_has_any_prefix=ipaddr_has_any_prefix, dstportnumber=dstportnumber, hostname_has_any=hostname_has_any, dvcaction=dvcaction, eventresult=eventresult, disabled=disabled, pack=pack) |
| ProcessEvent | Im_ProcessEventCustom |
_parser_name_ (starttime=starttime, endtime=endtime, commandline_has_any=commandline_has_any, commandline_has_all=commandline_has_all, commandline_has_any_ip_prefix=commandline_has_any_ip_prefix, actingprocess_has_any=actingprocess_has_any, targetprocess_has_any=targetprocess_has_any, parentprocess_has_any=parentprocess_has_any, actorusername_has=actorusername_has, targetusername_has=targetusername_has, dvcipaddr_has_any_prefix=dvcipaddr_has_any_prefix, dvchostname_has_any=dvchostname_has_any, hashes_has_any=hashes_has_any, eventtype= |
| RegistryEvent | Im_RegistryEventCustom |
_parser_name_ (starttime=starttime, endtime=endtime, eventtype_in=eventtype_in, actorusername_has_any=actorusername_has_any, registrykey_has_any=registrykey_has_any, registryvalue_has_any=registryvalue_has_any, registrydata_has_any=registrydata_has_any, dvchostname_has_any=dvchostname_has_any, disabled=disabled, pack=pack) |
| UserManagement | Im_UserManagementCustom |
_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, targetusername_has_any=targetusername_has_any, actorusername_has_any=actorusername_has_any, eventtype_in=eventtype_in, disabled=disabled, pack=pack) |
| WebSession | Im_WebSessionCustom |
_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, ipaddr_has_any_prefix=ipaddr_has_any_prefix, url_has_any=url_has_any, httpuseragent_has_any=httpuseragent_has_any, eventresultdetails_in=eventresultdetails_in, eventresult=eventresult, eventresultdetails_has_any=eventresultdetails_has_any, disabled=disabled, pack=pack) |
When adding an additional parser to a unifying custom parser that already references parsers, make sure you add a comma at the end of the previous line.
@@ -97,22 +97,23 @@ Make sure to add both a filtering custom parser (one that accepts filtering para
The syntax of the line to add is different for each schema. The following table uses the parameters defined in `ParserParams` in the corresponding `Parsers/ASim<SchemaName>/Parsers/im<SchemaName>.yaml` file in the [Microsoft Sentinel repository](https://github.com/Azure/Azure-Sentinel/tree/master/Parsers). Replace `_parser_name_` with your filtering source-specific parser's function name. Use named arguments to avoid binding values to the wrong parameters when declaration order differs.
-Before adding a line, verify that your custom unifying function declares the referenced parameters and that your source-specific parser accepts them. Forward `pack=pack` only if the source-specific parser supports it. If your custom unifying function declares `disabled`, also forward `disabled=disabled` to source-specific parsers that support it, even when it isn't included in the schema's row below.
+Before adding a line, verify that your custom unifying function declares the referenced parameters and that your source-specific parser accepts them. Forward `pack=pack` only if the source-specific parser supports it. If your custom unifying function declares `disabled`, also forward `disabled=disabled` to source-specific parsers that support it.
| Schema | Parser | Line to add |
| --- | --- | --- |
-| AgentEvent | `Im_AgentEventCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, agentid_has_any=agentid_has_any, agentname_has_any=agentname_has_any, username_has_any=username_has_any, pack=pack)` |
-| AlertEvent | `Im_AlertEventCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, ipaddr_has_any_prefix=ipaddr_has_any_prefix, hostname_has_any=hostname_has_any, username_has_any=username_has_any, attacktactics_has_any=attacktactics_has_any, attacktechniques_has_any=attacktechniques_has_any, threatcategory_has_any=threatcategory_has_any, alertverdict_has_any=alertverdict_has_any, eventseverity_has_any=eventseverity_has_any, pack=pack)` |
-| AssetEntity | `Im_AssetEntityCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, entityid_has_any=entityid_has_any, entityname_has_any=entityname_has_any, assettype_in=assettype_in, path_has_any=path_has_any, assetowner_has_any=assetowner_has_any, entitysource_has_any=entitysource_has_any, pack=pack)` |
-| AuditEvent | `Im_AuditEventCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, actorusername_has_any=actorusername_has_any, operation_has_any=operation_has_any, eventtype_in=eventtype_in, eventresult=eventresult, object_has_any=object_has_any, newvalue_has_any=newvalue_has_any, pack=pack)` |
-| Authentication | `Im_AuthenticationCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, username_has_any=username_has_any, targetappname_has_any=targetappname_has_any, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, srchostname_has_any=srchostname_has_any, eventtype_in=eventtype_in, eventresultdetails_in=eventresultdetails_in, eventresult=eventresult, pack=pack)` |
+| AgentEvent | `Im_AgentEventCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, agentid_has_any=agentid_has_any, agentname_has_any=agentname_has_any, username_has_any=username_has_any, disabled=disabled, pack=pack)` |
+| AlertEvent | `Im_AlertEventCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, ipaddr_has_any_prefix=ipaddr_has_any_prefix, hostname_has_any=hostname_has_any, username_has_any=username_has_any, attacktactics_has_any=attacktactics_has_any, attacktechniques_has_any=attacktechniques_has_any, threatcategory_has_any=threatcategory_has_any, alertverdict_has_any=alertverdict_has_any, eventseverity_has_any=eventseverity_has_any, disabled=disabled, pack=pack)` |
+| AssetEntity | `Im_AssetEntityCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, entityid_has_any=entityid_has_any, entityname_has_any=entityname_has_any, assettype_in=assettype_in, path_has_any=path_has_any, assetowner_has_any=assetowner_has_any, entitysource_has_any=entitysource_has_any, disabled=disabled, pack=pack)` |
+| AuditEvent | `Im_AuditEventCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, actorusername_has_any=actorusername_has_any, operation_has_any=operation_has_any, eventtype_in=eventtype_in, eventresult=eventresult, object_has_any=object_has_any, newvalue_has_any=newvalue_has_any, disabled=disabled, pack=pack)` |
+| Authentication | `Im_AuthenticationCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, username_has_any=username_has_any, targetappname_has_any=targetappname_has_any, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, srchostname_has_any=srchostname_has_any, eventtype_in=eventtype_in, eventresultdetails_in=eventresultdetails_in, eventresult=eventresult, disabled=disabled, pack=pack)` |
| DhcpEvent | `Im_DhcpEventCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, srchostname_has_any=srchostname_has_any, srcusername_has_any=srcusername_has_any, eventresult=eventresult, disabled=disabled, pack=pack)` |
-| Dns | `Im_DnsCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr=srcipaddr, domain_has_any=domain_has_any, responsecodename=responsecodename, response_has_ipv4=response_has_ipv4, response_has_any_prefix=response_has_any_prefix, eventtype=eventtype, pack=pack)` |
+| Dns | `Im_DnsCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr=srcipaddr, domain_has_any=domain_has_any, responsecodename=responsecodename, response_has_ipv4=response_has_ipv4, response_has_any_prefix=response_has_any_prefix, eventtype=eventtype, disabled=disabled, pack=pack)` |
+| EmailEvent | `Im_EmailEventCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, ipaddr_has_any_prefix=ipaddr_has_any_prefix, emailfromaddress_has_any=emailfromaddress_has_any, emailrecipient_has_any=emailrecipient_has_any, emailsubject_has_any=emailsubject_has_any, emaildirection_in=emaildirection_in, emaildeliverylocation_in=emaildeliverylocation_in, eventresult=eventresult, disabled=disabled, pack=pack)` |
| FileEvent | `Im_FileEventCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, eventtype_in=eventtype_in, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, actorusername_has_any=actorusername_has_any, targetfilepath_has_any=targetfilepath_has_any, srcfilepath_has_any=srcfilepath_has_any, hashes_has_any=hashes_has_any, dvchostname_has_any=dvchostname_has_any, disabled=disabled, pack=pack)` |
-| NetworkSession | `Im_NetworkSessionCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, dstipaddr_has_any_prefix=dstipaddr_has_any_prefix, ipaddr_has_any_prefix=ipaddr_has_any_prefix, dstportnumber=dstportnumber, hostname_has_any=hostname_has_any, dvcaction=dvcaction, eventresult=eventresult, pack=pack)` |
-| ProcessEvent | `Im_ProcessEventCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, commandline_has_any=commandline_has_any, commandline_has_all=commandline_has_all, commandline_has_any_ip_prefix=commandline_has_any_ip_prefix, actingprocess_has_any=actingprocess_has_any, targetprocess_has_any=targetprocess_has_any, parentprocess_has_any=parentprocess_has_any, actorusername_has=actorusername_has, targetusername_has=targetusername_has, dvcipaddr_has_any_prefix=dvcipaddr_has_any_prefix, dvchostname_has_any=dvchostname_has_any, hashes_has_any=hashes_has_any, eventtype=eventtype)` |
+| NetworkSession | `Im_NetworkSessionCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, dstipaddr_has_any_prefix=dstipaddr_has_any_prefix, ipaddr_has_any_prefix=ipaddr_has_any_prefix, dstportnumber=dstportnumber, hostname_has_any=hostname_has_any, dvcaction=dvcaction, eventresult=eventresult, disabled=disabled, pack=pack)` |
+| ProcessEvent | `Im_ProcessEventCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, commandline_has_any=commandline_has_any, commandline_has_all=commandline_has_all, commandline_has_any_ip_prefix=commandline_has_any_ip_prefix, actingprocess_has_any=actingprocess_has_any, targetprocess_has_any=targetprocess_has_any, parentprocess_has_any=parentprocess_has_any, actorusername_has=actorusername_has, targetusername_has=targetusername_has, dvcipaddr_has_any_prefix=dvcipaddr_has_any_prefix, dvchostname_has_any=dvchostname_has_any, hashes_has_any=hashes_has_any, eventtype=eventtype, disabled=disabled)` |
| RegistryEvent | `Im_RegistryEventCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, eventtype_in=eventtype_in, actorusername_has_any=actorusername_has_any, registrykey_has_any=registrykey_has_any, registryvalue_has_any=registryvalue_has_any, registrydata_has_any=registrydata_has_any, dvchostname_has_any=dvchostname_has_any, disabled=disabled, pack=pack)` |
-| UserManagement | `Im_UserManagementCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, targetusername_has_any=targetusername_has_any, actorusername_has_any=actorusername_has_any, eventtype_in=eventtype_in, pack=pack)` |
+| UserManagement | `Im_UserManagementCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, targetusername_has_any=targetusername_has_any, actorusername_has_any=actorusername_has_any, eventtype_in=eventtype_in, disabled=disabled, pack=pack)` |
| WebSession | `Im_WebSessionCustom` | `_parser_name_ (starttime=starttime, endtime=endtime, srcipaddr_has_any_prefix=srcipaddr_has_any_prefix, ipaddr_has_any_prefix=ipaddr_has_any_prefix, url_has_any=url_has_any, httpuseragent_has_any=httpuseragent_has_any, eventresultdetails_in=eventresultdetails_in, eventresult=eventresult, eventresultdetails_has_any=eventresultdetails_has_any, disabled=disabled, pack=pack)` |
When adding an additional parser to a unifying custom parser that already references parsers, make sure you add a comma at the end of the previous line.
