Back to listing

Palo Alto Networks

Content Pack

reighnman
free!

Published

25 Sep 15:44

Last Push

13 Aug 08:13

Marketplace Rating

Discussion

48 Comments

Your Rating

Please sign in to rate this add-on.

Comments

ddbnl 2 months ago

@ignisnigrum, nice find, I generated extractors from Palo Altos' documentation using a python script. I then make a content pack out of it:
https://marketplace.graylog.org/addons/f9facfdf-3d3d-423d-9bd0-4fba9db407ff. The dashboards should be more or less the same.

ignisnigrum 4 months ago

just my 2 cents:
https://www.paloaltonetworks.com/documentation/80/pan-os/pan-os/monitoring/use-syslog-for-monitoring/syslog-field-descriptions

description of the syslog fields by palo alto , maybe this helps.
as i seem to be unable to modify the grok pattern as everything in the greylog just seem to stop working :S.

PecnikMiha 5 months ago

Any news on this front?

PecnikMiha 6 months ago

That is great news, I can't wait for the new update.

TheJCamping 7 months ago

I've updated the GROK extractors to work with the changes in 7.0 and 8.0. It looks like its an issue with the time portion of the extractor. Once I've tested for a few days i'll try updating the Content Pack in full.

This is what I changed to get the Grok extractors working: %{IPORHOST:Source}%{NOTCOMMA},%{DATE_US2:LoggedDate} %{NOTCOMMA:LoggedTime},%{BASE10NUM:SerialNumber},%{NOTCOMMA:Type},%{NOTCOMMA:Subtype},%{NOTCOMMA},%{DATE_US2:EventDate} %{NOTCOMMA:EventTime}

If you replace the beginning of the extractor with that or for some of them shortened versions it should work.

NoX1De 7 months ago

Has anyone forked this already and updated for PANOS 7.1.x and/or 8.0.x compatibility? Don't want to reinvent the wheel...

reighnman 8 months ago

Really this content pack needs to be broken up by PANOS version since they like to change the structure of their logs on major releases. This was originally created back in PANOS 5 with older versions of graylog so some tweaks are likely necessary. Ideally it should be forged to a new v7+ plugin and updated so that users of old PANOS versions can continue to use the original.

blacs30 11 months ago

Thanks a log for this fix regarding the timeformat from @chrkli193. That seemed to have fixed more than 100.000 index errors a day.

reighnman 11 months ago

You can fork or you can just submit merge requests with fixes. I may be getting new PA devices in which case I'll probably have the opportunity to update it myself again.

dallanwagz about 1 year ago

@NoX1De Are you sending your logs to the input that the content pack created? The extractors need to be attached to the input.

PecnikMiha about 1 year ago

I too am interested what the future of this project is. The Graylog team will need to step up on their own end, be it with contributions or packs we can pay for, if they want to be considered as a Splunk competitor. Same issues as this one are pretty much happening all over, something gets released, it works for x.x version then stops, or lacks some fundemental functions such as custom community names in the SNMP pack.

Community contributions are great and all, but will not be enough in the long run.

luqmaanki over 1 year ago

Does anyone know if this project has been forked by anyone? I would really like to see it continue to live as PAN upgrades their FW.
Thanks

NoX1De over 1 year ago

And...my mistake, completely overlooked the grok patterns since none of them reference or designate they are for the Palo Alto Content Pack with something like PA_ in front of them...and the extractors are there, just missed them. Whoops!

NoX1De over 1 year ago

For clarification- what was meant is none of the extractors nor grok pattern(s) are appearing to be showing up after applying the content pack content.

NoX1De over 1 year ago

Actually, none of the Grok patterns nor any of the Extractors seem to be working after importing the Content Pack and activating the content...on Graylog v2.2.3+7adc951...?

NoX1De over 1 year ago

Activated the Content Pack after importing and the Grok Pattern is not showing up...anyone have that issue as well?

maritsas over 1 year ago

Hi everyone! I tested the pack on PAN-OS versions 7.0.9 and 7.0.5 and everything works fine (after I edited date/time/event properly, accoding to chrkli193's post), EXCEPT the PAN_THREAT grok pattern. Although, I am able to see the Threats/Threat logs that I get on my FWs, I am not getting them into Graylog (version 2.2.3). Is it a Graylog problem or a FW misconfig? I was also wondering if in order to see the threats, I have to map the log forwarding profile to a security policy (just like with the traffic extractor). It does not seem to be the same (and simple) config as with CONFIG or SYSTEM logs.

Thank you in advance for any answers!

@reighnman: thanks a lot for the code

tekctrl87 over 1 year ago

I can confirm I edited the .json content pack file and re-imported it in and my Pan OS 7.1.5 with Graylog 2.1.2 on CentOS 7 (thanks chrkli193!)

marciofoz almost 2 years ago

I have the same error in 2.1.1 that @speedst3r report and using the steps of @chrkli193 only stops the errors on the system/overview, but the dashboard not show any more data. Did you create a pattern for %{DATA} @chrkli193 ?

giannidaprile almost 2 years ago

I can confirm that chrkli193 fix works for me too (Graylog 2.1.1 on Centos 7).

chrkli193 almost 2 years ago

On Ubuntu 16.04 I've changed:
%{DATE_US2:LoggedDate} +%{TIME:LoggedTime}
to
%{DATA:LoggedDate;date;yyyy/MM/dd HH:mm:ss}

%{DATE_US2:EventDate} +%{TIME:EventTime}
to
%{DATA:EventDate;date;yyyy/MM/dd HH:mm:ss}
After that it's working

speedst3r almost 2 years ago

Getting the following error when indexing:
message [MapperParsingException[failed to parse [EventDate]]; nested: IllegalArgumentException[Invalid format: "2016/09/12" is malformed at "/09/12"];]

Vanilla Graylog install on Ubuntu 16.04. I have Netscreen logs insering without issue.

smallfish01 about 2 years ago

@Nisimar,

When I click "Configured extractors" on PAN_THREAT,I saw the right "Metrics" and "Converter time" are shown "No message passed through here yet", but I can see logs when I click "Show received messages", that means the logs received from Palo Alto firewall on Graylog server.

smallfish01 about 2 years ago

@Nisimar,
Thanks for your replied! did you means create rules on Gryloa web: System/Roles/Dashboard and add PaloAlto items and enable read/write´╝č

reighnman about 2 years ago

I am done traveling for a while and will begin looking at upgrading this to support the latest 7.1 fields time permitting :)

nisimar about 2 years ago

Hi @smallfish01 you need set into rules, transfer logs to graylog.

smallfish01 about 2 years ago

Hi @nisimar, I got the same issue as you, my Pan OS is 6.1, I'd like you know how you fix it?

nisimar about 2 years ago

it's working Tnx :)

nisimar about 2 years ago

Hi
The dashboard (Glary 2.0.3) does not display data with the PAN OS 7.1.3.

You can fix it?

Thank you :)

skear about 2 years ago

Working great with graylog2 and Panos 7.x. Very useful content pack!

ashishbar over 2 years ago

not working on graylog 2.0 .

thipsup over 2 years ago

@meoso I use graylog2 with PAN OS 7.0.4 but dashboard show nothing. Do you have any example of your content pack ?
Thanks.

bgarlock over 2 years ago

@meoso - Would you mind sharing your edits, and contribute back via github? This would be most appreciated and helpful.

meoso over 2 years ago

Thank you for this!!! Thank you for this!!!
Working on with PAN 7.0.4 and Graylog Appliance 1.3.3. - needs new 7.x fields; had to edit Login Summary dash.

clement201657 over 2 years ago

It doesn't work for me neither, anyone could help to debug ?
Nothing is collected to the input
Thx

kevin-sback over 2 years ago

I'm not seeing anything in my dashboards with PAN-OS 6.1.8. When I test the extractor in the "Field matches this regular expression" ^(.*,THREAT,.*), I get Does not match Extractor would not run. The Grok pattern passes. Same for Config ^(.*,CONFIG,.*). For System both Grok and expression does not pass. Any ideas. Sorry new to this. Thanks in advance.

jason-at-key over 2 years ago

works with pan os 7.0.2 and 7.0.3

jason-at-key almost 3 years ago

testing w/ panOS 7.0.2
dashboards seem to be working at 1st glance

tfriesen almost 3 years ago

Another minor bug. The PAN Threat extractor fails for flood type threats. Here is an updated Grok pattern:

%{IPORHOST:source} +: +%{BASE10NUM:Domain},%{DATE_US2:LoggedDate} +%{TIME:LoggedTime},%{HEXINT:SerialNumber},%{NOTCOMMA:Type},%{NOTCOMMA:Subtype},%{NOTCOMMA:ConfigVersion},%{DATE_US2:EventDate} +%{TIME:EventTime},%{IP:SourceIP},%{IP:DestinationIP},(%{IP:NATSourceIP})?,(%{IP:NATDestinationIP})?,(%{NOTCOMMA:RuleName})?,(%{NOTCOMMA:SourceUser})?,(%{NOTCOMMA:DestinationUser})?,(%{NOTCOMMA:Application})?,%{NOTCOMMA:VirtualSystem},%{NOTCOMMA:SourceZone},%{NOTCOMMA:DestinationZone},(%{NOTCOMMA:IngressInterface})?,(%{NOTCOMMA:EgressInterface})?,%{NOTCOMMA:LogForwardingProfile},%{NOTCOMMA:UNWANTED},%{BASE10NUM:SessionID},%{BASE10NUM:RepeatCount},(%{BASE10NUM:SourcePort})?,(%{BASE10NUM:DestinationPort})?,(%{BASE10NUM:NATSourcePort})?,(%{BASE10NUM:NATDestinationPort})?,%{NOTCOMMA:Flags},%{NOTCOMMA:Protocol},%{NOTCOMMA:Action},%{QSORNC:Miscellaneous},%{NOTCOMMA:ThreatID},%{NOTCOMMA:Category},%{NOTCOMMA:Severity},%{NOTCOMMA:Direction},%{BASE10NUM:Sequence},%{NOTCOMMA:ActionFlags},(%{NOTCOMMA:SourceLocation})?,(%{NOTCOMMA:DestinationLocation}).*

tfriesen almost 3 years ago

I've found a few minor bugs in the extractors in this pack. To start, the HOUR pattern is missing, which means it can't parse TIME. For the TRAFFIC extractor, SerialNumber can include hex characters, so it will sometimes fail BASE10NUM. Lastly, my logs don't seem to include a SessionEndReason field, so I had to omit that to get mine to work.

Thanks for the code, nonetheless! This will be really useful once I get all the gotchyas sorted out.

reighnman almost 3 years ago

You might want to download or install directly again as I went back through the grok patterns to make sure they were included as well as a few other updates.

bac36 almost 3 years ago

jdxnster & ifailalot, thank you both for your inputs this works for ThreatSummary + URL Filtering but the GlobalProtect Portal &Threat Summer High-Critical is not receiving any information. Did i miss something else?

jdxnster almost 3 years ago

Thanks for this, however the Grok pattern HOUR is missing from the content pack, I simply added the following pattern: "name" : "HOUR", "pattern" : "(?:[0-2][0-9])"

ifailalot almost 3 years ago

adding the pattern
YEAR (?:[1-2][0-9][0-9][0-9])
worked for me

rmurphyMLHC almost 3 years ago

neildavies44, thank you for your help. I changed the grok to look like this and it still does not import. Thank you for your input.

"name" : "YEAR",
"pattern" : "(?>\\d\\d){1,2}"

neildavies44 almost 3 years ago

I had the same error, but if you edit the file, at the end is the YEAR grok pattern and change it from \d\d to \\d\\d and then import, it should work.

rmurphyMLHC almost 3 years ago

I am getting the same error as ltb76.

Thanks.

ltb76 almost 3 years ago

I cannot seem to import this - I get the error: "Error! The uploaded bundle could not be applied: does it have the right format? "
I have tried re-downloading.

Please sign in to comment.

Back to listing