Embed Size (px)
DESCRIPTION
Citation preview
Cloud Application LoggingFor Forensics
ACM SAC 2011, Taichung - Taiwan
Raffael Marty - @zrlram
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
Agenda
2
•The Cloud and its Logs
•Logging Guidelines
Guidelines for logging infrastructures and application developmentTo enable the forensic process
•Logging Architecture
•Reference Setup
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
Logs for Computer Forensics
3
•Logs are part of the forensic process•Problems:- can’t find logs- logs got deleted- logs never got generated- logs are incomplete (e.g., no user name)- log format is unknown- archival and retention of logs (how long?)-knowledge of logging configuration-non compatible and random log formats (make correlation impossible)
2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: The current log file has a mismatching size/inode information, restarting from the beginning; filename='/mnt/log/apache2/www-access.log'2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: Follow-mode file source not found, deferring open; filename='/mnt/log/apache2/www-error.log'2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: The current log file has a mismatching size/inode information, restarting from the beginning; filename='/mnt/log/apache2/www-error.log'2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: The current log file has a mismatching size/inode information, restarting from the beginning; filename='/mnt/log/apache2/access.log'2010-12-28T18:15:53.258+00:00 frontend2-raffy syslog-ng[19632]: The current log file has a mismatching size/inode information, restarting from the beginning; filename='/mnt/log/apache2/error.log'
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
The Cloud
4
IaaS - Infrastructure
PaaS - Platform
SaaS - Software
LaaS - Logging
Complete control OS up
No control over OS
No or very limited control
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
Logs in the Cloud•Generally no infrastructure logs!-Routers, Firewalls, Load balancers, etc.
•PaaS only limited access to OS logging•SaaS generally no access to any logs•Volatility of machines / logs•Highly decentralized
5
Saturday, March 26, 2011
Raffael Marty - @zrlram
Applications Enable Visibility• If you can’t control the infrastructure, control your applications•Application logging-needs guidelines-better tools-education of developers and students?
6
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
What?
7
Mar 16 08:09:58 kernel: [! 0.000000] Normal!1048576 -> 1048576
Saturday, March 26, 2011
Logging Guidelines
8
•When •What •How
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
When to Log•Operations based logging-Errors are problems that impact a single application user-Critical conditions: situations that impacts all users of the application-System and application start, stop, and restart.-Changes to objects - attribute changes to an activity‣ Installation of a new application‣ Configuration change‣ Logging program code updates‣ Backup runs‣ Audit of log access
9
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
When to Log•Security (forensics) related logging - Login / logout (local and remote) - Password changes / authorization changes - Failed resource access (denied authorization) -All activity executed by a privileged account
•Regulatory and standards mandates- SOX (Financial system access)- PCI (Cardholder data access)- etc.
•Business relevant logging10
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
What to Log
• Timestamp
• Severity
• Categorization
• Application
• User
• Session ID
• Reason
11
when, what, who, and why
2010-05-13 13:03:47.123231PDT
info
object=input, action=create, status=success
loggly-indexing
zrlram
08BaswoAAQgAADVDG3IAAAAD
-
across tiersand applications!
see topic of eventcategorization
debuginfowarnerrorcrit
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
How to Log•Machine processable-field-identification-speed
•Field normalization -ranges (high, medium, low ==> 5, 3, 1)-terms (dropped, blocked, drop, denied)
•Encoding, see existing standards (e.g., syslog, CEE)
12
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
Log Formats-simple text --> key-value
-expressive text --> JSON
-binary --> special encoding
13
time=2010-05-13 13:03:47.123231PDT,session_id=08AADVDG3IAAAAD,severity=ERROR,user=zrlram, object=customer,action=delete,status=failure, reason=does not exist
{“time”:”2010-05-13 13:03:47.123231PDT”,“session_id”:”08AADVDG3IAAAAD”, “severity”=”ERROR”,”user”=”zrlram”,“category”: {“object”:”customer”,”action”:”delete”, ”status”:”failure”}, “reason”:”does not exist”}
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
Logging Architecture•Enable logging•Log transport- tcp vs. udp vs. relp vs. http?-encryption and compression?-Synchronized clocks across components
•Centralization of logs-preserve integrity
•Tune logging configurations-based on use-cases
14
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
Reference Setup
15
Elastic LoadBalancer
Frontends: Apache, Django
Clients: JavaScript
Amazon RDS
Backend: Java - log4j
Across machines: collectd, puppet, OS syslog
Saturday, March 26, 2011
© by Raffael MartyLogging as a Service
Future WorkAnalyzing framework for forensic log analysis-security visualization (see http://secviz.org)-forensic timeline analysis-log review-log correlation-policy monitoring
16
mobile-166 My syslog
Saturday, March 26, 2011
