The logging paradox
Organizations face a contradiction in logging. Security teams want comprehensive visibility. Finance teams want controlled costs. Compliance requirements demand specific retention periods. Operations teams struggle with signal-to-noise ratios.
The default response is often to log everything. Storage is cheap, after all. But "log everything" leads to SIEM costs that scale exponentially, alert fatigue from noisy data, and investigation delays when relevant events hide in noise.
Effective logging strategy requires deliberate choices about what to collect, how long to keep it, and how to use it.
What you must log
Some events are non-negotiable. Their absence makes incident response nearly impossible.
Authentication events: Successful and failed logins, MFA challenges, password changes, and account lockouts. These reveal compromise attempts and lateral movement.
Authorization changes: Permission grants, role changes, and privilege escalations. These capture insider threats and post-exploitation activity.
Administrative actions: Changes to systems, configurations, and security controls. These document what changed and who changed it.
Data access: Access to sensitive data, especially bulk access or unusual access patterns. These indicate data exfiltration or unauthorized snooping.
Network connections: External connections from critical systems, lateral movement between segments, and connections to known malicious infrastructure.
What you should probably log
Beyond the essentials, additional logging provides value but requires cost-benefit analysis.
Application errors: Stack traces and error logs help identify attacks exploiting application vulnerabilities. However, they can be noisy and may contain sensitive data requiring careful handling.
Endpoint activity: Process execution, file modifications, and registry changes provide visibility into malware behavior. Volume is high, but value is significant for threat hunting.
DNS queries: Domain lookups reveal command-and-control communication and data exfiltration attempts. Consider sampling for high-volume internal DNS.
Email flow: Message metadata (not full content) reveals phishing campaigns and business email compromise. Balance against privacy considerations.
What you can usually drop
Some logging generates cost without security value.
Debug logs in production: Development troubleshooting logs rarely help security investigations and often expose sensitive information.
Routine system health: CPU, memory, and disk metrics belong in monitoring systems, not security logs. Exception: anomalous resource usage that might indicate cryptomining.
Successful routine access: Log access to sensitive data, but routine access to public resources often adds noise without value.
Duplicate events: Multiple systems logging the same event (firewall, proxy, endpoint) may be redundant. Choose the authoritative source.
Retention strategy
Not all logs need the same retention period. Tier your strategy:
Hot storage (1-30 days): Recent logs for active investigation and real-time alerting. Keep in fast, expensive storage.
Warm storage (1-12 months): Logs for historical investigation and trend analysis. Move to cheaper storage with slower query performance.
Cold storage (1-7 years): Compliance-required logs. Archive to very low-cost storage with retrieval delays acceptable.
Delete: Logs beyond compliance requirements and investigation value. Define deletion policies and automate enforcement.
The logging strategy checklist
- Inventory all current log sources and volumes
- Document business justification for each log type
- Map logs to specific security use cases (detection, investigation, compliance)
- Define retention periods by log type and legal requirement
- Implement tiered storage to optimize costs
- Establish log parsing standards for consistency
- Create runbooks for common investigation scenarios
- Define what constitutes an actionable alert versus logged event
- Regularly review logging costs against value delivered
- Test log restoration from cold storage before you need it
Cost optimization techniques
Logging costs can spiral quickly. Control them strategically:
Sampling: For high-volume, low-security-value logs, sample rather than collect everything. One percent of DNS queries still reveals patterns.
Filtering at the source: Filter noise before it reaches your SIEM. Parse and filter on forwarders rather than sending everything to centralized systems.
Field extraction: Store full logs in cheap storage, parse and forward only relevant fields to expensive SIEM platforms.
Aggregation: Summarize high-volume events. Instead of every connection, log connection statistics per hour.
Cloud storage tiers: Use lifecycle policies to automatically move logs between storage classes based on age.
Balancing compliance and security
Compliance requirements often mandate specific logging and retention. Do not let compliance drive your entire strategy:
Separate compliance logging: Maintain minimal compliance logs separately from security logging if volumes differ significantly.
Understand requirements: Some regulations specify retention periods but not collection scope. Do not over-collect based on assumptions.
Document rationale: Be able to explain why you log what you log. Auditors appreciate thoughtful strategy over checkbox compliance.
Making logs useful
Collection is the beginning. Value comes from use:
Detection: Create alerts for specific patterns. Balance sensitivity: too many false positives train teams to ignore alerts.
Investigation: Ensure logs are searchable and correlated. An IP address in firewall logs should connect to user identity and endpoint activity.
Hunting: Proactively search logs for indicators of compromise. Threat hunting requires comprehensive, queryable data.
Trending: Analyze logs for patterns: failed login spikes, unusual data access, new external connections. Trends reveal evolving threats.
The right level of logging
There is no universal answer. The right logging strategy depends on your threat model, compliance requirements, budget, and technical capabilities. Start with must-have logs, add high-value sources as capacity allows, and continuously review whether each log source justifies its cost.
Organizations that log strategically respond faster to incidents, spend less on storage, and maintain clearer visibility than those that log indiscriminately. Thoughtful restraint beats comprehensive collection.
Disclaimer: This post is for educational and informational purposes only and does not constitute legal, compliance, or professional security advice.