Managing log events can be a challenge with cloud computing services for a number of reasons, not least the sheer volume of datapoints, as well as the number of applications involved, and reverse engineering identified errors to their source for correction.
Cloud logging services aim to make everything easier to manage by providing an interface through which your logs are routed so that the data they contain can be more easily sorted and subjected to reporting and analytics for information.
By using the cloud it means that monitoring remains independent of your network, so it should not be slowed by any errors or failures. Additionally, cloud storage makes it easier to save any historical datapoints you may need to reference later.
By themselves these should be enough reasons to make log management easier to work with, but additional ones include the ability to integrate it not just with other cloud services, but also useful business intelligence and data visualization tools for additional insights and communicating them to stakeholders.
Here then are the best services we've found for cloud logging and management.
- We've also highlighted the best cloud web hosting services
- Want your company or services to be added to this buyer’s guide? Please email your request to email@example.com with the URL of the buying guide in the subject line.
Best log management services - at a glance
- Azure Monitor
- AWS Centralized Logging
- Google Cloud Stackdriver Logging
Azure's powerful log management and analysis platform
Azure Monitor is Microsoft's cloud-based tool for providing a comprehensive way to monitor applications, network, and infrastructure. As you'd expect, it works with other Azure services to provide a focused overview from data and logs collected from a range of services, whether based in Azure or on-premises.
It's built to work especially well with Azure Security Center and Azure Automation, to push logs into Azure Monitor for analysis and visualization. You can set up alerts for specific events, as well as use extensive querying to work through data quickly and efficiently, working through multiple sources from tables, datasets, as well as logs.
Smart analytics and machine learning algorithms mean you can detect, respond to, and correct problems quickly and easily, as well as use them to mine for actionable insights.
As you'd expect with a platform as powerful as Azure, there's a lot of potential under the hood, but it will take experience and some time to work most effectively with it.
Amazon's log solution for the AWS powerhouse
AWS Centralized Logging is Amazon's log management solution for AWS which enables the collection, analysis, and visualization of data from logs from multiple accounts and regions.
AWS Centralized Logging uses AWS Elasticsearch and Kibana to collect data and provide analytics and visualizations accordingly. Working in conjunction with other AWs services offers the ability to customize not just for single users but for multiple accounts for analyzing logs from their AWS applications and services.
You can also use AWS CloudFormation templates to provide centralized logging management in a user-friendly dashboard, and control authentication using AWS Cognito when applied for multi-accounts.
AWS Centralized Logging can also work outside of AWS service logs and can be used for any host-level log files and VPC flow logs. Designed to scale with your business, preconfigured dashboards can also be used to simplify data visualization for stakeholders.
As above, this is a service for experience users, and those new to AWs should expect something of a learning curve to get the most out of the service.
Stackdriver Logging is the log management solution provided with Google Cloud that allows for the storing, searching, analysis and monitoring of log data not just on the Google Cloud platform. The API can also be used from other external sources, such as on-premises servers as well as other cloud services such as AWS.
As with other cloud services, Stackdriver Logging is a scales according to need, and can be used to ingest log data from thousands of virtual machines in real time. This means you can identify and resolve issues on the go using other Google Cloud services such as Monitoring, Trace, Error Reporting, and Debugger.
As well as working through logs, integration with these services mean you can also navigate through incidents, charts, and traces, to identity and correct issues in your infrastructure and applications.
As a fully-managed solution, Stackdriver Logging allows you to run applications in multi-cloud environments from a single platform and dashboard, simplifying the process of log management.
As ever, with cloud services, it may take a little time for even experienced cloud administrators to get the most of out the Stackdriver Logging platform's potential.
Easy cloud log management from Rapid7's insightOps
InsightOps from Rapid7 (formerly Logentries) aim to make cloud logging management easy by centralizing data, allowing easy visualization of metrics for monitoring, and using natural-language questions to identify problems for troubleshooting.
InsightOps collects logs across all cloud services, as well as directly from servers, not least for applications, DNS requests, VPNs, databases, and firewalls. It then normalizes even unstructured data into a consistent format. You can then monitor how much CPU and memory each asset is using to ensure smooth performance. Auditing logs makes it easier to ensure compliance, and InsightOps can easily integrate into your DevOps stack for automation.
There are visualization options for drilling down through data, and the Endpoint Interrogator allows the user to ask specific questions about what resources each asset is using. There are also alerts available, and the dashboard refreshes automatically in real-time.
Overall, InsightOps is easy to deploy across any network, and because it monitors the entire system, it's easy to spot malicious activity on any part of the network, even at file level.
InsightOps is available with a free trial.
Papertrail simple setup and beginner-friendly interface makes it ideal for less tech savvy users
Solarwinds is a company that develops software for businesses to help manage their networks, systems and information technology infrastructure. They acquired the log management service ‘Papertrail’ in 2014.
Papertrail allows users to log all apps, text log files and Syslog from one location. Searches and reports are done in real-time. Users can use a command line, browser or API.
Alerts are sent instantly to users which can then be used to detect trends.
Papertrail is easy to setup and implement. Users can gain visibility across all their systems quickly. Users can aggregate MySQL, cloud hosting, Apache, Tomcat, Heroku apps, routers and firewalls.
The lowest pricing tier starts at $7 for 1GB per month. This includes 1-week search and 1-year archive.
The mid-range tier starts at $75 for 8GB per month including 8 weeks search and 1-year archive.
The top tier weighs in at $230 for 25GB data per month and includes an additional 16GB for the first month, 2 weeks search and 1-year archive.
Papertrail forces users to save all searches.
Also consider these log management services
Splunk offers a wide range of log management tools such as log consolidation, retention, IT operations troubleshooting, application troubleshooting and compliance reporting. Users can collect, store, index, search, correlate, visualize and report on any machine-generated data. Subscribers can then resolve operational and security issues.
Sumologic provides real time dashboards to manage and monitor all data logs, metrics, assess graphs and generate reports from here. Dashboards can be customized with pie, bar, map and line charts. Sumologic’s machine learning uses algorithmic support to arrange the patterns in data and establish any inconsistencies, and can work with AWS, Google Cloud or any hybrid applications.
Sematext Logsense uses Elasticsearch API and integrated Kibana for log analysis and visualization. Alerts can be sent via email, ChatOps or WebHooks. Logsense has built-in integrations for PagerDuty, Slack and HipChat amongst others, and queries can be saved with custom tags, labels, or colors.
ManageEngine EventLog Analyzer aims to bring log management, auditing, and compliance together as part of a Security Information and Event Management (SIEM) solution. With the focus on security, this makes it easier to gain insights on potential attack routes and secure them before they become a problem. This means as well as log management it can be used to augment threat intelligence.