Back to Blog
Splunk itsi maintenance rest api6/29/2023 ![]() ![]() We can search by our intuitive “source” key again, this time by region. | stats sum(Sum) as sum by metric_dimensions dim_name metric_name Sourcetype=aws:cloudwatch (source="us-east-1:AWS/Lambda" OR source="us-east-1:AWS/ApiGateway") The resulting visualizations should look something like the following, with your API names substituted, and different distributions of values, etc.:Īnother search that might be interesting is mashing up our Lambda and API Gateway data to see the different metrics next to each other. The “rex” command is simply there for cosmetic reasons to make the API name easier to read. In this case, Splunk is summing the “Sum” value of each metric name by the API name. |stats sum(Sum) as sum by metric_name api_name | rex field=metric_dimensions "ApiName=\" First, it will be helpful to look at the different metric values across our “metric dimensions”, which is the name of our API in the AWS console: Now that we have API Gateway data flowing, let’s create a few simple searches to explore the data. You should see something similar to the screenshot below after a few minutes of the input running. This should return CloudWatch metric data across all regions for ApiGateway. The convention is typically :, so when exploring the data, you can narrow your searches very quickly by simply using a search such as this: This is where our “source” key becomes very helpful, as the Add-On does the heavy lifting of assigning an intuitive name for source. You should see some data, and if you’ve been using the Add-On previously, then likely we have more than just ApiGateway data from CloudWatch. Open to a search page on your Splunk instance and type in the following:Įarliest=-60m "sourcetype = aws:cloudwatch" It will be helpful to understand how the AWS add-on classifies the various data, which is informed by how we configured the input. Now that both AWS and the Add-On are configured, we can start looking at what data is flowing in. Please reference the Splunk documentation for what all these things mean in detail. ![]() The rest of the fields are up to you, but generally speaking, using 300 for “Metric Granularity” and “Minimum Polling Interval” should be sufficient for your first go (and you can adjust as needed). In the Splunk Add-On UI, you should have something similar to the below. The “Metric Statistics” you can just grab from the picklist (they are fairly universal across the different namespaces). Note that this is not part of the out of the box list provided by the add-on, but because the Splunk add-on is awesome, we can add new namespaces with a little configuration, and it will know how to collect metrics on them. The Metric Namespace we can just use the namespace as provided by Amazon, specifically: The “Metric Name” will be those metric names we saw in the AWS console, formatted as an array: Note that the syntax is similar to an array of JSON objects – with a regular expression sprinkled in (give me metrics for all my APIs by “name” of the API): The “ApiName” field is going to be our “Dimension Names” value. Services>CloudWatch>Metrics>you should be on the ”All Metrics” tab>scroll to “AWS Namepaces” section>select “ApiGateway”>Select “By Api Name”. Next we will want to review the various naming conventions that CloudWatch uses so we can properly configure things in the Splunk add-on. Repeat this step for any API and associated “stages” you see fit. ![]() “Services”>”API Gateway”>select the API you want to turn on metrics for>go to the “stage” sub-menu of that API>select the stage you want to add metrics to>check the box next to “Enable Detailed CloudWatch Metrics”. The rest of the blog will detail this short journey, and hopefully help you integrate Splunk into your AWS “serverless” infrastructure.įirst, we will need a quick primer on how to set this up on the AWS side so the Add-On has something to gather. In fact, it’s very feature-rich, seamlessly supporting data collection from new AWS services. The Splunk Add-On for AWS is no exception. What is an SE to do?! Much like AWS services, Splunk Add-Ons are also a thing of beauty that make the process of gathering and taming your data very simple. While we were using Splunk to monitor several EC2 servers with various bits of custom code via the Splunk App and Add-On for AWS, we realized (ex post facto) that while Lambda was supported out of the box by the Add-On, API Gateway was not. Both services are pretty robust, and while perhaps not perfect, to us they are a beautiful thing. Between the dramatic cost savings, and wonderful experience of not managing a server, making this move was a no brainer (facilitated as well by great frameworks like Zappa). Bill Bartlett (fellow Splunker) and I had the distinct pleasure of moving some workloads from AWS EC2 over to a combo of AWS Lambda and AWS API Gateway. ![]()
0 Comments
Read More
Leave a Reply. |