Last week, I outlined how to use Azure Cost analysis reduce spend with Sentinel data lake meters. I’ll now go through how to track costs natively in the data lake.
These blogs are shorter versions of the blogs in the series called A little slice of… which I co-wrote with my peer Jon Shectman, Principal Program Manager for Security at Microsoft, who published them on LinkedIn.
As I mentioned last week, many Security Operations Centre (SOC) practitioners have told us that visibility of costs and cost management are very important to them, which I completely understand. In a poll, 57% of people chose cost optimisation as their biggest challenge. This was way ahead of query performance, table management, and ecosystem integration. And from our experience of talking to cyber security analysts the world over, we know that SOCs are heading toward a data lake-first world.
Security data behaves differently. It grows in bursts, spikes without warning, and – most importantly – only makes sense when viewed through the lens of security value, not just spend. I believe it’s best to look at cost in a way that aligns with how security teams actually operate.
Be aware of the permission hurdle
Please note that you need Billing Administrator rights to be able to see the Cost Management blade. So grab your Azure admin and get those privileges first. Here’s a screenshot from my lab, showing the PIM setup:
The cost management feature
This feature is broken down into two distinct blades:
Usage:
Notifications:
Making sense of the data
Of the five meters, most of your data lake spend will be in these three:
- Ingestion: You see what’s flowing in, which tables are driving growth, and how onboarding new sources affects things
- Storage: Manage how much you store and how much you spend to save it
- Query & Processing: Queries are not free; everything is priced individually in the Sentinel data lake, which is part of its appeal.
Enabling SOC teams
All this gives security teams the ability to:
- Have visibility of cost without leaving Defender
- Create a data strategy and make better decisions
- Provide support for data lake-first architectures
- Get an early warning of drift before seeing the bill.
How to use Sentinel data lake cost management like a pro
- Establish your baseline
- Correlate with change
- Architect with cost in mind
Here are a few tips to help you:
- Ingestion is unpredictable unless you’re watching it
- Storage doesn’t tend to go down, it goes up
- Query costs spike when someone forgets to filter or scope
In summary, the Sentinel data lake Cost Management feature is a decision making tool to help you optimise your storage space.
Delve deeper into the details
If you wish to go deeper into this subject, please take a look at A Little Slice of…Tracking Cost with Microsoft Sentinel data lake Meters (Part 2).
Looking ahead
Jon and I have launched a poll to find out what Sentinel data lake users would like to know more about. We’ll use this feedback to decide which subjects to cover during the year. As always, feel free to contact us at Quorum Cyber if you would like to talk about any other aspects of cyber security and data security for your organisation.
