2

I use Azure Kubernetes service, which is connected to Log Analytics Workspace.

Log Analytics Workspace collects too much data, what is quite expensive.

After googling the ways to reduce the costs I found a few recommendations, but most of them about reducing the ContainerLogs size. My case is different. I do need all ContainerLogs, but nothing else.

On the image you can see the result of running this query:

union withsource = tt *
| where TimeGenerated > ago(1day)
| where _IsBillable == true
| summarize BillableDataMBytes = sum(_BilledSize)/ (1000. * 1000.) by tt
| render piechart

log analytics workspace usage

As you can see I need only 3% of all stored data, the rest 97% I want to disable or reduce (take new values much rare). Is it possible?

Is it possible to disable/reduce at least the "Perf"?

1 Answer 1

0

Basic Logs and Archive Logs options can help you. Please take a look at this article.

You can find explanation below regarding Basic Logs.

The first option is the introduction of “Basic Logs”. This sits alongside the current option, called “Analytics Logs”. Basic logs allow you to designate specific tables in your Log Analytics workspace as “basic”. This then means that the cost of the data ingestion is reduced; however, some of the features of Log Analytics are not available on that data. This means that you cannot configure alerts on this data, and the options for querying the data are limited. You also pay a cost for running a query against the data.

enter image description here

For the Archive Logs:

The second change is adding an option for archiving logs. It is often necessary to keep certain logs for an extended period due to company or regulatory requirements. Log Analytics does support retention of logs for up to two years, but you pay a retention cost that is relatively high because the data is kept in live tables that can be accessed at any time.

Archive logs allow you to move the data into an offline state where it cannot be accessed directly but is significantly cheaper. Archive data is charged at $0.025 per GB per month, compared to $0.12 per GB per month for standard data retention. If you need to access the archive data, you pay an additional fee for either querying the archive or restoring it to active tables. This costs $0.007 per GB of data scanned for querying or $0.123 per GB per day of data restored (so effectively the same as standard data retention.). These prices were correct at the time of publishing, please check the Azure Monitor pricing page.

1
  • Thanks for the answer. However, a day before your answer I tested it and had an error with the message similar to "Not permitted operation for this table". Now I am thinking about setting up my own logs and metrics monitoring tools inside the cluster.
    – Farad
    Commented Apr 7, 2022 at 14:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.