I’m excited to announce a new log streaming feature in Daml Hub which allows ledger component logs to be streamed to a customer owned Datadog account for easier consumption and parsing. We’ve made strides in this direction over the past months by making logs available for ledger components in the Daml Hub console to view and download but log streaming into a log aggregation tool has been highly anticipated by our customers. I learned that Datadog was the only tool used by our customers so it was easy to decide on creating a direct integration into Datadog making it really easy to set up.

Log streaming now enables developers and support teams with a Daml Hub Enterprise account to stream the logs for their high-capacity ledgers and store them into a customer owned Datadog account where they can be more easily searched, filtered and analyzed. Logs for the ledger components can still be viewed and downloaded from within the Daml Hub console, documentation on how to do that can be found here.

Enterprise customers can set up a log stream to Datadog for their high-capacity ledgers by accessing the Integrations tab from within their Ledger Details page and entering a Datadog API key. More information about how to do this can be read in the documentation here and by watching a demo video on Youtube that I recorded here.

FAQ

How do I set up a log stream to my Datadog account?

Enterprise customers can set up a log stream to Datadog for their high-capacity ledgers by accessing the Integrations tab from within their Ledger Details page and entering a Datadog API key. More information about how to do this can be read in the documentation here.

Can I set up log streaming for my ledgers if I’m on the free tier?

No, since log streaming to Datadog is more of a production feature it is only possible on high-capacity ledgers for Daml Hub Enterprise tier customers.

Can I set up log streaming for my standard capacity ledgers?

No, since log streaming to Datadog is more of a production feature it is only possible on high-capacity ledgers for Daml Hub Enterprise tier customers.

Can I stream historical logs?

No, you will receive logs from the point that you start to stream them and you cannot collect logs prior to that point.

Which logging applications and services are supported for collecting logs?

Datadog is the only service that is supported at this time. More options will be added based on customer demand.

Can I query / filter to only collect a subset of log detail?

No filtering can happen in the Daml Hub service, however the logs can be filtered in Datadog once they have been received from Daml Hub.

What happens if my ledger is paused?

If your ledger is paused then no logs are generated and therefore no logs are sent. However, after the ledger is resumed the logs will begin streaming again - there’s no need to re-configure the Datadog connection.

If I downsize my ledger from a high-capacity ledger to a standard-capacity ledger will I still continue to receive logs?

No, if a ledger is downsized from a high-capacity ledger to a standard-capacity ledger then log streaming will stop and it will need to be set up again if the ledger is re-sized back up to a high-capacity ledger.