Release of DAML SDK 1.7.0

DAML SDK 1.7.0 has been released on November 11th 2020. You can install it using:

daml install latest

If you're using the DAML Triggers Early Access you'll need to migrate to the new API. No other mandatory changes are required for this release but other impacts and migrations are detailed below.

Highlights

• DAML Connect has been introduced as a logical grouping for all those components a developer needs to connect to a DAML network.
• JSON API, DAML Script, and the JavaScript Client Libraries now support reading as multiple parties.
• daml start can now perform code generation, and has a quick-reload feature for fast iterative app development
• Support for multi-key/query streaming queries in React Hooks
  • New query functions accepting multiple keys supersede the old single-key/query versions, which are now deprecated.
• DAML Triggers (Early Access) have an overhauled API that is more aligned with DAML Script
  • This change requires a migration detailed below.

Impact and Migration

• The compiler now emits warnings if you use advanced and undocumented language features which are not supported by data-dependencies. This only affects you if you use language extensions not documented on docs.daml.com or import DA.Generics. If you receive such warnings, it is recommended that you move off them. If you are getting unexpected warnings, or don’t know how to migrate, please get in touch with us via the public forum or support.digitalasset.com (for registered users).
• If you are using stream queries in the React Hooks, we recommend you migrate to the new multi-key versions. The migration is detailed below. The old functions are now deprecated, meaning they may be removed with a major release 12 months from now.
• If you are using DAML Triggers, you’ll need to migrate them to the new API.

What’s New

Clearer Segmentation of the DAML stack with DAML Connect

Background

With Release 1.6 work started to clarify the DAML stack in many respects: What it consists of, what state different pieces are in, and what stability and compatibility guarantees users can expect. However, the ecosystem overview page highlighted that there was a lack of good terminology of the sets of components users needed to either establish a DAML network, or to connect to a DAML network. Previously the label “SDK” was sometimes used to refer to the latter, but that was inaccurate since the SDK only contains components intended to be used at development time. With release 1.7 this has been tidied up with clearer terminology for different layers of the stack. DAML Connect describes all those pieces that are needed to connect to a DAML network:

• Runtime Components like the JSON API
• Libraries like the Java Ledger API bindings and JavaScript client library
• Generated Code from the daml codegen commands
• Developer tools like the IDE, collectively called the SDK

Specific Changes

The ecosystem overview docs page has been improved, and naming has been adjusted throughout docs and artifacts to incorporate these changes.

Impact and Migration

This change only affects documentation and help text. There are no API changes.

Multi-Party Read in JSON API, Script, and JS Client Libs

Background

The DAML Ledger API has supported multi-party transaction subscriptions for a long time, but until now, the runtime components like JSON API, and DAML Script and REPL were only able to query as a single party. As a first step towards enhancing DAML’s capabilities in sharing data in more flexible ways than the observer concept, the runtime components have now gained features to query contracts on behalf of multiple parties at the same time. Since read-access via the JSON API is controlled using JWTs, any client of the JSON API, including the JavaScript client libraries, can profit from these improvements provided the token issuer is able to issue appropriate tokens.

Specific Changes

• The JSON API now accepts tokens with multiple parties specified in the readAs field. In queries, contracts known to any of the known parties will be returned. In command submissions, the readAs field is ignored. actAs is still limited to a single party.
  • Before this change, the JSON API would accept tokens with only a readAs party, and no actAs party set for command submission endpoints. When working with unauthenticated ledgers, this could lead to successful command submission without an actAs party set. This bug has been fixed as part of this change. If you were previously exploiting this bug during development, please move the submitting party from readAs to actAs.
• In DAML Script, the query, queryContractId, and queryContractKey now accept multiple parties using the IsParties abstraction used by signatory, observer, and other fields. Specifically this means they accept single parties, lists of parties, and sets of parties interchangeably. They will return all contracts for which any of the given parties is a stakeholder.

Impact and Migration

As mentioned above, a bug was fixed in which a command submission could be successful via the JSON API if only the readAs field in the JWT was set, and the underlying Ledger API was running in unauthenticated mode. This is no longer possible. The submitting party has to be set in the actAs field.

Better (re)build with daml start

Background

daml start is the primary command allowing quick testing of a DAML application. It compiles DAML contracts, starts up a Sandbox and a Navigator, and runs initialization scripts. However, it didn’t run code-generation, nor did it have any functionality for re-building and deploying after a change, requiring developers to manually shut it down, re-generate code, and restart. With this release, that manual and error-prone process has been replaced with automatic refreshes including codegen.

Note that if you use Navigator, you might still want to do a full restart to clear old contracts and packages.

Specific Changes

You can now press 'r' (or 'r' + 'Enter' on Windows) in the terminal where daml start is running to rebuild the DAR package and generate JavaScript/Java/Scala bindings and upload the new package to the Sandbox. This frees the user from killing and restarting daml start.

The daml start now runs all the code generators specified in the daml.yaml project configuration file under the codegen stanza. This frees the user from having to do so manually on every change to the DAML model.

For example, instead of running the JavaScript codegen manually as part of the create-daml-app, it is now included in the daml.yaml.

Before: Manual run of

daml codegen js .daml/dist/create-daml-app-0.1.0.dar -o daml.js

After: stanza in daml.yaml

Impact and Migration

This is a purely additive change.

API Alignment of Triggers (Early Access)

Background

The DAML API of Triggers has been brought further inline with those of DAML Script and REPL, at the same time allowing for more efficient implementation of high-level triggers under the hood. Before, all the information consumable by the trigger rule, initialization, and update functions was passed in as function arguments. With this iteration, state, active contracts, and time are accessible using actions (using <- notation) instead.

Specific Changes

• Trigger updateState, rule, and initialize functions no longer accept an ACS argument; instead, they must use the query action to query the active contract set, similar to the same function in DAML Script. See issue #7632.
• Instead of taking a state argument, the TriggerA action now has functions to get, put and modify user defined state.
• The Time argument was removed from the trigger rule function; instead, it can be fetched within the TriggerA do block by getTime, as with Update and Script.
• The "commands in flight" of type Map CommandId [Command] argument has been removed from high-level trigger rule functions; instead, the current commands-in-flight can be retrieved with the new getCommandsInFlight function.  See issue #7600.
• initialize is now a TriggerInitializeA action, which is able to query the ledger using query and related functions, and returns the initial state.
• The updateState function now takes just a message and returns a TriggerStateA which allows querying the ledger using query and related functions, and modifying state using the get, put, and modify functions. See issue #7621.
• Two new functions are available for querying the ledger: queryContractId, for looking up a contract by ID, and queryContractKey for looking one up by key. See issue #7726.

Impact and Migration

1. To migrate an existing initialize function acs -> expr, write a do block with query taking the place of any getContracts occurrences. For example

For triggers without custom state (ie s == ()), pure () is the right expression for initialize.

2. To migrate an existing updateState function, deal with ACS access as in 1. In addition, replace the state argument by calls with a call to modify.

For triggers without custom state (ie s == ()), \msg -> pure () is the right expression for initialize.

3. To migrate an existing rule function, deal with ACS access as in 1., replace the state argument with a call to get, and in addition replace time and commands-in-flight access using the new accessor functions.
time, commandsInFlight, and state are each optional; if they are not used, the lines that “get” them may be removed.

React Hooks Multi-query streams

Background

Since version 1.6 the JSON API and JavaScript client libraries have had support for stream requests with multiple keys or queries. As of 1.7, this functionality is also available in the React Hooks, allowing for more efficient binding of ledger data to UI elements.

Specific Changes

• The React bindings now expose the recent addition of multi-key and multi-query streams, with the new  useStreamQueries and useStreamFetchByKeys hooks in @daml/react mapping to, respectively, streamQueries and streamFetchByKeys in @daml/ledger.
• The singular versions are marked as deprecated as they have become redundant.

Impact and Migration

The change is fully backwards compatible as the old singular functions are merely deprecated, not removed. The below describes the upgrade path for the new multi-query versions.

Upgrading useStreamQuery is straightforward: the query factory remains optional, but if specified it should return an array of queries instead of a single query. The array may be empty, which will return all contracts for that template, similar to not passing in a query factory. The return values of useStreamQuery and useStreamQueries have the same type.

Upgrading useStreamFetchByKey is only slightly more involved as the return type of useStreamFetchByKeys is different,  called FetchByKeysResult instead of the existing FetchResult. FetchByKeysResult differs from FetchResult in that it contains a contracts field with an array of contracts instead of a singular contract field. It differs from QueryResult in that each element of the returned array can also be null, if there is no corresponding active contract. Call sites can be updated as follows:

Minor Improvements

• The Ledger API is now independently versioned instead of inheriting its version from the release of the Integration Kit components. This allows for cleaner cross-compatibility guarantees as described in the compatibility documentation. Every participant node from this release onwards advertises its Ledger API version via the version service. 
• The daml ledger list-parties command can now query the ledger for known parties via the HTTP JSON API instead of the gRPC API. This requires setting the --json-api flag.
• The JSON API’s JDBC url can now also be specified via an environment variable set with the CLI flag --query-store-jdbc-config-env.
• The JSON API has gained /livez and /readyz health check endpoints for easier integration with k8s and other schedulers.
• The JavaScript client library’s stream queries useStreamQueries and useStreamFetchByKeys as well as their (deprecated) singular counterparts now accept an optional closeHandler callback, which will be called if the underlying WebSocket connection get closed due to an error or because close was called.
  
  The same functions previously logged every close event as an error. However, there are legitimate cases for the connection to be closed (e.g. the component has been unmounted). The default behaviour will now be to log only unexpected disconnects and be silent on deliberate connection closes. This can be customized using the closeHandler.
• The JavaScript client library’s reconnectThreshold can now be configured through the LedgerProps of the React wrapper.
• The Standard Library’s DA.Date type now has Enum and Bounded instances allowing the use of ranges. Eg
  -- Create a list of all Sundays in 2020
[date 2020 Jan 5, date 2020 Jan 12 .. date 2020 Dec 31]
• Performance improvements in the DAML Engine when using typeclasses.
• From this release, the digitalasset/daml-sdk docker image on DockerHub will be signed.

Bug Fixes

• The DAML compiler shows the correct column numbers in error locations produced by command line tools like daml build.
• The DAML Engine now properly enforces that the list of maintainers is non-empty. Before this version, it was possible on some DAML Ledgers to create contracts with empty maintainer lists. Such contracts can still be used when referenced by ContractId, but none of the key operations will work, and no new contracts with an empty maintainer list can be created.
• A bug in the JavaScript Client Libraries was fixed, which caused the useStreamFetchByKeys hook to sometimes report a "ready" state (i.e. loading: false) even though the underlying connection had not yet been fully established.
• A bug in DAML REPL was fixed, which caused errors about ambiguous names, even if names were fully qualified.
• The compiler now writes the proper SDK version in the DAR manifest for snapshot releases instead of a sanitized version.
• daml ledger upload-dar now exits with a non-zero exit code on failures and no longer prints "DAR upload succeeded" in error cases. This was a regression.
• Contract Key lookup mismatches are now consistently reported as Inconsistent rather than Disputed. Sandbox-classic, in particular, previously reported Disputed, which implies malicious intent or malfunctioning of the submitter.

DAML Driver for PostgreSQL

• New metrics tracking the pending submissions and completions on the CommandService have been added. Check out the monitoring section in the documentation for more details. The new metrics are
  • daml.commands.<party_name>.input_buffer_size
  • daml.commands.<party_name>.input_buffer_saturation
  • daml.commands.<party_name>.max_in_flight_size
  • daml.commands.<party_name>.max_in_flight_saturation
• Add new metrics for measuring the number of concurrent command executions. The metrics are:
  • daml.commands.submissions_running
  • daml.execution.total_running
  • daml.execution.engine_running

Integration Kit

• The Ledger API test tool’s --ledger-clock-granularity option now takes a time duration (e.g. "10s" or "5m"), rather than an integer number of milliseconds.
• The Ledger API test tool now defaults to a ledger clock granularity of 1 second, not 10s, in line with most ledger implementations.
• The Ledger API test tool has a new command line argument skip-dar-upload. See docs.
• The Integration Kit’s ResourceOwner type is now parameterized by a Context, which is filled in by the corresponding Context class in the ledger-resources dependency. This allows us to pass extra information through resource acquisition.
• The kvutils have a new metric daml.kvutils.committer.package_upload.validate_timer to track package validation time.
• The Ledger API Server’s previously hardcoded timeout for party allocation and package uploads can now be configured via ParticipantConfig and the default value is now set to 2 minutes. See issue #6880.