Release of Daml Connect 1.8.0

DAML Connect 1.8.0 has been released on December 16th. You can install it using:

daml install 1.8.1

Note: Daml Connect 1.8.0 contained a bug that is now fixed and noted in the 1.8.1 release. Daml Connect 1.8.0 is no longer supported.

Want to know what's happening in our developer community? Check out the latest massive update for this month.

Highlights

• The API coverage of DAML Script, the JavaScript Client Libraries, and the DAML Assistant have been improved.
• DAML Driver for PostgreSQL Community Edition is now stable.
  • Action required unless you were already using the --implicit-party-allocation=No flag.
  • Running the Sandbox with persistence is now deprecated.

Impact and Migration

The DAML compiler now enforces sequential ordering for let expressions with multiple bindings. This fixes a bug where let expressions would be evaluated in the wrong order. In rare cases it may be necessary to re-order your let expressions if they relied on forward references.

In the old behavior the code below would rely on forward references and result in `x` being assigned the value `1` despite `y` being assigned this value after `x`. In the fix this is now a compile-time error.

let x = y

    y = 1

in x

To correct this code you would change the order of the assignments so:

let y = 1

    x = y

in x

There are no other backwards incompatible changes to any stable components.

DAML Driver for PostgreSQL (daml-on-sql) Community Edition ledger has been downloadable as Early Access from GitHub releases since SDK 1.4.0. The option --implicit-party-allocation used to be on by default, but has now been removed. Users that were using the DAML Driver for PostgreSQL with the implicit party allocation option will need to explicitly allocate parties now.

Users running DAML Triggers (Early Access) against authenticated ledgers may now need to handle authentication errors externally.

What’s New

Better API coverage in Script, Libraries, and Assistant

Background

This change addresses several gaps in the API coverage of DAML Connect tooling. In particular, it adds support for the party and package management APIs to the JavaScript Client libraries, easing the development of client applications that perform administrative tasks.

Specific Changes

• The JavaScript Client Libraries’ Ledger object, returned by useLedger, now has three new methods covering the package management API, and three new methods covering the party management API:
  • listPackages returns a list of all known packageIDs,
  • getPackage returns the binary data of the corresponding DALF, and
  • uploadDarFile takes binary data and uploads it to the ledger. Note that uploadDarFile requires admin access.
  • getParties allows users to, based on a party id (or party ids, as the name suggests) fetch more information about the party or check for its existence,
  • listKnownParties will return a list of all known parties, and 
  • allocateParty will allocate a new party.
• The JavaScript Client Libraries’ Ledger object now also exposes the API method createAndExercise, which creates a contract and exercises a choice on it in the same transaction.
• daml ledger can now also be run against the JSON API instead of the gRPC API.
• The listKnownParties function in DAML Script is now also supported when running over the JSON API.

Impact and Migration

This change is fully backwards compatible.

DAML Driver for PostgreSQL Community Edition now stable

Background

DAML Driver for PostgreSQL (daml-on-sql) Community Edition ledger has been downloadable as Early Access from GitHub releases since SDK 1.4.0. With release 1.8.0 it is now stable and completes the separation of the Sandbox development and test tool from a SQL-based ledger intended for production use. Accordingly, as announced in the release notes of SDK 1.4.0, the persistence mode in Sandbox is now deprecated.

Documentation is available at https://docs.daml.com/daml-driver-for-postgresql/.

Specific Changes

• The --implicit-party-allocation option, previously enabled by default, is no longer supported by DAML for PostgreSQL.
• The PostgreSQL JDBC URL can now be passed in through an environment variable via --sql-backend-jdbcurl-env. For example, to instruct the DAML Driver to read the PostgreSQL JDBC URL from JDBC_URL, use --sql-backend-jdbcurl-env "JDBC_URL".
• The feature to run Sandbox with persistence using the --sql-backend-jdbcurl flag is now deprecated.

Impact and Migration

Anyone wanting to run an open source DAML ledger on SQL is advised to migrate to DAML Driver for PostgreSQL. SQL support in Sandbox may be removed with a major version release following the usual 12-month deprecation cycle.

Implicit party allocation was previously the default mode of operation in DAML Driver for PostgreSQL. Users of DAML Driver for PostgreSQL should now allocate parties explicitly using the admin endpoints of the JSON or gRPC APIs. The combination of persistence and implicit party allocation will not be supported after the deprecation cycle; implicit party allocation will continue to be supported in Sandbox. 

Minor Improvements

• The JSON API’s /v1/fetch endpoint now uses the Postgres database, if configured, to look up contracts by ID or key, except when querying a contract by ID without its corresponding template ID. The fallback in-memory version of /v1/fetch is also significantly more efficient for large datasets, though still linear.
  
  You may optionally re-create the JSON API database to take full advantage of this change. See issue #7993.
• The Navigator’s password option in the config file is now deprecated. Note that the option has not had an effect since SDK 1.0.0.
• If no parties are in the Navigator config or daml.yaml, Navigator will now pick up parties from the party management service. Those parties are periodically refreshed.
• The JavaScript Client Libraries will now log warnings received from the JSON API.
• The SDK Docker image is now signed. Note that this is a dev-only image, not intended for production use.
  
  To verify the signature, use the docker trust inspect command. You can also set the DOCKER_CONTENT_TRUST environment variable to 1 to instruct Docker commands to only pull and run signed images. Keep in mind, however, that this only checks that there is a signature, not that the signer is who you expect it to be. For optimal security, you should manually check the signature once with docker trust inspect --pretty and then pin the image hash rather than relying on tags.
  
  The expected output of the docker sign inspect command should mention a signer named automation with a public key ID matching 533a6e09faa512f974f217668580da1ceb6aa5b00aad34ea1240afc7d249703f (note that the --pretty output only shows the first 12 chars) and a repository key matching f5dc2aee6aed2d05d7eda75db7aa2b3fac7fc67afbb880d03535d5a5295a0d3b.
• The gRPC Ledger API proto definitions and documentation pages now document the error codes endpoints may return.
• DAML-LF protobuf definitions are now published to Maven Central in JARs. The artifact names follow the format "<name>_proto_jar" under group "com.daml".
• The *.proto files containing the gRPC definitions are now provided by a new Maven Central artifact, with the group "com.daml" and the artifact name "ledger-api-proto".

Early Access Features

• Support choice observers in DAML-LF 1.dev. DAML-LF 1.dev can be activated by adding the build option --target=1.dev to your daml.yaml, but note that packages compiled to DAML-LF 1.dev cannot be deployed to production ledgers.
  
  Choice observers, documented on the reference page for choices, add an additional keyword observer to the choice … syntax. Parties designated choice observers using that keyword are guaranteed to see the exercise of the choice. This can be useful for defining custom events, for example.
  
  nonconsuming choice MyEvent : ()
  with
    message: Text
    sender: Party
    receivers: [Party]
  observer receivers
  controller sender
  do
    return ()
  
  Here the parties in sender will all see an exercise node of type MyEvent with a payload containing message.
• Trigger Service
  • Fixed a bug where complex models resulted in a fatal error when restoring the state from the database due to an incorrect protobuf recursion limit.
  • The application id used by a trigger can now be configured by an optional applicationId in the start request.
  • The trigger status endpoint /v1/triggers/:id now includes metadata about the trigger like the party and the trigger id. The logs field has been replaced by a status field.
  • Endpoints have been rearranged to be more consistent:
    
    

    New endpoint	Old endpoint	Functionality
    GET /v1/triggers	/v1/list	List triggers
    POST /v1/triggers	/v1/start	Start trigger
    GET /v1/triggers/:id	/v1/status/:id	Trigger status
    DELETE /v1/triggers/:id	/v1/triggers/:id	Stop/delete trigger
    POST /v1/packages	/v1/upload_dar	Upload DAR
    GET /livez	/v1/health	liveness check

  • The trigger service now has a --port-file option matching the corresponding option in the JSON API.
  • The trigger service now accepts multiple --dar options.
• Triggers
  • The Daml.Trigger module now re-exports Event which avoids having to import Daml.Trigger.LowLevel for implementing a non-trivial updateState function.
  • UNAUTHENTICATED errors will now terminate the trigger. These errors are no longer available for handling in the trigger DAML code. Instead, they are forwarded to the trigger service for handling, e.g., access token refresh.
• Command submission requests to the gRPC Ledger API now accept some optional fields for use in the upcoming multi-party submissions feature. Such submissions currently return UNIMPLEMENTED errors, but they will be enabled in the future.

Bugfixes

• The DAML compiler now enforces sequential ordering for let expressions with multiple bindings. This fixes a bug where let expressions would be evaluated in the wrong order.
• Fixed a bug where trace statements from a failing transaction were not displayed in DAML Studio.
• Fixed a regression in the HTTP JSON API introduced in SDK 1.7.0, where using a party multiple times in the same JWT token (e.g., readAs and actAs) broke database queries for that party. Note that there is never a reason to include a party multiple times since actAs implies readAs.

Integration Kit

• When using a PostgreSQL-based index, leveraging native parallel unnesting allows you to more efficiently index new transactions.
• The kvutils Protobuf definition is now published to Maven Central in a JAR, under the group "com.daml", with the artifact name "participant-state-kvutils-proto".
• The Scala JARs containing the gRPC definitions no longer contain the *.proto files used to generate the ScalaPB-based classes.
• New CLI option --cert-revocation-checking for enabling the TLS certificate revocation checking in the LedgerApiServer.
• kvutils reports LookupByKey node mismatches during validation as Inconsistent instead of Disputed if they can be due to contention on the contract key.
• Bugfix: daml.index.db.store_transaction metrics were keeping track of empty insertions, skewing down the numbers.
• Performance: minimizing the number of traversals of a transaction to index it, more efficient indexing.
• The integrity checker now validates the contents of each write set, regardless of whether it is used subsequently.
• Pipelining in the indexing process improves throughput by up to 15%.
• Correct some remaining package name references to com.digitalasset.platform in logback and readme file.
• Fix a bug resulting in the error message "Dispatcher is closed" on participant shutdown (#7986).
• The preview of ParticipantPruningService enables ledger participants to prune the "front" of ledger state at the participant including the ledger api server index.

What’s Next

The eagle eyed reader may have noticed that some features have appeared in the “What’s Next” section for some time, and that there hasn’t been a DAML-LF release since SDK 1.0.0 and DAML-LF 1.8. This will change with one of the next releases because several features that require a new DAML-LF version are currently being finalized:

• Choice observers (see Early Access section above)
• Generic Maps
• Better Exception Handling in DAML

Work also continues to move DAML Triggers and the Trigger Service (see Early Access section above) to general availability.

Lastly, the multi-party read features on the gRPC Ledger and JSON APIs will be supplemented with multi-party writes, allowing the submission of commands involving multiple parties via the APIs as long as they are all hosted on the same node.