Mulesoft DataWeave Language Overview and In depth Knowledge

• The Transform (or Transform Message) component converts input data to a new output structure or format. You can use the component’s visual mapper to build your transformation implicitly by dragging and dropping elements of the incoming data structure, or you can explicitly write a transformation script in DataWeave expression language.

• Graphical View (Drag-and-Drop editor): Two tree views show the expected metadata structures of the input and output. Mappings between these two trees are represented by lines that connect input to output fields. The lines can be created by dragging and dropping an element from the input to another element to the output.

• DataWeave can read and write many types of data formats, such as JSON, XML, and many others. Before you begin, note that 2.x versions of DataWeave are used by Mule 4 apps.

• DataWeave can read input data as a whole by loading it into memory or by indexing it in local storage and, for some data formats, DataWeave can read data sequentially in parts by streaming the input. The indexed reading strategy implements a mechanism to avoid out-of-memory issues when processing large files. The streaming strategy can improve performance but restricts file access to sequential access (which disallows random access) because you can access only the part of the file that is being read and stored in memory.

• In some cases, it is necessary to modify or specify aspects of the format through format-specific properties. For example, you can specify CSV input and output properties, such as the separator (or delimiter) to use in the CSV file. For Cobol copybook, you need to specify the path to a schema file using the schemaPath property.

• DataWeave selectors traverse the structures of objects and arrays and return matching values. Before you begin, note that 2.x versions of DataWeave are used by Mule 4 apps.

• In DataWeave, a name that matches the key of an object (such as the key "someName" in the JSON object { "someName" : "Data Weave" }) can select data from that object.

• This selector matches the value of the first key-value pair in which the key matches the given selector name. The selector can be applied to an Object or to an array. On an array, the selector applies to all Object values inside the array and ignores all values that are not Object values. If no key matches, the value null is returned.

• DataWeave supports several operators, including mathematical operators, equality operators, and operators such as prepend, append and update. Before you begin, note that 2.x versions of DataWeave are used by Mule 4 apps.

• In addition to operating with numbers, the (-) and (+) operators can also operate with complex data structures like arrays, objects, and dates.

• Though the semantics of not and ! are the same, their precedence differs. not true or true is executed as not (true or true), so it returns false, whereas !true or true returns true because the ! only applies to the first true. !(true or true) returns false.

You can use the following operators within any DataWeave expression:

• do

• if else

• else if

  
  

• A do statement creates a scope in which new variables, functions, annotations, or namespaces can be declared and used. The syntax is similar to a mapping in that it is composed of a header and body separated by ---. Its header is where all the declarations are defined, and its body is the result of the expression.

• An if statement evaluates a conditional expression and returns the value under the if only if the conditional expression returns true. Otherwise, it returns the expression under else. Every if expression must have a matching else expression.

• Iterates over an object using a mapper that acts on keys, values, or indices of that object.

• Iterates a list of key-value pairs in an object and applies an expression that returns only matching objects, filtering out the rest from the output.

• The expression must return true or false. If the expression returns true for a key, value, or index of an object, the object gets captured in the output. If it returns false for any of them, the object gets filtered out of the output. If there are no matches, the output array will be empty.

• Useful for mapping an object into an array, pluck iterates over an object and returns an array of keys, values, or indices from the object.

• It is an alternative to mapObject, which is similar but returns an object, instead of an array.

• Iterates over an array and applies an expression that returns matching values. The expression must return true or false. If the expression returns true for a value or index in the array, the value gets captured in the output array. If it returns false for a value or index in the array, that item gets filtered out of the output. If there are no matches, the output array will be empty.

• Iterates over an array and returns the unique elements in it. DataWeave uses the result of applying the provided lambda as the uniqueness criteria.

• This version of distinctBy finds unique values in an array. Other versions act on an object and handle a null value.

• Iterates over items in an array and outputs the results into a new array.

Dates in DataWeave follow the ISO-8601 standard and literals are defined between | characters.

The language has the following native date types:

• Date

• DateTime

• LocalDateTime

• LocalTime

• Period

• Time

• TimeZone

You can use DataWeave to change the format of date and time input. You can combine formatting characters, such as MM and dd, to write supported date and time formats. The example uses the as operator to write the dates and times as a string.

You can define your own DataWeave functions using the fun declaration in the header of a DataWeave script.

To define a function in DataWeave use the following syntax:

fun myFunction(param1, param2, ...) = <code to execute>

• The fun keyword starts the definition of a function.

• myFunction is the name you define for the function.

  Function names must be valid identifiers. For additional details about valid identifiers, see Rules for Declaring Valid Identifiers.

• (param1, param2, …​ , paramn) represents the parameters that your function accepts.

  You can specify from zero to any number of parameters, separated by commas (,) and enclosed in parentheses.

• The = sign marks the beginning of the code block to execute when the function is called.

• <code to execute> represents the actual code that you define for your function.

In addition to using the built-in DataWeave function modules (such as dw::Core and dw::Crypto), you can also create and use custom modules and mapping files. The examples demonstrate common data extraction and transformation approaches.

You write modules and mapping files in a DataWeave Language (.dwl) file and import into your Mule app through DataWeave scripts in Mule components. Both modules and mapping files are useful when you need to reuse the same functionality or feature over and over again.

• Custom modules can define functions, variables, types, and namespaces. You can import these modules into a DataWeave script to use the features.

• Custom mapping files are a type of module that contains a complete DataWeave script that you can import and use in another DataWeave script or reference in a Mule component.

Fields in many Mule connectors and components accept DataWeave expressions and scripts.

Note that if you want to import and use a built-in DataWeave function module, and not a custom one

From a DataWeave statement, you can call Java methods from any Java class that’s in your Mule project. Note that you can only call Static methods via DataWeave (methods that belong to a Java class, not methods that belong to a specific instance of a class).

Before you can call a method, you must first import the class it belongs to into your DataWeave code. You can import Java classes just as you import DataWeave modules by including java! into the statement.

To create the MyClass example in a Studio project:

1. Create a project for the class in Studio by clicking New → Mule Project and providing a name, such as my-app, and clicking Finish.

2. From my-app, create the org.mycompany.utils package by right-clicking src/main/java from Studio’s Package Explorer, selecting New → Package, and naming the package org.mycompany.utils.

3. Create the MyClass class in the your new org.mycompany.utils package by right-clicking org.mycompany.utils, selecting New → Class, naming that class MyClass, and clicking Finish.

4. In the MyClass.java tab that opens in Studio, copy, paste, and save the contents MyClass.

The following DataWeave examples show how to convert a file stream into Base64 and to convert a Base64 string into a file stream. They use a PDF file as the input and output.

To use this module, you must import it to your DataWeave code, for example, by adding the line import * from dw::core::Binaries to the header of your DataWeave script.

Functions:

• fromBase64

• fromHex

• toBase64

• toHex

DataWeave supports end-to-end streaming through a flow in a Mule application. Streaming speeds the processing of large documents without overloading memory.

DataWeave processes streamed data as its bytes arrive instead of scanning the entire document to index it. When in deferred mode, DataWeave can also pass streamed output data directly to a message processor without saving it to the disk. This behavior enables DataWeave and Mule to process data faster and consume fewer resources than the default processes for reading and writing data.

To stream successfully, it is important to understand the following:

• The basic unit of the stream is specific to the data format.
  The unit is a record in a CSV document, an element of an array in a JSON document, or a collection in an XML document.

• Streaming accesses each unit of the stream sequentially.
  Streaming does not support random access to a document.

Streaming is not enabled by default. You can use two configuration properties to stream data in a supported data format:

• streaming property, for reading source data as a stream

• deferred writer property, for passing an output stream directly to the next message processor in a flow

Below are a series of examples that demonstrate various data transformation approaches expressed in DataWeave. DataWeave code is typically written inside the Transform Message component, which is accessible in Anypoint Studio.

https://docs.mulesoft.com/dataweave/1.2/dataweave-examples