PluginsPlugin SerdesProtobuf

ProtobufToIon​Protobuf​To​Ion

Convert a Protobuf file into Amazon Ion.

The plugin reads one or more Protobuf messages from a binary file or stream, decodes them using a provided descriptor and message type name, and then serializes the result as Ion data based on ProtoJSON Format. It requires the following information regarding the Protobuf message:

  • A descriptor file (.desc), generated with --descriptor_set_out, that contains the compiled Protobuf message definitions.
  • A type name (e.g., com.company.Product) corresponding to the root message to decode.

Here's an example of how to generate a descriptor file from a Protobuf file using protoc:

text
protoc --proto_path=src/main/proto        --descriptor_set_out=products.desc        src/main/proto/products.proto
yaml
type: "io.kestra.plugin.serdes.protobuf.ProtobufToIon"

Convert a Protobuf file to the Amazon Ion format.

yaml
id: protobuf_to_ion
namespace: company.team

tasks:
  - id: http_download
    type: io.kestra.plugin.core.http.Download
    uri: https://example.com/data/products.pb

  - id: to_ion
    type: io.kestra.plugin.serdes.protobuf.ProtobufToIon
    from: "{{ outputs.http_download.uri }}"
    descriptorFile: "kestra:///path/to/proto.desc"
    typeName: com.company.Product
Properties

Protobuf Descriptor File

Protobuf descriptor file containing message definitions.

Source file URI

The URI of the input Protobuf file to read.

Fully qualified Protobuf message type name

For example: com.company.Product.

Default false

Is the input a stream of length-delimited messages?

If true, the input file is expected to contain multiple length-delimited Protobuf messages. If false, it will be parsed as a single Protobuf message.

Default false

Whether to error on unknown fields

If true, an error will be thrown if the input contains unknown fields.

Format uri

URI of a temporary result file

Number of Protobuf messages converted