Linux command
avro-tools 命令
文件
涉及管道、覆盖或删除，执行前请先确认路径和参数。
常用示例
Convert JSON to Avro
avro-tools fromjson --schema-file [schema.avsc] [input.json] > [output.avro]
Convert Avro to JSON
avro-tools tojson [input.avro] > [output.json]
Display Avro file schema
avro-tools getschema [file.avro]
Display file metadata
avro-tools getmeta [file.avro]
Concatenate Avro files
avro-tools concat [file1.avro] [file2.avro] [output.avro]
Compile Avro schema to Java
avro-tools compile schema [schema.avsc] [output_directory]
Generate random data from schema
avro-tools random --schema-file [schema.avsc] --count [100] [output.avro]
说明
avro-tools is a collection of utilities for working with Apache Avro data files. Avro is a data serialization system that provides rich data structures, a compact binary format, and schema evolution capabilities. The tools enable conversion between Avro's binary format and human-readable JSON, schema extraction and validation, file manipulation, and code generation. Avro files are self-describing, containing their schema alongside the data, which avro-tools can extract and display. Common workflows include inspecting Avro files produced by data pipelines, converting data between formats for debugging, generating test data, and compiling schemas to language-specific classes for application development.
参数
- fromjson
- Convert JSON to Avro format.
- tojson
- Convert Avro to JSON format.
- getschema
- Extract and print the schema from an Avro file.
- getmeta
- Print metadata from an Avro file.
- concat
- Concatenate multiple Avro files with compatible schemas.
- compile
- Compile schema to Java source files.
- idl
- Compile Avro IDL to schema.
- random
- Generate random data based on a schema.
- recodec
- Change the codec of an Avro file.
- --schema-file _file_
- Path to Avro schema file (.avsc).
- --count _n_
- Number of records to generate (for random).
- --codec _name_
- Compression codec (null, deflate, snappy, bzip2, zstd, xz).
FAQ
What is the avro-tools command used for?
avro-tools is a collection of utilities for working with Apache Avro data files. Avro is a data serialization system that provides rich data structures, a compact binary format, and schema evolution capabilities. The tools enable conversion between Avro's binary format and human-readable JSON, schema extraction and validation, file manipulation, and code generation. Avro files are self-describing, containing their schema alongside the data, which avro-tools can extract and display. Common workflows include inspecting Avro files produced by data pipelines, converting data between formats for debugging, generating test data, and compiling schemas to language-specific classes for application development.
How do I run a basic avro-tools example?
Run `avro-tools fromjson --schema-file [schema.avsc] [input.json] > [output.avro]` in a terminal, then adjust file names, paths, flags, or remote targets for your system.
What does fromjson do in avro-tools?
Convert JSON to Avro format.