mirror of
https://github.com/openappsec/openappsec.git
synced 2025-09-29 19:24:26 +03:00
Support local managment for embedded agent on nginx
This commit is contained in:
253
external/yq/pkg/yqlib/doc/usage/convert.md
vendored
Executable file
253
external/yq/pkg/yqlib/doc/usage/convert.md
vendored
Executable file
@@ -0,0 +1,253 @@
|
||||
# JSON
|
||||
|
||||
Encode and decode to and from JSON. Supports multiple JSON documents in a single file (e.g. NDJSON).
|
||||
|
||||
Note that YAML is a superset of (single document) JSON - so you don't have to use the JSON parser to read JSON when there is only one JSON document in the input. You will probably want to pretty print the result in this case, to get idiomatic YAML styling.
|
||||
|
||||
|
||||
## Parse json: simple
|
||||
JSON is a subset of yaml, so all you need to do is prettify the output
|
||||
|
||||
Given a sample.json file of:
|
||||
```json
|
||||
{"cat": "meow"}
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -P '.' sample.json
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
cat: meow
|
||||
```
|
||||
|
||||
## Parse json: complex
|
||||
JSON is a subset of yaml, so all you need to do is prettify the output
|
||||
|
||||
Given a sample.json file of:
|
||||
```json
|
||||
{"a":"Easy! as one two three","b":{"c":2,"d":[3,4]}}
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -P '.' sample.json
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: Easy! as one two three
|
||||
b:
|
||||
c: 2
|
||||
d:
|
||||
- 3
|
||||
- 4
|
||||
```
|
||||
|
||||
## Encode json: simple
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
cat: meow
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=json '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```json
|
||||
{
|
||||
"cat": "meow"
|
||||
}
|
||||
```
|
||||
|
||||
## Encode json: simple - in one line
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
cat: meow # this is a comment, and it will be dropped.
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=json -I=0 '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```json
|
||||
{"cat":"meow"}
|
||||
```
|
||||
|
||||
## Encode json: comments
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
cat: meow # this is a comment, and it will be dropped.
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=json '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```json
|
||||
{
|
||||
"cat": "meow"
|
||||
}
|
||||
```
|
||||
|
||||
## Encode json: anchors
|
||||
Anchors are dereferenced
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
cat: &ref meow
|
||||
anotherCat: *ref
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=json '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```json
|
||||
{
|
||||
"cat": "meow",
|
||||
"anotherCat": "meow"
|
||||
}
|
||||
```
|
||||
|
||||
## Encode json: multiple results
|
||||
Each matching node is converted into a json doc. This is best used with 0 indent (json document per line)
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
things: [{stuff: cool}, {whatever: cat}]
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=json -I=0 '.things[]' sample.yml
|
||||
```
|
||||
will output
|
||||
```json
|
||||
{"stuff":"cool"}
|
||||
{"whatever":"cat"}
|
||||
```
|
||||
|
||||
## Roundtrip NDJSON
|
||||
Unfortunately the json encoder strips leading spaces of values.
|
||||
|
||||
Given a sample.json file of:
|
||||
```json
|
||||
{"this": "is a multidoc json file"}
|
||||
{"each": ["line is a valid json document"]}
|
||||
{"a number": 4}
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=json -o=json -I=0 sample.json
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{"this":"is a multidoc json file"}
|
||||
{"each":["line is a valid json document"]}
|
||||
{"a number":4}
|
||||
```
|
||||
|
||||
## Roundtrip multi-document JSON
|
||||
The NDJSON parser can also handle multiple multi-line json documents in a single file!
|
||||
|
||||
Given a sample.json file of:
|
||||
```json
|
||||
{
|
||||
"this": "is a multidoc json file"
|
||||
}
|
||||
{
|
||||
"it": [
|
||||
"has",
|
||||
"consecutive",
|
||||
"json documents"
|
||||
]
|
||||
}
|
||||
{
|
||||
"a number": 4
|
||||
}
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=json -o=json -I=2 sample.json
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{
|
||||
"this": "is a multidoc json file"
|
||||
}
|
||||
{
|
||||
"it": [
|
||||
"has",
|
||||
"consecutive",
|
||||
"json documents"
|
||||
]
|
||||
}
|
||||
{
|
||||
"a number": 4
|
||||
}
|
||||
```
|
||||
|
||||
## Update a specific document in a multi-document json
|
||||
Documents are indexed by the `documentIndex` or `di` operator.
|
||||
|
||||
Given a sample.json file of:
|
||||
```json
|
||||
{"this": "is a multidoc json file"}
|
||||
{"each": ["line is a valid json document"]}
|
||||
{"a number": 4}
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=json -o=json -I=0 '(select(di == 1) | .each ) += "cool"' sample.json
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{"this":"is a multidoc json file"}
|
||||
{"each":["line is a valid json document","cool"]}
|
||||
{"a number":4}
|
||||
```
|
||||
|
||||
## Find and update a specific document in a multi-document json
|
||||
Use expressions as you normally would.
|
||||
|
||||
Given a sample.json file of:
|
||||
```json
|
||||
{"this": "is a multidoc json file"}
|
||||
{"each": ["line is a valid json document"]}
|
||||
{"a number": 4}
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=json -o=json -I=0 '(select(has("each")) | .each ) += "cool"' sample.json
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{"this":"is a multidoc json file"}
|
||||
{"each":["line is a valid json document","cool"]}
|
||||
{"a number":4}
|
||||
```
|
||||
|
||||
## Decode NDJSON
|
||||
Given a sample.json file of:
|
||||
```json
|
||||
{"this": "is a multidoc json file"}
|
||||
{"each": ["line is a valid json document"]}
|
||||
{"a number": 4}
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=json sample.json
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
this: is a multidoc json file
|
||||
---
|
||||
each:
|
||||
- line is a valid json document
|
||||
---
|
||||
a number: 4
|
||||
```
|
||||
|
208
external/yq/pkg/yqlib/doc/usage/csv-tsv.md
vendored
Executable file
208
external/yq/pkg/yqlib/doc/usage/csv-tsv.md
vendored
Executable file
@@ -0,0 +1,208 @@
|
||||
# CSV
|
||||
Encode/Decode/Roundtrip CSV and TSV files.
|
||||
|
||||
## Encode
|
||||
Currently supports arrays of homogenous flat objects, that is: no nesting and it assumes the _first_ object has all the keys required:
|
||||
|
||||
```yaml
|
||||
- name: Bobo
|
||||
type: dog
|
||||
- name: Fifi
|
||||
type: cat
|
||||
```
|
||||
|
||||
As well as arrays of arrays of scalars (strings/numbers/booleans):
|
||||
|
||||
```yaml
|
||||
- [Bobo, dog]
|
||||
- [Fifi, cat]
|
||||
```
|
||||
|
||||
## Decode
|
||||
Decode assumes the first CSV/TSV row is the header row, and all rows beneath are the entries.
|
||||
The data will be coded into an array of objects, using the header rows as keys.
|
||||
|
||||
```csv
|
||||
name,type
|
||||
Bobo,dog
|
||||
Fifi,cat
|
||||
```
|
||||
|
||||
|
||||
## Encode CSV simple
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- [i, like, csv]
|
||||
- [because, excel, is, cool]
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=csv sample.yml
|
||||
```
|
||||
will output
|
||||
```csv
|
||||
i,like,csv
|
||||
because,excel,is,cool
|
||||
```
|
||||
|
||||
## Encode TSV simple
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- [i, like, csv]
|
||||
- [because, excel, is, cool]
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=tsv sample.yml
|
||||
```
|
||||
will output
|
||||
```tsv
|
||||
i like csv
|
||||
because excel is cool
|
||||
```
|
||||
|
||||
## Encode array of objects to csv
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- name: Gary
|
||||
numberOfCats: 1
|
||||
likesApples: true
|
||||
height: 168.8
|
||||
- name: Samantha's Rabbit
|
||||
numberOfCats: 2
|
||||
likesApples: false
|
||||
height: -188.8
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=csv sample.yml
|
||||
```
|
||||
will output
|
||||
```csv
|
||||
name,numberOfCats,likesApples,height
|
||||
Gary,1,true,168.8
|
||||
Samantha's Rabbit,2,false,-188.8
|
||||
```
|
||||
|
||||
## Encode array of objects to custom csv format
|
||||
Add the header row manually, then the we convert each object into an array of values - resulting in an array of arrays. Pick the columns and call the header whatever you like.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- name: Gary
|
||||
numberOfCats: 1
|
||||
likesApples: true
|
||||
height: 168.8
|
||||
- name: Samantha's Rabbit
|
||||
numberOfCats: 2
|
||||
likesApples: false
|
||||
height: -188.8
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=csv '[["Name", "Number of Cats"]] + [.[] | [.name, .numberOfCats ]]' sample.yml
|
||||
```
|
||||
will output
|
||||
```csv
|
||||
Name,Number of Cats
|
||||
Gary,1
|
||||
Samantha's Rabbit,2
|
||||
```
|
||||
|
||||
## Encode array of objects to csv - missing fields behaviour
|
||||
First entry is used to determine the headers, and it is missing 'likesApples', so it is not included in the csv. Second entry does not have 'numberOfCats' so that is blank
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- name: Gary
|
||||
numberOfCats: 1
|
||||
height: 168.8
|
||||
- name: Samantha's Rabbit
|
||||
height: -188.8
|
||||
likesApples: false
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=csv sample.yml
|
||||
```
|
||||
will output
|
||||
```csv
|
||||
name,numberOfCats,height
|
||||
Gary,1,168.8
|
||||
Samantha's Rabbit,,-188.8
|
||||
```
|
||||
|
||||
## Parse CSV into an array of objects
|
||||
First row is assumed to be the header row.
|
||||
|
||||
Given a sample.csv file of:
|
||||
```csv
|
||||
name,numberOfCats,likesApples,height
|
||||
Gary,1,true,168.8
|
||||
Samantha's Rabbit,2,false,-188.8
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=csv sample.csv
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- name: Gary
|
||||
numberOfCats: 1
|
||||
likesApples: true
|
||||
height: 168.8
|
||||
- name: Samantha's Rabbit
|
||||
numberOfCats: 2
|
||||
likesApples: false
|
||||
height: -188.8
|
||||
```
|
||||
|
||||
## Parse TSV into an array of objects
|
||||
First row is assumed to be the header row.
|
||||
|
||||
Given a sample.tsv file of:
|
||||
```tsv
|
||||
name numberOfCats likesApples height
|
||||
Gary 1 true 168.8
|
||||
Samantha's Rabbit 2 false -188.8
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=tsv sample.tsv
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- name: Gary
|
||||
numberOfCats: 1
|
||||
likesApples: true
|
||||
height: 168.8
|
||||
- name: Samantha's Rabbit
|
||||
numberOfCats: 2
|
||||
likesApples: false
|
||||
height: -188.8
|
||||
```
|
||||
|
||||
## Round trip
|
||||
Given a sample.csv file of:
|
||||
```csv
|
||||
name,numberOfCats,likesApples,height
|
||||
Gary,1,true,168.8
|
||||
Samantha's Rabbit,2,false,-188.8
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=csv -o=csv '(.[] | select(.name == "Gary") | .numberOfCats) = 3' sample.csv
|
||||
```
|
||||
will output
|
||||
```csv
|
||||
name,numberOfCats,likesApples,height
|
||||
Gary,3,true,168.8
|
||||
Samantha's Rabbit,2,false,-188.8
|
||||
```
|
||||
|
6
external/yq/pkg/yqlib/doc/usage/headers/convert.md
vendored
Executable file
6
external/yq/pkg/yqlib/doc/usage/headers/convert.md
vendored
Executable file
@@ -0,0 +1,6 @@
|
||||
# JSON
|
||||
|
||||
Encode and decode to and from JSON. Supports multiple JSON documents in a single file (e.g. NDJSON).
|
||||
|
||||
Note that YAML is a superset of (single document) JSON - so you don't have to use the JSON parser to read JSON when there is only one JSON document in the input. You will probably want to pretty print the result in this case, to get idiomatic YAML styling.
|
||||
|
30
external/yq/pkg/yqlib/doc/usage/headers/csv-tsv.md
vendored
Executable file
30
external/yq/pkg/yqlib/doc/usage/headers/csv-tsv.md
vendored
Executable file
@@ -0,0 +1,30 @@
|
||||
# CSV
|
||||
Encode/Decode/Roundtrip CSV and TSV files.
|
||||
|
||||
## Encode
|
||||
Currently supports arrays of homogenous flat objects, that is: no nesting and it assumes the _first_ object has all the keys required:
|
||||
|
||||
```yaml
|
||||
- name: Bobo
|
||||
type: dog
|
||||
- name: Fifi
|
||||
type: cat
|
||||
```
|
||||
|
||||
As well as arrays of arrays of scalars (strings/numbers/booleans):
|
||||
|
||||
```yaml
|
||||
- [Bobo, dog]
|
||||
- [Fifi, cat]
|
||||
```
|
||||
|
||||
## Decode
|
||||
Decode assumes the first CSV/TSV row is the header row, and all rows beneath are the entries.
|
||||
The data will be coded into an array of objects, using the header rows as keys.
|
||||
|
||||
```csv
|
||||
name,type
|
||||
Bobo,dog
|
||||
Fifi,cat
|
||||
```
|
||||
|
5
external/yq/pkg/yqlib/doc/usage/headers/properties.md
vendored
Executable file
5
external/yq/pkg/yqlib/doc/usage/headers/properties.md
vendored
Executable file
@@ -0,0 +1,5 @@
|
||||
# Properties
|
||||
|
||||
Encode/Decode/Roundtrip to/from a property file. Line comments on value nodes will be copied across.
|
||||
|
||||
By default, empty maps and arrays are not encoded - see below for an example on how to encode a value for these.
|
40
external/yq/pkg/yqlib/doc/usage/headers/xml.md
vendored
Executable file
40
external/yq/pkg/yqlib/doc/usage/headers/xml.md
vendored
Executable file
@@ -0,0 +1,40 @@
|
||||
# XML
|
||||
|
||||
Encode and decode to and from XML. Whitespace is not conserved for round trips - but the order of the fields are.
|
||||
|
||||
Consecutive xml nodes with the same name are assumed to be arrays.
|
||||
|
||||
XML content data, attributes processing instructions and directives are all created as plain fields.
|
||||
|
||||
This can be controlled by:
|
||||
|
||||
| Flag | Default |Sample XML |
|
||||
| -- | -- | -- |
|
||||
| `--xml-attribute-prefix` | `+` (changing to `+@` soon) | Legs in ```<cat legs="4"/>``` |
|
||||
| `--xml-content-name` | `+content` | Meow in ```<cat>Meow <fur>true</true></cat>``` |
|
||||
| `--xml-directive-name` | `+directive` | ```<!DOCTYPE config system "blah">``` |
|
||||
| `--xml-proc-inst-prefix` | `+p_` | ```<?xml version="1"?>``` |
|
||||
|
||||
|
||||
{% hint style="warning" %}
|
||||
Default Attribute Prefix will be changing in v4.30!
|
||||
In order to avoid name conflicts (e.g. having an attribute named "content" will create a field that clashes with the default content name of "+content") the attribute prefix will be changing to "+@".
|
||||
|
||||
This will affect users that have not set their own prefix and are not roundtripping XML changes.
|
||||
|
||||
{% endhint %}
|
||||
|
||||
## Encoder / Decoder flag options
|
||||
|
||||
In addition to the above flags, there are the following xml encoder/decoder options controlled by flags:
|
||||
|
||||
| Flag | Default | Description |
|
||||
| -- | -- | -- |
|
||||
| `--xml-strict-mode` | false | Strict mode enforces the requirements of the XML specification. When switched off the parser allows input containing common mistakes. See [the Golang xml decoder ](https://pkg.go.dev/encoding/xml#Decoder) for more details.|
|
||||
| `--xml-keep-namespace` | true | Keeps the namespace of attributes |
|
||||
| `--xml-raw-token` | true | Does not verify that start and end elements match and does not translate name space prefixes to their corresponding URLs. |
|
||||
| `--xml-skip-proc-inst` | false | Skips over processing instructions, e.g. `<?xml version="1"?>` |
|
||||
| `--xml-skip-directives` | false | Skips over directives, e.g. ```<!DOCTYPE config system "blah">``` |
|
||||
|
||||
|
||||
See below for examples
|
195
external/yq/pkg/yqlib/doc/usage/properties.md
vendored
Executable file
195
external/yq/pkg/yqlib/doc/usage/properties.md
vendored
Executable file
@@ -0,0 +1,195 @@
|
||||
# Properties
|
||||
|
||||
Encode/Decode/Roundtrip to/from a property file. Line comments on value nodes will be copied across.
|
||||
|
||||
By default, empty maps and arrays are not encoded - see below for an example on how to encode a value for these.
|
||||
|
||||
## Encode properties
|
||||
Note that empty arrays and maps are not encoded by default.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
# block comments come through
|
||||
person: # neither do comments on maps
|
||||
name: Mike Wazowski # comments on values appear
|
||||
pets:
|
||||
- cat # comments on array values appear
|
||||
food: [pizza] # comments on arrays do not
|
||||
emptyArray: []
|
||||
emptyMap: []
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=props sample.yml
|
||||
```
|
||||
will output
|
||||
```properties
|
||||
# block comments come through
|
||||
# comments on values appear
|
||||
person.name = Mike Wazowski
|
||||
|
||||
# comments on array values appear
|
||||
person.pets.0 = cat
|
||||
person.food.0 = pizza
|
||||
```
|
||||
|
||||
## Encode properties: scalar encapsulation
|
||||
Note that string values with blank characters in them are encapsulated with double quotes
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
# block comments come through
|
||||
person: # neither do comments on maps
|
||||
name: Mike Wazowski # comments on values appear
|
||||
pets:
|
||||
- cat # comments on array values appear
|
||||
food: [pizza] # comments on arrays do not
|
||||
emptyArray: []
|
||||
emptyMap: []
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=props --unwrapScalar=false sample.yml
|
||||
```
|
||||
will output
|
||||
```properties
|
||||
# block comments come through
|
||||
# comments on values appear
|
||||
person.name = "Mike Wazowski"
|
||||
|
||||
# comments on array values appear
|
||||
person.pets.0 = cat
|
||||
person.food.0 = pizza
|
||||
```
|
||||
|
||||
## Encode properties: no comments
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
# block comments come through
|
||||
person: # neither do comments on maps
|
||||
name: Mike Wazowski # comments on values appear
|
||||
pets:
|
||||
- cat # comments on array values appear
|
||||
food: [pizza] # comments on arrays do not
|
||||
emptyArray: []
|
||||
emptyMap: []
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=props '... comments = ""' sample.yml
|
||||
```
|
||||
will output
|
||||
```properties
|
||||
person.name = Mike Wazowski
|
||||
person.pets.0 = cat
|
||||
person.food.0 = pizza
|
||||
```
|
||||
|
||||
## Encode properties: include empty maps and arrays
|
||||
Use a yq expression to set the empty maps and sequences to your desired value.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
# block comments come through
|
||||
person: # neither do comments on maps
|
||||
name: Mike Wazowski # comments on values appear
|
||||
pets:
|
||||
- cat # comments on array values appear
|
||||
food: [pizza] # comments on arrays do not
|
||||
emptyArray: []
|
||||
emptyMap: []
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=props '(.. | select( (tag == "!!map" or tag =="!!seq") and length == 0)) = ""' sample.yml
|
||||
```
|
||||
will output
|
||||
```properties
|
||||
# block comments come through
|
||||
# comments on values appear
|
||||
person.name = Mike Wazowski
|
||||
|
||||
# comments on array values appear
|
||||
person.pets.0 = cat
|
||||
person.food.0 = pizza
|
||||
emptyArray =
|
||||
emptyMap =
|
||||
```
|
||||
|
||||
## Decode properties
|
||||
Given a sample.properties file of:
|
||||
```properties
|
||||
# block comments come through
|
||||
# comments on values appear
|
||||
person.name = Mike Wazowski
|
||||
|
||||
# comments on array values appear
|
||||
person.pets.0 = cat
|
||||
person.food.0 = pizza
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=props sample.properties
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
person:
|
||||
# block comments come through
|
||||
# comments on values appear
|
||||
name: Mike Wazowski
|
||||
pets:
|
||||
# comments on array values appear
|
||||
- cat
|
||||
food:
|
||||
- pizza
|
||||
```
|
||||
|
||||
## Decode properties - array should be a map
|
||||
If you have a numeric map key in your property files, use array_to_map to convert them to maps.
|
||||
|
||||
Given a sample.properties file of:
|
||||
```properties
|
||||
things.10 = mike
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=props '.things |= array_to_map' sample.properties
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
things:
|
||||
10: mike
|
||||
```
|
||||
|
||||
## Roundtrip
|
||||
Given a sample.properties file of:
|
||||
```properties
|
||||
# block comments come through
|
||||
# comments on values appear
|
||||
person.name = Mike Wazowski
|
||||
|
||||
# comments on array values appear
|
||||
person.pets.0 = cat
|
||||
person.food.0 = pizza
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=props -o=props '.person.pets.0 = "dog"' sample.properties
|
||||
```
|
||||
will output
|
||||
```properties
|
||||
# block comments come through
|
||||
# comments on values appear
|
||||
person.name = Mike Wazowski
|
||||
|
||||
# comments on array values appear
|
||||
person.pets.0 = dog
|
||||
person.food.0 = pizza
|
||||
```
|
||||
|
509
external/yq/pkg/yqlib/doc/usage/xml.md
vendored
Executable file
509
external/yq/pkg/yqlib/doc/usage/xml.md
vendored
Executable file
@@ -0,0 +1,509 @@
|
||||
# XML
|
||||
|
||||
Encode and decode to and from XML. Whitespace is not conserved for round trips - but the order of the fields are.
|
||||
|
||||
Consecutive xml nodes with the same name are assumed to be arrays.
|
||||
|
||||
XML content data, attributes processing instructions and directives are all created as plain fields.
|
||||
|
||||
This can be controlled by:
|
||||
|
||||
| Flag | Default |Sample XML |
|
||||
| -- | -- | -- |
|
||||
| `--xml-attribute-prefix` | `+` (changing to `+@` soon) | Legs in ```<cat legs="4"/>``` |
|
||||
| `--xml-content-name` | `+content` | Meow in ```<cat>Meow <fur>true</true></cat>``` |
|
||||
| `--xml-directive-name` | `+directive` | ```<!DOCTYPE config system "blah">``` |
|
||||
| `--xml-proc-inst-prefix` | `+p_` | ```<?xml version="1"?>``` |
|
||||
|
||||
|
||||
{% hint style="warning" %}
|
||||
Default Attribute Prefix will be changing in v4.30!
|
||||
In order to avoid name conflicts (e.g. having an attribute named "content" will create a field that clashes with the default content name of "+content") the attribute prefix will be changing to "+@".
|
||||
|
||||
This will affect users that have not set their own prefix and are not roundtripping XML changes.
|
||||
|
||||
{% endhint %}
|
||||
|
||||
## Encoder / Decoder flag options
|
||||
|
||||
In addition to the above flags, there are the following xml encoder/decoder options controlled by flags:
|
||||
|
||||
| Flag | Default | Description |
|
||||
| -- | -- | -- |
|
||||
| `--xml-strict-mode` | false | Strict mode enforces the requirements of the XML specification. When switched off the parser allows input containing common mistakes. See [the Golang xml decoder ](https://pkg.go.dev/encoding/xml#Decoder) for more details.|
|
||||
| `--xml-keep-namespace` | true | Keeps the namespace of attributes |
|
||||
| `--xml-raw-token` | true | Does not verify that start and end elements match and does not translate name space prefixes to their corresponding URLs. |
|
||||
| `--xml-skip-proc-inst` | false | Skips over processing instructions, e.g. `<?xml version="1"?>` |
|
||||
| `--xml-skip-directives` | false | Skips over directives, e.g. ```<!DOCTYPE config system "blah">``` |
|
||||
|
||||
|
||||
See below for examples
|
||||
|
||||
## Parse xml: simple
|
||||
Notice how all the values are strings, see the next example on how you can fix that.
|
||||
|
||||
Given a sample.xml file of:
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<cat>
|
||||
<says>meow</says>
|
||||
<legs>4</legs>
|
||||
<cute>true</cute>
|
||||
</cat>
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=xml '.' sample.xml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
+p_xml: version="1.0" encoding="UTF-8"
|
||||
cat:
|
||||
says: meow
|
||||
legs: "4"
|
||||
cute: "true"
|
||||
```
|
||||
|
||||
## Parse xml: number
|
||||
All values are assumed to be strings when parsing XML, but you can use the `from_yaml` operator on all the strings values to autoparse into the correct type.
|
||||
|
||||
Given a sample.xml file of:
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<cat>
|
||||
<says>meow</says>
|
||||
<legs>4</legs>
|
||||
<cute>true</cute>
|
||||
</cat>
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=xml ' (.. | select(tag == "!!str")) |= from_yaml' sample.xml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
+p_xml: version="1.0" encoding="UTF-8"
|
||||
cat:
|
||||
says: meow
|
||||
legs: 4
|
||||
cute: true
|
||||
```
|
||||
|
||||
## Parse xml: array
|
||||
Consecutive nodes with identical xml names are assumed to be arrays.
|
||||
|
||||
Given a sample.xml file of:
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<animal>cat</animal>
|
||||
<animal>goat</animal>
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=xml '.' sample.xml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
+p_xml: version="1.0" encoding="UTF-8"
|
||||
animal:
|
||||
- cat
|
||||
- goat
|
||||
```
|
||||
|
||||
## Parse xml: attributes
|
||||
Attributes are converted to fields, with the default attribute prefix '+'. Use '--xml-attribute-prefix` to set your own.
|
||||
|
||||
Given a sample.xml file of:
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<cat legs="4">
|
||||
<legs>7</legs>
|
||||
</cat>
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=xml '.' sample.xml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
+p_xml: version="1.0" encoding="UTF-8"
|
||||
cat:
|
||||
+@legs: "4"
|
||||
legs: "7"
|
||||
```
|
||||
|
||||
## Parse xml: attributes with content
|
||||
Content is added as a field, using the default content name of `+content`. Use `--xml-content-name` to set your own.
|
||||
|
||||
Given a sample.xml file of:
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<cat legs="4">meow</cat>
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=xml '.' sample.xml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
+p_xml: version="1.0" encoding="UTF-8"
|
||||
cat:
|
||||
+content: meow
|
||||
+@legs: "4"
|
||||
```
|
||||
|
||||
## Parse xml: custom dtd
|
||||
DTD entities are processed as directives.
|
||||
|
||||
Given a sample.xml file of:
|
||||
```xml
|
||||
|
||||
<?xml version="1.0"?>
|
||||
<!DOCTYPE root [
|
||||
<!ENTITY writer "Blah.">
|
||||
<!ENTITY copyright "Blah">
|
||||
]>
|
||||
<root>
|
||||
<item>&writer;©right;</item>
|
||||
</root>
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=xml -o=xml '.' sample.xml
|
||||
```
|
||||
will output
|
||||
```xml
|
||||
<?xml version="1.0"?>
|
||||
<!DOCTYPE root [
|
||||
<!ENTITY writer "Blah.">
|
||||
<!ENTITY copyright "Blah">
|
||||
]>
|
||||
<root>
|
||||
<item>&writer;&copyright;</item>
|
||||
</root>
|
||||
```
|
||||
|
||||
## Parse xml: skip custom dtd
|
||||
DTDs are directives, skip over directives to skip DTDs.
|
||||
|
||||
Given a sample.xml file of:
|
||||
```xml
|
||||
|
||||
<?xml version="1.0"?>
|
||||
<!DOCTYPE root [
|
||||
<!ENTITY writer "Blah.">
|
||||
<!ENTITY copyright "Blah">
|
||||
]>
|
||||
<root>
|
||||
<item>&writer;©right;</item>
|
||||
</root>
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=xml -o=xml --xml-skip-directives '.' sample.xml
|
||||
```
|
||||
will output
|
||||
```xml
|
||||
<?xml version="1.0"?>
|
||||
<root>
|
||||
<item>&writer;&copyright;</item>
|
||||
</root>
|
||||
```
|
||||
|
||||
## Parse xml: with comments
|
||||
A best attempt is made to preserve comments.
|
||||
|
||||
Given a sample.xml file of:
|
||||
```xml
|
||||
|
||||
<!-- before cat -->
|
||||
<cat>
|
||||
<!-- in cat before -->
|
||||
<x>3<!-- multi
|
||||
line comment
|
||||
for x --></x>
|
||||
<!-- before y -->
|
||||
<y>
|
||||
<!-- in y before -->
|
||||
<d><!-- in d before -->z<!-- in d after --></d>
|
||||
|
||||
<!-- in y after -->
|
||||
</y>
|
||||
<!-- in_cat_after -->
|
||||
</cat>
|
||||
<!-- after cat -->
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=xml '.' sample.xml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
# before cat
|
||||
cat:
|
||||
# in cat before
|
||||
x: "3" # multi
|
||||
# line comment
|
||||
# for x
|
||||
# before y
|
||||
|
||||
y:
|
||||
# in y before
|
||||
# in d before
|
||||
d: z # in d after
|
||||
# in y after
|
||||
# in_cat_after
|
||||
# after cat
|
||||
```
|
||||
|
||||
## Parse xml: keep attribute namespace
|
||||
Defaults to true
|
||||
|
||||
Given a sample.xml file of:
|
||||
```xml
|
||||
<?xml version="1.0"?>
|
||||
<map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url"></map>
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=xml -o=xml --xml-keep-namespace=false '.' sample.xml
|
||||
```
|
||||
will output
|
||||
```xml
|
||||
<?xml version="1.0"?>
|
||||
<map xmlns="some-namespace" xsi="some-instance" schemaLocation="some-url"></map>
|
||||
```
|
||||
|
||||
instead of
|
||||
```xml
|
||||
<?xml version="1.0"?>
|
||||
<map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url"></map>
|
||||
```
|
||||
|
||||
## Parse xml: keep raw attribute namespace
|
||||
Defaults to true
|
||||
|
||||
Given a sample.xml file of:
|
||||
```xml
|
||||
<?xml version="1.0"?>
|
||||
<map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url"></map>
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=xml -o=xml --xml-raw-token=false '.' sample.xml
|
||||
```
|
||||
will output
|
||||
```xml
|
||||
<?xml version="1.0"?>
|
||||
<map xmlns="some-namespace" xmlns:xsi="some-instance" some-instance:schemaLocation="some-url"></map>
|
||||
```
|
||||
|
||||
instead of
|
||||
```xml
|
||||
<?xml version="1.0"?>
|
||||
<map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url"></map>
|
||||
```
|
||||
|
||||
## Encode xml: simple
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
cat: purrs
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=xml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```xml
|
||||
<cat>purrs</cat>
|
||||
```
|
||||
|
||||
## Encode xml: array
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
pets:
|
||||
cat:
|
||||
- purrs
|
||||
- meows
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=xml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```xml
|
||||
<pets>
|
||||
<cat>purrs</cat>
|
||||
<cat>meows</cat>
|
||||
</pets>
|
||||
```
|
||||
|
||||
## Encode xml: attributes
|
||||
Fields with the matching xml-attribute-prefix are assumed to be attributes.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
cat:
|
||||
+@name: tiger
|
||||
meows: true
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=xml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```xml
|
||||
<cat name="tiger">
|
||||
<meows>true</meows>
|
||||
</cat>
|
||||
```
|
||||
|
||||
## Encode xml: attributes with content
|
||||
Fields with the matching xml-content-name is assumed to be content.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
cat:
|
||||
+@name: tiger
|
||||
+content: cool
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=xml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```xml
|
||||
<cat name="tiger">cool</cat>
|
||||
```
|
||||
|
||||
## Encode xml: comments
|
||||
A best attempt is made to copy comments to xml.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
# header comment
|
||||
# above_cat
|
||||
cat: # inline_cat
|
||||
# above_array
|
||||
array: # inline_array
|
||||
- val1 # inline_val1
|
||||
# above_val2
|
||||
- val2 # inline_val2
|
||||
# below_cat
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=xml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```xml
|
||||
<!--
|
||||
header comment
|
||||
above_cat
|
||||
--><!-- inline_cat --><cat><!-- above_array inline_array -->
|
||||
<array>val1<!-- inline_val1 --></array>
|
||||
<array><!-- above_val2 -->val2<!-- inline_val2 --></array>
|
||||
</cat><!-- below_cat -->
|
||||
```
|
||||
|
||||
## Encode: doctype and xml declaration
|
||||
Use the special xml names to add/modify proc instructions and directives.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
+p_xml: version="1.0"
|
||||
+directive: 'DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" '
|
||||
apple:
|
||||
+p_coolioo: version="1.0"
|
||||
+directive: 'CATYPE meow purr puss '
|
||||
b: things
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=xml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```xml
|
||||
<?xml version="1.0"?>
|
||||
<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
|
||||
<apple><?coolioo version="1.0"?><!CATYPE meow purr puss >
|
||||
<b>things</b>
|
||||
</apple>
|
||||
```
|
||||
|
||||
## Round trip: with comments
|
||||
A best effort is made, but comment positions and white space are not preserved perfectly.
|
||||
|
||||
Given a sample.xml file of:
|
||||
```xml
|
||||
|
||||
<!-- before cat -->
|
||||
<cat>
|
||||
<!-- in cat before -->
|
||||
<x>3<!-- multi
|
||||
line comment
|
||||
for x --></x>
|
||||
<!-- before y -->
|
||||
<y>
|
||||
<!-- in y before -->
|
||||
<d><!-- in d before -->z<!-- in d after --></d>
|
||||
|
||||
<!-- in y after -->
|
||||
</y>
|
||||
<!-- in_cat_after -->
|
||||
</cat>
|
||||
<!-- after cat -->
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=xml -o=xml '.' sample.xml
|
||||
```
|
||||
will output
|
||||
```xml
|
||||
<!-- before cat --><cat><!-- in cat before -->
|
||||
<x>3<!-- multi
|
||||
line comment
|
||||
for x --></x><!-- before y -->
|
||||
<y><!-- in y before
|
||||
in d before -->
|
||||
<d>z<!-- in d after --></d><!-- in y after -->
|
||||
</y><!-- in_cat_after -->
|
||||
</cat><!-- after cat -->
|
||||
```
|
||||
|
||||
## Roundtrip: with doctype and declaration
|
||||
yq parses XML proc instructions and directives into nodes.
|
||||
Unfortunately the underlying XML parser loses whitespace information.
|
||||
|
||||
Given a sample.xml file of:
|
||||
```xml
|
||||
<?xml version="1.0"?>
|
||||
<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
|
||||
<apple>
|
||||
<?coolioo version="1.0"?>
|
||||
<!CATYPE meow purr puss >
|
||||
<b>things</b>
|
||||
</apple>
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=xml -o=xml '.' sample.xml
|
||||
```
|
||||
will output
|
||||
```xml
|
||||
<?xml version="1.0"?>
|
||||
<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
|
||||
<apple><?coolioo version="1.0"?><!CATYPE meow purr puss >
|
||||
<b>things</b>
|
||||
</apple>
|
||||
```
|
||||
|
Reference in New Issue
Block a user