v0.6.4

Fixed

• Fix internal client pools (metastore, searcher, etc.) when nodes restart

v0.6.3

Added

• Multilang tokenizer
• Source code tokenizer
• Sort by two fields
• Parse Unix timestamps passed as a string.

Fixed

• Fix disabling of ANSI characters in logs
• Fix OTEL logs default tokenizer of body.message field

Changed

• Default fast field tokenizer for fast fields is the raw tokenizer with remove filter above 256 bytes

v0.6.2

Added

• Add cluster info endpoint at _elastic
• Add support for custom tokenizers ngram and regex
• Add storage flavors
• Allow dots in identifier

Fixed

• Fix mrecordlog corruption bug

• Fix merge planner issue #3576 by fetching only immature splits.
  A short note on this fix. The upgrade will trigger a Postgresql migration, adding the column maturity_timestamp to the splits table.
  By default, the maturity_timestamp is set to 1970-01-01 00:00:00. All your splits will be considered mature and won't
  undergo merge operations. This is primarily true for your splits, but... it may not be for splits created recently. You can do an additional
  operation, like set the maturity_timestamp of recent splits with a low number of documents to a timestamp in the future, like now + 48h.
  For the file-backed metastore, there is no migration process, the upgrade will add to a new field maturity for splits in the metastore.json file. By default it will be set to {"type": "mature"} but, as for Postgresql, you may want to set a different value for the split maturity like {"type": "immature", "maturation_period_millis":172800000}.

• Fix storage and metastore configuration

v0.6.1

Added

• Support phrase prefix queries in the query language.

Fixed

• Fix timestamp field which was not allowed when defined in an object mapping.
• Fix querying of integer on a JSON field (no documents were returned).

nightly

Bump tempfile from 3.5.0 to 3.6.0 in /quickwit (#3514)

Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.5.0 to 3.6.0.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.5.0...v3.6.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

v0.6.0

Added

• Elasticsearch/Opensearch compatible API.
• New columnar format:
  • Fast fields can now have any cardinality (Optional, Multivalued, restricted). In fact cardinality is now only used to format the output.
  • Dynamic Fields are now fast fields.
• String fast fields now can be normalized.
• Various parameters of object storages can now be configured.
• The ingest API makes it possible to force a commit, or wait for a scheduled commit to occur.
• Ability to parse non-JSON data using VRL to extract some structure from documents.
• Object storage can now use the virtual-hosted–style.
• date_histogram aggregation.
• percentiles aggregation.
• Added support for Prefix Phrase query.
• Added support for range queries.
• The query language now supports different date formats.
• Added support for base16 input/output configuration for bytes field. You can search for bytes fields using base16 encoded values.
• Autotagging: fields used in the partition key are automatically added to tags.
• Added arm64 docker image.
• Added CORS configuration for the REST API.

Fixed

• Major bug fix that required to restart quickwit when deleting and recreating an index with the same name.
• The number of concurrent GET requests to object stores is now limited. This fixes a bug observed with when requested a lot of documents from MinIO.
• Quickwit now searches into resource attributes when receiving a Jaeger request carrying tags
• Object storage can be figured to:
  • avoid Bulk delete API (workaround for Google Cloud Storage).
  • Use virtual-host style addresses (workaround for Alibaba Object Storage Service).
• Fix aggregation min doc_count empty merge bug.
• Fix: Sort order for term aggregations.
• Switch to ms in histogram for date type (aligning with ES).

Improvements

• Search performance improvement.
• Aggregation performance improvement.
• Aggregation memory improvement.

More details in tantivy's changelog.

Changed

• Datetime now have up to a nanosecond precision.
• By default, quickwit now uses the node's hostname as the default node ID.
• By default, Quickwit is in dynamic mode and all dynamic fields are marked as fast fields.
• JSON field uses by default the raw tokanizer and is set to fast field.
• Various performance/compression improvements.
• OTEL indexes Trace ID and Span ID are now bytes fields.
• OTEL indexes stores timestamps with nanosecond precision.
• pan status is now indexed in the OTEL trace index.
• Default and raw tokenizers filter tokesn longer than 255 bytes instead of 40 bytes.

v0.5.2

Fixed

• Fix conversion from OTLP attributes to Jaeger tags for arrays

v0.5.1

Added

• Add arm64 docker image
• Extract OTLP arrays of primitive values

Fixed

• Fix OTEL index pipelines crashes (source actor timeout)

Changed

• Update mrecordlog to 0.3

v0.5.0

Added

• gRPC OpenTelemetry Protocol support for traces
• gRPC OpenTelemetry Protocol support for logs
• Control plane (indexing tasks scheduling)
• Ingest API rate limiter
• Pulsar source
• VRL transform for data sources
• REST API enhanced to fully manage indexes, sources, and splits
• OpenAPI specification and swagger UI for all REST available endpoints
• Large responses from REST API can be compressed
• Add bulk stage splits method to metastore
• MacOS M1 binary
• Doc mapping field names starting with _ are now valid

Fixed

• Fix UI index completion on search page
• Fix CLI index describe command to show stats on published splits
• Fix REST API to always return on error a body formatted as {"message": "error message"}
• Fixed REST status code when deleting unexisting index, source and when fetching splits on unexisting index

Changed

• Source config schema (breaking or not? use serde rename to be not breaking?)
• RocksDB replaced by mrecordlog to store ingest API queues records
• (Breaking) Indexing partition key new DSL
• (Breaking) Helm chart updated with the new CLI
• (Breaking) CLI indexes, sources, and splits commands use the REST API
• (Breaking) Index new format: you need to reindex all your data

v0.4.0

Update changelog