OpenSearch Platform
The OpenSearch Platform is presented as a set of integrated components that users combine to build search and observability applications. The project describes the approach as indexing once, then searching, retrieving and analysing across everything a user builds and runs. OpenSearch Dashboards provides the visualisation and management interface for the suite, which supports full-text search, application monitoring, log analytics and vector search.
- Apache 2.0-licensed open source suite
- Machine learning and AI capabilities
- Search, including enterprise and document search
- Observability stack and log analysis
- Security analytics with threat intelligence and event correlation
- OpenSearch Dashboards for visualisation and management
OpenSearch Software Foundation
The OpenSearch Software Foundation was formed in September 2024 as the next chapter for the OpenSearch Project. It is a project of the Linux Foundation, and the project describes the move as opening development up further to the community. Linux Foundation Research and the OpenSearch Software Foundation jointly produced the 2026 Open Data Infrastructure Report, which examines whether organisations' data platforms are built for the AI era.
- Formed in September 2024
- A project of the Linux Foundation
- Co-publisher of the 2026 Open Data Infrastructure Report
Vector search and HNSW
Vector search in OpenSearch stores embeddings as dense vector fields and retrieves them using k-nearest neighbour algorithms, with similarity measured by functions such as inner product, cosine similarity or Euclidean distance. Because exact k-NN is computationally expensive at scale, OpenSearch uses approximate nearest neighbour methods, notably Hierarchical Navigable Small Worlds (HNSW). HNSW behaviour is controlled by the hyperparameters M, efSearch and efConstruction, which trade search quality against query throughput and index build time. Vector search also underpins retrieval-augmented generation, where relevant content is retrieved from a large document collection to improve large language model responses.
- M – maximum graph edges per vector
- efSearch – candidate queue size during search
- efConstruction – candidate queue size during index construction
- Recall@k used to measure search quality
- Query throughput used to measure search speed
Quantisation and hardware acceleration
OpenSearch provides quantisation options that reduce the memory footprint of vector indexes. From OpenSearch 2.13, a k-NN index can be configured to use the Faiss scalar quantizer (SQfp16), which converts each vector dimension from 32-bit to 16-bit floating point and saves roughly 50% of memory with minimal reduction in recall. Disk-based vector search adds online binary quantisation together with an on_disk mode that includes a rescoring stage. Intel AVX-512, introduced in OpenSearch 2.18, accelerates vector workloads; benchmarks run with OpenSearch Benchmark showed gains of up to 15% for indexing and up to 13% for search compared with AVX2.
- Faiss SQfp16 scalar quantisation available from OpenSearch 2.13
- Encoder set via method.parameters.encoder.name = sq
- clip parameter controls out-of-range values
- Disk-based vector search with on_disk and in_memory modes
- Intel AVX-512 support introduced in OpenSearch 2.18
Hybrid search and pagination
Hybrid queries, introduced in OpenSearch 2.10, combine full-text lexical search using BM25 with semantic search over dense vector embeddings. Scores from the different subqueries are made comparable through rank normalisation, for which OpenSearch provides L2 and min-max techniques, alongside rank-based combination using reciprocal rank fusion. OpenSearch 2.19 added the pagination_depth parameter, which sets the maximum number of results retrieved from each shard for every subquery and works with the from and size parameters. OpenSearch supports four pagination techniques overall.
- Hybrid queries introduced in OpenSearch 2.10
- BM25 as the default lexical scoring algorithm
- L2 and min-max score normalisation
- Reciprocal rank fusion for rank-based normalisation
- pagination_depth introduced in OpenSearch 2.19
- Pagination via from/size, scroll, search_after, and Point in Time
Neural sparse search
Neural sparse search is a semantic retrieval method introduced in OpenSearch 2.11. Sparse encoding models encode text into (token, weight) entries so that OpenSearch can build indexes and run searches using Lucene's inverted index. The v2 series of neural sparse models is available in both OpenSearch and Hugging Face, with the v2-distill model reducing parameters by 50% and the v2-mini model by 75%. The v2-distill architecture supports both doc-only and bi-encoder modes, while v2-mini supports doc-only mode.
- Introduced in OpenSearch 2.11
- v2-distill: 50% fewer parameters, doc-only and bi-encoder modes
- v2-mini: 75% fewer parameters, doc-only mode
- Models published on OpenSearch and Hugging Face
Anomaly Detection plugin
The OpenSearch Anomaly Detection plugin is powered by the Random Cut Forest (RCF) algorithm. False alarms reduce the algorithm's practical value because users must verify and dismiss them, so in OpenSearch 2.17 the Anomaly Detection team introduced four enhancements to RCF. These include statistical tracking of candidate anomalies, grouped alerting, the use of approximate nearest neighbours, and rescoring using expected value vectors. The project reports a 94.3% reduction in false positives from these changes, evaluated against the Numenta Anomaly Benchmark.
- Built on the Random Cut Forest algorithm
- Four RCF enhancements introduced in OpenSearch 2.17
- Default shingle size of 8
- Evaluated with the Numenta Anomaly Benchmark
Piped Processing Language (PPL)
Piped Processing Language is OpenSearch's query language for interactive data exploration, used particularly in observability and log analytics. OpenSearch 3.0 added the experimental lookup, join and subsearch commands, which enrich, correlate and filter logs. The commands are powered by Apache Calcite, which also improves query planning and execution, and require Calcite to be enabled on the OpenSearch instance.
- lookup – enriches logs with external reference data
- join – correlates logs from distinct indexes
- subsearch – builds dynamic query conditions from results
- Enabled via the plugins.calcite.enabled setting
Sources
-
OpenSearch and OpenSearch Dashboards | OpenSearch Documentation https://docs.opensearch.org/latest/about/ Verified 17 Oct 2026
-
Enhanced log analysis with OpenSearch PPL: Introducing lookup, join, and subsearch - OpenSearch https://opensearch.org/blog/enhanced-log-analysis-with-opensearch-ppl-introducing-lookup-join-and-subsearch/ Verified 01 Nov 2026
-
Navigating pagination in hybrid queries with the pagination_depth parameter - OpenSearch https://opensearch.org/blog/navigating-pagination-in-hybrid-queries-with-the-pagination_depth-parameter/ Verified 01 Nov 2026
-
Building effective hybrid search in OpenSearch: Techniques and best practices - OpenSearch https://opensearch.org/blog/building-effective-hybrid-search-in-opensearch-techniques-and-best-practices/ Verified 01 Nov 2026
-
A practical guide to selecting HNSW hyperparameters - OpenSearch https://opensearch.org/blog/a-practical-guide-to-selecting-hnsw-hyperparameters/ Verified 01 Nov 2026
-
Boost OpenSearch vector search performance with Intel AVX-512 - OpenSearch https://opensearch.org/blog/boost-opensearch-vector-search-performance-with-intel-avx512/ Verified 01 Nov 2026
-
Reduce costs with disk-based vector search - OpenSearch https://opensearch.org/blog/reduce-cost-with-disk-based-vector-search/ Verified 01 Nov 2026
-
Enhancing OpenSearch anomaly detection: Reducing false positives through algorithmic improvements - OpenSearch https://opensearch.org/blog/reducing-false-positives-through-algorithmic-improvements/ Verified 01 Nov 2026
-
An overview of rank normalization in hybrid search - OpenSearch https://opensearch.org/blog/how-does-the-rank-normalization-work-in-hybrid-search/ Verified 01 Nov 2026
-
Improving search efficiency and accuracy with the newest v2 neural sparse models - OpenSearch https://opensearch.org/blog/neural-sparse-v2-models/ Verified 01 Nov 2026
-
Optimizing OpenSearch with Faiss FP16 scalar quantization: Enhancing memory efficiency and cost-effectiveness - OpenSearch https://opensearch.org/blog/optimizing-opensearch-with-fp16-quantization/ Verified 01 Nov 2026
Last verified 01 Nov 2026. This entry is compiled from the public web pages listed above. Nothing here is stated that those pages do not, and each of them was read on the date shown.