ElasticSearch教程FG010-ElasticSearch DSL查询语法实战

Part01-基础概念与理论知识

1.1 DSL查询概念

DSL（Domain Specific Language）是ElasticSearch提供的一种基于JSON的查询语言，用于构建复杂的查询。DSL查询是ElasticSearch的核心功能之一，支持各种复杂的查询场景。更多视频教程www.fgedu.net.cn

DSL查询的特点：

基于JSON格式
支持多种查询类型
可以组合多个查询
支持复杂的过滤条件

1.2 查询类型

ElasticSearch支持多种查询类型：

全文查询：match, match_phrase, multi_match等
术语查询：term, terms, range等
复合查询：bool, must, should, must_not等
聚合查询：sum, avg, max, min, terms等
特殊查询：geo, script等

1.3 查询结构

DSL查询的基本结构：

查询部分（query）：用于评分和相关性排序
过滤部分（filter）：用于过滤文档，不影响评分
聚合部分（aggs）：用于数据统计和分析
排序部分（sort）：用于结果排序
分页部分（from, size）：用于结果分页

学习交流加群风哥微信: itpux-com

Part02-生产环境规划与建议

2.1 查询性能优化

查询性能优化的方法：

使用过滤查询（filter）减少评分计算
合理使用缓存
避免深度分页
使用合适的查询类型
优化索引结构

2.2 查询缓存策略

查询缓存策略：

使用filter查询自动缓存
合理设置缓存大小
监控缓存命中率
避免缓存膨胀

2.3 生产环境查询规范

生产环境查询规范：

限制查询结果大小
避免使用通配符查询
合理使用聚合
监控查询性能
设置查询超时

学习交流加群风哥QQ113257174

Part03-生产环境项目实施方案

3.1 基础查询实战

基础查询操作：

# 全文查询
curl -X GET “http://192.168.1.10:9200/fgedu-products/_search” -H “Content-Type: application/json” -d ‘{
“query”: {
“match”: {
“product_name”: “ElasticSearch”
}
}
}’

# 执行
# 输出日志
{
“took”: 12,
“timed_out”: false,
“_shards”: {
“total”: 5,
“successful”: 5,
“skipped”: 0,
“failed”: 0
},
“hits”: {
“total”: {
“value”: 2,
“relation”: “eq”
},
“max_score”: 0.6931472,
“hits”: [
{
“_index”: “fgedu-products”,
“_id”: “1”,
“_score”: 0.6931472,
“_source”: {
“product_id”: “P001”,
“product_name”: “ElasticSearch实战指南”,
“price”: 99.9,
“category”: “技术书籍”,
“stock”: 85,
“created_at”: “2024-01-01 10:00:00”
}
},
{
“_index”: “fgedu-products”,
“_id”: “4”,
“_score”: 0.5753642,
“_source”: {
“product_id”: “P004”,
“product_name”: “ElasticStack实战”,
“price”: 119.9,
“category”: “技术书籍”,
“stock”: 50,
“created_at”: “2024-01-01 12:00:00”
}
}
]
}
}

# 术语查询
curl -X GET “http://192.168.1.10:9200/fgedu-products/_search” -H “Content-Type: application/json” -d ‘{
“query”: {
“term”: {
“category”: “技术书籍”
}
}
}’

# 执行
# 输出日志
{
“took”: 5,
“timed_out”: false,
“_shards”: {
“total”: 5,
“successful”: 5,
“skipped”: 0,
“failed”: 0
},
“hits”: {
“total”: {
“value”: 3,
“relation”: “eq”
},
“max_score”: 0.2876821,
“hits”: [
{
“_index”: “fgedu-products”,
“_id”: “1”,
“_score”: 0.2876821,
“_source”: {
“product_id”: “P001”,
“product_name”: “ElasticSearch实战指南”,
“price”: 99.9,
“category”: “技术书籍”,
“stock”: 85,
“created_at”: “2024-01-01 10:00:00”
}
},
{
“_index”: “fgedu-products”,
“_id”: “2”,
“_score”: 0.2876821,
“_source”: {
“product_id”: “P002”,
“product_name”: “Kibana权威指南”,
“price”: 79.9,
“category”: “技术书籍”,
“stock”: 80,
“created_at”: “2024-01-01 10:30:00”
}
},
{
“_index”: “fgedu-products”,
“_id”: “4”,
“_score”: 0.2876821,
“_source”: {
“product_id”: “P004”,
“product_name”: “ElasticStack实战”,
“price”: 119.9,
“category”: “技术书籍”,
“stock”: 50,
“created_at”: “2024-01-01 12:00:00”
}
}
]
}
}

3.2 复合查询实战

复合查询操作：

# Bool查询
curl -X GET “http://192.168.1.10:9200/fgedu-products/_search” -H “Content-Type: application/json” -d ‘{
“query”: {
“bool”: {
“must”: [
{
“match”: {
“product_name”: “Elastic”
}
}
],
“filter”: [
{
“range”: {
“price”: {
“lte”: 100
}
}
}
]
}
}
}’

# 执行
# 输出日志
{
“took”: 8,
“timed_out”: false,
“_shards”: {
“total”: 5,
“successful”: 5,
“skipped”: 0,
“failed”: 0
},
“hits”: {
“total”: {
“value”: 1,
“relation”: “eq”
},
“max_score”: 0.6931472,
“hits”: [
{
“_index”: “fgedu-products”,
“_id”: “1”,
“_score”: 0.6931472,
“_source”: {
“product_id”: “P001”,
“product_name”: “ElasticSearch实战指南”,
“price”: 99.9,
“category”: “技术书籍”,
“stock”: 85,
“created_at”: “2024-01-01 10:00:00”
}
}
]
}
}

# 范围查询
curl -X GET “http://192.168.1.10:9200/fgedu-products/_search” -H “Content-Type: application/json” -d ‘{
“query”: {
“range”: {
“price”: {
“gte”: 80,
“lte”: 120
}
}
}
}’

# 执行
# 输出日志
{
“took”: 6,
“timed_out”: false,
“_shards”: {
“total”: 5,
“successful”: 5,
“skipped”: 0,
“failed”: 0
},
“hits”: {
“total”: {
“value”: 2,
“relation”: “eq”
},
“max_score”: 1.0,
“hits”: [
{
“_index”: “fgedu-products”,
“_id”: “1”,
“_score”: 1.0,
“_source”: {
“product_id”: “P001”,
“product_name”: “ElasticSearch实战指南”,
“price”: 99.9,
“category”: “技术书籍”,
“stock”: 85,
“created_at”: “2024-01-01 10:00:00”
}
},
{
“_index”: “fgedu-products”,
“_id”: “4”,
“_score”: 1.0,
“_source”: {
“product_id”: “P004”,
“product_name”: “ElasticStack实战”,
“price”: 119.9,
“category”: “技术书籍”,
“stock”: 50,
“created_at”: “2024-01-01 12:00:00”
}
}
]
}
}

3.3 聚合查询实战

聚合查询操作：

# 术语聚合
curl -X GET “http://192.168.1.10:9200/fgedu-products/_search” -H “Content-Type: application/json” -d ‘{
“size”: 0,
“aggs”: {
“category_stats”: {
“terms”: {
“field”: “category”
},
“aggs”: {
“avg_price”: {
“avg”: {
“field”: “price”
}
}
}
}
}
}’

# 执行
# 输出日志
{
“took”: 15,
“timed_out”: false,
“_shards”: {
“total”: 5,
“successful”: 5,
“skipped”: 0,
“failed”: 0
},
“hits”: {
“total”: {
“value”: 3,
“relation”: “eq”
},
“max_score”: null,
“hits”: []
},
“aggregations”: {
“category_stats”: {
“doc_count_error_upper_bound”: 0,
“sum_other_doc_count”: 0,
“buckets”: [
{
“key”: “技术书籍”,
“doc_count”: 3,
“avg_price”: {
“value”: 99.9
}
}
]
}
}
}

# 范围聚合
curl -X GET “http://192.168.1.10:9200/fgedu-products/_search” -H “Content-Type: application/json” -d ‘{
“size”: 0,
“aggs”: {
“price_ranges”: {
“range”: {
“field”: “price”,
“ranges”: [
{
“to”: 80
},
{
“from”: 80,
“to”: 100
},
{
“from”: 100
}
]
}
}
}
}’

# 执行
# 输出日志
{
“took”: 10,
“timed_out”: false,
“_shards”: {
“total”: 5,
“successful”: 5,
“skipped”: 0,
“failed”: 0
},
“hits”: {
“total”: {
“value”: 3,
“relation”: “eq”
},
“max_score”: null,
“hits”: []
},
“aggregations”: {
“price_ranges”: {
“buckets”: [
{
“key”: “*-80.0”,
“to”: 80.0,
“doc_count”: 1
},
{
“key”: “80.0-100.0”,
“from”: 80.0,
“to”: 100.0,
“doc_count”: 1
},
{
“key”: “100.0-*”,
“from”: 100.0,
“doc_count”: 1
}
]
}
}
}

更多学习教程公众号风哥教程itpux_com

Part04-生产案例与实战讲解

4.1 电商搜索查询

电商搜索场景的查询：

# 电商搜索查询
curl -X GET “http://192.168.1.10:9200/fgedu-products/_search” -H “Content-Type: application/json” -d ‘{
“query”: {
“bool”: {
“must”: [
{
“multi_match”: {
“query”: “ElasticSearch实战”,
“fields”: [“product_name^3”, “description”]
}
}
],
“filter”: [
{
“term”: {
“category”: “技术书籍”
}
},
{
“range”: {
“price”: {
“lte”: 150
}
}
},
{
“range”: {
“stock”: {
“gt”: 0
}
}
}
]
}
},
“sort”: [
{
“price”: {
“order”: “asc”
}
}
],
“from”: 0,
“size”: 10
}’

# 执行
# 输出日志
{
“took”: 12,
“timed_out”: false,
“_shards”: {
“total”: 5,
“successful”: 5,
“skipped”: 0,
“failed”: 0
},
“hits”: {
“total”: {
“value”: 2,
“relation”: “eq”
},
“max_score”: null,
“hits”: [
{
“_index”: “fgedu-products”,
“_id”: “1”,
“_score”: null,
“sort”: [
99.9
],
“_source”: {
“product_id”: “P001”,
“product_name”: “ElasticSearch实战指南”,
“price”: 99.9,
“category”: “技术书籍”,
“stock”: 85,
“created_at”: “2024-01-01 10:00:00”
}
},
{
“_index”: “fgedu-products”,
“_id”: “4”,
“_score”: null,
“sort”: [
119.9
],
“_source”: {
“product_id”: “P004”,
“product_name”: “ElasticStack实战”,
“price”: 119.9,
“category”: “技术书籍”,
“stock”: 50,
“created_at”: “2024-01-01 12:00:00”
}
}
]
}
}

4.2 日志分析查询

日志分析场景的查询：

# 日志分析查询
curl -X GET “http://192.168.1.10:9200/fgedu-logs-2024.01/_search” -H “Content-Type: application/json” -d ‘{
“query”: {
“bool”: {
“must”: [
{
“term”: {
“service”: “fgedu-api”
}
},
{
“range”: {
“timestamp”: {
“gte”: “2024-01-01 00:00:00”,
“lte”: “2024-01-01 23:59:59”
}
}
}
],
“should”: [
{
“term”: {
“level”: “ERROR”
}
},
{
“term”: {
“level”: “WARN”
}
}
]
}
},
“sort”: [
{
“timestamp”: {
“order”: “desc”
}
}
],
“aggs”: {
“level_stats”: {
“terms”: {
“field”: “level”
}
}
}
}’

# 执行
# 输出日志
{
“took”: 8,
“timed_out”: false,
“_shards”: {
“total”: 3,
“successful”: 3,
“skipped”: 0,
“failed”: 0
},
“hits”: {
“total”: {
“value”: 1,
“relation”: “eq”
},
“max_score”: 1.0,
“hits”: [
{
“_index”: “fgedu-logs-2024.01”,
“_id”: “_cQH044BMQz8e7k9eJ6-“,
“_score”: 1.0,
“_source”: {
“timestamp”: “2024-01-01 10:00:00.123”,
“level”: “INFO”,
“message”: “Application started successfully”,
“service”: “fgedu-api”
}
}
]
},
“aggregations”: {
“level_stats”: {
“doc_count_error_upper_bound”: 0,
“sum_other_doc_count”: 0,
“buckets”: [
{
“key”: “INFO”,
“doc_count”: 1
}
]
}
}
}

4.3 性能调优实战

查询性能调优：

# 优化查询 – 使用filter缓存
curl -X GET “http://192.168.1.10:9200/fgedu-products/_search” -H “Content-Type: application/json” -d ‘{
“query”: {
“bool”: {
“filter”: [
{
“term”: {
“category”: “技术书籍”
}
},
{
“range”: {
“price”: {
“lte”: 100
}
}
}
]
}
},
“size”: 10
}’

# 执行
# 输出日志
{
“took”: 3,
“timed_out”: false,
“_shards”: {
“total”: 5,
“successful”: 5,
“skipped”: 0,
“failed”: 0
},
“hits”: {
“total”: {
“value”: 2,
“relation”: “eq”
},
“max_score”: 0.0,
“hits”: [
{
“_index”: “fgedu-products”,
“_id”: “1”,
“_score”: 0.0,
“_source”: {
“product_id”: “P001”,
“product_name”: “ElasticSearch实战指南”,
“price”: 99.9,
“category”: “技术书籍”,
“stock”: 85,
“created_at”: “2024-01-01 10:00:00”
}
},
{
“_index”: “fgedu-products”,
“_id”: “2”,
“_score”: 0.0,
“_source”: {
“product_id”: “P002”,
“product_name”: “Kibana权威指南”,
“price”: 79.9,
“category”: “技术书籍”,
“stock”: 80,
“created_at”: “2024-01-01 10:30:00”
}
}
]
}
}

# 优化查询 – 限制返回字段
curl -X GET “http://192.168.1.10:9200/fgedu-products/_search” -H “Content-Type: application/json” -d ‘{
“query”: {
“match”: {
“product_name”: “Elastic”
}
},
“_source”: [“product_id”, “product_name”, “price”]
}’

# 执行
# 输出日志
{
“took”: 5,
“timed_out”: false,
“_shards”: {
“total”: 5,
“successful”: 5,
“skipped”: 0,
“failed”: 0
},
“hits”: {
“total”: {
“value”: 2,
“relation”: “eq”
},
“max_score”: 0.6931472,
“hits”: [
{
“_index”: “fgedu-products”,
“_id”: “1”,
“_score”: 0.6931472,
“_source”: {
“product_id”: “P001”,
“product_name”: “ElasticSearch实战指南”,
“price”: 99.9
}
},
{
“_index”: “fgedu-products”,
“_id”: “4”,
“_score”: 0.5753642,
“_source”: {
“product_id”: “P004”,
“product_name”: “ElasticStack实战”,
“price”: 119.9
}
}
]
}
}

风哥提示：使用filter查询可以利用缓存提高性能，减少评分计算开销

Part05-风哥经验总结与分享

5.1 DSL查询最佳实践

优先使用filter查询提高性能
合理使用缓存策略
避免深度分页
使用合适的查询类型
限制返回字段减少网络传输

5.2 常见查询问题

查询性能慢：检查查询复杂度和数据量
内存溢出：控制聚合大小和查询结果数
查询结果不准确：检查字段类型和分词器
缓存命中率低：优化查询结构
查询超时：设置合理的超时时间

5.3 生产环境调优建议

监控查询性能指标
使用慢查询日志分析问题
优化索引结构和Mapping
合理配置缓存大小
使用索引别名和路由提高查询效率

from ElasticSearch视频:www.itpux.com

本文由风哥教程整理发布,仅用于学习测试使用,转载注明出处:http://www.fgedu.net.cn/10327.html