一、部署ES6分布式集群¶

1、

2、结果验证

二、基于_reindex的API迁移¶

2.1 环境准备¶

1、搭建好两套多实例集群

[root@elk121 ~]# curl 192.168.1.121:9200/_cat/nodes
192.168.1.121 58 25 0 0.00 0.01 0.05 cdfhilmrstw - elk121
192.168.1.122 68 26 0 0.00 0.01 0.05 cdfhilmrstw * elk122
192.168.1.123 56 34 0 0.05 0.03 0.05 cdfhilmrstw - elk123
[root@elk121 ~]# curl 192.168.1.121:19200/_cat/nodes
192.168.1.121 55 25 0 0.00 0.01 0.05 mdi - elk121
192.168.1.122 55 26 0 0.00 0.01 0.05 mdi - elk122
192.168.1.123 66 34 0 0.04 0.03 0.05 mdi * elk123

2、在ES6环境创建文档

{
"name": "痴汉",
"age": 35,
"hobby": ["吃鸡","丝袜","rap"]
}
{
"name": "李四",
"age": 30,
"hobby": ["吃鸡","丝袜","rap"]
}
{
"name": "张三",
"age": 26,
"hobby": ["吃鸡","丝袜","rap"]
}
{
"name": "李文轩",
"age": 20,
"hobby": ["吃鸡","丝袜","rap"]
}

注意：在Postman执行时需要单个单个执行，上面只是为了方便，放到一起而已

完成后，打开ES Head扩展程序查看结果，观察到成功创建文档到ES6

2.2 同集群迁移实战¶

2.2.1 ES6集群索引拷贝¶

1、填写POST请求http://192.168.1.121:19200/_reindex,在ES6集群进行索引拷贝

{
    "source": {
        "index": "es111"
    },
    "dest": {
        "index": "es111-new"
    }
}

2、打开ES Head扩展程序输入http://192.168.1.121:19200/查看结果，观察到ES6数据已经够成功拷贝

2.2.2 ES7集群索引拷贝¶

1、填写POST请求http://192.168.1.121:9200/_reindex,在ES7集群进行索引拷贝

{
    "source": {
        "index": "es-shopping"
    },
    "dest": {
        "index": "es-shopping-new"
    }
}

2、打开ES Head扩展程序输入http://192.168.1.121:9200/查看结果，观察到ES7数据已经够成功拷贝

2.3 不同集群迁移实战(ES6到ES7)¶

1、在elk121节点上修改配置文件

[root@elk121 ~]# vim /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml
…
…
reindex.remote.whitelist: "192.168.1.*:19200"

说明：192.168.1.*是集群主机的网段

配置完成后，如下：

[root@elk121 ~]# egrep -v '^$|^#' /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml
cluster.name: es
path.data: /es/data/es7
path.logs: /es/logs/es7
network.host: 192.168.1.121
discovery.seed_hosts: ["elk121","elk122","elk123"]
cluster.initial_master_nodes: ["elk121","elk122","elk123"]
reindex.remote.whitelist: "192.168.1.*:19200"

2、在elk121节点上同步配置文件

[root@elk121 ~]# data_rsync.sh /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml

3、在elk122节点和elk123节点上修改network.host为各自IP，修改完成后，配置内容如下

[root@elk122 ~]# egrep -v '^$|^#' /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml
cluster.name: es
path.data: /es/data/es7
path.logs: /es/logs/es7
network.host: 192.168.1.122
discovery.seed_hosts: ["elk121","elk122","elk123"]
cluster.initial_master_nodes: ["elk121","elk122","elk123"]
reindex.remote.whitelist: "192.168.1.*:19200"
[root@elk123 ~]# egrep -v '^$|^#' /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml
cluster.name: es
path.data: /es/data/es7
path.logs: /es/logs/es7
network.host: 192.168.1.123
discovery.seed_hosts: ["elk121","elk122","elk123"]
cluster.initial_master_nodes: ["elk121","elk122","elk123"]
reindex.remote.whitelist: "192.168.1.*:19200"

4、在elk121节点、lk122节点和elk123节点上重启ES7服务

[root@elk121 ~]# systemctl restart es7
[root@elk122~]# systemctl restart es7
[root@elk123~]# systemctl restart es7

重启后进行集群状态检测

[root@elk121 ~]# curl 192.168.1.121:19200/_cat/nodes
192.168.1.121 44 25 0 0.13 0.08 0.06 mdi - elk121
192.168.1.122 59 26 0 0.19 0.10 0.08 mdi - elk122
192.168.1.123 58 32 0 0.16 0.08 0.07 mdi * elk123
[root@elk121 ~]# curl 192.168.1.121:9200/_cat/nodes
192.168.1.121 65 25 2 0.12 0.07 0.06 cdfhilmrstw - elk121
192.168.1.122 56 26 3 0.19 0.10 0.08 cdfhilmrstw * elk122
192.168.1.123 40 32 3 0.16 0.08 0.07 cdfhilmrstw - elk123

5、填写POST请求http://192.168.1.121:9200/_reindex,在ES6集群索引es111中age大于25的数据导入到ES7中，并重新定义名为es7-111的索引

{
    "source": {
        "index": "es111",
        "remote": {
            "host": "http://192.168.1.121:19200"
        },
        "query": {
            "bool": {
                "filter": {
                    "range": {
                        "age": {
                            "gt": 25
                        }
                    }
                }
            }
        }
    },
    "dest": {
        "index": "es7-111"
    }
}

6、打开ES Head扩展程序输入http://192.168.1.121:9200/查看结果，观察到ES6数据已经够成功迁移到ES7

三、基于1ogstash实现索引跨集群迁移¶

温馨提示:

对于低版本的数据迁移到高版本时,比如从Es5迁移到ES7,应该注意不同点:

(1)默认的分片数量和副本数量;

(2)默认的文档类型是否相同,尤其是在Es7版本中移除了type类型,仅保留了"_doc"这一种内直类型;

Elasticsearch集群迁移实战：_reindex与Logstash跨集群同步