Spring Boot与Spark、Cassandra系统集成开发示例

本文演示以Spark作为分析引擎,Cassandra作为数据存储,而使用Spring Boot来开发驱动程序的示例。

1.前置条件

创建keyspace

1	`CREATE KEYSPACE hfcb WITH REPLICATION = {` `'class'` `:` `'SimpleStrategy',` `'replication_factor'` `:` `3` `};`

创建table

CREATE TABLE person (

id text PRIMARY KEY,

first_name text,

last_name text

);

插入测试数据

insert into person (id,first_name,last_name) values('1','wang','yunfei');

insert into person (id,first_name,last_name) values('2','peng','chao');

insert into person (id,first_name,last_name) values('3','li','jian');

insert into person (id,first_name,last_name) values('4','zhang','jie');

insert into person (id,first_name,last_name) values('5','liang','wei');

2.spark-cassandra-connector安装

让Spark-1.5.1能够使用Cassandra作为数据存储，需要加上下面jar包的依赖(示例将包放置于 /opt/spark/managed-lib/ 目录，可任意):