1.2.20 • Published 22 days ago
@qinrun/datafactory v1.2.20
datafactory
当前监听mongodb的change stream https://www.mongodb.com/docs/manual/changeStreams/ 来实时处理数据
主要作用为将 Staging 层的数据按照一定的格式进行建模,存储到Data Vault层
本地开发
- 修改本地 hosts, mongo1, mongo2, mongo3 都指向 127.0.0.1
- 使用
start-db.sh
在本地启动mongodb副本集和mysql数据库 - run main.go
从mongodb官网安装mongosh
然后 mongosh mongosh mongodb://mongo1:27017,mongo2:27018,mongo3:27019
连接到mongo数据库
use tushare;
db.stock_basic.insert({ timestamp: 1662949539, dataType: 'Stock', dataSource: 'tushare', data: { ts_code: '000001.SZ', symbol: '000001', name: '平安银行', area: '深圳', industry: '银行', fullname: '平安银行股份有限公司', enname: 'Ping An Bank Co., Ltd.', cnspell: 'payh', market: '主板', exchange: 'SZSE', curr_type: 'CNY', list_status: 'L', list_date: 19910403, delist_date: null, is_hs: 'S' } });
test
1.2.20
22 days ago
1.2.19
24 days ago
1.2.18
1 month ago
1.2.17
2 months ago
1.2.16
2 months ago
1.2.14
2 years ago
1.2.15
2 years ago
1.2.13
2 years ago
1.2.12
2 years ago
1.2.11
2 years ago
1.2.10
2 years ago
1.2.9
2 years ago
1.2.8
2 years ago
1.2.7
2 years ago
1.2.6
2 years ago
1.2.5
2 years ago
1.2.4
2 years ago
1.2.3
2 years ago
1.2.2
2 years ago
1.2.1
2 years ago
1.2.0
2 years ago
1.1.0
2 years ago
1.0.8
2 years ago
1.0.7
2 years ago
1.0.6
2 years ago
1.0.5
2 years ago
1.0.4
2 years ago
1.0.3
2 years ago
1.0.2
2 years ago
1.0.1
2 years ago
1.0.0
2 years ago