TIGER QA CheckList for DataStage Programs

本文提供了一系列数据处理的最佳实践,包括文件命名、版权说明、代码注释等通用规则;并针对并行作业和服务器作业提出了具体的技术建议,如避免使用过多的Transformer阶段、合理使用链接排序等,以提高数据处理效率。

 General Rules
1 File and identifiers named according to standards
2 Header documented with copyright, overview and amendment history
3 Comments added on amendment history and around code
4 Use of Transformer Stages avoided if there are more than 500,000 records
5 Use of multiple Transformer Stages where the logic can be combined into a single stage avoided
6 Use $PROJDEF as the value of job parameter that derived from environment variable, if possible
7 Define password variable as encrypted type
8 Keep parallel/server job containing no more than 50 stages, including those in containers
9 Disable job monitor by setting $APT_NO_JOBMON=TRUE, if possible
10 Use Sequential File Stage’s filter option to read part of the file content
11 Perform sort before Join, Merge and Aggregator Stages
12 Unneeded columns removed as early as possible in the job flow
13 Combine operations around the same sort keys if possible
14 The null field value and length for nullable column in Sequential File Stage is defined
15 Cross link avoided
16 Use of server jobs avoided, if possible
 Parallel Jobs
17 Run time column propagation avoided
18 Repartition avoided
19 Use link sort with repartition, if possible
20 Use link sort instead of sort stage, if possible
21 Define links with less data as reference link for Lookup Stage
22 Use one node configuration for jobs that deal with small amount of data
23 Only use stage-wise config (.apt) for reading files in a specific machine
24 Use of Basic Transformer Stage avoided
25 Use Change Capture Stage instead of Compare Stage, if possible
26 Use persistent data set for sharing between parallel jobs and large volumes of data processed
27 Use of data set for long-term backup and recovery of source data avoided
28 Reading from sequential files using the Same partition method avoided
29 Use Join Stage instead of Lookup Stage if datasets are larger than available memory resources
30 Use Copy Stage for simple operations such as renaming columns, dropping columns and implicit type conversions
31 Use Modify Stage for single column operations such as keep/drop column, type conversion and null handling
32 Use Modify Stage for explicit type conversion and null handling
33 Every parallel job should have $APT_CONFIG_FILE as project parameter and with the value $PROJDEF
34 Write to a data set or file set instead of a sequential file if the data is going to be read back in parallel
35 Modify of buffering mode avoided
 Server Jobs
36 Use an Inter Process Stage instead of a Sequential File Stage if the data is going to be read back
 Job Senquence
37 Sequencer wrapped by StartLoop/EndLoop Activities with large loop number avoided
 Container
38 Use of Server Shared Container in parallel job avoided, if possible
 Routine
39 Attach the job to get the new jobhandler if it is reset in the routine
 Multiple Instance
40 Use of multiple instance avoided, if possible
41 Table/file lock avoided
42 Invocation id is given in each calling program
43 Duplicate invocation id avoided
44 Misarranged log entry avoided
 Meta Data
45 Date and timestamp type defined correctly (timestamp's length is 19 for Oracle)

 

项目概述 这是一个基于 Spring Boot 2.2.2 构建的网上书城系统,采用经典的 SSM(Spring + SpringMVC + MyBatis Plus) 架构模式,实现了完整的图书电商业务流程。项目包含用户管理、图书管理、购物车、订单处理、收藏评论等核心功能模块。 核心功能模块 1. 用户管理模块(Yonghu) 功能特性: 用户注册、登录、信息修改 用户余额管理 用户头像上传 2. 图书管理模块(Tushuxinxi) 功能特性: 图书信息的增删改查 图书分类与类型管理 点击量统计 库存管理 3. 购物车模块(Cart) 功能特性: 添加商品到购物车 修改购买数量 删除购物车商品 批量结算 4. 订单模块(Orders) 功能特性: 创建订单 订单状态管理(未支付/已支付/已发货/已完成/已取消/已退款) 订单支付(支持支付宝) 订单查询 5. 收藏与评论模块 收藏功能(Storeup): 用户收藏图书 收藏列表管理 评论功能(Discusstushuxinxi): 图书评论 评论回复 6. 公告资讯模块(News) 系统公告发布 资讯列表展示 项目亮点 完整的业务流程:从用户注册、浏览图书、加入购物车、提交订单到支付完成,实现完整闭环。 安全机制: Token认证防止未授权访问 密码加密存储 SQL注入防护 推荐算法:实现基于用户购买记录的协同过滤推荐(autoSort2接口)。 扩展性设计: 支持多数据库切换(MySQL/SQL Server) 模块化架构便于功能扩展 支付集成:预留支付宝支付接口,便于后续接入。 总结 该网上书城项目是一个功能完整、架构清晰的Spring Boot实战项目,适合作为学习Spring Boot开发的参考案例,也可作为毕业设计或课程设计的基础框架。项目代码结构规范,注释完善,具备良好的可维护性和扩展性。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值