diff --git a/.gitignore b/.gitignore index fbc1fdffff..97b1d8a064 100644 --- a/.gitignore +++ b/.gitignore @@ -19,4 +19,10 @@ s3.properties *.iws .*.swp .DS_Store +.springBeans +build +.gradle +pom.xml +out +/.gradletasknamecache diff --git a/.travis.yml b/.travis.yml deleted file mode 100644 index 6bf47b00a8..0000000000 --- a/.travis.yml +++ /dev/null @@ -1,4 +0,0 @@ -language: java - -install: mvn -U install --quiet -DskipTests=true -P bootstrap -script: mvn clean test -P bootstrap diff --git a/CODE_OF_CONDUCT.adoc b/CODE_OF_CONDUCT.adoc new file mode 100644 index 0000000000..d5e7178131 --- /dev/null +++ b/CODE_OF_CONDUCT.adoc @@ -0,0 +1,44 @@ += Contributor Code of Conduct + +As contributors and maintainers of this project, and in the interest of fostering an open +and welcoming community, we pledge to respect all people who contribute through reporting +issues, posting feature requests, updating documentation, submitting pull requests or +patches, and other activities. + +We are committed to making participation in this project a harassment-free experience for +everyone, regardless of level of experience, gender, gender identity and expression, +sexual orientation, disability, personal appearance, body size, race, ethnicity, age, +religion, or nationality. + +Examples of unacceptable behavior by participants include: + +* The use of sexualized language or imagery +* Personal attacks +* Trolling or insulting/derogatory comments +* Public or private harassment +* Publishing other's private information, such as physical or electronic addresses, + without explicit permission +* Other unethical or unprofessional conduct + +Project maintainers have the right and responsibility to remove, edit, or reject comments, +commits, code, wiki edits, issues, and other contributions that are not aligned to this +Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors +that they deem inappropriate, threatening, offensive, or harmful. + +By adopting this Code of Conduct, project maintainers commit themselves to fairly and +consistently applying these principles to every aspect of managing this project. Project +maintainers who do not follow or enforce the Code of Conduct may be permanently removed +from the project team. + +This Code of Conduct applies both within project spaces and in public spaces when an +individual is representing the project or its community. + +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by +contacting a project maintainer at spring-code-of-conduct@pivotal.io . All complaints will +be reviewed and investigated and will result in a response that is deemed necessary and +appropriate to the circumstances. Maintainers are obligated to maintain confidentiality +with regard to the reporter of an incident. + +This Code of Conduct is adapted from the +https://contributor-covenant.org[Contributor Covenant], version 1.3.0, available at +https://contributor-covenant.org/version/1/3/0/[contributor-covenant.org/version/1/3/0/] \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000000..0ab0cb732f --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,193 @@ +Contributor Guidelines +====================== + +Have something you'd like to contribute to **Spring Batch**? We welcome pull requests, but ask that you carefully read this document first to understand how best to submit them; what kind of changes are likely to be accepted; and what to expect from the Spring team when evaluating your submission. + +Please refer back to this document as a checklist before issuing any pull request; this will save time for everyone! + +## Understand the basics + +Not sure what a *pull request* is, or how to submit one? Take a look at GitHub's excellent [help documentation][] first. + +## Search JIRA first; create an issue if necessary + +Is there already an issue that addresses your concern? Do a bit of searching in our [JIRA issue tracker][] to see if you can find something similar. If not, please create a new issue before submitting a pull request unless the change is truly trivial, e.g. typo fixes, removing compiler warnings, etc. + +## Sign the contributor license agreement + +If you have not previously done so, please fill out and +submit the [Contributor License Agreement](https://cla.pivotal.io/sign/spring). + +## Fork the Repository + +1. Go to [https://github.com/spring-projects/spring-batch](https://github.com/spring-projects/spring-batch) +2. Hit the "fork" button and choose your own github account as the target +3. For more details see [https://help.github.com/fork-a-repo/](https://help.github.com/fork-a-repo/) + +## Setup your Local Development Environment + +1. `git clone git@github.com:/spring-batch.git` +2. `cd spring-batch` +3. `git remote show` +_you should see only 'origin' - which is the fork you created for your own github account_ +4. `git remote add upstream git@github.com:spring-projects/spring-batch.git` +5. `git remote show` +_you should now see 'upstream' in addition to 'origin' where 'upstream' is the *spring-projects*repository from which releases are built_ +6. `git fetch --all` +7. `git branch -a` +_you should see branches on origin as well as upstream, including 'master'_ + +## A Day in the Life of a Contributor + +* _Always_ work on topic branches (Typically use the Jira ticket ID as the branch name). + - For example, to create and switch to a new branch for issue BATCH-123: `git checkout -b BATCH-123` +* You might be working on several different topic branches at any given time, but when at a stopping point for one of those branches, commit (a local operation). +* Please follow the "Commit Guidelines" described in this chapter of Pro Git: [https://git-scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project](https://git-scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project#_commit_guidelines) +* Then to begin working on another issue (say BATCH-101): `git checkout BATCH-101`. The _-b_ flag is not needed if that branch already exists in your local repository. +* When ready to resolve an issue or to collaborate with others, you can push your branch to origin (your fork), e.g.: `git push origin BATCH-123` +* If you want to collaborate with another contributor, have them fork your repository (add it as a remote) and `git fetch ` to grab your branch. Alternatively, they can use `git fetch --all` to sync their local state with all of their remotes. +* If you grant that collaborator push access to your repository, they can even apply their changes to your branch. +* When ready for your contribution to be reviewed for potential inclusion in the master branch of the canonical *spring-batch* repository (what you know as 'upstream'), issue a pull request to the *spring-projects* repository (for more detail, see [https://help.github.com/send-pull-requests/](https://help.github.com/send-pull-requests/)). +* The project lead may merge your changes into the upstream master branch as-is, he may keep the pull request open yet add a comment about something that should be modified, or he might reject the pull request by closing it. +* A prerequisite for any pull request is that it will be cleanly merge-able with the upstream master's current state. **This is the responsibility of any contributor.** If your pull request cannot be applied cleanly, the project lead will most likely add a comment requesting that you make it merge-able. For a full explanation, see the Pro Git section on rebasing: [https://git-scm.com/book/en/v2/Git-Branching-Rebasing](https://git-scm.com/book/en/v2/Git-Branching-Rebasing). As stated there: "> Often, you’ll do this to make sure your commits apply cleanly on a remote branch — perhaps in a project to which you’re trying to contribute but that you don’t maintain." + +## Keeping your Local Code in Sync +* As mentioned above, you should always work on topic branches (since 'master' is a moving target). However, you do want to always keep your own 'origin' master branch in synch with the 'upstream' master. +* Within your local working directory, you can sync up all remotes' branches with: `git fetch --all` +* While on your own local master branch: `git pull upstream master` (which is the equivalent of fetching upstream/master and merging that into the branch you are in currently) +* Now that you're in synch, switch to the topic branch where you plan to work, e.g.: `git checkout -b BATCH-123` +* When you get to a stopping point: `git commit` +* If changes have occurred on the upstream/master while you were working you can synch again: + - Switch back to master: `git checkout master` + - Then: `git pull upstream master` + - Switch back to the topic branch: `git checkout BATCH-123` (no -b needed since the branch already exists) + - Rebase the topic branch to minimize the distance between it and your recently synched master branch: `git rebase master` +(Again, for more detail see the Pro Git section on rebasing: [https://git-scm.com/book/en/v2/Git-Branching-Rebasing](https://git-scm.com/book/en/v2/Git-Branching-Rebasing)) +* **Note** You cannot rebase if you have already pushed your branch to your remote because you'd be rewriting history (see **'The Perils of Rebasing'** in the article). If you rebase by mistake, you can undo it as discussed [in this stackoverflow discussion](https://stackoverflow.com/questions/134882/undoing-a-git-rebase). Once you have published your branch, you need to merge in the master rather than rebasing. +* Now, if you issue a pull request, it is much more likely to be merged without conflicts. Most likely, any pull request that would produce conflicts will be deferred until the issuer of that pull request makes these adjustments. +* Assuming your pull request is merged into the 'upstream' master, you will actually end up pulling that change into your own master eventually, and at that time, you may decide to delete the topic branch from your local repository and your fork (origin) if you pushed it there. + - to delete the local branch: `git branch -d BATCH-123` + - to delete the branch from your origin: `git push origin :BATCH-123` + +## Maintain a linear commit history + +When issuing pull requests, please ensure that your commit history is linear. From the command line you can check this using: + +```` +log --graph --pretty=oneline +```` + +As this may cause lots of typing, we recommend creating a global alias, e.g. `git logg` for this: + +```` +git config --global alias.logg 'log --graph --pretty=oneline' +```` + +This command, will provide the following output, which in this case shows a nice linear history: + +```` +* e1f6de38e04a5227fea2d4df193a5b50beaf2d00 BATCH-2002: Initial support for complex conditional replacements +* 65d2df652abaae2ca309d96e3026c2d67312655f Add ability to set a custom TaskExecutor impl, remove unused namespaces from JSR bootst +* 85807568575c24d8878ad605a344f2bc35bb2b13 Update to allow restart parameters to override previous parameters in JsrJobOperator an +* a21df75ce9dfc92e9768353b827da4248aefe425 BATCH-2049: Support multiple fragmentRootElementNames in StaxEventItemReader +* 7f1130c9a265a3ce18a46cbbc122e6573167a036 Fix TCK test JobOperatorTests.testJobOperatorRestartJobAlreadyAbandoned +* c4231c4cc861bbcc43437c80a03ddd9b7b2897a3 Fixed no executions returned check and added a unit test +```` +If you see intersecting lines, that usually means that you forgot to rebase you branch. As mentioned earlier, **please rebase against master** before issuing a pull request. + +## Code style + +Please carefully follow the same [code style as Spring Framework](https://github.com/spring-projects/spring-framework/wiki/Code-Style). + +## Add Apache license header to all new classes + +```java +/* + * Copyright 2002-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package ...; +``` + +## Update license header to modified files as necessary + +Always check the date range in the Apache license header. For example, if you've modified a file in 2013 whose header still reads + +```java + * Copyright 2002-2011 the original author or authors. +``` + +then be sure to update it to 2013 appropriately + +```java + * Copyright 2002-2013 the original author or authors. +``` + +## Use @since tags + +Use @since tags for newly-added public API types and methods e.g. + +```java +/** + * ... + * + * @author First Last + * @since 3.0 + * @see ... + */ +``` + +## Submit JUnit test cases for all behavior changes + +Search the codebase to find related unit tests and add additional @Test methods within. It is also acceptable to submit test cases on a per JIRA issue basis. + +## Squash commits + +Use `git rebase --interactive`, `git add --patch` and other tools to "squash" multiple commits into atomic changes. In addition to the man pages for git, there are many resources online to help you understand how these tools work. Here is one: https://book.git-scm.com/book/en/v2/Git-Tools-Rewriting-History#_squashing . + +## Use your real name in git commits + +Please configure git to use your real first and last name for any commits you intend to submit as pull requests. For example, this is not acceptable: + + Author: Nickname + +Rather, please include your first and last name, properly capitalized, as submitted against the Spring contributor license agreement: + + Author: First Last + +This helps ensure traceability against the CLA, and also goes a long way to ensuring useful output from tools like `git shortlog` and others. + +You can configure this globally via the account admin area GitHub (useful for fork-and-edit cases); globally with + + git config --global user.name "First Last" + git config --global user.email user@mail.com + +or locally for the *spring-batch repository only by omitting the '--global' flag: + + cd spring-batch + git config user.name "First Last" + git config user.email user@mail.com + +## Run all tests prior to submission + +See the [checking out and building][] section of the README for instructions. Make sure that all tests pass prior to submitting your pull request. + +## Mention your pull request on the associated JIRA issue + +Add a comment to the associated JIRA issue(s) linking to your new pull request. + +[help documentation]: https://help.github.com/send-pull-requests +[JIRA issue tracker]: https://jira.spring.io/browse/BATCH +[checking out and building]: https://github.com/spring-projects/spring-batch#building-from-source diff --git a/README.md b/README.md index e275573f80..d36568450d 100644 --- a/README.md +++ b/README.md @@ -1,102 +1,54 @@ -# Spring Batch + + +# Spring Batch [![build status](https://build.spring.io/plugins/servlet/wittified/build-status/BATCH-GRAD)](https://build.spring.io/browse/BATCH-GRAD) Spring Batch is a lightweight, comprehensive batch framework designed to enable the development of robust batch applications vital for the daily operations of enterprise systems. Spring Batch builds upon the productivity, POJO-based development approach, and general ease of use capabilities people have come to know from the [Spring Framework](https://github.com/SpringSource/spring-framework), while making it easy for developers to access and leverage more advanced enterprise services when necessary. -If you are looking for a runtime container for your Batch applications, or need a management console to view current and historic executions, take a look at [Spring Batch Admin](http://www.springsource.org/spring-batch-admin). It is a set of services (Java, JSON, JMX) and an optional web UI that you can use to manage and monitor a Batch system. +If you are looking for a runtime orchestration tool for your Batch applications, or need a management console to view current and historic executions, take a look at [Spring Cloud Data Flow](https://cloud.spring.io/spring-cloud-dataflow/). It is an orchestration tool for deploying and executing data integration based microservices including Spring Batch applications. # Building from Source Clone the git repository using the URL on the Github home page: - $ git clone git://github.com/SpringSource/spring-batch.git + $ git clone git@github.com:spring-projects/spring-batch.git $ cd spring-batch ## Command Line -Use Maven 2.2 or 3.0, then on the command line: - - $ mvn install - -or, the first time (to download the stuff that isn't in the Maven Central repository): +Gradle is the build tool used for Spring Batch. You can perform a full build of Spring Batch via the command: - $ mvn install -P bootstrap - -To perform a full build of all modules (required prior to issuing a pull request) execute the all profile + $ ./gradlew build - $ mvn install -P all +## Spring Tool Suite (STS) +In STS (or any Eclipse distro or other IDE with Gradle support), import the module directories as existing projects. They should compile and the tests should run with no additional steps. -## SpringSource Tool Suite (STS) -In STS (or any Eclipse distro or other IDE with Maven support), import the module directories as existing projects. They should compile and the tests should run with no additional steps. +# Getting Started Using Spring Boot +This is the quickest way to get started with a new Spring Batch project. You find the Getting Started Guide for Spring +Batch on Spring.io: [Creating a Batch Service](https://spring.io/guides/gs/batch-processing/) -# Getting Started Using SpringSource Tool Suite (STS) +# Getting Started Using Spring Tool Suite (STS) - This is the quickest way to get started. It requires an internet connection for download, and access to a Maven repository (remote or local). +It requires an internet connection for download, and access to a Maven repository (remote or local). -* Download STS version 2.5.* (or better) from the [SpringSource website](http://www.springsource.com/products/sts). STS is a free Eclipse bundle with many features useful for Spring developers. +* Download STS version 3.4.* (or better) from the [Spring website](https://spring.io/tools/sts/). STS is a free Eclipse bundle with many features useful for Spring developers. * Go to `File->New->Spring Template Project` from the menu bar (in the Spring perspective). * The wizard has a drop down with a list of template projects. One of them is a "Simple Spring Batch Project". Select it and follow the wizard. * A project is created with all dependencies and a simple input/output job configuration. It can be run using a unit test, or on the command line (see instructions in the pom.xml). # Getting Help -Read the main project [website](http://www.springsource.org/spring-batch) and the [User Guide](http://static.springsource.org/spring-batch/reference). Look at the source code and the Javadocs. For more detailed questions, use the [forum](http://forum.springsource.org/forumdisplay.php?f=41). If you are new to Spring as well as to Spring Batch, look for information about [Spring projects](http://www.springsource.org/projects). +Read the main project [website](https://projects.spring.io/spring-batch/) and the [User Guide](https://docs.spring.io/spring-batch/trunk/reference/). Look at the source code and the Javadocs. For more detailed questions, use the [forum](https://forum.spring.io/forum/spring-projects/batch). If you are new to Spring as well as to Spring Batch, look for information about [Spring projects](https://spring.io/projects). # Contributing to Spring Batch Here are some ways for you to get involved in the community: -* Get involved with the Spring community on the Spring Community Forums. Please help out on the [forum](http://forum.springsource.org/forumdisplay.php?f=41) by responding to questions and joining the debate. -* Create [JIRA](https://jira.springsource.org/browse/BATCH) tickets for bugs and new features and comment and vote on the ones that you are interested in. -* Github is for social coding: if you want to write code, we encourage contributions through pull requests from [forks of this repository](http://help.github.com/forking/). If you want to contribute code this way, please familiarize yourself with the process oulined for contributing to Spring projects here: [Contributor Guidelines](https://github.com/SpringSource/spring-integration/wiki/Contributor-Guidelines). -* Watch for upcoming articles on Spring by [subscribing](http://www.springsource.org/node/feed) to springframework.org +* Get involved with the Spring community on the Spring Community Forums. Please help out on the [forum](https://forum.spring.io/forum/spring-projects/batch) by responding to questions and joining the debate. +* Create [JIRA](https://jira.spring.io/browse/BATCH) tickets for bugs and new features and comment and vote on the ones that you are interested in. +* Github is for social coding: if you want to write code, we encourage contributions through pull requests from [forks of this repository](https://help.github.com/forking/). If you want to contribute code this way, please familiarize yourself with the process outlined for contributing to Spring projects here: [Contributor Guidelines](https://github.com/spring-projects/spring-batch/blob/master/CONTRIBUTING.md). +* Watch for upcoming articles on Spring by [subscribing](feed://assets.spring.io/drupal/node/feed.xml) to spring.io Before we accept a non-trivial patch or pull request we will need you to sign the [contributor's agreement](https://support.springsource.com/spring_committer_signup). Signing the contributor's agreement does not grant anyone commit rights to the main repository, but it does mean that we can accept your contributions, and you will get an author credit if we do. Active contributors might be asked to join the core team, and given the ability to merge pull requests. - -# Getting Started Using the Samples - -A convenient way to get started quickly with Spring Batch is to run the samples which are packaged in the samples module. There is also a simple command line sample (or "archetype") which has a bare bones but complete implementation of a simpel job. The source code for the samples (and the other modules) is available either from the [Zip assembly](http://static.springsource.org/spring-batch/downloads.html) or from [Git](http://www.springsource.org/spring-batch/source-repository.html). - -## Using the .Zip Distribution - -### With Maven and Eclipse - -* Download the "no-dependencies" version of the distribution and unzip to create a directory `org.springframework.batch-. -* Get the [m2eclipse plugin](http://m2eclipse.sonatype.org/update) - (installed in STS out of the box). If you can't or don't want to - install this plugin, you can use the [Maven Eclipse - Plugin](http://maven.apache.org/plugins/maven-eclipse-plugin) to - create the classpath entries you need. -* Open Eclipse and create a workspace as for the non-Mavenized version. -* Import the samples and archetype projects from the samples sub-directory in the directory you just unpacked. -* The project should build cleanly without having to fix the dependencies. If it doesn't, make sure you are online, and maybe try building on the command line first to make sure all the dependencies are downloaded. See the [build instructions](http://static.springsource.org/spring-batch/building.html) if you run into difficulty. - -(N.B. the "archetype" is not a real Maven archetype, just a template project that can be used as a starting point for a self-contained batch job. It is the same project that can be imported into STS using the Project Template wizard.) - -### With Maven on the Command Line - -* Download the distribution as above. -* Then run Maven in the spring-batch-samples directory, e.g. - - $ cd spring-batch-samples - $ mvn test - ... - -### With Eclipse and without Maven - -Similar instructions would apply to other IDEs. - -* Download the "no-dependencies" package and unzip to create a directory `spring-batch-. -* Open Eclipse and make a workspace in the directory you just created. -* Import the `org.springframework.batch.samples` project from the samples directory. -* Find all the compile scope and non-optional runtime jar files listed in the [core dependencies report](http://static.springsource.org/spring-batch/spring-batch-core/dependencies.html) and [infrastructure dependencies](http://static.springsource.org/spring-batch/spring-batch-infrastructure/dependencies.html), and import them into the project. -* Force the workspace to build (e.g. Project -> Clean...) -* Run the unit tests in your project under src/test/java. N.B. the FootbalJobFunctionTests takes quite a long time to run. - -You can get a pretty good idea about how to set up a job by examining the unit tests in the `org.springframework.batch.sample` package (in `src/main/java`) and the configuration in `src/main/resources/jobs`. - -To launch a job from the command line instead of a unit test use the `CommandLineJobRunner.main()` method (see Javadocs included in that class). - -## Using Maven and Git - -* Check out the Spring Batch project from Git (instructions are available [here](https://github.com/SpringSource/spring-batch)). -* Run Maven from the command line in the samples directory. There are additional building instructions and suggestions about what to do if it goes wrong [here](http://static.springsource.org/spring-batch/building.html). +# Code of Conduct + This project adheres to the [Contributor Covenant ](https://github.com/spring-projects/spring-batch/blob/master/CODE_OF_CONDUCT.adoc). By participating, you are expected to uphold this code. Please report unacceptable behavior to spring-code-of-conduct@pivotal.io. + \ No newline at end of file diff --git a/archetypes/common-test.xml b/archetypes/common-test.xml deleted file mode 100644 index cc38edb16e..0000000000 --- a/archetypes/common-test.xml +++ /dev/null @@ -1,47 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/archetypes/pom.xml b/archetypes/pom.xml deleted file mode 100644 index 03f12915db..0000000000 --- a/archetypes/pom.xml +++ /dev/null @@ -1,31 +0,0 @@ - - - 4.0.0 - spring-batch-archetype - pom - Archetypes - http://static.springframework.org/spring-batch/${artifactId} - Spring Batch archetypes are simple project templates containing just enough code to get you started running a job. For more detailed examples of using particular features of the framework, look at the Spring Batch Samples project. Currently the archetypes are deployed as regular Maven projects (not archetypes), because of limitations in the archetype plugin (or our understanding of how it works). For most effective use, copy one into Eclipse and use Q4E to manage the dependencies. - - org.springframework.batch - spring-batch-parent - 2.2.2.BUILD-SNAPSHOT - ../spring-batch-parent - - - simple-cli - - - - - - org.apache.maven.plugins - maven-surefire-plugin - - junit:junit - - - - - - diff --git a/archetypes/simple-cli/.springBeans b/archetypes/simple-cli/.springBeans deleted file mode 100644 index f2c1c7d8a4..0000000000 --- a/archetypes/simple-cli/.springBeans +++ /dev/null @@ -1,16 +0,0 @@ - - - 1 - - - - - - - src/test/resources/test-context.xml - src/main/resources/META-INF/spring/module-context.xml - src/main/resources/launch-context.xml - - - - diff --git a/archetypes/simple-cli/pom.xml b/archetypes/simple-cli/pom.xml deleted file mode 100644 index 3d3a0e7a82..0000000000 --- a/archetypes/simple-cli/pom.xml +++ /dev/null @@ -1,286 +0,0 @@ - - - 4.0.0 - org.springframework.batch - spring-batch-simple-cli - 2.2.2.BUILD-SNAPSHOT - jar - Commandline - http://www.springframework.org/spring-batch/archetypes/simple-cli-archetype - This project is a command line batch sample from Spring Batch demonstrating a typical batch job. - The job can be invoked via "mvn exec:java" to see the job run from the command line. - During an "mvn package", a distribution archive in the format defined in src/main/assembly/descriptor.xml will - be created in the target directory which can be deployed and extracted to your target location of choice. - The ./bin/runJob.(sh|bat) script from the extracted deployment archive can be invoked to run the job. - - - 3.2.0.RELEASE - 2.2.2.BUILD-SNAPSHOT - false - 4.10 - 1.6 - 1.6 - UTF-8 - - - - staging - - - staging - file:///${user.dir}/target/staging - - - staging - file:///${user.dir}/target/staging - - - - - bootstrap - - - Codehaus - http://repository.codehaus.org/ - - false - - - - com.springsource.repository.bundles.release - SpringSource Enterprise Bundle Repository - SpringSource Bundle Releases - http://repository.springsource.com/maven/bundles/release - - false - - - - - - - - junit - junit - ${junit.version} - test - - - org.springframework - spring-test - ${spring.framework.version} - test - - - org.springframework - spring-aop - ${spring.framework.version} - - - org.springframework - spring-jdbc - ${spring.framework.version} - - - org.springframework.batch - spring-batch-core - ${spring.batch.version} - - - org.springframework.batch - spring-batch-infrastructure - ${spring.batch.version} - - - org.springframework.batch - spring-batch-test - ${spring.batch.version} - test - - - commons-dbcp - commons-dbcp - 1.2.2 - - - org.slf4j - slf4j-log4j12 - 1.5.8 - true - - - log4j - log4j - 1.2.14 - true - - - commons-io - commons-io - 1.4 - - - org.hsqldb - hsqldb - 2.2.9 - - - org.aspectj - aspectjrt - 1.6.8 - - - org.aspectj - aspectjweaver - 1.6.8 - - - - - - org.springframework.build.aws - org.springframework.build.aws.maven - 3.0.0.RELEASE - - - - - - maven-assembly-plugin - false - - - project - - - - - - org.eclipse.m2e - lifecycle-mapping - 1.0.0 - - - - - - org.apache.maven.plugins - maven-dependency-plugin - [1.0,) - - copy-dependencies - - - - - - - - - - - - - - - org.apache.maven.plugins - maven-compiler-plugin - 2.3.2 - - ${compiler.source.version} - ${compiler.target.version} - - - - org.apache.maven.plugins - maven-surefire-plugin - - - **/*Tests.java - - - **/Abstract*.java - - junit:junit - - - - org.codehaus.mojo - exec-maven-plugin - 1.1 - - org.springframework.batch.core.launch.support.CommandLineJobRunner - - classpath:/launch-context.xml - personJob - - - - - org.apache.maven.plugins - maven-dependency-plugin - - - copy-dependencies - package - - copy-dependencies - - - ${project.build.directory}/lib - - - - - - maven-assembly-plugin - 2.3 - - - src/main/assembly/descriptor.xml - - - - - make-distribution - package - - single - - - - - - org.apache.maven.plugins - maven-jar-plugin - - - false - - org.springframework.batch.core.launch.support.CommandLineJobRunner - true - lib/ - - - - - - - - http://www.springframework.org/download - - staging - file:///${user.dir}/target/staging/org.springframework.batch.archetype/${project.artifactId} - - - spring-release - Spring Release Repository - file:///${user.dir}/target/staging/release - - - spring-snapshot - Spring Snapshot Repository - file:///${user.dir}/target/staging/snapshot - - - diff --git a/archetypes/simple-cli/src/main/assembly/descriptor.xml b/archetypes/simple-cli/src/main/assembly/descriptor.xml deleted file mode 100644 index c28873cb29..0000000000 --- a/archetypes/simple-cli/src/main/assembly/descriptor.xml +++ /dev/null @@ -1,25 +0,0 @@ - - distribution - - tar.gz - - false - - - src/main/scripts - bin - true - - - src/main/resources - resources - true - true - - - - - lib - - - diff --git a/archetypes/simple-cli/src/main/java/example/Person.java b/archetypes/simple-cli/src/main/java/example/Person.java deleted file mode 100644 index 43787449db..0000000000 --- a/archetypes/simple-cli/src/main/java/example/Person.java +++ /dev/null @@ -1,47 +0,0 @@ -/* - * Copyright 2013 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package example; - -/** - *

- * Domain object representing information about a person. - *

- */ -public class Person { - private String lastName; - private String firstName; - - public void setFirstName(final String firstName) { - this.firstName = firstName; - } - - public String getFirstName() { - return firstName; - } - - public void setLastName(final String lastName) { - this.lastName = lastName; - } - - public String getLastName() { - return lastName; - } - - @Override - public String toString() { - return "firstName: " + firstName + ", lastName: " + lastName; - } -} diff --git a/archetypes/simple-cli/src/main/java/example/PersonItemProcessor.java b/archetypes/simple-cli/src/main/java/example/PersonItemProcessor.java deleted file mode 100644 index b9a25bdeab..0000000000 --- a/archetypes/simple-cli/src/main/java/example/PersonItemProcessor.java +++ /dev/null @@ -1,38 +0,0 @@ -/* - * Copyright 2013 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package example; - -import org.springframework.batch.item.ItemProcessor; - -/** - *

- * An example {@link org.springframework.batch.item.ItemProcessor} implementation that upper cases attributes on the - * provided {@link Person} object. - *

- */ -public class PersonItemProcessor implements ItemProcessor { - @Override - public Person process(final Person person) throws Exception { - final String firstName = person.getFirstName().toUpperCase(); - final String lastName = person.getLastName().toUpperCase(); - - final Person transformedPerson = new Person(); - transformedPerson.setFirstName(firstName); - transformedPerson.setLastName(lastName); - - return transformedPerson; - } -} diff --git a/archetypes/simple-cli/src/main/resources/META-INF/spring/module-context.xml b/archetypes/simple-cli/src/main/resources/META-INF/spring/module-context.xml deleted file mode 100644 index 4d20c2fe7f..0000000000 --- a/archetypes/simple-cli/src/main/resources/META-INF/spring/module-context.xml +++ /dev/null @@ -1,22 +0,0 @@ - - - - Example job providing a skeleton for a typical batch application. - - - - - - - - - - - diff --git a/archetypes/simple-cli/src/main/resources/batch.properties b/archetypes/simple-cli/src/main/resources/batch.properties deleted file mode 100644 index dea386513f..0000000000 --- a/archetypes/simple-cli/src/main/resources/batch.properties +++ /dev/null @@ -1,15 +0,0 @@ -# Placeholders batch.* -# for HSQLDB: -batch.jdbc.driver=org.hsqldb.jdbcDriver -batch.jdbc.url=jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true -# use this one for a separate server process so you can inspect the results -# (or add it to system properties with -D to override at run time). -# batch.jdbc.url=jdbc:hsqldb:hsql://localhost:9005/samples -batch.jdbc.user=sa -batch.jdbc.password= -batch.schema.script=classpath:org/springframework/batch/core/schema-hsqldb.sql -batch.drop.script=classpath:org/springframework/batch/core/schema-drop-hsqldb.sql -person.sql.location=classpath:support/person.sql -person.test.data.location=classpath:support/sample-data.csv -person.insert.sql=INSERT INTO people (first_name, last_name) VALUES (:firstName, :lastName) -step1.commit.interval=5 diff --git a/archetypes/simple-cli/src/main/resources/launch-context.xml b/archetypes/simple-cli/src/main/resources/launch-context.xml deleted file mode 100644 index f47c948ced..0000000000 --- a/archetypes/simple-cli/src/main/resources/launch-context.xml +++ /dev/null @@ -1,69 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/archetypes/simple-cli/src/main/resources/log4j.properties b/archetypes/simple-cli/src/main/resources/log4j.properties deleted file mode 100644 index 2cdeeb81ba..0000000000 --- a/archetypes/simple-cli/src/main/resources/log4j.properties +++ /dev/null @@ -1,8 +0,0 @@ -log4j.rootCategory=INFO, stdout - -log4j.appender.stdout=org.apache.log4j.ConsoleAppender -log4j.appender.stdout.layout=org.apache.log4j.PatternLayout -log4j.appender.stdout.layout.ConversionPattern=%d %p %t [%c] - <%m>%n - -log4j.category.org.springframework.batch=DEBUG -log4j.category.org.springframework.transaction=INFO diff --git a/archetypes/simple-cli/src/main/resources/support/person.sql b/archetypes/simple-cli/src/main/resources/support/person.sql deleted file mode 100644 index 4e405e6de6..0000000000 --- a/archetypes/simple-cli/src/main/resources/support/person.sql +++ /dev/null @@ -1,7 +0,0 @@ -DROP TABLE people IF EXISTS; - -CREATE TABLE people ( - person_id BIGINT IDENTITY NOT NULL PRIMARY KEY, - first_name VARCHAR(20), - last_name VARCHAR(20) -); diff --git a/archetypes/simple-cli/src/main/scripts/runJob.bat b/archetypes/simple-cli/src/main/scripts/runJob.bat deleted file mode 100644 index 72ed089a47..0000000000 --- a/archetypes/simple-cli/src/main/scripts/runJob.bat +++ /dev/null @@ -1,8 +0,0 @@ -@echo off - -IF NOT DEFINED JAVA_HOME ( - echo Error: JAVA_HOME environment variable is not set. - EXIT /B -) - -%JAVA_HOME%\bin\java -cp resources\;lib\* org.springframework.batch.core.launch.support.CommandLineJobRunner classpath:/launch-context.xml personJob diff --git a/archetypes/simple-cli/src/main/scripts/runJob.sh b/archetypes/simple-cli/src/main/scripts/runJob.sh deleted file mode 100644 index fe9e1c8bdd..0000000000 --- a/archetypes/simple-cli/src/main/scripts/runJob.sh +++ /dev/null @@ -1,8 +0,0 @@ -#!/bin/bash - -if [ "$JAVA_HOME" = "" ]; then - echo "Error: JAVA_HOME environment variable is not set." - exit 1 -fi - -$JAVA_HOME/bin/java -cp resources/:lib/* org.springframework.batch.core.launch.support.CommandLineJobRunner classpath:/launch-context.xml personJob diff --git a/archetypes/simple-cli/src/site/site.xml b/archetypes/simple-cli/src/site/site.xml deleted file mode 100644 index 42b3b297eb..0000000000 --- a/archetypes/simple-cli/src/site/site.xml +++ /dev/null @@ -1,20 +0,0 @@ - - - - Spring Batch: ${project.name} - index.html - - - - org.springframework.maven.skins - maven-spring-skin - 1.0.5 - - - - - - - - - diff --git a/archetypes/simple-cli/src/test/java/example/PersonItemProcessorTest.java b/archetypes/simple-cli/src/test/java/example/PersonItemProcessorTest.java deleted file mode 100644 index 49f0d3da03..0000000000 --- a/archetypes/simple-cli/src/test/java/example/PersonItemProcessorTest.java +++ /dev/null @@ -1,38 +0,0 @@ -/* - * Copyright 2013 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package example; - -import org.junit.Test; -import static org.junit.Assert.assertEquals; - -/** - *

- * Example test case processing a {@link Person} using the {@link PersonItemProcessor}. - *

- */ -public class PersonItemProcessorTest { - @Test - public void testProcessedPersonRecord() throws Exception { - final Person person = new Person(); - person.setFirstName("Jane"); - person.setLastName("Doe"); - - final Person processedPerson = new PersonItemProcessor().process(person); - - assertEquals("First name does not match expected value.", "JANE", processedPerson.getFirstName()); - assertEquals("Last name does not match expected value.", "DOE", processedPerson.getLastName()); - } -} diff --git a/archetypes/simple-cli/src/test/java/example/PersonJobConfigurationTest.java b/archetypes/simple-cli/src/test/java/example/PersonJobConfigurationTest.java deleted file mode 100644 index 96fbf4cce8..0000000000 --- a/archetypes/simple-cli/src/test/java/example/PersonJobConfigurationTest.java +++ /dev/null @@ -1,96 +0,0 @@ -/* - * Copyright 2006-2013 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package example; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.springframework.batch.core.BatchStatus; -import org.springframework.batch.core.Job; -import org.springframework.batch.core.JobExecution; -import org.springframework.batch.core.JobInstance; -import org.springframework.batch.core.JobParameters; -import org.springframework.batch.core.launch.JobLauncher; -import org.springframework.batch.core.launch.JobOperator; -import org.springframework.batch.test.JobLauncherTestUtils; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.test.context.ContextConfiguration; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; - -/** - *

- * Test cases asserting on the example job's configuration. - *

- */ -@ContextConfiguration(locations = "/test-context.xml") -@RunWith(SpringJUnit4ClassRunner.class) -public class PersonJobConfigurationTest { - @Autowired - private JobLauncher jobLauncher; - - @Autowired - private Job job; - - @Autowired - private JobOperator jobOperator; - - @Autowired - private JobLauncherTestUtils jobLauncherTestUtils; - - /** - *

- * Creates a new {@link JobExecution} using a {@link JobLauncher}. - *

- * - * @throws Exception if any {@link Exception}'s occur - */ - @Test - public void testLaunchJobWithJobLauncher() throws Exception { - final JobExecution jobExecution = jobLauncher.run(job, new JobParameters()); - assertEquals("Batch status not COMPLETED", BatchStatus.COMPLETED, jobExecution.getStatus()); - } - - /** - *

- * Create a unique job instance and check it's execution completes successfully. - * Uses the convenience methods provided by the testing superclass. - *

- * - * @throws Exception if any {@link Exception}'s occur - */ - @Test - public void testLaunchJob() throws Exception { - final JobExecution jobExecution = jobLauncherTestUtils.launchJob(jobLauncherTestUtils.getUniqueJobParameters()); - assertEquals("Batch status not COMPLETED", BatchStatus.COMPLETED, jobExecution.getStatus()); - } - - /** - *

- * Execute a fresh {@link JobInstance} using {@link JobOperator} which is closer to - * a remote invocation scenario. - *

- * - * @throws Exception if any {@link Exception}'s occur - */ - @Test - public void testLaunchByJobOperator() throws Exception { - final long jobExecutionId = jobOperator.startNextInstance(jobLauncherTestUtils.getJob().getName()); - - final String result = jobOperator.getSummary(jobExecutionId); - assertTrue("Result does not contain status=COMPLETED", result.contains("status=" + BatchStatus.COMPLETED)); - } -} diff --git a/archetypes/simple-cli/src/test/resources/test-context.xml b/archetypes/simple-cli/src/test/resources/test-context.xml deleted file mode 100644 index e14b4c2d62..0000000000 --- a/archetypes/simple-cli/src/test/resources/test-context.xml +++ /dev/null @@ -1,38 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - diff --git a/archetypes/src/site/site.xml b/archetypes/src/site/site.xml deleted file mode 100644 index 62ec37d46a..0000000000 --- a/archetypes/src/site/site.xml +++ /dev/null @@ -1,20 +0,0 @@ - - - - - Spring Batch: ${project.name} - index.html - - - - - - - - - - - - - - diff --git a/build-spring-batch/build.xml b/build-spring-batch/build.xml deleted file mode 100644 index 5006155d55..0000000000 --- a/build-spring-batch/build.xml +++ /dev/null @@ -1,26 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/build-spring-batch/package-bundle.xml b/build-spring-batch/package-bundle.xml deleted file mode 100644 index 67330f04d7..0000000000 --- a/build-spring-batch/package-bundle.xml +++ /dev/null @@ -1,12 +0,0 @@ - - - - - - - - - - - - diff --git a/build.gradle b/build.gradle new file mode 100644 index 0000000000..0083e7baf0 --- /dev/null +++ b/build.gradle @@ -0,0 +1,947 @@ +//description = 'Spring Batch' + +//apply plugin: 'base' + +buildscript { + repositories { + maven { url 'https://repo.spring.io/plugins-release' } + maven { url 'https://plugins.gradle.org/m2/' } + } + dependencies { + classpath 'org.springframework.build.gradle:propdeps-plugin:0.0.7' + classpath 'io.spring.gradle:spring-io-plugin:0.0.5.RELEASE' + classpath "io.spring.gradle:dependency-management-plugin:0.6.0.RELEASE" + classpath "org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:2.6.2" + classpath "org.asciidoctor:asciidoctor-gradle-plugin:1.5.6" + classpath "org.asciidoctor:asciidoctorj-pdf:1.5.0-alpha.16" + } +} + +plugins { + id 'org.sonarqube' version '2.6.2' +} + +ext { + linkHomepage = 'https://projects.spring.io/spring-batch/' + linkCi = 'https://build.spring.io/browse/BATCH' + linkIssue = 'https://jira.spring.io/browse/BATCH' + linkScmUrl = 'https://github.com/spring-projects/spring-batch' + linkScmConnection = 'git://github.com/spring-projects/spring-batch.git' + linkScmDevConnection = 'git@github.com:spring-projects/spring-batch.git' + + mainProjects = subprojects.findAll { !it.name.endsWith('tests') && !it.name.endsWith('samples') && it.name.startsWith('spring-batch-')} +} + +allprojects { + group = 'org.springframework.batch' + + repositories { + maven { url 'https://repo.spring.io/libs-snapshot' } + maven { url 'https://repo.spring.io/libs-milestone' } + maven { url 'https://repo.spring.io/plugins-release' } + mavenCentral() + } + + ext { + + environmentProperty = project.hasProperty('environment') ? getProperty('environment') : 'hsql' + + springVersionDefault = '5.2.0.RELEASE' + springVersion = project.hasProperty('springVersion') ? getProperty('springVersion') : springVersionDefault + springRetryVersion = '1.2.4.RELEASE' + springAmqpVersion = '2.2.0.RELEASE' + springDataCommonsVersion = '2.2.0.RELEASE' + springDataGemfireVersion = '2.2.0.RELEASE' + springDataJpaVersion = '2.2.0.RELEASE' + springDataMongodbVersion = '2.2.0.RELEASE' + springDataNeo4jVersion = '5.2.0.RELEASE' + springIntegrationVersion = '5.2.0.RELEASE' + springKafkaVersion = '2.3.0.RELEASE' + springLdapVersion = '2.3.2.RELEASE' + + activemqVersion = '5.15.10' + apacheAvroVersion ='1.9.1' + aspectjVersion = '1.9.4' + commonsDdbcpVersion = '2.7.0' + commonsIoVersion = '2.6' + commonsLangVersion = '3.9' + commonsPoolVersion = '1.6' + derbyVersion = '10.14.2.0' + groovyVersion = '2.5.8' + hamcrestVersion = '2.1' + h2databaseVersion = '1.4.199' + hibernateVersion = '5.4.5.Final' + hibernateValidatorVersion = '6.0.17.Final' + javaxElVersion = '3.0.0' + hsqldbVersion = '2.5.0' + jackson2Version = '2.10.0' + gsonVersion = '2.8.5' + javaMailVersion = '1.6.2' + javaxBatchApiVersion = '1.0' + javaxAnnotationApiVersion = '1.3.2' + javaxInjectVersion = '1' + javaxTransactionVersion = '1.3' + jbatchTckSpi = '1.0' + jettisonVersion = '1.2' //? => upgrade to 1.4 and the build fails (deprecated anyway via xstream) + jmsVersion = '2.0.1' + junitVersion = '4.12' + log4jVersion = '2.12.1' + mysqlVersion = '8.0.17' + mockitoVersion = '3.0.0' + postgresqlVersion = '42.2.8' + quartzVersion = '2.3.1' + servletApiVersion = '4.0.1' + sqlfireclientVersion = '1.0.3' + sqliteVersion = '3.28.0' + woodstoxVersion = '5.2.0' + xmlunitVersion = '2.6.3' + xstreamVersion = '1.4.11.1' + jrubyVersion = '1.7.27' + beanshellVersion = '2.0b5' + jaxbApiVersion = '2.3.1' + jaxbImplVersion = '2.3.0.1' + micrometerVersion = '1.3.0' + prometheusPushgatewayVersion = '0.7.0' + docResourcesVersion = '0.1.1.RELEASE' + assertjVersion='3.13.2' + } + + apply plugin: 'idea' +} + +configure(subprojects - project(":spring-build-src")) { subproject -> + + apply plugin: 'java' + apply from: "${rootProject.projectDir}/publish-maven.gradle" + apply plugin: 'jacoco' + apply plugin: 'propdeps-idea' + apply plugin: 'propdeps-eclipse' + apply plugin: 'merge' + + jacoco { + toolVersion = "0.8.2" + } + + compileJava { + sourceCompatibility=1.8 + targetCompatibility=1.8 + options.encoding='UTF-8' + } + + compileTestJava { + sourceCompatibility=1.8 + targetCompatibility=1.8 + options.encoding='UTF-8' + } + + eclipse { + project { + natures += 'org.springframework.ide.eclipse.core.springnature' + } + } + + sourceSets { + test { + resources { + srcDirs = ['src/test/resources', 'src/test/java'] + } + } + } + + // enable all compiler warnings; individual projects may customize further + ext.xLintArg = '-Xlint:all' + [compileJava, compileTestJava]*.options*.compilerArgs = [xLintArg] + '-proc:none' + + tasks.withType(Test).all { + // suppress all console output during testing unless running `gradle -i` + logging.captureStandardOutput(LogLevel.INFO) + systemProperty "ENVIRONMENT", environmentProperty + + jacoco { + append = false + destinationFile = file("$buildDir/jacoco.exec") + } + + include '**/*Tests.class' + exclude '**/Abstract*.class' + } + + test { + jacoco { + append = false + destinationFile = file("$buildDir/jacoco.exec") + } + +// testLogging { +// showStandardStreams = true +// } + } + + task sourcesJar(type: Jar) { + classifier = 'sources' + from sourceSets.main.allJava + } + + task javadocJar(type: Jar) { + classifier = 'javadoc' + from javadoc + } + + task checkTestConfigs { + doLast { + def configFiles = [] + sourceSets.test.java.srcDirs.each { + fileTree(it).include('**/*.xml').exclude('**/log4j.xml').each { configFile -> + def configXml = new XmlParser(false, false).parse(configFile) + + if (configXml.@'xsi:schemaLocation' ==~ /.*spring-[a-z-]*\d\.\d\.xsd.*/) { + configFiles << configFile + } + } + } + if (configFiles) { + throw new InvalidUserDataException('Hardcoded XSD version in the config files:\n' + + configFiles.collect { relativePath(it) }.join('\n') + + '\nPlease, use versionless schemaLocations for Spring XSDs to avoid issues with builds on different versions of dependencies.') + } + } + } + + jar { + manifest.attributes["Created-By"] = + "${System.getProperty("java.version")} (${System.getProperty("java.specification.vendor")})" + manifest.attributes["Implementation-Title"] = subproject.name + manifest.attributes["Implementation-Version"] = subproject.version + + from("${rootProject.projectDir}/src/dist") { + include "license.txt" + include "notice.txt" + into "META-INF" + expand(copyright: new Date().format("yyyy"), version: project.version) + } + } + + test.dependsOn checkTestConfigs + + artifacts { + archives sourcesJar + archives javadocJar + } +} + +subprojects { + task allDeps(type: DependencyReportTask) {} +} + +configure(mainProjects) { + if (project.hasProperty('platformVersion')) { + apply plugin: 'spring-io' + + repositories { + maven { url "https://repo.spring.io/libs-snapshot" } + } + + dependencyManagement { + springIoTestRuntime { + imports { + mavenBom "io.spring.platform:platform-bom:${platformVersion}" + } + } + } + } +} + +project("spring-build-src") { + description = "Exposes gradle buildSrc for IDE support" + apply plugin: "groovy" + + dependencies { + compile gradleApi() + compile localGroovy() + } + + configurations.archives.artifacts.clear() +} + +project('spring-batch-core') { + description = 'Spring Batch Core' + + dependencies { + compile project(":spring-batch-infrastructure") + + compile "com.fasterxml.jackson.core:jackson-databind:${jackson2Version}" + compile ("org.codehaus.jettison:jettison:$jettisonVersion") { + exclude group: 'stax', module: 'stax-api' + } + compile "org.springframework:spring-aop:$springVersion" + compile "org.springframework:spring-beans:$springVersion" + compile "org.springframework:spring-context:$springVersion" + compile "org.springframework:spring-core:$springVersion" + compile "org.springframework:spring-tx:$springVersion" + compile "javax.batch:javax.batch-api:$javaxBatchApiVersion" + compile "io.micrometer:micrometer-core:$micrometerVersion" + + testCompile "org.springframework:spring-test:$springVersion" + testCompile "org.mockito:mockito-core:$mockitoVersion" + testCompile "javax.inject:javax.inject:$javaxInjectVersion" + testCompile "org.hsqldb:hsqldb:$hsqldbVersion" + testCompile "com.h2database:h2:$h2databaseVersion" + testCompile "commons-io:commons-io:$commonsIoVersion" + testCompile "org.apache.commons:commons-dbcp2:$commonsDdbcpVersion" + testCompile "junit:junit:${junitVersion}" + testCompile "org.hamcrest:hamcrest-library:$hamcrestVersion" + optional "com.ibm.jbatch:com.ibm.jbatch-tck-spi:$jbatchTckSpi" + optional "com.thoughtworks.xstream:xstream:$xstreamVersion" + optional "org.aspectj:aspectjrt:$aspectjVersion" + optional "org.aspectj:aspectjweaver:$aspectjVersion" + optional "org.springframework:spring-jdbc:$springVersion" + optional "org.apache.logging.log4j:log4j-api:$log4jVersion" + optional "org.apache.logging.log4j:log4j-core:$log4jVersion" + optional "javax.annotation:javax.annotation-api:$javaxAnnotationApiVersion" + // JSR-305 only used for non-required meta-annotations + compileOnly("com.google.code.findbugs:jsr305:3.0.2") + testCompileOnly("com.google.code.findbugs:jsr305:3.0.2") + } +} + +project('spring-batch-infrastructure') { + description = 'Spring Batch Infrastructure' + + dependencies { + + compile "org.springframework:spring-core:$springVersion" + compile "org.springframework.retry:spring-retry:$springRetryVersion" + + testCompile "org.apache.logging.log4j:log4j-api:$log4jVersion" + testCompile "org.apache.logging.log4j:log4j-core:$log4jVersion" + testCompile "commons-io:commons-io:$commonsIoVersion" + testCompile "org.apache.commons:commons-dbcp2:$commonsDdbcpVersion" + testCompile "org.hsqldb:hsqldb:$hsqldbVersion" + testCompile "com.h2database:h2:$h2databaseVersion" + testCompile "org.apache.derby:derby:$derbyVersion" + testCompile "org.springframework:spring-test:$springVersion" + testCompile "junit:junit:${junitVersion}" + testCompile "org.hamcrest:hamcrest-library:$hamcrestVersion" + testCompile "org.aspectj:aspectjrt:$aspectjVersion" + testCompile "org.aspectj:aspectjweaver:$aspectjVersion" + testCompile "org.mockito:mockito-core:$mockitoVersion" + testCompile "org.xerial:sqlite-jdbc:$sqliteVersion" + testCompile "javax.xml.bind:jaxb-api:$jaxbApiVersion" + testCompile "org.springframework.kafka:spring-kafka-test:$springKafkaVersion" + testCompile "org.assertj:assertj-core:$assertjVersion" + + testRuntime "com.sun.mail:javax.mail:$javaMailVersion" + testRuntime "org.codehaus.groovy:groovy-jsr223:$groovyVersion" + testRuntime "org.jruby:jruby:$jrubyVersion" + testRuntime "org.hibernate.validator:hibernate-validator:$hibernateValidatorVersion" + testRuntime "org.glassfish:javax.el:$javaxElVersion" + testRuntime "org.beanshell:bsh:$beanshellVersion" + testRuntime "com.sun.xml.bind:jaxb-core:$jaxbImplVersion" + testRuntime "com.sun.xml.bind:jaxb-impl:$jaxbImplVersion" + + optional "javax.jms:javax.jms-api:$jmsVersion" + optional "com.fasterxml.jackson.core:jackson-databind:${jackson2Version}" + optional "com.google.code.gson:gson:${gsonVersion}" + compile("org.hibernate:hibernate-core:$hibernateVersion") { dep -> + optional dep + exclude group: 'org.jboss.spec.javax.transaction', module: 'jboss-transaction-api_1.1_spec' + } + compile("org.hibernate:hibernate-entitymanager:$hibernateVersion") { dep -> + optional dep + exclude group: 'org.jboss.spec.javax.transaction', module: 'jboss-transaction-api_1.1_spec' + } + optional "org.hibernate.validator:hibernate-validator:$hibernateValidatorVersion" + optional "javax.transaction:javax.transaction-api:$javaxTransactionVersion" + optional "javax.mail:javax.mail-api:$javaMailVersion" + optional "javax.batch:javax.batch-api:$javaxBatchApiVersion" + compile("org.springframework:spring-oxm:$springVersion") { dep -> + optional dep + exclude group: 'commons-lang', module: 'commons-lang' + } + optional "org.springframework:spring-aop:$springVersion" + optional "org.springframework:spring-context:$springVersion" + compile("org.springframework:spring-context-support:$springVersion") { dep -> + optional dep + } + optional "org.springframework:spring-jdbc:$springVersion" + optional "org.springframework:spring-jms:$springVersion" + optional "org.springframework:spring-orm:$springVersion" + optional "org.springframework:spring-tx:$springVersion" + optional "org.springframework.data:spring-data-commons:$springDataCommonsVersion" + optional "org.springframework.data:spring-data-mongodb:$springDataMongodbVersion" + optional ("org.springframework.data:spring-data-neo4j:$springDataNeo4jVersion") { + exclude group: "org.springframework", module: "spring-aspects" + } + optional "org.springframework.data:spring-data-gemfire:$springDataGemfireVersion" + compile("com.fasterxml.woodstox:woodstox-core:$woodstoxVersion") { dep -> + optional dep + exclude group: 'stax', module: 'stax-api' + } + optional "org.springframework.amqp:spring-amqp:$springAmqpVersion" + optional ("org.springframework.amqp:spring-rabbit:$springAmqpVersion") { + exclude group: "org.springframework", module: "spring-messaging" + exclude group: "org.springframework", module: "spring-web" + } + optional "org.springframework.ldap:spring-ldap-core:$springLdapVersion" + optional "org.springframework.ldap:spring-ldap-core-tiger:$springLdapVersion" + optional "org.springframework.ldap:spring-ldap-ldif-core:$springLdapVersion" + optional "org.springframework.kafka:spring-kafka:$springKafkaVersion" + optional "org.apache.avro:avro:$apacheAvroVersion" + + // JSR-305 only used for non-required meta-annotations + compileOnly("com.google.code.findbugs:jsr305:3.0.2") + testCompileOnly("com.google.code.findbugs:jsr305:3.0.2") + } + + if (project.hasProperty('platformVersion')) { + def dataNeo4jVersion = dependencyManagement.springIoTestRuntime.managedVersions['org.springframework.data:spring-data-neo4j'] + if (dataNeo4jVersion?.startsWith('4.')) { + tasks.withType(Test).matching { it.name =~ "springIoJdk.Test" }.all { + exclude '**/Neo4jItemReaderTests.class' + } + } + } +} + +project('spring-batch-docs') { + description = 'Spring Batch Docs' +} + +project('spring-batch-core-tests') { + description = 'Spring Batch Core Tests' + project.tasks.findByPath("artifactoryPublish")?.enabled = false + dependencies { + compile project(":spring-batch-core") + compile project(":spring-batch-infrastructure") + compile "org.apache.commons:commons-dbcp2:$commonsDdbcpVersion" + compile "org.springframework:spring-jdbc:$springVersion" + compile "org.springframework.retry:spring-retry:$springRetryVersion" + compile "org.springframework:spring-tx:$springVersion" + compile "org.springframework:spring-aop:$springVersion" + + testCompile "org.hsqldb:hsqldb:$hsqldbVersion" + testCompile "commons-io:commons-io:$commonsIoVersion" + testCompile "org.apache.derby:derby:$derbyVersion" + testCompile "junit:junit:${junitVersion}" + testCompile "org.hamcrest:hamcrest-library:$hamcrestVersion" + testCompile "org.apache.logging.log4j:log4j-api:$log4jVersion" + testCompile "org.apache.logging.log4j:log4j-core:$log4jVersion" + testCompile "org.springframework:spring-test:$springVersion" + testCompile "org.springframework:spring-jdbc:$springVersion" + + runtime "mysql:mysql-connector-java:$mysqlVersion" + runtime "org.postgresql:postgresql:$postgresqlVersion" + runtime "javax.batch:javax.batch-api:$javaxBatchApiVersion" + + optional "org.aspectj:aspectjrt:$aspectjVersion" + optional "org.aspectj:aspectjweaver:$aspectjVersion" + optional "org.springframework.ldap:spring-ldap-core:$springLdapVersion" + optional "org.springframework.ldap:spring-ldap-core-tiger:$springLdapVersion" + optional "org.springframework.ldap:spring-ldap-ldif-core:$springLdapVersion" + } + test { + enabled = project.hasProperty('alltests') + } +} + +project('spring-batch-infrastructure-tests') { + description = 'Spring Batch Infrastructure Tests' + project.tasks.findByPath("artifactoryPublish")?.enabled = false + dependencies { + compile project(":spring-batch-infrastructure") + compile "javax.jms:javax.jms-api:$jmsVersion" + compile "org.apache.commons:commons-dbcp2:$commonsDdbcpVersion" + compile "org.springframework:spring-tx:$springVersion" + compile "org.springframework:spring-aop:$springVersion" + + testCompile "org.hsqldb:hsqldb:$hsqldbVersion" + testCompile "commons-io:commons-io:$commonsIoVersion" + testCompile "org.apache.derby:derby:$derbyVersion" + testCompile "org.apache.activemq:activemq-broker:$activemqVersion" + testCompile "org.apache.activemq:activemq-kahadb-store:$activemqVersion" + testCompile "junit:junit:${junitVersion}" + testCompile "org.hamcrest:hamcrest-library:$hamcrestVersion" + testCompile "org.apache.geronimo.specs:geronimo-j2ee-management_1.1_spec:1.0.1" + testCompile "org.xmlunit:xmlunit-core:$xmlunitVersion" + testCompile "org.xmlunit:xmlunit-matchers:$xmlunitVersion" + testCompile "org.apache.logging.log4j:log4j-api:$log4jVersion" + testCompile "org.apache.logging.log4j:log4j-core:$log4jVersion" + testCompile "com.thoughtworks.xstream:xstream:$xstreamVersion" + testCompile("com.fasterxml.woodstox:woodstox-core:$woodstoxVersion") { + exclude group: 'stax', module: 'stax-api' + } + testCompile "org.apache.commons:commons-lang3:$commonsLangVersion" + testCompile("org.springframework:spring-oxm:$springVersion") { + exclude group: 'commons-lang', module: 'commons-lang' + } + testCompile "org.springframework:spring-jdbc:$springVersion" + testCompile "org.springframework:spring-test:$springVersion" + testCompile "org.mockito:mockito-core:$mockitoVersion" + testCompile "javax.xml.bind:jaxb-api:$jaxbApiVersion" + + compile("org.hibernate:hibernate-core:$hibernateVersion") { dep -> + optional dep + exclude group: 'org.jboss.spec.javax.transaction', module: 'jboss-transaction-api_1.1_spec' + } + compile("org.hibernate:hibernate-entitymanager:$hibernateVersion") { dep -> + optional dep + exclude group: 'org.jboss.spec.javax.transaction', module: 'jboss-transaction-api_1.1_spec' + } + optional "javax.transaction:javax.transaction-api:1.2" + optional "org.springframework:spring-orm:$springVersion" + optional "org.springframework:spring-jms:$springVersion" + + runtime "mysql:mysql-connector-java:$mysqlVersion" + runtime "org.postgresql:postgresql:$postgresqlVersion" + } + test { + enabled = project.hasProperty('alltests') + } +} + +//Domain for batch job testing +project('spring-batch-test') { + description = 'Spring Batch Test' + + dependencies { + compile project(":spring-batch-core") + + compile "junit:junit:${junitVersion}" + compile "org.springframework:spring-test:$springVersion" + compile "org.springframework:spring-jdbc:$springVersion" + + testCompile "org.hamcrest:hamcrest-library:$hamcrestVersion" + testCompile "org.apache.commons:commons-dbcp2:$commonsDdbcpVersion" + testCompile "org.hsqldb:hsqldb:$hsqldbVersion" + testCompile "org.mockito:mockito-core:$mockitoVersion" + + optional "org.aspectj:aspectjrt:$aspectjVersion" + optional "javax.batch:javax.batch-api:$javaxBatchApiVersion" + } +} + +project('spring-batch-integration') { + description = 'Batch Integration' + + dependencies { + compile project(":spring-batch-core") + + compile "org.springframework.retry:spring-retry:$springRetryVersion" + compile "org.springframework:spring-context:$springVersion" + compile "org.springframework:spring-messaging:$springVersion" + compile "org.springframework:spring-aop:$springVersion" + compile "org.springframework.integration:spring-integration-core:$springIntegrationVersion" + compile "org.springframework:spring-tx:$springVersion" + + testCompile project(":spring-batch-test") + + testCompile "org.apache.activemq:activemq-broker:$activemqVersion" + testCompile "org.apache.activemq:activemq-kahadb-store:$activemqVersion" + testCompile "junit:junit:${junitVersion}" + testCompile "org.hamcrest:hamcrest-library:$hamcrestVersion" + testCompile "org.aspectj:aspectjrt:$aspectjVersion" + testCompile "org.aspectj:aspectjweaver:$aspectjVersion" + testCompile "org.apache.commons:commons-dbcp2:$commonsDdbcpVersion" + testCompile "com.h2database:h2:$h2databaseVersion" + testCompile "mysql:mysql-connector-java:$mysqlVersion" + testCompile "org.apache.derby:derby:$derbyVersion" + testCompile "org.hsqldb:hsqldb:$hsqldbVersion" + testCompile "org.springframework:spring-test:$springVersion" + testCompile "org.mockito:mockito-core:$mockitoVersion" + testCompile("org.springframework.integration:spring-integration-test:$springIntegrationVersion") { + exclude group: 'junit', module: 'junit-dep' + exclude group:'org.hamcrest', module:'hamcrest-all' + } + testCompile "org.springframework.integration:spring-integration-jdbc:$springIntegrationVersion" + + optional "javax.jms:javax.jms-api:$jmsVersion" + optional "org.apache.logging.log4j:log4j-api:$log4jVersion" + optional "org.apache.logging.log4j:log4j-core:$log4jVersion" + optional "org.springframework.integration:spring-integration-jms:$springIntegrationVersion" + optional "org.springframework:spring-jms:$springVersion" + + optional "javax.batch:javax.batch-api:$javaxBatchApiVersion" + + // JSR-305 only used for non-required meta-annotations + compileOnly("com.google.code.findbugs:jsr305:3.0.2") + } +} + +project('spring-batch-samples') { + description = 'Batch Batch Samples' + project.tasks.findByPath("artifactoryPublish")?.enabled = false + + dependencies { + + compile project(":spring-batch-integration") + compile "org.aspectj:aspectjrt:$aspectjVersion" + compile "org.aspectj:aspectjweaver:$aspectjVersion" + compile "org.quartz-scheduler:quartz:$quartzVersion" + compile "commons-io:commons-io:$commonsIoVersion" + compile "org.apache.commons:commons-dbcp2:$commonsDdbcpVersion" + compile "com.thoughtworks.xstream:xstream:$xstreamVersion" + compile("com.fasterxml.woodstox:woodstox-core:$woodstoxVersion") { + exclude group: 'stax', module: 'stax-api' + } + compile("org.hibernate:hibernate-core:$hibernateVersion") { dep -> + optional dep + exclude group: 'org.jboss.spec.javax.transaction', module: 'jboss-transaction-api_1.1_spec' + } + compile("org.hibernate:hibernate-entitymanager:$hibernateVersion") { dep -> + optional dep + exclude group: 'org.jboss.spec.javax.transaction', module: 'jboss-transaction-api_1.1_spec' + } + compile "javax.transaction:javax.transaction-api:$javaxTransactionVersion" + compile "org.springframework:spring-aop:$springVersion" + compile("org.springframework:spring-oxm:$springVersion") { + exclude group: 'commons-lang', module: 'commons-lang' + } + compile "org.springframework:spring-core:$springVersion" + compile "org.springframework:spring-context-support:$springVersion" + compile "org.springframework:spring-jdbc:$springVersion" + compile "org.springframework:spring-orm:$springVersion" + compile "org.springframework:spring-tx:$springVersion" + compile "org.springframework.data:spring-data-jpa:$springDataJpaVersion" + compile "javax.mail:javax.mail-api:$javaMailVersion" + compile "org.apache.activemq:activemq-client:$activemqVersion" + compile "org.apache.activemq:activemq-broker:$activemqVersion" + compile "io.micrometer:micrometer-registry-prometheus:$micrometerVersion" + compile "io.prometheus:simpleclient_pushgateway:$prometheusPushgatewayVersion" + + testCompile "org.xmlunit:xmlunit-core:$xmlunitVersion" + testCompile "org.xmlunit:xmlunit-matchers:$xmlunitVersion" + testCompile project(":spring-batch-test") + testCompile "junit:junit:${junitVersion}" + testCompile "org.hamcrest:hamcrest-library:$hamcrestVersion" + testCompile "org.hsqldb:hsqldb:$hsqldbVersion" + testCompile "org.apache.logging.log4j:log4j-api:$log4jVersion" + testCompile "org.apache.logging.log4j:log4j-core:$log4jVersion" + testCompile "org.codehaus.groovy:groovy:$groovyVersion" + testCompile "org.codehaus.groovy:groovy-ant:$groovyVersion" + testCompile "org.springframework:spring-test:$springVersion" + testCompile "org.mockito:mockito-core:$mockitoVersion" + testCompile "org.apache.activemq:activemq-kahadb-store:$activemqVersion" + + testRuntime "com.sun.mail:javax.mail:$javaMailVersion" + testRuntime "org.hibernate.validator:hibernate-validator:$hibernateValidatorVersion" + testRuntime "org.glassfish:javax.el:$javaxElVersion" + testRuntime "javax.xml.bind:jaxb-api:$jaxbApiVersion" + testRuntime "com.sun.xml.bind:jaxb-core:$jaxbImplVersion" + testRuntime "com.sun.xml.bind:jaxb-impl:$jaxbImplVersion" + + provided "mysql:mysql-connector-java:$mysqlVersion" + provided "com.h2database:h2:$h2databaseVersion" + provided "javax.servlet:javax.servlet-api:$servletApiVersion" + + optional "com.vmware.sqlfire:sqlfireclient:$sqlfireclientVersion" + optional "org.apache.derby:derby:$derbyVersion" + optional "org.postgresql:postgresql:$postgresqlVersion" + optional "org.springframework:spring-web:$springVersion" + optional "org.springframework.data:spring-data-commons:$springDataCommonsVersion" + optional "org.springframework.amqp:spring-amqp:$springAmqpVersion" + optional ("org.springframework.amqp:spring-rabbit:$springAmqpVersion") { + exclude group: "org.springframework", module: "spring-messaging" + exclude group: "org.springframework", module: "spring-web" + } + optional "javax.inject:javax.inject:1" + + optional "javax.batch:javax.batch-api:$javaxBatchApiVersion" + } +} + +apply plugin: "org.asciidoctor.convert" + +asciidoctorj { + version = '1.5.5' +} + +configurations { + docs +} + +dependencies { + docs "io.spring.docresources:spring-doc-resources:${docResourcesVersion}@zip" +} + +task prepareAsciidocBuild(type: Sync) { + dependsOn configurations.docs + // copy doc resources + from { + configurations.docs.collect { zipTree(it) } + } + // and doc sources + from "spring-batch-docs/asciidoc/" + // to a build directory of your choice + into "$buildDir/asciidoc/assemble" +} + +task('makePDF', type: org.asciidoctor.gradle.AsciidoctorTask){ + dependsOn prepareAsciidocBuild + backends 'pdf' + sourceDir "$buildDir/asciidoc/assemble" + sources { + include 'index-single.adoc' + } + options doctype: 'book', eruby: 'erubis' + logDocuments = true + attributes 'icons': 'font', + 'sectanchors': '', + 'sectnums': '', + 'toc': '', + 'source-highlighter' : 'coderay', + revnumber: project.version +} + +asciidoctor { + dependsOn makePDF + backends 'html5' + sourceDir "$buildDir/asciidoc/assemble" + resources { + from(sourceDir) { + include 'images/*', 'css/**', 'js/**' + } + } + options doctype: 'book', eruby: 'erubis' + logDocuments = true + attributes 'docinfo': 'shared', + // use provided stylesheet + stylesdir: "css/", + stylesheet: 'spring.css', + 'linkcss': true, + 'icons': 'font', + 'sectanchors': '', + // use provided highlighter + 'source-highlighter=highlight.js', + 'highlightjsdir=js/highlight', + 'highlightjs-theme=atom-one-dark-reasonable', + 'idprefix': '', + 'idseparator': '-', + 'spring-version': project.version, + 'allow-uri-read': '', + revnumber: project.version +} + + +apply plugin: 'org.sonarqube' + +sonarqube { + properties { + property "sonar.projectName", "Spring Batch" + property "sonar.jacoco.reportPath", "${buildDir.name}/jacoco.exec" + property "sonar.links.homepage", linkHomepage + property "sonar.links.ci", linkCi + property "sonar.links.issue", linkIssue + property "sonar.links.scm", linkScmUrl + property "sonar.links.scm_dev", linkScmDevConnection + property "sonar.java.coveragePlugin", "jacoco" + } +} + +task api(type: Javadoc) { + group = 'Documentation' + description = 'Generates aggregated Javadoc API documentation.' + title = "Spring Batch ${version} API" + options.memberLevel = org.gradle.external.javadoc.JavadocMemberLevel.PROTECTED + options.author = true + options.header = 'Spring Batch' + options.overview = 'src/api/overview.html' + if (JavaVersion.current().isJava8Compatible()) { + options.addStringOption('Xdoclint:none', '-quiet') + } + + source subprojects + .findAll { it.name != 'spring-batch-samples' && !it.name.endsWith("-tests") } + .collect { project -> project.sourceSets.main.allJava } + destinationDir = new File(buildDir, "api") + classpath = files(subprojects.collect { project -> + project.sourceSets.main.compileClasspath + }) +} + +task schemaZip(type: Zip) { + group = 'Distribution' + classifier = 'schema' + description = "Builds -${classifier} archive containing all " + + "XSDs for deployment at static.springframework.org/schema." + + subprojects.each { subproject -> + def Properties schemas = new Properties(); + def shortName = subproject.name.replaceFirst("${rootProject.name}-", '') + if (subproject.name.endsWith("-core")) { + shortName = '' + } + + subproject.sourceSets.main.resources.find { + it.path.endsWith("META-INF${File.separator}spring.schemas") + }?.withInputStream { schemas.load(it) } + + for (def key : schemas.keySet()) { + File xsdFile = subproject.sourceSets.main.resources.find { + it.path.replaceAll('\\\\', '/').endsWith(schemas.get(key)) + } + assert xsdFile != null + into ("batch/${shortName}") { + from xsdFile.path + } + } + } +} + +task docsZip(type: Zip) { + group = 'Distribution' + classifier = 'docs' + description = "Builds -${classifier} archive containing api and reference " + + "for deployment at static.springframework.org/spring-batch/reference." + + from('src/dist') { + include 'changelog.txt' + } + + from (api) { + into 'api' + } + + from (makePDF) { + include "index-single.pdf" + rename 'index-single.pdf', 'spring-batch-reference.pdf' + into 'reference/pdf' + } + + from (asciidoctor) { + include "*.html" + include "css/**" + include "js/**" + include "images/**" + into 'reference/html' + } +} + +task distZip(type: Zip, dependsOn: [docsZip, schemaZip]) { + group = 'Distribution' + classifier = 'dist' + description = "Builds -${classifier} archive, containing all jars and docs, " + + "suitable for community download page." + + ext.baseDir = "${project.name}-${project.version}"; + + from('src/dist') { + include 'readme.txt' + include 'license.txt' + include 'notice.txt' + into "${baseDir}" + expand(copyright: new Date().format("yyyy"), version: project.version) + } + + from(zipTree(docsZip.archivePath)) { + into "${baseDir}/docs" + } + + from(zipTree(schemaZip.archivePath)) { + into "${baseDir}/schema" + } + + [project(':spring-batch-core'), project(':spring-batch-infrastructure'), project(':spring-batch-test'), project(':spring-batch-integration')].each { subproject -> + into ("${baseDir}/libs") { + from subproject.jar + from subproject.sourcesJar + from subproject.javadocJar + } + } +} + +// Create an optional "with dependencies" distribution. +// Not published by default; only for use when building from source. +task depsZip(type: Zip, dependsOn: distZip) { zipTask -> + group = 'Distribution' + classifier = 'dist-with-deps' + description = "Builds -${classifier} archive, containing everything " + + "in the -${distZip.classifier} archive plus all dependencies." + + from zipTree(distZip.archivePath) + + gradle.taskGraph.whenReady { taskGraph -> + if (taskGraph.hasTask(":${zipTask.name}")) { + def projectNames = rootProject.subprojects*.name + def artifacts = new HashSet() + subprojects.each { subproject -> + subproject.configurations.runtime.resolvedConfiguration.resolvedArtifacts.each { artifact -> + def dependency = artifact.moduleVersion.id + if (!projectNames.contains(dependency.name)) { + artifacts << artifact.file + } + } + } + + zipTask.from(artifacts) { + into "${distZip.baseDir}/deps" + } + } + } +} + +artifacts { + archives distZip + archives docsZip + archives schemaZip +} + +task dist(dependsOn: assemble) { + group = 'Distribution' + description = 'Builds -dist, -docs and -schema distribution archives.' +} + +task runTck(dependsOn: subprojects.compileJava) { + + configurations { + tck { + transitive = true + } + antcp { + transitive = true + exclude module: 'ant' + } + } + + dependencies { + tck project(":spring-batch-core") + + tck "commons-pool:commons-pool:$commonsPoolVersion" + tck "javax.batch:javax.batch-api:$javaxBatchApiVersion" + tck "org.springframework:spring-core:$springVersion" + tck "org.springframework:spring-context:$springVersion" + tck "org.springframework:spring-beans:$springVersion" + tck 'commons-logging:commons-logging-api:1.1' + tck "org.springframework:spring-aop:$springVersion" + tck "org.springframework:spring-tx:$springVersion" + tck "org.springframework.retry:spring-retry:$springRetryVersion" + tck "org.hsqldb:hsqldb:$hsqldbVersion" + tck "org.springframework:spring-jdbc:$springVersion" + tck "com.thoughtworks.xstream:xstream:$xstreamVersion" + tck "org.codehaus.jettison:jettison:$jettisonVersion" + tck "org.apache.commons:commons-dbcp2:$commonsDdbcpVersion" + tck "org.apache.derby:derby:$derbyVersion" + + antcp "ant-contrib:ant-contrib:1.0b3" + } + + doLast { + logger.info('tck dependencies: ' + configurations.tck.asPath) + def tckHome = project.hasProperty('TCK_HOME') ? getProperty('TCK_HOME') : System.getenv("JSR_352_TCK_HOME") + + assert tckHome : '''\ +tckHome is not set. Please set either the environment variable 'JSR_352_TCK_HOME' +or specify the Gradle property `TCK_HOME`, e.g: ./gradlew runTck -PTCK_HOME=/path/to/tck''' + + println "Using the JSR 352 TCK at: '$tckHome'" + + ant.taskdef resource: "net/sf/antcontrib/antcontrib.properties", + classpath: configurations.antcp.asPath + ant.properties['batch.impl.classes'] = configurations.tck.asPath + ant.ant antfile: "$tckHome/build.xml", target: "run", dir: "$tckHome" + } +} + +wrapper { + description = 'Generates gradlew[.bat] scripts' + gradleVersion = '4.10.3' +} diff --git a/buildSrc/src/main/groovy/org/springframework/build/gradle/MergePlugin.groovy b/buildSrc/src/main/groovy/org/springframework/build/gradle/MergePlugin.groovy new file mode 100644 index 0000000000..1933dd0fb1 --- /dev/null +++ b/buildSrc/src/main/groovy/org/springframework/build/gradle/MergePlugin.groovy @@ -0,0 +1,159 @@ +/* + * Copyright 2002-2015 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.build.gradle + +import org.gradle.api.Project +import org.gradle.api.Action +import org.gradle.api.Plugin +import org.gradle.api.invocation.Gradle +import org.gradle.api.artifacts.Configuration +import org.gradle.api.artifacts.ProjectDependency +import org.gradle.api.artifacts.maven.Conf2ScopeMapping +import org.gradle.api.plugins.MavenPlugin +import org.gradle.plugins.ide.eclipse.EclipsePlugin +import org.gradle.plugins.ide.idea.IdeaPlugin +/** + * Gradle plugin that allows projects to merged together. Primarily developed to + * allow Spring to support multiple incompatible versions of third-party + * dependencies (for example Hibernate v3 and v4). + *

+ * The 'merge' extension should be used to define how projects are merged, for example: + *

+ * configure(subprojects) {
+ *     apply plugin: MergePlugin
+ * }
+ *
+ * project("myproject") {
+ * }
+ *
+ * project("myproject-extra") {
+ *     merge.into = project("myproject")
+ * }
+ * 
+ *

+ * This plugin adds two new configurations: + *

    + *
  • merging - Contains the projects being merged into this project
  • + *
  • runtimeMerge - Contains all dependencies that are merge projects. These are used + * to allow an IDE to reference merge projects.
  • + *
      + * + * @author Rob Winch + * @author Phillip Webb + */ +class MergePlugin implements Plugin { + + private static boolean attachedProjectsEvaluated; + + public void apply(Project project) { + project.plugins.apply(MavenPlugin) + project.plugins.apply(EclipsePlugin) + project.plugins.apply(IdeaPlugin) + + MergeModel model = project.extensions.create("merge", MergeModel) + project.configurations.create("merging") + Configuration runtimeMerge = project.configurations.create("runtimeMerge") + + // Ensure the IDE can reference merged projects + project.eclipse.classpath.plusConfigurations += [ runtimeMerge ] + project.idea.module.scopes.PROVIDED.plus += [ runtimeMerge ] + + // Hook to perform the actual merge logic + project.afterEvaluate{ + if (it.merge.into != null) { + setup(it) + } + } + + // Hook to build runtimeMerge dependencies + if (!attachedProjectsEvaluated) { + project.gradle.projectsEvaluated{ + postProcessProjects(it) + } + attachedProjectsEvaluated = true; + } + } + + private void setup(Project project) { + project.merge.into.dependencies.add("merging", project) + project.dependencies.add("provided", project.merge.into.sourceSets.main.output) + project.dependencies.add("runtimeMerge", project.merge.into) + setupTaskDependencies(project) + setupMaven(project) + } + + private void setupTaskDependencies(Project project) { + // invoking a task will invoke the task with the same name on 'into' project + ["sourcesJar", "jar", "javadocJar", "javadoc", "install", "artifactoryPublish"].each { + def task = project.tasks.findByPath(it) + if (task) { + task.enabled = false + task.dependsOn(project.merge.into.tasks.findByPath(it)) + } + } + + // update 'into' project artifacts to contain the source artifact contents + project.merge.into.sourcesJar.from(project.sourcesJar.source) + project.merge.into.jar.from(project.sourceSets.main.output) + project.merge.into.javadoc { + source += project.javadoc.source + classpath += project.javadoc.classpath + } + } + + private void setupMaven(Project project) { + project.configurations.each { configuration -> + Conf2ScopeMapping mapping = project.conf2ScopeMappings.getMapping([configuration]) + if (mapping.scope) { + Configuration intoConfiguration = project.merge.into.configurations.create( + project.name + "-" + configuration.name) + configuration.excludeRules.each { + configuration.exclude([ + (ExcludeRule.GROUP_KEY) : it.group, + (ExcludeRule.MODULE_KEY) : it.module]) + } + configuration.dependencies.each { + def intoCompile = project.merge.into.configurations.getByName("compile") + // Protect against changing a compile scope dependency (SPR-10218) + if (!intoCompile.dependencies.contains(it)) { + intoConfiguration.dependencies.add(it) + } + } + def index = project.parent.childProjects.findIndexOf {p -> p.getValue() == project} + project.merge.into.install.repositories.mavenInstaller.pom.scopeMappings.addMapping( + mapping.priority + 100 + index, intoConfiguration, mapping.scope) + } + } + } + + private postProcessProjects(Gradle gradle) { + gradle.rootProject.subprojects(new Action() { + public void execute(Project project) { + project.configurations.getByName("runtime").allDependencies.withType(ProjectDependency).each{ + Configuration dependsOnMergedFrom = it.dependencyProject.configurations.getByName("merging"); + dependsOnMergedFrom.dependencies.each{ dep -> + project.dependencies.add("runtimeMerge", dep.dependencyProject) + } + } + } + }); + } +} + +class MergeModel { + Project into; +} \ No newline at end of file diff --git a/buildSrc/src/main/resources/META-INF/gradle-plugins/merge.properties b/buildSrc/src/main/resources/META-INF/gradle-plugins/merge.properties new file mode 100644 index 0000000000..9cef804165 --- /dev/null +++ b/buildSrc/src/main/resources/META-INF/gradle-plugins/merge.properties @@ -0,0 +1 @@ +implementation-class=org.springframework.build.gradle.MergePlugin diff --git a/dictionary.txt b/dictionary.txt index 63e4bda797..9aba8f63e9 100644 --- a/dictionary.txt +++ b/dictionary.txt @@ -99,3 +99,215 @@ accessors subclassing ajax javascript +chris +schaefer +unescaped +namespace +autowired +registrar +ref +configurer +callable +classpath +config +registries +stephane +nicoll +href +fischer +sebastien +gerard +xml +gt +garrette +thomas +risberg +josh +preconfigured +lob +hydrated +hydrate +subflow +validator +subflows +wildcard +foo +wildcards +pre +morten +andersen +gott +validators +enum +init +elvis +refactored +autowire +batchlet +upfront +glenn +renfro +arg +chunking +spec +honour +ids +exiter +stdin +stijn +maller +schipp +javadocs +prioritising +recognise +summarise +summarising +analyse +no-op +multicaster +optimisation +deserializes +david +turanski +serializer +multi +charsets +jettison +deserialization +jimmy +praet +behaviour +unregistration +initialised +hydrating +aop +analysed +ignorable +tarred +generics +memorise +modularise +skippability +panicking +eric +evans +deserialize +wonky +rawtypes +internet +inline +foobar +sergey +shcherbakov +jsr +blah +txt +spam +punk +min +downgrade +weirdnesses +varchar +warmup +tobias +flohre +backstop +timestamp +recognised +attrs +ouch +tripped +barf +baz +phil +webb +costin +leau +hsql +vs +payload +mongo +cypher +iterable +anatoly +polinsky +sybase +thexton +postgres +luke +taylor +rowid +num +externalized +juergen +hoeller +capitalised +capitalisation +tokenization +tomas +slanina +tokenizing +alef +arjen +arendsen +poutsma +levenshtein +unset +unparseable +tokenizers +parseable +regex +customisation +waseem +malik +synced +charset +juliusz +brzostek +marshallers +unmarshaller +marshaller +uri +namespaces +xmlns +woodstox +inits +marshalled +precache +douglas +kaminsky +endeavour +uncategorised +specialisation +tenuous +checkpointing +mutex +signalling +customise +rethrowing +queueing +runnables +resourceless +initialisation +args +params +delegator +val +asc +desc +unmarshalling +unmarshallers +resettable +dataset +signalled +injectable +asynch +flakey +usecase +tokenize +debit +isin +fieldsets +mockito +mappers +adhoc diff --git a/gradle.properties b/gradle.properties new file mode 100644 index 0000000000..6acf520a9d --- /dev/null +++ b/gradle.properties @@ -0,0 +1 @@ +version=4.2.0.RELEASE diff --git a/gradle/wrapper/gradle-wrapper.jar b/gradle/wrapper/gradle-wrapper.jar new file mode 100644 index 0000000000..28861d273a Binary files /dev/null and b/gradle/wrapper/gradle-wrapper.jar differ diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties new file mode 100644 index 0000000000..ae45383b6d --- /dev/null +++ b/gradle/wrapper/gradle-wrapper.properties @@ -0,0 +1,5 @@ +distributionBase=GRADLE_USER_HOME +distributionPath=wrapper/dists +distributionUrl=https\://services.gradle.org/distributions/gradle-4.10.3-all.zip +zipStoreBase=GRADLE_USER_HOME +zipStorePath=wrapper/dists diff --git a/gradlew b/gradlew new file mode 100755 index 0000000000..4453ccea33 --- /dev/null +++ b/gradlew @@ -0,0 +1,172 @@ +#!/usr/bin/env sh + +############################################################################## +## +## Gradle start up script for UN*X +## +############################################################################## + +# Attempt to set APP_HOME +# Resolve links: $0 may be a link +PRG="$0" +# Need this for relative symlinks. +while [ -h "$PRG" ] ; do + ls=`ls -ld "$PRG"` + link=`expr "$ls" : '.*-> \(.*\)$'` + if expr "$link" : '/.*' > /dev/null; then + PRG="$link" + else + PRG=`dirname "$PRG"`"/$link" + fi +done +SAVED="`pwd`" +cd "`dirname \"$PRG\"`/" >/dev/null +APP_HOME="`pwd -P`" +cd "$SAVED" >/dev/null + +APP_NAME="Gradle" +APP_BASE_NAME=`basename "$0"` + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS="" + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD="maximum" + +warn ( ) { + echo "$*" +} + +die ( ) { + echo + echo "$*" + echo + exit 1 +} + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "`uname`" in + CYGWIN* ) + cygwin=true + ;; + Darwin* ) + darwin=true + ;; + MINGW* ) + msys=true + ;; + NONSTOP* ) + nonstop=true + ;; +esac + +CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD="$JAVA_HOME/jre/sh/java" + else + JAVACMD="$JAVA_HOME/bin/java" + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD="java" + which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." +fi + +# Increase the maximum file descriptors if we can. +if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then + MAX_FD_LIMIT=`ulimit -H -n` + if [ $? -eq 0 ] ; then + if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then + MAX_FD="$MAX_FD_LIMIT" + fi + ulimit -n $MAX_FD + if [ $? -ne 0 ] ; then + warn "Could not set maximum file descriptor limit: $MAX_FD" + fi + else + warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT" + fi +fi + +# For Darwin, add options to specify how the application appears in the dock +if $darwin; then + GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\"" +fi + +# For Cygwin, switch paths to Windows format before running java +if $cygwin ; then + APP_HOME=`cygpath --path --mixed "$APP_HOME"` + CLASSPATH=`cygpath --path --mixed "$CLASSPATH"` + JAVACMD=`cygpath --unix "$JAVACMD"` + + # We build the pattern for arguments to be converted via cygpath + ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null` + SEP="" + for dir in $ROOTDIRSRAW ; do + ROOTDIRS="$ROOTDIRS$SEP$dir" + SEP="|" + done + OURCYGPATTERN="(^($ROOTDIRS))" + # Add a user-defined pattern to the cygpath arguments + if [ "$GRADLE_CYGPATTERN" != "" ] ; then + OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)" + fi + # Now convert the arguments - kludge to limit ourselves to /bin/sh + i=0 + for arg in "$@" ; do + CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -` + CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option + + if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition + eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"` + else + eval `echo args$i`="\"$arg\"" + fi + i=$((i+1)) + done + case $i in + (0) set -- ;; + (1) set -- "$args0" ;; + (2) set -- "$args0" "$args1" ;; + (3) set -- "$args0" "$args1" "$args2" ;; + (4) set -- "$args0" "$args1" "$args2" "$args3" ;; + (5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;; + (6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;; + (7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;; + (8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;; + (9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;; + esac +fi + +# Escape application args +save ( ) { + for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done + echo " " +} +APP_ARGS=$(save "$@") + +# Collect all arguments for the java command, following the shell quoting and substitution rules +eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS" + +# by default we should be in the correct project dir, but when run from Finder on Mac, the cwd is wrong +if [ "$(uname)" = "Darwin" ] && [ "$HOME" = "$PWD" ]; then + cd "$(dirname "$0")" +fi + +exec "$JAVACMD" "$@" diff --git a/gradlew.bat b/gradlew.bat new file mode 100755 index 0000000000..e95643d6a2 --- /dev/null +++ b/gradlew.bat @@ -0,0 +1,84 @@ +@if "%DEBUG%" == "" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables with windows NT shell +if "%OS%"=="Windows_NT" setlocal + +set DIRNAME=%~dp0 +if "%DIRNAME%" == "" set DIRNAME=. +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS= + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if "%ERRORLEVEL%" == "0" goto init + +echo. +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto init + +echo. +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:init +@rem Get command-line arguments, handling Windows variants + +if not "%OS%" == "Windows_NT" goto win9xME_args + +:win9xME_args +@rem Slurp the command line arguments. +set CMD_LINE_ARGS= +set _SKIP=2 + +:win9xME_args_slurp +if "x%~1" == "x" goto execute + +set CMD_LINE_ARGS=%* + +:execute +@rem Setup the command line + +set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar + +@rem Execute Gradle +"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %CMD_LINE_ARGS% + +:end +@rem End local scope for the variables with windows NT shell +if "%ERRORLEVEL%"=="0" goto mainEnd + +:fail +rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of +rem the _cmd.exe /c_ return code! +if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1 +exit /b 1 + +:mainEnd +if "%OS%"=="Windows_NT" endlocal + +:omega diff --git a/pom.xml b/pom.xml deleted file mode 100644 index a0355621f4..0000000000 --- a/pom.xml +++ /dev/null @@ -1,652 +0,0 @@ - - - 4.0.0 - org.springframework.batch - spring-batch - Spring Batch - Spring Batch provides tools for enterprise batch or bulk processing. It can be used to wire up jobs, and track - their execution, or simply as an optimization for repetitive processing in a transactional environment. Spring Batch is part of the Spring Portfolio. - 2.2.2.BUILD-SNAPSHOT - pom - - spring-batch-parent - spring-batch-infrastructure - spring-batch-core - spring-batch-test - - http://www.springsource.org/spring-batch - - SpringSource - http://www.springsource.com - - - http://github.com/SpringSource/spring-batch - scm:git:git://github.com/SpringSource/spring-batch.git - scm:git:ssh://git@github.com/SpringSource/spring-batch.git - HEAD - - - JIRA - http://opensource.atlassian.com/projects/spring/browse/BATCH - - - - Spring Batch Forum - http://forum.springframework.org/forumdisplay.php?f=41 - http://forum.springframework.org/forumdisplay.php?f=41 - - - - Bamboo - https://build.springframework.org/bamboo/browse/BATCH - - - - Apache 2.0 - http://www.apache.org/licenses/LICENSE-2.0.txt - - - - UTF-8 - false - spring-batch - Spring Batch - BATCH - ${project.version} - ${dist.id}-${dist.version}-no-dependencies - ${dist.finalName}.zip - target/${dist.fileName} - dist.springframework.org - - - - all - - spring-batch-samples - archetypes - spring-batch-infrastructure-tests - spring-batch-core-tests - - - - release - - spring-batch-samples - archetypes - - - - - maven-assembly-plugin - false - - - zip-files - false - package - - single - - - - src/assembly/no-dependencies.xml - - - - - - - com.agilejava.docbkx - docbkx-maven-plugin - 2.0.8 - - - single-page - - generate-html - - - ${basedir}/src/docbkx/resources/xsl/html.xsl - - - - - - - - - - - - - - - - - - - - - - - - pre-site - - - single-pdf - - generate-pdf - - - src/site/docbook/reference/ - src/docbkx/resources/images/ - - - - - pre-site - - - multi-page - - generate-html - - - true - ${basedir}/src/docbkx/resources/xsl/html_chunk.xsl - - - - - - - - - - - - - - - - - - - - - - - - pre-site - - - - - org.docbook - docbook-xml - 4.4 - runtime - - - org.apache.xmlgraphics - fop - 0.93 - - - - index.xml - false - - css/html.css - ${basedir}/src/site/docbook/reference - ${basedir}/src/docbkx/resources/xsl/fopdf.xsl - true - - - version - ${project.version} - - - - - - org.apache.maven.plugins - maven-antrun-plugin - - - copy-parent-pom - generate-sources - - - - - - - run - - - - upload-dist - deploy - - run - - - - - - - - - - - - - - - - verify - - false - - - - - org.apache.maven.plugins - maven-antrun-plugin - - - extract - verify - - run - - - - - - - - - - - - - - - - - - - - - - - - - - org.apache.maven.plugins - maven-invoker-plugin - - target/it/no-dep - target/local-repo - true - ${basedir}/src/it/settings.xml - - test - - - samples/*/pom.xml - - - - - integration-test - verify - - run - - - - - - - - - build - - false - - - - - - org.apache.maven.plugins - maven-invoker-plugin - - ${basedir} - target/local-repo - true - ${basedir}/src/it/settings.xml - - clean - install - - - pom.xml - - - - - integration-test - verify - - run - - - - - - - - - samples - - spring-batch-samples - archetypes - - - - milestone - - - spring-milestone - Spring Milestone Repository - s3://maven.springframework.org/milestone - - - - - central - - - sonatype-nexus-snapshots - Sonatype Nexus Snapshots - https://oss.sonatype.org/content/repositories/snapshots/ - - - sonatype-nexus-staging - Nexus Release Repository - https://oss.sonatype.org/service/local/staging/deploy/maven2/ - - - - - staging - - /${user.dir}/target/staging/dist - - - - staging - file:///${user.dir}/target/staging - - - staging - file:///${user.dir}/target/staging - - - staging - file:///${user.dir}/target/staging - - - - - - org.apache.maven.plugins - maven-deploy-plugin - - false - - - - - ${dist.finalName} - - - - manifest - - false - - - - - com.springsource.bundlor - com.springsource.bundlor.maven - - - bundlor-manifest - package - - manifest - - - - - - - - - - - static.springframework.org - scp://static.springframework.org/var/www/domains/springframework.org/static/htdocs/spring-batch/trunk - - - spring-release - Spring Release Repository - s3://maven.springframework.org/release - - - spring-snapshot - Spring Snapshot Repository - s3://maven.springframework.org/snapshot - - - - - - org.springframework.build.aws - org.springframework.build.aws.maven - 3.0.0.RELEASE - - - - - - org.apache.maven.plugins - maven-release-plugin - 2.3 - - - org.apache.maven.plugins - maven-site-plugin - 2.2 - - - org.apache.maven.wagon - wagon-ssh - 1.0-beta-7 - - - - - org.apache.maven.plugins - maven-antrun-plugin - 1.4 - - - org.springframework.build - org.springframework.build.aws.ant - 3.0.5.RELEASE - - - net.java.dev.jets3t - jets3t - 0.7.2 - - - org.apache.ant - ant - 1.7.0 - - - org.apache.ant - ant-apache-regexp - 1.7.0 - - - - - org.apache.maven.plugins - maven-surefire-plugin - - junit:junit - - - - - - - org.apache.maven.plugins - maven-deploy-plugin - 2.7 - - true - - - - - - true - - - org.apache.maven.plugins - maven-project-info-reports-plugin - 2.1.1 - - - org.apache.maven.plugins - maven-surefire-report-plugin - 2.3 - - - org.codehaus.mojo - jdepend-maven-plugin - 2.0-beta-2 - - - maven-javadoc-plugin - 2.8.1 - - - html - - javadoc - - - - - true - true - - http://download.oracle.com/javaee/5/api - http://download.oracle.com/javase/1.5.0/docs/api - http://jakarta.apache.org/commons/collections/apidocs-COLLECTIONS_3_0/ - http://jakarta.apache.org/commons/dbcp/apidocs/ - http://jakarta.apache.org/commons/fileupload/apidocs/ - http://jakarta.apache.org/commons/httpclient/apidocs/ - http://jakarta.apache.org/commons/pool/apidocs/ - http://jakarta.apache.org/commons/logging/apidocs/ - http://junit.sourceforge.net/javadoc/ - http://logging.apache.org/log4j/docs/api/ - http://jakarta.apache.org/regexp/apidocs/ - http://jakarta.apache.org/velocity/api/ - http://static.springsource.org/spring/docs/3.1.2.RELEASE/javadoc-api/ - http://static.springframework.org/spring-batch/apidocs/ - http://static.springframework.org/spring-ws/site/apidocs/ - - test.*:*.example:*.sample - - - - org.apache.maven.plugins - maven-jxr-plugin - 2.3 - - - - - - dsyer - Dave Syer - david.syer@springsource.com - - - nebhale - Ben Hale - ben.hale@springsource.com - 0 - - - lward - Lucas Ward - lucas.l.ward@accenture.com - - - robokaso - Robert Kasanicky - robokaso@gmail.com - - - trisberg - Thomas Risberg - thomas.risberg@springsource.com - - - dhgarrette - Dan Garrette - dhgarrette@gmail.com - - - mminella - Michael Minella - mminella@vmware.com - - - - - com.springsource.repository.bundles.milestone - SpringSource Enterprise Bundle Repository - SpringSource Bundle Milestones - http://repository.springsource.com/maven/bundles/milestone - - false - - - - com.springsource.repository.bundles.snapshot - SpringSource Enterprise Bundle Repository - SpringSource Bundle Snapshots - http://repository.springsource.com/maven/bundles/snapshot - - - com.springsource.repository.bundles.release - SpringSource Enterprise Bundle Repository - SpringSource Bundle Releases - http://repository.springsource.com/maven/bundles/release - - false - - - - diff --git a/publish-maven.gradle b/publish-maven.gradle new file mode 100644 index 0000000000..6c3625aa0d --- /dev/null +++ b/publish-maven.gradle @@ -0,0 +1,94 @@ +apply plugin: "propdeps-maven" + +install { + repositories.mavenInstaller { + customizePom(pom, project) + } +} + +def customizePom(pom, gradleProject) { + pom.whenConfigured { generatedPom -> + + // eliminate test-scoped dependencies (no need in maven central poms) + generatedPom.dependencies.removeAll { dep -> + dep.scope == 'test' + } + + // sort to make pom dependencies order consistent to ease comparison of older poms + generatedPom.dependencies = generatedPom.dependencies.sort { dep -> + "$dep.scope:$dep.groupId:$dep.artifactId" + } + + // add all items necessary for maven central publication + generatedPom.project { + name = gradleProject.description + description = gradleProject.description + url = linkHomepage + organization { + name = 'Spring' + url = 'https://spring.io/' + } + licenses { + license { + name 'The Apache Software License, Version 2.0' + url 'https://www.apache.org/licenses/LICENSE-2.0.txt' + distribution 'repo' + } + } + + scm { + url = linkScmUrl + connection = 'scm:git:' + linkScmConnection + developerConnection = 'scm:git:' + linkScmDevConnection + } + + developers { + developer { + id = 'dsyer' + name = 'Dave Syer' + email = 'dsyer@pivotal.io' + } + developer { + id = 'nebhale' + name = 'Ben Hale' + email = 'bhale@gpivotal.io' + } + developer { + id = 'lward' + name = 'Lucas Ward' + } + developer { + id = 'robokaso' + name = 'Robert Kasanicky' + email = 'robokaso@gmail.com' + } + developer { + id = 'trisberg' + name = 'Thomas Risberg' + email = 'trisberg@pivotal.io' + } + developer { + id = 'dhgarrette' + name = 'Dan Garrette' + email = 'dhgarrette@gmail.com' + } + developer { + id = 'mminella' + name = 'Michael Minella' + email = 'mminella@pivotal.io' + roles = ["project lead"] + } + developer { + id = 'chrisjs' + name = 'Chris Schaefer' + email = 'cschaefer@pivotal.io' + } + developer { + id = 'benas' + name = 'Mahmoud Ben Hassine' + email = 'mbenhassine@pivotal.io' + } + } + } + } +} diff --git a/settings.gradle b/settings.gradle new file mode 100644 index 0000000000..8cf9aadd38 --- /dev/null +++ b/settings.gradle @@ -0,0 +1,13 @@ +rootProject.name = 'spring-batch' + +include 'spring-batch-core' +include 'spring-batch-core-tests' +include 'spring-batch-docs' +include 'spring-batch-infrastructure' +include 'spring-batch-infrastructure-tests' +include 'spring-batch-test' +include 'spring-batch-integration' +include 'spring-batch-samples' + +include "buildSrc" +rootProject.children.find{ it.name == "buildSrc" }.name = "spring-build-src" diff --git a/spring-batch-core-tests/.springBeans b/spring-batch-core-tests/.springBeans index 5f141c431d..b4afd61c9f 100644 --- a/spring-batch-core-tests/.springBeans +++ b/spring-batch-core-tests/.springBeans @@ -1,240 +1,299 @@ - - - 1 - - - - - - - src/test/resources/data-source-context.xml - src/test/resources/simple-job-launcher-context.xml - src/test/resources/org/springframework/batch/core/test/step/SplitJobMapRepositoryIntegrationTests-context.xml - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - - true - false - - - - - + + + 1 + + + + + + + src/test/resources/data-source-context.xml + src/test/resources/simple-job-launcher-context.xml + src/test/resources/org/springframework/batch/core/test/step/SplitJobMapRepositoryIntegrationTests-context.xml + src/main/resources/META-INF/batch/footballJob.xml + src/main/resources/META-INF/batch/footballSkipJob.xml + src/main/resources/META-INF/batch/parallelJob.xml + src/main/resources/META-INF/batch/timeoutJob.xml + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + + true + false + + + + + + + diff --git a/spring-batch-core-tests/pom.xml b/spring-batch-core-tests/pom.xml deleted file mode 100644 index 67f418a9df..0000000000 --- a/spring-batch-core-tests/pom.xml +++ /dev/null @@ -1,172 +0,0 @@ - - - 4.0.0 - spring-batch-core-tests - Core Tests - Integration tests for the Spring Batch Core - http://static.springframework.org/spring-batch/${project.artifactId} - - org.springframework.batch - spring-batch-parent - 2.2.2.BUILD-SNAPSHOT - ../spring-batch-parent - - - - default - - true - - - - - org.apache.maven.plugins - maven-surefire-plugin - - true - - - - - - - test - - - - org.apache.maven.plugins - maven-surefire-plugin - - - - job.commit.interval - 100 - - - games.file.name - games.csv - - - player.file.name - player.csv - - - ENVIRONMENT - ${environment} - - - - - - - - - derby - - derby - - - - - - org.springframework.batch - spring-batch-core - ${project.version} - - - commons-dbcp - commons-dbcp - - - org.hsqldb - hsqldb - - - mysql - mysql-connector-java - 5.1.6 - runtime - - - postgresql - postgresql - 8.3-603.jdbc3 - true - runtime - - - commons-io - commons-io - test - - - org.apache.derby - derby - - - junit - junit - - - - - - - log4j - log4j - test - - - org.springframework - spring-jdbc - - - org.springframework.retry - spring-retry - - - org.springframework - spring-test - test - - - org.springframework - spring-tx - - - org.springframework - spring-aop - - - org.aspectj - aspectjrt - true - - - org.aspectj - aspectjweaver - true - - - cglib - cglib-nodep - true - - - - - - - org.apache.maven.plugins - maven-surefire-plugin - - junit:junit - - - - - - - hsql - - diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/Game.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/Game.java index 1f74b2c581..2624ae6b5a 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/Game.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/Game.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,6 +18,7 @@ import java.io.Serializable; +@SuppressWarnings("serial") public class Game implements Serializable { private String id; @@ -260,6 +261,7 @@ public void setTotalTd(int totalTd) { this.totalTd = totalTd; } + @Override public String toString() { return "Game: ID=" + id + " " + team + " vs. " + opponent + " - " + year; diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/Player.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/Player.java index 5314be5bae..53616e6437 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/Player.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/Player.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,6 +18,7 @@ import java.io.Serializable; +@SuppressWarnings("serial") public class Player implements Serializable { private String id; @@ -27,6 +28,7 @@ public class Player implements Serializable { private int birthYear; private int debutYear; + @Override public String toString() { return "PLAYER:id=" + id + ",Last Name=" + lastName + diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/PlayerDao.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/PlayerDao.java index 5fedace708..7ca29d0d3d 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/PlayerDao.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/PlayerDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/PlayerSummary.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/PlayerSummary.java index 2d906a5dd8..089623564f 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/PlayerSummary.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/football/PlayerSummary.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -113,6 +113,7 @@ public void setTotalTd(int totalTd) { } + @Override public String toString() { return "Player Summary: ID=" + id + " Year=" + year + "[" + completes + ";" + attempts + ";" + passingYards + ";" + passingTd + ";" + interceptions + ";" + rushes + ";" + rushYards + ";" + receptions + diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/timeout/LoggingItemWriter.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/timeout/LoggingItemWriter.java new file mode 100644 index 0000000000..fe2a6d3b7b --- /dev/null +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/timeout/LoggingItemWriter.java @@ -0,0 +1,33 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.test.timeout; + +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.item.ItemWriter; + +public class LoggingItemWriter implements ItemWriter { + + protected Log logger = LogFactory.getLog(LoggingItemWriter.class); + + @Override + public void write(List items) throws Exception { + logger.info(items); + } + +} diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/timeout/SleepingItemProcessor.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/timeout/SleepingItemProcessor.java new file mode 100644 index 0000000000..e801bf37f7 --- /dev/null +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/timeout/SleepingItemProcessor.java @@ -0,0 +1,36 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.test.timeout; + +import org.springframework.batch.item.ItemProcessor; +import org.springframework.lang.Nullable; + +public class SleepingItemProcessor implements ItemProcessor { + + private long millisToSleep; + + @Nullable + @Override + public I process(I item) throws Exception { + Thread.sleep(millisToSleep); + return item; + } + + public void setMillisToSleep(long millisToSleep) { + this.millisToSleep = millisToSleep; + } + +} diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/timeout/SleepingTasklet.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/timeout/SleepingTasklet.java new file mode 100644 index 0000000000..037ea7ee3d --- /dev/null +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/core/test/timeout/SleepingTasklet.java @@ -0,0 +1,40 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.test.timeout; + +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; + +public class SleepingTasklet implements Tasklet { + + private long millisToSleep; + + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, + ChunkContext chunkContext) throws Exception { + Thread.sleep(millisToSleep); + return RepeatStatus.FINISHED; + } + + public void setMillisToSleep(long millisToSleep) { + this.millisToSleep = millisToSleep; + } + +} diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/FootballExceptionHandler.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/FootballExceptionHandler.java index a1ff7afcbb..420dd450d5 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/FootballExceptionHandler.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/FootballExceptionHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,6 +26,7 @@ public class FootballExceptionHandler implements ExceptionHandler { private static final Log logger = LogFactory .getLog(FootballExceptionHandler.class); + @Override public void handleException(RepeatContext context, Throwable throwable) throws Throwable { diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/GameFieldSetMapper.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/GameFieldSetMapper.java index 197d4f5251..e2b767ff81 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/GameFieldSetMapper.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/GameFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ public class GameFieldSetMapper implements FieldSetMapper { + @Override public Game mapFieldSet(FieldSet fs) { if(fs == null){ diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcGameDao.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcGameDao.java index 2e8e48ef41..a3b76bac75 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcGameDao.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcGameDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ public class JdbcGameDao extends JdbcDaoSupport implements ItemWriter { private SimpleJdbcInsert insertGame; + @Override protected void initDao() throws Exception { super.initDao(); insertGame = new SimpleJdbcInsert(getDataSource()).withTableName("GAMES").usingColumns("player_id", "year_no", @@ -36,6 +37,7 @@ protected void initDao() throws Exception { "rushes", "rush_yards", "receptions", "receptions_yards", "total_td"); } + @Override public void write(List games) { for (Game game : games) { diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerDao.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerDao.java index 281f346b15..f712752e66 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerDao.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,7 +35,8 @@ public class JdbcPlayerDao implements PlayerDao { private NamedParameterJdbcTemplate namedParameterJdbcTemplate; - public void savePlayer(Player player) { + @Override + public void savePlayer(Player player) { namedParameterJdbcTemplate.update(INSERT_PLAYER, new BeanPropertySqlParameterSource(player)); } diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerSummaryDao.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerSummaryDao.java index beb8bb40e7..f7e19a8c36 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerSummaryDao.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerSummaryDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,7 +34,8 @@ public class JdbcPlayerSummaryDao implements ItemWriter { private NamedParameterJdbcTemplate namedParameterJdbcTemplate; - public void write(List summaries) { + @Override + public void write(List summaries) { for (PlayerSummary summary : summaries) { diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerFieldSetMapper.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerFieldSetMapper.java index 6b40468b4e..b3203d9193 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerFieldSetMapper.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ public class PlayerFieldSetMapper implements FieldSetMapper { + @Override public Player mapFieldSet(FieldSet fs) { if(fs == null){ diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerItemWriter.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerItemWriter.java index 13eda4b09f..088e85382f 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerItemWriter.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,6 +26,7 @@ public class PlayerItemWriter implements ItemWriter { private PlayerDao playerDao; + @Override public void write(List players) throws Exception { for (Player player : players) { playerDao.savePlayer(player); diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryMapper.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryMapper.java index ae14ccef01..eaade25023 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryMapper.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryMapper.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,19 +19,21 @@ import java.sql.SQLException; import org.springframework.batch.core.test.football.PlayerSummary; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; /** - * RowMapper used to map a ResultSet to a {@link PlayerSummary} + * RowMapper used to map a ResultSet to a {@link org.springframework.batch.core.test.football.PlayerSummary} * * @author Lucas Ward + * @author Mahmoud Ben Hassine * */ -public class PlayerSummaryMapper implements ParameterizedRowMapper { +public class PlayerSummaryMapper implements RowMapper { /* (non-Javadoc) * @see org.springframework.jdbc.core.RowMapper#mapRow(java.sql.ResultSet, int) */ + @Override public PlayerSummary mapRow(ResultSet rs, int rowNum) throws SQLException { PlayerSummary summary = new PlayerSummary(); diff --git a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryRowMapper.java b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryRowMapper.java index 11cf08b72b..e49cd22645 100644 --- a/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryRowMapper.java +++ b/spring-batch-core-tests/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryRowMapper.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,17 +22,19 @@ import org.springframework.jdbc.core.RowMapper; /** - * RowMapper used to map a ResultSet to a {@link PlayerSummary} + * RowMapper used to map a ResultSet to a {@link org.springframework.batch.core.test.football.PlayerSummary} * * @author Lucas Ward + * @author Mahmoud Ben Hassine * */ -public class PlayerSummaryRowMapper implements RowMapper { +public class PlayerSummaryRowMapper implements RowMapper { /* (non-Javadoc) * @see org.springframework.jdbc.core.RowMapper#mapRow(java.sql.ResultSet, int) */ - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { + @Override + public PlayerSummary mapRow(ResultSet rs, int rowNum) throws SQLException { PlayerSummary summary = new PlayerSummary(); diff --git a/spring-batch-core-tests/src/main/resources/META-INF/batch/footballJob.xml b/spring-batch-core-tests/src/main/resources/META-INF/batch/footballJob.xml index 8d45077b45..ba8e595b32 100644 --- a/spring-batch-core-tests/src/main/resources/META-INF/batch/footballJob.xml +++ b/spring-batch-core-tests/src/main/resources/META-INF/batch/footballJob.xml @@ -3,9 +3,9 @@ xmlns:aop="http://www.springframework.org/schema/aop" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/aop%20http://www.springframework.org/schema/aop/spring-aop-3.1.xsd-http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/aop%20https://www.springframework.org/schema/aop/spring-aop-3.1.xsd+http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core-tests/src/main/resources/META-INF/batch/footballSkipJob.xml b/spring-batch-core-tests/src/main/resources/META-INF/batch/footballSkipJob.xml index e6e0f7c737..97942bc196 100644 --- a/spring-batch-core-tests/src/main/resources/META-INF/batch/footballSkipJob.xml +++ b/spring-batch-core-tests/src/main/resources/META-INF/batch/footballSkipJob.xml @@ -1,9 +1,9 @@ + xsi:schemaLocation="http://www.springframework.org/schema/aop%20https://www.springframework.org/schema/aop/spring-aop-3.1.xsd+http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core-tests/src/main/resources/META-INF/batch/parallelJob.xml b/spring-batch-core-tests/src/main/resources/META-INF/batch/parallelJob.xml index 76de2415fe..37570802d7 100644 --- a/spring-batch-core-tests/src/main/resources/META-INF/batch/parallelJob.xml +++ b/spring-batch-core-tests/src/main/resources/META-INF/batch/parallelJob.xml @@ -3,9 +3,9 @@ xmlns:aop="http://www.springframework.org/schema/aop" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/aop%20http://www.springframework.org/schema/aop/spring-aop-3.1.xsd-http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/aop%20https://www.springframework.org/schema/aop/spring-aop-3.1.xsd+http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core-tests/src/main/resources/META-INF/batch/timeoutJob.xml b/spring-batch-core-tests/src/main/resources/META-INF/batch/timeoutJob.xml new file mode 100644 index 0000000000..01786d89ce --- /dev/null +++ b/spring-batch-core-tests/src/main/resources/META-INF/batch/timeoutJob.xml @@ -0,0 +1,55 @@ + + + + + + + + + + + + + + + + + + + + + + + + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core-tests/src/main/resources/log4j.properties b/spring-batch-core-tests/src/main/resources/log4j.properties index f8c8855ebc..ab37ea0326 100644 --- a/spring-batch-core-tests/src/main/resources/log4j.properties +++ b/spring-batch-core-tests/src/main/resources/log4j.properties @@ -1,7 +1,7 @@ log4j.rootCategory=INFO, stdout -log4j.appender.stdout=org.apache.log4j.ConsoleAppender -log4j.appender.stdout.layout=org.apache.log4j.PatternLayout +log4j.appender.stdout=org.apache.logging.log4j.core.appender.ConsoleAppender +log4j.appender.stdout.layout=org.apache.logging.log4j.core.layout.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %t %c{1}:%L - %m%n log4j.category.org.apache.activemq=ERROR diff --git a/spring-batch-core-tests/src/site/apt/index.apt b/spring-batch-core-tests/src/site/apt/index.apt deleted file mode 100644 index 45d636ad3c..0000000000 --- a/spring-batch-core-tests/src/site/apt/index.apt +++ /dev/null @@ -1,15 +0,0 @@ - ------ - Spring Batch Integration Tests - ------ - Dave Syer - ------ - March 2008 - -Overview of the Spring Batch Integration Tests - - This module contains integration tests for the Spring Batch Infrastructure module. In the tests we exercise some extended and mostly realistic scenarios using the infrastructure components. - - * Message-pipeline tests. There are a bunch of tests using JMS and RDBMS in the same transaction, exercising various failure and retry scenarios. The basic premise is that throughput in such systems is greatly increased by widening the transactio nboundaries to process more than one message in each transaction. If there is a failure we may want to retry the operation in the next transaction. - - * OXM integration tests. There are also some tests of the XML reader and writers in Spring Batch with more extensive configuration than in the unit tests. - diff --git a/spring-batch-core-tests/src/site/site.xml b/spring-batch-core-tests/src/site/site.xml deleted file mode 100644 index 29c2887830..0000000000 --- a/spring-batch-core-tests/src/site/site.xml +++ /dev/null @@ -1,17 +0,0 @@ - - - - - - - - - - - - - - - - - diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/AbstractIntegrationTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/AbstractIntegrationTests.java new file mode 100644 index 0000000000..e136803682 --- /dev/null +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/AbstractIntegrationTests.java @@ -0,0 +1,41 @@ +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.test; + +import javax.sql.DataSource; + +import org.junit.Before; + +import org.springframework.core.io.ClassPathResource; +import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator; + +/** + * @author Mahmoud Ben Hassine + */ +public class AbstractIntegrationTests { + + protected DataSource dataSource; + + @Before + public void setUp() { + ResourceDatabasePopulator databasePopulator = new ResourceDatabasePopulator(); + databasePopulator.addScript(new ClassPathResource("/org/springframework/batch/core/schema-drop-hsqldb.sql")); + databasePopulator.addScript(new ClassPathResource("/org/springframework/batch/core/schema-hsqldb.sql")); + databasePopulator.addScript(new ClassPathResource("/business-schema-hsqldb.sql")); + databasePopulator.execute(this.dataSource); + } + +} diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/IgnoredTestSuite.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/IgnoredTestSuite.java index 513b54d9f5..48730ec659 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/IgnoredTestSuite.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/IgnoredTestSuite.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/concurrent/ConcurrentTransactionTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/concurrent/ConcurrentTransactionTests.java new file mode 100644 index 0000000000..e7c0898553 --- /dev/null +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/concurrent/ConcurrentTransactionTests.java @@ -0,0 +1,224 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.test.concurrent; + +import java.sql.Connection; +import java.sql.Driver; +import java.sql.SQLException; +import java.sql.Statement; +import javax.sql.DataSource; + +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.configuration.annotation.DefaultBatchConfigurer; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.core.job.builder.FlowBuilder; +import org.springframework.batch.core.job.flow.Flow; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.repository.support.JobRepositoryFactoryBean; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.core.io.DefaultResourceLoader; +import org.springframework.core.io.ResourceLoader; +import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.core.task.TaskExecutor; +import org.springframework.jdbc.datasource.embedded.ConnectionProperties; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseConfigurer; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory; +import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator; +import org.springframework.lang.Nullable; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.util.ClassUtils; + +import static org.junit.Assert.assertEquals; + +/** + * @author Michael Minella + */ +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(classes = ConcurrentTransactionTests.ConcurrentJobConfiguration.class) +public class ConcurrentTransactionTests { + + @Autowired + private Job concurrentJob; + + @Autowired + private JobLauncher jobLauncher; + + @DirtiesContext + @Test + public void testConcurrentLongRunningJobExecutions() throws Exception { + + JobExecution jobExecution = jobLauncher.run(concurrentJob, new JobParameters()); + + assertEquals(jobExecution.getStatus(), BatchStatus.COMPLETED); + } + + @Configuration + @EnableBatchProcessing + public static class ConcurrentJobConfiguration extends DefaultBatchConfigurer { + + @Autowired + private JobBuilderFactory jobBuilderFactory; + + @Autowired + private StepBuilderFactory stepBuilderFactory; + + @Bean + public TaskExecutor taskExecutor() { + return new SimpleAsyncTaskExecutor(); + } + + /** + * This datasource configuration configures the HSQLDB instance using MVCC. When + * configured using the default behavior, transaction serialization errors are + * thrown (default configuration example below). + * + * return new PooledEmbeddedDataSource(new EmbeddedDatabaseBuilder(). + * addScript("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql"). + * addScript("classpath:org/springframework/batch/core/schema-hsqldb.sql"). + * build()); + + * @return + */ + @Bean + DataSource dataSource() { + ResourceLoader defaultResourceLoader = new DefaultResourceLoader(); + EmbeddedDatabaseFactory embeddedDatabaseFactory = new EmbeddedDatabaseFactory(); + embeddedDatabaseFactory.setDatabaseConfigurer(new EmbeddedDatabaseConfigurer() { + + @Override + @SuppressWarnings("unchecked") + public void configureConnectionProperties(ConnectionProperties properties, String databaseName) { + try { + properties.setDriverClass((Class) ClassUtils.forName("org.hsqldb.jdbcDriver", this.getClass().getClassLoader())); + } + catch (Exception e) { + e.printStackTrace(); + } + properties.setUrl("jdbc:hsqldb:mem:" + databaseName + ";hsqldb.tx=mvcc"); + properties.setUsername("sa"); + properties.setPassword(""); + } + + @Override + public void shutdown(DataSource dataSource, String databaseName) { + try { + Connection connection = dataSource.getConnection(); + Statement stmt = connection.createStatement(); + stmt.execute("SHUTDOWN"); + } + catch (SQLException ex) { + } + } + }); + + ResourceDatabasePopulator databasePopulator = new ResourceDatabasePopulator(); + databasePopulator.addScript(defaultResourceLoader.getResource("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql")); + databasePopulator.addScript(defaultResourceLoader.getResource("classpath:org/springframework/batch/core/schema-hsqldb.sql")); + embeddedDatabaseFactory.setDatabasePopulator(databasePopulator); + + return embeddedDatabaseFactory.getDatabase(); + } + + @Bean + public Flow flow() { + return new FlowBuilder("flow") + .start(stepBuilderFactory.get("flow.step1") + .tasklet(new Tasklet() { + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + return RepeatStatus.FINISHED; + } + }).build() + ).next(stepBuilderFactory.get("flow.step2") + .tasklet(new Tasklet() { + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + return RepeatStatus.FINISHED; + } + }).build() + ).build(); + } + + @Bean + public Step firstStep() { + return stepBuilderFactory.get("firstStep") + .tasklet(new Tasklet() { + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + System.out.println(">> Beginning concurrent job test"); + return RepeatStatus.FINISHED; + } + }).build(); + } + + @Bean + public Step lastStep() { + return stepBuilderFactory.get("lastStep") + .tasklet(new Tasklet() { + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + System.out.println(">> Ending concurrent job test"); + return RepeatStatus.FINISHED; + } + }).build(); + } + + @Bean + public Job concurrentJob() { + Flow splitFlow = new FlowBuilder("splitflow").split(new SimpleAsyncTaskExecutor()).add(flow(), flow(), flow(), flow(), flow(), flow(), flow()).build(); + + return jobBuilderFactory.get("concurrentJob") + .start(firstStep()) + .next(stepBuilderFactory.get("splitFlowStep") + .flow(splitFlow) + .build()) + .next(lastStep()) + .build(); + } + + @Override + protected JobRepository createJobRepository() throws Exception { + JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean(); + factory.setDataSource(dataSource()); + factory.setIsolationLevelForCreate("ISOLATION_READ_COMMITTED"); + factory.setTransactionManager(getTransactionManager()); + factory.afterPropertiesSet(); + return factory.getObject(); + } + } +} diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/football/FootballJobIntegrationTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/football/FootballJobIntegrationTests.java index 240fa23424..d3d56d1e80 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/football/FootballJobIntegrationTests.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/football/FootballJobIntegrationTests.java @@ -1,84 +1,76 @@ -/* - * Copyright 2006-2009 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.core.test.football; - -import static org.junit.Assert.assertEquals; - -import javax.sql.DataSource; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.springframework.batch.core.BatchStatus; -import org.springframework.batch.core.Job; -import org.springframework.batch.core.JobExecution; -import org.springframework.batch.core.JobParametersBuilder; -import org.springframework.batch.core.StepExecution; -import org.springframework.batch.core.launch.JobLauncher; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.test.context.ContextConfiguration; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; -import org.springframework.test.jdbc.JdbcTestUtils; - -/** - * @author Dave Syer - * - */ -@RunWith(SpringJUnit4ClassRunner.class) -@ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/META-INF/batch/footballJob.xml" }) -public class FootballJobIntegrationTests { - - /** Logger */ - private final Log logger = LogFactory.getLog(getClass()); - - private JdbcTemplate jdbcTemplate; - - @Autowired - private JobLauncher jobLauncher; - - @Autowired - private Job job; - - @Autowired - public void setDataSource(DataSource dataSource) { - this.jdbcTemplate = new JdbcTemplate(dataSource); - } - - @Before - public void clear() { - JdbcTestUtils.deleteFromTables(jdbcTemplate, "PLAYER_SUMMARY", "GAMES", "PLAYERS"); - } - - @Test - public void testLaunchJob() throws Exception { - JobExecution execution = jobLauncher.run(job, new JobParametersBuilder().addLong("commit.interval", 10L) - .toJobParameters()); - assertEquals(BatchStatus.COMPLETED, execution.getStatus()); - for (StepExecution stepExecution : execution.getStepExecutions()) { - logger.info("Processed: " + stepExecution); - if (stepExecution.getStepName().equals("playerload")) { - // The effect of the retries - assertEquals(new Double(Math.ceil(stepExecution.getReadCount() / 10. + 1)).intValue(), - stepExecution.getCommitCount()); - } - } - } - -} +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.core.test.football; + +import static org.junit.Assert.assertEquals; + +import javax.sql.DataSource; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.test.AbstractIntegrationTests; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +/** + * @author Dave Syer + * @author Mahmoud Ben Hassine + * + */ +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/META-INF/batch/footballJob.xml" }) +public class FootballJobIntegrationTests extends AbstractIntegrationTests { + + /** Logger */ + private final Log logger = LogFactory.getLog(getClass()); + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private Job job; + + @Autowired + public void setDataSource(DataSource dataSource) { + this.dataSource = dataSource; + } + + @Test + public void testLaunchJob() throws Exception { + JobExecution execution = jobLauncher.run(job, new JobParametersBuilder().addLong("commit.interval", 10L) + .toJobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + for (StepExecution stepExecution : execution.getStepExecutions()) { + logger.info("Processed: " + stepExecution); + if (stepExecution.getStepName().equals("playerload")) { + // The effect of the retries + assertEquals(new Double(Math.ceil(stepExecution.getReadCount() / 10. + 1)).intValue(), + stepExecution.getCommitCount()); + } + } + } + +} diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/football/FootballJobSkipIntegrationTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/football/FootballJobSkipIntegrationTests.java index feceecc206..2565dddd46 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/football/FootballJobSkipIntegrationTests.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/football/FootballJobSkipIntegrationTests.java @@ -1,116 +1,112 @@ -/* - * Copyright 2006-2009 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.core.test.football; - -import static org.junit.Assert.assertEquals; - -import javax.sql.DataSource; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.springframework.batch.core.BatchStatus; -import org.springframework.batch.core.Job; -import org.springframework.batch.core.JobExecution; -import org.springframework.batch.core.JobParametersBuilder; -import org.springframework.batch.core.StepExecution; -import org.springframework.batch.core.launch.JobLauncher; -import org.springframework.batch.support.DatabaseType; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.test.context.ContextConfiguration; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; -import org.springframework.test.jdbc.JdbcTestUtils; - -/** - * @author Dave Syer - * - */ -@RunWith(SpringJUnit4ClassRunner.class) -@ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/META-INF/batch/footballSkipJob.xml" }) -public class FootballJobSkipIntegrationTests { - - /** Logger */ - private final Log logger = LogFactory.getLog(getClass()); - - private JdbcTemplate jdbcTemplate; - - @Autowired - private JobLauncher jobLauncher; - - @Autowired - private Job job; - - private DatabaseType databaseType; - - @Autowired - public void setDataSource(DataSource dataSource) throws Exception { - this.jdbcTemplate = new JdbcTemplate(dataSource); - databaseType = DatabaseType.fromMetaData(dataSource); - } - - @Before - public void clear() { - JdbcTestUtils.deleteFromTables(jdbcTemplate, "PLAYER_SUMMARY", "GAMES", "PLAYERS"); - } - - @Test - public void testLaunchJob() throws Exception { - try { - if (databaseType == DatabaseType.POSTGRES || databaseType == DatabaseType.ORACLE) { - // Extra special test for these platforms (would have failed - // the job with UNKNOWN status in Batch 2.0): - jdbcTemplate.update("SET CONSTRAINTS ALL DEFERRED"); - } - } - catch (Exception e) { - // Ignore (wrong platform) - } - JobExecution execution = jobLauncher.run(job, new JobParametersBuilder().addLong("skip.limit", 0L) - .toJobParameters()); - assertEquals(BatchStatus.COMPLETED, execution.getStatus()); - for (StepExecution stepExecution : execution.getStepExecutions()) { - logger.info("Processed: " + stepExecution); - } - // They all skip on the second execution because of a primary key - // violation - long retryLimit = 2L; - execution = jobLauncher.run(job, - new JobParametersBuilder().addLong("skip.limit", 100000L).addLong("retry.limit", retryLimit) - .toJobParameters()); - assertEquals(BatchStatus.COMPLETED, execution.getStatus()); - for (StepExecution stepExecution : execution.getStepExecutions()) { - logger.info("Processed: " + stepExecution); - if (stepExecution.getStepName().equals("playerload")) { - // The effect of the retries is to increase the number of - // rollbacks - int commitInterval = stepExecution.getReadCount() / (stepExecution.getCommitCount() - 1); - // Account for the extra empty commit if the read count is - // commensurate with the commit interval - int effectiveCommitCount = stepExecution.getReadCount() % commitInterval == 0 ? stepExecution - .getCommitCount() - 1 : stepExecution.getCommitCount(); - long expectedRollbacks = Math.max(1, retryLimit) * effectiveCommitCount + stepExecution.getReadCount(); - assertEquals(expectedRollbacks, stepExecution.getRollbackCount()); - assertEquals(stepExecution.getReadCount(), stepExecution.getWriteSkipCount()); - } - } - - } - -} +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.core.test.football; + +import static org.junit.Assert.assertEquals; + +import javax.sql.DataSource; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.test.AbstractIntegrationTests; +import org.springframework.batch.support.DatabaseType; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +/** + * @author Dave Syer + * @author Mahmoud Ben Hassine + * + */ +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/META-INF/batch/footballSkipJob.xml" }) +public class FootballJobSkipIntegrationTests extends AbstractIntegrationTests { + + /** Logger */ + private final Log logger = LogFactory.getLog(getClass()); + + private JdbcTemplate jdbcTemplate; + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private Job job; + + private DatabaseType databaseType; + + @Autowired + public void setDataSource(DataSource dataSource) throws Exception { + this.dataSource = dataSource; + this.jdbcTemplate = new JdbcTemplate(dataSource); + databaseType = DatabaseType.fromMetaData(dataSource); + } + + @Test + public void testLaunchJob() throws Exception { + try { + if (databaseType == DatabaseType.POSTGRES || databaseType == DatabaseType.ORACLE) { + // Extra special test for these platforms (would have failed + // the job with UNKNOWN status in Batch 2.0): + jdbcTemplate.update("SET CONSTRAINTS ALL DEFERRED"); + } + } + catch (Exception e) { + // Ignore (wrong platform) + } + JobExecution execution = jobLauncher.run(job, new JobParametersBuilder().addLong("skip.limit", 0L) + .toJobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + for (StepExecution stepExecution : execution.getStepExecutions()) { + logger.info("Processed: " + stepExecution); + } + // They all skip on the second execution because of a primary key + // violation + long retryLimit = 2L; + execution = jobLauncher.run(job, + new JobParametersBuilder().addLong("skip.limit", 100000L).addLong("retry.limit", retryLimit) + .toJobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + for (StepExecution stepExecution : execution.getStepExecutions()) { + logger.info("Processed: " + stepExecution); + if (stepExecution.getStepName().equals("playerload")) { + // The effect of the retries is to increase the number of + // rollbacks + int commitInterval = stepExecution.getReadCount() / (stepExecution.getCommitCount() - 1); + // Account for the extra empty commit if the read count is + // commensurate with the commit interval + int effectiveCommitCount = stepExecution.getReadCount() % commitInterval == 0 ? stepExecution + .getCommitCount() - 1 : stepExecution.getCommitCount(); + long expectedRollbacks = Math.max(1, retryLimit) * effectiveCommitCount + stepExecution.getReadCount(); + assertEquals(expectedRollbacks, stepExecution.getRollbackCount()); + assertEquals(stepExecution.getReadCount(), stepExecution.getWriteSkipCount()); + } + } + + } + +} diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/football/ParallelJobIntegrationTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/football/ParallelJobIntegrationTests.java index 61c590d4f4..fd85cf08b5 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/football/ParallelJobIntegrationTests.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/football/ParallelJobIntegrationTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/LdifReaderTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/LdifReaderTests.java new file mode 100644 index 0000000000..148c593207 --- /dev/null +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/LdifReaderTests.java @@ -0,0 +1,108 @@ +/* + * Copyright 2005-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.test.ldif; + +import static org.junit.Assert.assertEquals; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; +import java.net.MalformedURLException; + +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.core.io.ClassPathResource; +import org.springframework.core.io.Resource; +import org.springframework.core.io.UrlResource; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.util.Assert; + +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/applicationContext-test1.xml"}) +public class LdifReaderTests { + + private Resource expected; + private Resource actual; + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + @Qualifier("job1") + private Job job1; + + @Autowired + @Qualifier("job2") + private Job job2; + + public LdifReaderTests() throws MalformedURLException { + expected = new ClassPathResource("/expectedOutput.ldif"); + actual = new UrlResource("file:target/test-outputs/output.ldif"); + } + + @Before + public void checkFiles() { + Assert.isTrue(expected.exists(), "Expected does not exist."); + } + + @Test + public void testValidRun() throws Exception { + JobExecution jobExecution = jobLauncher.run(job1, new JobParameters()); + + //Ensure job completed successfully. + Assert.isTrue(jobExecution.getExitStatus().equals(ExitStatus.COMPLETED), "Step Execution did not complete normally: " + jobExecution.getExitStatus()); + + //Check output. + Assert.isTrue(actual.exists(), "Actual does not exist."); + compareFiles(expected.getFile(), actual.getFile()); + } + + @Test + public void testResourceNotExists() throws Exception { + JobExecution jobExecution = jobLauncher.run(job2, new JobParameters()); + + Assert.isTrue(jobExecution.getExitStatus().getExitCode().equals("FAILED"), "The job exit status is not FAILED."); + Assert.isTrue(jobExecution.getAllFailureExceptions().get(0).getMessage().contains("Failed to initialize the reader"), "The job failed for the wrong reason."); + } + + private void compareFiles(File expected, File actual) throws Exception { + BufferedReader expectedReader = new BufferedReader(new FileReader(expected)); + BufferedReader actualReader = new BufferedReader(new FileReader(actual)); + try { + int lineNum = 1; + for (String expectedLine = null; (expectedLine = expectedReader.readLine()) != null; lineNum++) { + String actualLine = actualReader.readLine(); + assertEquals("Line number " + lineNum + " does not match.", expectedLine, actualLine); + } + + String actualLine = actualReader.readLine(); + assertEquals("More lines than expected. There should not be a line number " + lineNum + ".", null, actualLine); + } + finally { + expectedReader.close(); + actualReader.close(); + } + } +} \ No newline at end of file diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/MappingLdifReaderTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/MappingLdifReaderTests.java new file mode 100644 index 0000000000..a0f7f41e24 --- /dev/null +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/MappingLdifReaderTests.java @@ -0,0 +1,120 @@ +/* + * Copyright 2005-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.test.ldif; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileInputStream; +import java.io.InputStreamReader; +import java.net.MalformedURLException; + +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.core.io.ClassPathResource; +import org.springframework.core.io.Resource; +import org.springframework.core.io.UrlResource; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.util.Assert; + +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/applicationContext-test2.xml"}) +public class MappingLdifReaderTests { + private static Logger log = LoggerFactory.getLogger(MappingLdifReaderTests.class); + + private Resource expected; + private Resource actual; + + @Autowired + private JobLauncher launcher; + + @Autowired + @Qualifier("job1") + private Job job1; + + @Autowired + @Qualifier("job2") + private Job job2; + + public MappingLdifReaderTests() throws MalformedURLException { + expected = new ClassPathResource("/expectedOutput.ldif"); + actual = new UrlResource("file:target/test-outputs/output.ldif"); + } + + @Before + public void checkFiles() { + Assert.isTrue(expected.exists(), "Expected does not exist."); + } + + @Test + public void testValidRun() throws Exception { + JobExecution jobExecution = launcher.run(job1, new JobParameters()); + + //Ensure job completed successfully. + Assert.isTrue(jobExecution.getExitStatus().equals(ExitStatus.COMPLETED), "Step Execution did not complete normally: " + jobExecution.getExitStatus()); + + //Check output. + Assert.isTrue(actual.exists(), "Actual does not exist."); + Assert.isTrue(compareFiles(expected.getFile(), actual.getFile()), "Files were not equal"); + } + + @Test + public void testResourceNotExists() throws Exception { + JobExecution jobExecution = launcher.run(job2, new JobParameters()); + + Assert.isTrue(jobExecution.getExitStatus().getExitCode().equals("FAILED"), "The job exit status is not FAILED."); + Assert.isTrue(jobExecution.getAllFailureExceptions().get(0).getMessage().contains("Failed to initialize the reader"), "The job failed for the wrong reason."); + } + + + private boolean compareFiles(File expected, File actual) throws Exception { + boolean equal = true; + + FileInputStream expectedStream = new FileInputStream(expected); + FileInputStream actualStream = new FileInputStream(actual); + + //Construct BufferedReader from InputStreamReader + BufferedReader expectedReader = new BufferedReader(new InputStreamReader(expectedStream)); + BufferedReader actualReader = new BufferedReader(new InputStreamReader(actualStream)); + + String line = null; + while ((line = expectedReader.readLine()) != null) { + if(!line.equals(actualReader.readLine())) { + equal = false; + break; + } + } + + if(actualReader.readLine() != null) { + equal = false; + } + + expectedReader.close(); + + return equal; + } +} \ No newline at end of file diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/MyMapper.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/MyMapper.java new file mode 100644 index 0000000000..cbbb2a9cc4 --- /dev/null +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/MyMapper.java @@ -0,0 +1,37 @@ +/* + * Copyright 2005-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.test.ldif; + +import org.springframework.batch.item.ldif.RecordMapper; +import org.springframework.lang.Nullable; +import org.springframework.ldap.core.LdapAttributes; + +/** + * This default implementation simply returns the LdapAttributes object and is only intended for test. As its not required + * to return an object of a specific type to make the MappingLdifReader implementation work, this basic setting is sufficient + * to demonstrate its function. + * + * @author Keith Barlow + * + */ +public class MyMapper implements RecordMapper { + + @Nullable + public LdapAttributes mapRecord(LdapAttributes attributes) { + return attributes; + } + +} diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/builder/LdifReaderBuilderTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/builder/LdifReaderBuilderTests.java new file mode 100644 index 0000000000..5f9abe1b84 --- /dev/null +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/builder/LdifReaderBuilderTests.java @@ -0,0 +1,164 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.core.test.ldif.builder; + +import org.junit.After; +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.ldif.LdifReader; +import org.springframework.batch.item.ldif.RecordCallbackHandler; +import org.springframework.batch.item.ldif.builder.LdifReaderBuilder; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.ApplicationContext; +import org.springframework.context.annotation.Configuration; +import org.springframework.ldap.core.LdapAttributes; +import org.springframework.test.context.junit4.SpringRunner; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; + +/** + * @author Glenn Renfro + */ +@RunWith(SpringRunner.class) +public class LdifReaderBuilderTests { + + @Autowired + private ApplicationContext context; + + private LdifReader ldifReader; + + private String callbackAttributeName; + + @After + public void tearDown() { + this.callbackAttributeName = null; + if (this.ldifReader != null) { + this.ldifReader.close(); + } + } + + @Test + public void testSkipRecord() throws Exception { + this.ldifReader = new LdifReaderBuilder().recordsToSkip(1).resource(context.getResource("classpath:/test.ldif")) + .name("foo").build(); + LdapAttributes ldapAttributes = firstRead(); + assertEquals("The attribute name for the second record did not match expected result", + "cn=Bjorn Jensen, ou=Accounting, dc=airius, dc=com", ldapAttributes.getName().toString()); + } + + @Test + public void testBasicRead() throws Exception { + this.ldifReader = new LdifReaderBuilder().resource(context.getResource("classpath:/test.ldif")).name("foo").build(); + LdapAttributes ldapAttributes = firstRead(); + assertEquals("The attribute name for the first record did not match expected result", + "cn=Barbara Jensen, ou=Product Development, dc=airius, dc=com", ldapAttributes.getName().toString()); + } + + @Test + public void testCurrentItemCount() throws Exception { + this.ldifReader = new LdifReaderBuilder().currentItemCount(3) + .resource(context.getResource("classpath:/test.ldif")).name("foo").build(); + LdapAttributes ldapAttributes = firstRead(); + assertEquals("The attribute name for the third record did not match expected result", + "cn=Gern Jensen, ou=Product Testing, dc=airius, dc=com", ldapAttributes.getName().toString()); + } + + @Test + public void testMaxItemCount() throws Exception { + this.ldifReader = new LdifReaderBuilder().maxItemCount(1).resource(context.getResource("classpath:/test.ldif")) + .name("foo").build(); + LdapAttributes ldapAttributes = firstRead(); + assertEquals("The attribute name for the first record did not match expected result", + "cn=Barbara Jensen, ou=Product Development, dc=airius, dc=com", ldapAttributes.getName().toString()); + ldapAttributes = this.ldifReader.read(); + assertNull("The second read should have returned null", ldapAttributes); + } + + @Test + public void testSkipRecordCallback() throws Exception { + this.ldifReader = new LdifReaderBuilder().recordsToSkip(1).skippedRecordsCallback(new TestCallBackHandler()) + .resource(context.getResource("classpath:/test.ldif")).name("foo").build(); + firstRead(); + assertEquals("The attribute name from the callback handler did not match the expected result", + "cn=Barbara Jensen, ou=Product Development, dc=airius, dc=com", this.callbackAttributeName); + } + + @Test + public void testSaveState() throws Exception { + this.ldifReader = new LdifReaderBuilder().resource(context.getResource("classpath:/test.ldif")).name("foo").build(); + ExecutionContext executionContext = new ExecutionContext(); + firstRead(executionContext); + this.ldifReader.update(executionContext); + assertEquals("foo.read.count did not have the expected result", 1, + executionContext.getInt("foo.read.count")); + } + + @Test + public void testSaveStateDisabled() throws Exception { + this.ldifReader = new LdifReaderBuilder().saveState(false).resource(context.getResource("classpath:/test.ldif")) + .build(); + ExecutionContext executionContext = new ExecutionContext(); + firstRead(executionContext); + this.ldifReader.update(executionContext); + assertEquals("ExecutionContext should have been empty", 0, executionContext.size()); + } + + @Test + public void testStrict() { + // Test that strict when enabled will throw an exception. + try { + this.ldifReader = new LdifReaderBuilder().resource(context.getResource("classpath:/teadsfst.ldif")).name("foo").build(); + this.ldifReader.open(new ExecutionContext()); + fail("IllegalStateException should have been thrown, because strict was set to true"); + } + catch (ItemStreamException ise) { + assertEquals("IllegalStateException message did not match the expected result.", + "Failed to initialize the reader", ise.getMessage()); + } + // Test that strict when disabled will still allow the ldap resource to be opened. + this.ldifReader = new LdifReaderBuilder().strict(false) + .resource(context.getResource("classpath:/teadsfst.ldif")).name("foo").build(); + this.ldifReader.open(new ExecutionContext()); + } + + private LdapAttributes firstRead() throws Exception { + return firstRead(new ExecutionContext()); + } + + private LdapAttributes firstRead(ExecutionContext executionContext) throws Exception { + this.ldifReader.open(executionContext); + return this.ldifReader.read(); + } + + @Configuration + public static class LdifConfiguration { + + } + + public class TestCallBackHandler implements RecordCallbackHandler { + + @Override + public void handleRecord(LdapAttributes attributes) { + callbackAttributeName = attributes.getName().toString(); + } + } +} diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/builder/MappingLdifReaderBuilderTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/builder/MappingLdifReaderBuilderTests.java new file mode 100644 index 0000000000..aef6c5bb19 --- /dev/null +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/ldif/builder/MappingLdifReaderBuilderTests.java @@ -0,0 +1,220 @@ +/* + * Copyright 2017-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.core.test.ldif.builder; + +import org.junit.After; +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.ldif.MappingLdifReader; +import org.springframework.batch.item.ldif.RecordCallbackHandler; +import org.springframework.batch.item.ldif.RecordMapper; +import org.springframework.batch.item.ldif.builder.MappingLdifReaderBuilder; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.ApplicationContext; +import org.springframework.context.annotation.Configuration; +import org.springframework.lang.Nullable; +import org.springframework.ldap.core.LdapAttributes; +import org.springframework.test.context.junit4.SpringRunner; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; + +/** + * @author Glenn Renfro + */ +@RunWith(SpringRunner.class) +public class MappingLdifReaderBuilderTests { + @Autowired + private ApplicationContext context; + + private MappingLdifReader mappingLdifReader; + + private String callbackAttributeName; + + @After + public void tearDown() { + this.callbackAttributeName = null; + if (this.mappingLdifReader != null) { + this.mappingLdifReader.close(); + } + } + + @Test + public void testSkipRecord() throws Exception { + this.mappingLdifReader = new MappingLdifReaderBuilder() + .recordsToSkip(1) + .recordMapper(new TestMapper()) + .resource(context.getResource("classpath:/test.ldif")) + .name("foo") + .build(); + LdapAttributes ldapAttributes = firstRead(); + assertEquals("The attribute name for the second record did not match expected result", + "cn=Bjorn Jensen, ou=Accounting, dc=airius, dc=com", ldapAttributes.getName().toString()); + } + + @Test + public void testBasicRead() throws Exception { + this.mappingLdifReader = new MappingLdifReaderBuilder() + .recordMapper(new TestMapper()) + .resource(context.getResource("classpath:/test.ldif")) + .name("foo") + .build(); + LdapAttributes ldapAttributes = firstRead(); + assertEquals("The attribute name for the first record did not match expected result", + "cn=Barbara Jensen, ou=Product Development, dc=airius, dc=com", ldapAttributes.getName().toString()); + } + + @Test + public void testCurrentItemCount() throws Exception { + this.mappingLdifReader = new MappingLdifReaderBuilder() + .currentItemCount(3) + .recordMapper(new TestMapper()) + .resource(context.getResource("classpath:/test.ldif")) + .name("foo") + .build(); + LdapAttributes ldapAttributes = firstRead(); + assertEquals("The attribute name for the third record did not match expected result", + "cn=Gern Jensen, ou=Product Testing, dc=airius, dc=com", ldapAttributes.getName().toString()); + } + + @Test + public void testMaxItemCount() throws Exception { + this.mappingLdifReader = new MappingLdifReaderBuilder() + .maxItemCount(1) + .recordMapper(new TestMapper()) + .resource(context.getResource("classpath:/test.ldif")) + .name("foo") + .build(); + LdapAttributes ldapAttributes = firstRead(); + assertEquals("The attribute name for the first record did not match expected result", + "cn=Barbara Jensen, ou=Product Development, dc=airius, dc=com", ldapAttributes.getName().toString()); + ldapAttributes = this.mappingLdifReader.read(); + assertNull("The second read should have returned null", ldapAttributes); + } + + @Test + public void testSkipRecordCallback() throws Exception { + this.mappingLdifReader = new MappingLdifReaderBuilder() + .recordsToSkip(1) + .recordMapper(new TestMapper()) + .skippedRecordsCallback(new TestCallBackHandler()) + .resource(context.getResource("classpath:/test.ldif")) + .name("foo") + .build(); + firstRead(); + assertEquals("The attribute name from the callback handler did not match the expected result", + "cn=Barbara Jensen, ou=Product Development, dc=airius, dc=com", this.callbackAttributeName); + } + + @Test + public void testSaveState() throws Exception { + this.mappingLdifReader = new MappingLdifReaderBuilder() + .recordMapper(new TestMapper()) + .resource(context.getResource("classpath:/test.ldif")) + .name("foo") + .build(); + ExecutionContext executionContext = new ExecutionContext(); + firstRead(executionContext); + this.mappingLdifReader.update(executionContext); + assertEquals("foo.read.count did not have the expected result", 1, + executionContext.getInt("foo.read.count")); + } + + @Test + public void testSaveStateDisabled() throws Exception { + this.mappingLdifReader = new MappingLdifReaderBuilder() + .saveState(false) + .recordMapper(new TestMapper()) + .resource(context.getResource("classpath:/test.ldif")) + .build(); + ExecutionContext executionContext = new ExecutionContext(); + firstRead(executionContext); + this.mappingLdifReader.update(executionContext); + assertEquals("ExecutionContext should have been empty", 0, executionContext.size()); + } + + @Test + public void testStrict() { + // Test that strict when enabled will throw an exception. + try { + this.mappingLdifReader = new MappingLdifReaderBuilder() + .recordMapper(new TestMapper()) + .resource(context.getResource("classpath:/teadsfst.ldif")) + .name("foo") + .build(); + this.mappingLdifReader.open(new ExecutionContext()); + fail("IllegalStateException should have been thrown, because strict was set to true"); + } + catch (ItemStreamException ise) { + assertEquals("IllegalStateException message did not match the expected result.", + "Failed to initialize the reader", ise.getMessage()); + } + // Test that strict when disabled will still allow the ldap resource to be opened. + this.mappingLdifReader = new MappingLdifReaderBuilder().strict(false).name("foo") + .recordMapper(new TestMapper()).resource(context.getResource("classpath:/teadsfst.ldif")).build(); + this.mappingLdifReader.open(new ExecutionContext()); + } + + @Test + public void testNullRecordMapper() { + try { + this.mappingLdifReader = new MappingLdifReaderBuilder() + .resource(context.getResource("classpath:/teadsfst.ldif")) + .build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException ise) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "RecordMapper is required.", ise.getMessage()); + } + + } + + private LdapAttributes firstRead() throws Exception { + return firstRead(new ExecutionContext()); + } + + private LdapAttributes firstRead(ExecutionContext executionContext) throws Exception { + this.mappingLdifReader.open(executionContext); + return this.mappingLdifReader.read(); + } + + @Configuration + public static class LdifConfiguration { + + } + + public class TestCallBackHandler implements RecordCallbackHandler { + + @Override + public void handleRecord(LdapAttributes attributes) { + callbackAttributeName = attributes.getName().toString(); + } + } + + public class TestMapper implements RecordMapper { + @Nullable + @Override + public LdapAttributes mapRecord(LdapAttributes attributes) { + return attributes; + } + } +} diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/repository/ConcurrentMapExecutionContextDaoTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/repository/ConcurrentMapExecutionContextDaoTests.java index 1fe63dd590..71e741f805 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/repository/ConcurrentMapExecutionContextDaoTests.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/repository/ConcurrentMapExecutionContextDaoTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -60,8 +60,9 @@ public void testSaveUpdate() throws Exception { public void testTransactionalSaveUpdate() throws Exception { final StepExecution stepExecution = new StepExecution("step", new JobExecution(11L)); stepExecution.setId(123L); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + @Override + public Void doInTransaction(TransactionStatus status) { stepExecution.getExecutionContext().put("foo", "bar"); dao.saveExecutionContext(stepExecution); return null; @@ -76,7 +77,7 @@ public Object doInTransaction(TransactionStatus status) { public void testConcurrentTransactionalSaveUpdate() throws Exception { ExecutorService executor = Executors.newFixedThreadPool(3); - CompletionService completionService = new ExecutorCompletionService(executor); + CompletionService completionService = new ExecutorCompletionService<>(executor); final int outerMax = 10; final int innerMax = 100; @@ -89,6 +90,7 @@ public void testConcurrentTransactionalSaveUpdate() throws Exception { stepExecution2.setId(1234L + i); completionService.submit(new Callable() { + @Override public StepExecution call() throws Exception { for (int i = 0; i < innerMax; i++) { String value = "bar" + i; @@ -99,6 +101,7 @@ public StepExecution call() throws Exception { }); completionService.submit(new Callable() { + @Override public StepExecution call() throws Exception { for (int i = 0; i < innerMax; i++) { String value = "spam" + i; @@ -119,8 +122,9 @@ public StepExecution call() throws Exception { private void saveAndAssert(final StepExecution stepExecution, final String value) { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + @Override + public Void doInTransaction(TransactionStatus status) { stepExecution.getExecutionContext().put("foo", value); dao.saveExecutionContext(stepExecution); return null; diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/repository/JdbcJobRepositoryTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/repository/JdbcJobRepositoryTests.java index 8614b2e4f3..ac0b268e91 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/repository/JdbcJobRepositoryTests.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/repository/JdbcJobRepositoryTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,47 +15,48 @@ */ package org.springframework.batch.core.test.repository; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - import java.io.Serializable; import java.sql.Timestamp; import java.util.ArrayList; +import java.util.Date; import java.util.HashSet; import java.util.List; import java.util.Set; - import javax.sql.DataSource; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; -import org.junit.After; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.test.AbstractIntegrationTests; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = { "/simple-job-launcher-context.xml" }) -public class JdbcJobRepositoryTests { +public class JdbcJobRepositoryTests extends AbstractIntegrationTests { private JobSupport job; - private Set jobExecutionIds = new HashSet(); + private Set jobExecutionIds = new HashSet<>(); - private Set jobIds = new HashSet(); + private Set jobIds = new HashSet<>(); - private List list = new ArrayList(); + private List list = new ArrayList<>(); private JdbcTemplate jdbcTemplate; @@ -67,35 +68,15 @@ public class JdbcJobRepositoryTests { @Autowired public void setDataSource(DataSource dataSource) { + this.dataSource = dataSource; this.jdbcTemplate = new JdbcTemplate(dataSource); } @Before public void onSetUpInTransaction() throws Exception { + super.setUp(); job = new JobSupport("test-job"); job.setRestartable(true); - jdbcTemplate.update("DELETE FROM BATCH_STEP_EXECUTION_CONTEXT"); - jdbcTemplate.update("DELETE FROM BATCH_JOB_EXECUTION_CONTEXT"); - jdbcTemplate.update("DELETE FROM BATCH_STEP_EXECUTION"); - jdbcTemplate.update("DELETE FROM BATCH_JOB_EXECUTION_PARAMS"); - jdbcTemplate.update("DELETE FROM BATCH_JOB_EXECUTION"); - jdbcTemplate.update("DELETE FROM BATCH_JOB_INSTANCE"); - } - - @After - public void onTearDownAfterTransaction() throws Exception { - for (Long id : jobExecutionIds) { - jdbcTemplate.update("DELETE FROM BATCH_JOB_EXECUTION_CONTEXT where JOB_EXECUTION_ID=?", id); - jdbcTemplate.update("DELETE FROM BATCH_JOB_EXECUTION where JOB_EXECUTION_ID=?", id); - } - for (Long id : jobIds) { - jdbcTemplate.update("DELETE FROM BATCH_JOB_INSTANCE where JOB_INSTANCE_ID=?", id); - } - for (Long id : jobIds) { - int count = jdbcTemplate.queryForInt( - "SELECT COUNT(*) FROM BATCH_JOB_INSTANCE where JOB_INSTANCE_ID=?", id); - assertEquals(0, count); - } } @Test @@ -103,7 +84,7 @@ public void testFindOrCreateJob() throws Exception { job.setName("foo"); int before = 0; JobExecution execution = repository.createJobExecution(job.getName(), new JobParameters()); - int after = jdbcTemplate.queryForInt("SELECT COUNT(*) FROM BATCH_JOB_INSTANCE"); + int after = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM BATCH_JOB_INSTANCE", Integer.class); assertEquals(before + 1, after); assertNotNull(execution.getId()); } @@ -115,7 +96,7 @@ public void testFindOrCreateJobWithExecutionContext() throws Exception { JobExecution execution = repository.createJobExecution(job.getName(), new JobParameters()); execution.getExecutionContext().put("foo", "bar"); repository.updateExecutionContext(execution); - int after = jdbcTemplate.queryForInt("SELECT COUNT(*) FROM BATCH_JOB_EXECUTION_CONTEXT"); + int after = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM BATCH_JOB_EXECUTION_CONTEXT", Integer.class); assertEquals(before + 1, after); assertNotNull(execution.getId()); JobExecution last = repository.getLastJobExecution(job.getName(), new JobParameters()); @@ -145,7 +126,7 @@ public void testFindOrCreateJobConcurrently() throws Exception { assertNotNull(execution); - int after = jdbcTemplate.queryForInt("SELECT COUNT(*) FROM BATCH_JOB_INSTANCE"); + int after = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM BATCH_JOB_INSTANCE", Integer.class); assertNotNull(execution.getId()); assertEquals(before + 1, after); @@ -166,7 +147,7 @@ public void testFindOrCreateJobConcurrentlyWhenJobAlreadyExists() throws Excepti repository.update(execution); execution.setStatus(BatchStatus.FAILED); - int before = jdbcTemplate.queryForInt("SELECT COUNT(*) FROM BATCH_JOB_INSTANCE"); + int before = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM BATCH_JOB_INSTANCE", Integer.class); assertEquals(1, before); long t0 = System.currentTimeMillis(); @@ -179,7 +160,7 @@ public void testFindOrCreateJobConcurrentlyWhenJobAlreadyExists() throws Excepti } long t1 = System.currentTimeMillis(); - int after = jdbcTemplate.queryForInt("SELECT COUNT(*) FROM BATCH_JOB_INSTANCE"); + int after = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM BATCH_JOB_INSTANCE", Integer.class); assertNotNull(execution.getId()); assertEquals(before, after); @@ -197,10 +178,16 @@ private void cacheJobIds(JobExecution execution) { private JobExecution doConcurrentStart() throws Exception { new Thread(new Runnable() { + @Override public void run() { try { JobExecution execution = repository.createJobExecution(job.getName(), new JobParameters()); + + //simulate running execution + execution.setStartTime(new Date()); + repository.update(execution); + cacheJobIds(execution); list.add(execution); Thread.sleep(1000); diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/repository/JobSupport.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/repository/JobSupport.java index 9183ce255e..b8c3284a0b 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/repository/JobSupport.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/repository/JobSupport.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,6 +27,7 @@ import org.springframework.batch.core.UnexpectedJobExecutionException; import org.springframework.batch.core.job.DefaultJobParametersValidator; import org.springframework.beans.factory.BeanNameAware; +import org.springframework.lang.Nullable; import org.springframework.util.ClassUtils; /** @@ -40,7 +41,7 @@ */ public class JobSupport implements BeanNameAware, Job { - private List steps = new ArrayList(); + private List steps = new ArrayList<>(); private String name; @@ -77,6 +78,7 @@ public JobSupport(String name) { * * @see org.springframework.beans.factory.BeanNameAware#setBeanName(java.lang.String) */ + @Override public void setBeanName(String name) { if (this.name == null) { this.name = name; @@ -97,6 +99,7 @@ public void setName(String name) { /* (non-Javadoc) * @see org.springframework.batch.core.domain.IJob#getName() */ + @Override public String getName() { return name; } @@ -139,6 +142,7 @@ public void setRestartable(boolean restartable) { /* (non-Javadoc) * @see org.springframework.batch.core.domain.IJob#isRestartable() */ + @Override public boolean isRestartable() { return restartable; } @@ -146,10 +150,13 @@ public boolean isRestartable() { /* (non-Javadoc) * @see org.springframework.batch.core.Job#getJobParametersIncrementer() */ + @Nullable + @Override public JobParametersIncrementer getJobParametersIncrementer() { return null; } + @Override public JobParametersValidator getJobParametersValidator() { return jobParametersValidator; } @@ -157,10 +164,12 @@ public JobParametersValidator getJobParametersValidator() { /* (non-Javadoc) * @see org.springframework.batch.core.domain.Job#run(org.springframework.batch.core.domain.JobExecution) */ + @Override public void execute(JobExecution execution) throws UnexpectedJobExecutionException { throw new UnsupportedOperationException("JobSupport does not provide an implementation of run(). Use a smarter subclass."); } + @Override public String toString() { return ClassUtils.getShortName(getClass()) + ": [name=" + name + "]"; } diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/FaultTolerantStepFactoryBeanIntegrationTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/FaultTolerantStepFactoryBeanIntegrationTests.java index cb2ce008aa..0e5f5fd00f 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/FaultTolerantStepFactoryBeanIntegrationTests.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/FaultTolerantStepFactoryBeanIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.test.step; import static org.junit.Assert.assertEquals; @@ -17,6 +32,7 @@ import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; @@ -30,8 +46,9 @@ import org.springframework.batch.item.ParseException; import org.springframework.batch.item.UnexpectedInputException; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.jdbc.core.RowMapper; +import org.springframework.lang.Nullable; import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @@ -75,7 +92,7 @@ public void setUp() throws Exception { writer = new SkipWriterStub(dataSource); processor = new SkipProcessorStub(dataSource); - factory = new FaultTolerantStepFactoryBean(); + factory = new FaultTolerantStepFactoryBean<>(); factory.setBeanName("stepName"); factory.setTransactionManager(transactionManager); @@ -129,17 +146,17 @@ public void testMultithreadedSunnyDay() throws Throwable { try { - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); stepExecution = jobExecution.createStepExecution(factory.getName()); repository.add(stepExecution); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); - List committed = new ArrayList(writer.getCommitted()); + List committed = new ArrayList<>(writer.getCommitted()); Collections.sort(committed); assertEquals("[1, 2, 3, 4, 5]", committed.toString()); - List processed = new ArrayList(processor.getCommitted()); + List processed = new ArrayList<>(processor.getCommitted()); Collections.sort(processed); assertEquals("[1, 2, 3, 4, 5]", processed.toString()); assertEquals(0, stepExecution.getSkipCount()); @@ -173,6 +190,8 @@ public void clear() { counter = -1; } + @Nullable + @Override public synchronized String read() throws Exception, UnexpectedInputException, ParseException { counter++; if (counter >= items.length) { @@ -185,7 +204,7 @@ public synchronized String read() throws Exception, UnexpectedInputException, Pa private static class SkipWriterStub implements ItemWriter { - private List written = new ArrayList(); + private List written = new ArrayList<>(); private Collection failures = Collections.emptySet(); @@ -197,7 +216,8 @@ public SkipWriterStub(DataSource dataSource) { public List getCommitted() { return jdbcTemplate.query("SELECT MESSAGE from ERROR_LOG where STEP_NAME='written'", - new ParameterizedRowMapper() { + new RowMapper() { + @Override public String mapRow(ResultSet rs, int rowNum) throws SQLException { return rs.getString(1); } @@ -209,6 +229,7 @@ public void clear() { jdbcTemplate.update("DELETE FROM ERROR_LOG where STEP_NAME='written'"); } + @Override public void write(List items) throws Exception { for (String item : items) { written.add(item); @@ -228,7 +249,7 @@ private static class SkipProcessorStub implements ItemProcessor private final Log logger = LogFactory.getLog(getClass()); - private List processed = new ArrayList(); + private List processed = new ArrayList<>(); private JdbcTemplate jdbcTemplate; @@ -241,7 +262,8 @@ public SkipProcessorStub(DataSource dataSource) { public List getCommitted() { return jdbcTemplate.query("SELECT MESSAGE from ERROR_LOG where STEP_NAME='processed'", - new ParameterizedRowMapper() { + new RowMapper() { + @Override public String mapRow(ResultSet rs, int rowNum) throws SQLException { return rs.getString(1); } @@ -253,6 +275,8 @@ public void clear() { jdbcTemplate.update("DELETE FROM ERROR_LOG where STEP_NAME='processed'"); } + @Nullable + @Override public String process(String item) throws Exception { processed.add(item); logger.debug("Processed item: "+item); diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/FaultTolerantStepFactoryBeanRollbackIntegrationTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/FaultTolerantStepFactoryBeanRollbackIntegrationTests.java index dd223f94ae..f5603daa73 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/FaultTolerantStepFactoryBeanRollbackIntegrationTests.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/FaultTolerantStepFactoryBeanRollbackIntegrationTests.java @@ -1,7 +1,20 @@ +/* + * Copyright 2010-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.test.step; -import static org.junit.Assert.assertEquals; - import java.sql.ResultSet; import java.sql.SQLException; import java.util.ArrayList; @@ -12,7 +25,6 @@ import java.util.List; import java.util.Map; import java.util.concurrent.CopyOnWriteArrayList; - import javax.sql.DataSource; import org.apache.commons.logging.Log; @@ -20,6 +32,7 @@ import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; @@ -33,8 +46,9 @@ import org.springframework.batch.item.ParseException; import org.springframework.batch.item.UnexpectedInputException; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.jdbc.core.RowMapper; +import org.springframework.lang.Nullable; import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @@ -42,6 +56,8 @@ import org.springframework.transaction.PlatformTransactionManager; import org.springframework.util.Assert; +import static org.junit.Assert.assertEquals; + /** * Tests for {@link FaultTolerantStepFactoryBean}. */ @@ -78,7 +94,7 @@ public void setUp() throws Exception { writer = new SkipWriterStub(dataSource); processor = new SkipProcessorStub(dataSource); - factory = new FaultTolerantStepFactoryBean(); + factory = new FaultTolerantStepFactoryBean<>(); factory.setBeanName("stepName"); factory.setTransactionManager(transactionManager); @@ -143,7 +159,7 @@ public void testMultithreadedSkipInWriter() throws Throwable { writer.setFailures("1", "2", "3", "4", "5"); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); stepExecution = jobExecution.createStepExecution(factory.getName()); repository.add(stepExecution); @@ -152,7 +168,7 @@ public void testMultithreadedSkipInWriter() throws Throwable { assertEquals("[]", writer.getCommitted().toString()); assertEquals("[]", processor.getCommitted().toString()); - List processed = new ArrayList(processor.getProcessed()); + List processed = new ArrayList<>(processor.getProcessed()); Collections.sort(processed); assertEquals("[1, 1, 2, 2, 3, 3, 4, 4, 5, 5]", processed.toString()); assertEquals(5, stepExecution.getSkipCount()); @@ -167,8 +183,9 @@ public void testMultithreadedSkipInWriter() throws Throwable { } + @SuppressWarnings("unchecked") private Map, Boolean> getExceptionMap(Class... args) { - Map, Boolean> map = new HashMap, Boolean>(); + Map, Boolean> map = new HashMap<>(); for (Class arg : args) { map.put(arg, true); } @@ -194,6 +211,8 @@ public void clear() { counter = -1; } + @Nullable + @Override public synchronized String read() throws Exception, UnexpectedInputException, ParseException { counter++; if (counter >= items.length) { @@ -206,7 +225,7 @@ public synchronized String read() throws Exception, UnexpectedInputException, Pa private static class SkipWriterStub implements ItemWriter { - private List written = new CopyOnWriteArrayList(); + private List written = new CopyOnWriteArrayList<>(); private Collection failures = Collections.emptySet(); @@ -222,7 +241,8 @@ public void setFailures(String... failures) { public List getCommitted() { return jdbcTemplate.query("SELECT MESSAGE from ERROR_LOG where STEP_NAME='written'", - new ParameterizedRowMapper() { + new RowMapper() { + @Override public String mapRow(ResultSet rs, int rowNum) throws SQLException { return rs.getString(1); } @@ -234,6 +254,7 @@ public void clear() { jdbcTemplate.update("DELETE FROM ERROR_LOG where STEP_NAME='written'"); } + @Override public void write(List items) throws Exception { for (String item : items) { written.add(item); @@ -253,7 +274,7 @@ private static class SkipProcessorStub implements ItemProcessor private final Log logger = LogFactory.getLog(getClass()); - private List processed = new CopyOnWriteArrayList(); + private List processed = new CopyOnWriteArrayList<>(); private JdbcTemplate jdbcTemplate; @@ -273,7 +294,8 @@ public List getProcessed() { public List getCommitted() { return jdbcTemplate.query("SELECT MESSAGE from ERROR_LOG where STEP_NAME='processed'", - new ParameterizedRowMapper() { + new RowMapper() { + @Override public String mapRow(ResultSet rs, int rowNum) throws SQLException { return rs.getString(1); } @@ -285,6 +307,8 @@ public void clear() { jdbcTemplate.update("DELETE FROM ERROR_LOG where STEP_NAME='processed'"); } + @Nullable + @Override public String process(String item) throws Exception { processed.add(item); logger.debug("Processed item: " + item); diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/FaultTolerantStepIntegrationTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/FaultTolerantStepIntegrationTests.java new file mode 100644 index 0000000000..50d6aa74c5 --- /dev/null +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/FaultTolerantStepIntegrationTests.java @@ -0,0 +1,226 @@ +package org.springframework.batch.core.test.step; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.step.builder.FaultTolerantStepBuilder; +import org.springframework.batch.core.step.skip.SkipLimitExceededException; +import org.springframework.batch.core.step.skip.SkipPolicy; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.ListItemReader; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.lang.Nullable; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.transaction.PlatformTransactionManager; + +import static org.junit.Assert.assertEquals; + +/** + * Tests for fault tolerant {@link org.springframework.batch.core.step.item.ChunkOrientedTasklet}. + */ +@ContextConfiguration(locations = "/simple-job-launcher-context.xml") +@RunWith(SpringJUnit4ClassRunner.class) +public class FaultTolerantStepIntegrationTests { + + private static final int TOTAL_ITEMS = 30; + private static final int CHUNK_SIZE = TOTAL_ITEMS; + + @Autowired + private JobRepository jobRepository; + + @Autowired + private PlatformTransactionManager transactionManager; + + private SkipPolicy skipPolicy; + + private FaultTolerantStepBuilder stepBuilder; + + @Before + public void setUp() { + ItemReader itemReader = new ListItemReader<>(createItems()); + ItemProcessor itemProcessor = item -> item > 20 ? null : item; + ItemWriter itemWriter = chunk -> { + if (chunk.contains(1)) { + throw new IllegalArgumentException(); + } + }; + skipPolicy = new SkipIllegalArgumentExceptionSkipPolicy(); + stepBuilder = new StepBuilderFactory(jobRepository, transactionManager).get("step") + .chunk(CHUNK_SIZE) + .reader(itemReader) + .processor(itemProcessor) + .writer(itemWriter) + .faultTolerant(); + } + + @Test + public void testFilterCountWithTransactionalProcessorWhenSkipInWrite() throws Exception { + // Given + Step step = stepBuilder + .skipPolicy(skipPolicy) + .build(); + + // When + StepExecution stepExecution = execute(step); + + // Then + assertEquals(TOTAL_ITEMS, stepExecution.getReadCount()); + assertEquals(10, stepExecution.getFilterCount()); + assertEquals(19, stepExecution.getWriteCount()); + assertEquals(1, stepExecution.getWriteSkipCount()); + } + + @Test + public void testFilterCountWithNonTransactionalProcessorWhenSkipInWrite() throws Exception { + // Given + Step step = stepBuilder + .skipPolicy(skipPolicy) + .processorNonTransactional() + .build(); + + // When + StepExecution stepExecution = execute(step); + + // Then + assertEquals(TOTAL_ITEMS, stepExecution.getReadCount()); + assertEquals(10, stepExecution.getFilterCount()); + assertEquals(19, stepExecution.getWriteCount()); + assertEquals(1, stepExecution.getWriteSkipCount()); + } + + @Test + public void testFilterCountOnRetryWithTransactionalProcessorWhenSkipInWrite() throws Exception { + // Given + Step step = stepBuilder + .retry(IllegalArgumentException.class) + .retryLimit(2) + .skipPolicy(skipPolicy) + .build(); + + // When + StepExecution stepExecution = execute(step); + + // Then + assertEquals(TOTAL_ITEMS, stepExecution.getReadCount()); + // filter count is expected to be counted on each retry attempt + assertEquals(20, stepExecution.getFilterCount()); + assertEquals(19, stepExecution.getWriteCount()); + assertEquals(1, stepExecution.getWriteSkipCount()); + } + + @Test + public void testFilterCountOnRetryWithNonTransactionalProcessorWhenSkipInWrite() throws Exception { + // Given + Step step = stepBuilder + .retry(IllegalArgumentException.class) + .retryLimit(2) + .skipPolicy(skipPolicy) + .processorNonTransactional() + .build(); + + // When + StepExecution stepExecution = execute(step); + + // Then + assertEquals(TOTAL_ITEMS, stepExecution.getReadCount()); + // filter count is expected to be counted on each retry attempt + assertEquals(20, stepExecution.getFilterCount()); + assertEquals(19, stepExecution.getWriteCount()); + assertEquals(1, stepExecution.getWriteSkipCount()); + } + + @Test(timeout = 3000) + public void testExceptionInProcessDuringChunkScan() throws Exception { + // Given + ListItemReader itemReader = new ListItemReader<>(Arrays.asList(1, 2, 3, 4, 5, 6, 7)); + + ItemProcessor itemProcessor = new ItemProcessor() { + private int cpt; + + @Nullable + @Override + public Integer process(Integer item) throws Exception { + cpt++; + if (cpt == 7) { // item 2 succeeds the first time but fails during the scan + throw new Exception("Error during process"); + } + return item; + } + }; + + ItemWriter itemWriter = new ItemWriter() { + private int cpt; + + @Override + public void write(List items) throws Exception { + cpt++; + if (cpt == 1) { + throw new Exception("Error during write"); + } + } + }; + + Step step = new StepBuilderFactory(jobRepository, transactionManager).get("step") + .chunk(5) + .reader(itemReader) + .processor(itemProcessor) + .writer(itemWriter) + .faultTolerant() + .skip(Exception.class) + .skipLimit(3) + .build(); + + // When + StepExecution stepExecution = execute(step); + + // Then + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(ExitStatus.COMPLETED, stepExecution.getExitStatus()); + assertEquals(7, stepExecution.getReadCount()); + assertEquals(6, stepExecution.getWriteCount()); + assertEquals(1, stepExecution.getProcessSkipCount()); + } + + private List createItems() { + List items = new ArrayList<>(TOTAL_ITEMS); + for (int i = 1; i <= TOTAL_ITEMS; i++) { + items.add(i); + } + return items; + } + + private StepExecution execute(Step step) throws Exception { + JobExecution jobExecution = jobRepository.createJobExecution( + "job" + Math.random(), new JobParameters()); + StepExecution stepExecution = jobExecution.createStepExecution("step"); + jobRepository.add(stepExecution); + step.execute(stepExecution); + return stepExecution; + } + + private class SkipIllegalArgumentExceptionSkipPolicy implements SkipPolicy { + + @Override + public boolean shouldSkip(Throwable throwable, int skipCount) + throws SkipLimitExceededException { + return throwable instanceof IllegalArgumentException; + } + + } +} diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/MapRepositoryFaultTolerantStepFactoryBeanRollbackTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/MapRepositoryFaultTolerantStepFactoryBeanRollbackTests.java index cfd47fa1ac..9fc279d253 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/MapRepositoryFaultTolerantStepFactoryBeanRollbackTests.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/MapRepositoryFaultTolerantStepFactoryBeanRollbackTests.java @@ -1,7 +1,20 @@ +/* + * Copyright 2010-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.test.step; -import static org.junit.Assert.assertEquals; - import java.util.ArrayList; import java.util.Arrays; import java.util.Collection; @@ -15,6 +28,7 @@ import org.apache.commons.logging.LogFactory; import org.junit.Before; import org.junit.Test; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; @@ -26,13 +40,14 @@ import org.springframework.batch.item.ItemProcessor; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ItemWriter; -import org.springframework.batch.item.ParseException; -import org.springframework.batch.item.UnexpectedInputException; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; +import org.springframework.lang.Nullable; import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor; import org.springframework.transaction.PlatformTransactionManager; import org.springframework.util.Assert; +import static org.junit.Assert.assertEquals; + /** * Tests for {@link FaultTolerantStepFactoryBean}. */ @@ -66,7 +81,7 @@ public void setUp() throws Exception { writer = new SkipWriterStub(); processor = new SkipProcessorStub(); - factory = new FaultTolerantStepFactoryBean(); + factory = new FaultTolerantStepFactoryBean<>(); factory.setTransactionManager(transactionManager); factory.setBeanName("stepName"); @@ -104,7 +119,7 @@ public void testMultithreadedSkipInWrite() throws Throwable { if (i%100==0) { logger.info("Starting step: "+i); - repository = new MapJobRepositoryFactoryBean(transactionManager).getJobRepository(); + repository = new MapJobRepositoryFactoryBean(transactionManager).getObject(); factory.setJobRepository(repository); jobExecution = repository.createJobExecution("vanillaJob", new JobParameters()); } @@ -121,7 +136,7 @@ public void testMultithreadedSkipInWrite() throws Throwable { try { - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); stepExecution = jobExecution.createStepExecution(factory.getName()); repository.add(stepExecution); @@ -129,7 +144,7 @@ public void testMultithreadedSkipInWrite() throws Throwable { assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); assertEquals(5, stepExecution.getSkipCount()); - List processed = new ArrayList(processor.getProcessed()); + List processed = new ArrayList<>(processor.getProcessed()); Collections.sort(processed); assertEquals("[1, 1, 2, 2, 3, 3, 4, 4, 5, 5]", processed.toString()); @@ -162,13 +177,14 @@ public void clear() { counter = -1; } - public synchronized String read() throws Exception, UnexpectedInputException, ParseException { + @Nullable + @Override + public synchronized String read() throws Exception { counter++; if (counter >= items.length) { return null; } - String item = items[counter]; - return item; + return items[counter]; } } @@ -176,7 +192,7 @@ private static class SkipWriterStub implements ItemWriter { private final Log logger = LogFactory.getLog(getClass()); - private List written = new CopyOnWriteArrayList(); + private List written = new CopyOnWriteArrayList<>(); private Collection failures = Collections.emptySet(); @@ -192,6 +208,7 @@ public void clear() { written.clear(); } + @Override public void write(List items) throws Exception { for (String item : items) { logger.trace("Writing: "+item); @@ -211,7 +228,7 @@ private static class SkipProcessorStub implements ItemProcessor private final Log logger = LogFactory.getLog(getClass()); - private List processed = new CopyOnWriteArrayList(); + private List processed = new CopyOnWriteArrayList<>(); public List getProcessed() { return processed; @@ -221,6 +238,8 @@ public void clear() { processed.clear(); } + @Nullable + @Override public String process(String item) throws Exception { processed.add(item); logger.debug("Processed item: "+item); @@ -228,8 +247,9 @@ public String process(String item) throws Exception { } } + @SuppressWarnings("unchecked") private Map, Boolean> getExceptionMap(Class... args) { - Map, Boolean> map = new HashMap, Boolean>(); + Map, Boolean> map = new HashMap<>(); for (Class arg : args) { map.put(arg, true); } diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/MapRepositoryFaultTolerantStepFactoryBeanTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/MapRepositoryFaultTolerantStepFactoryBeanTests.java index 6044d1f138..669b441426 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/MapRepositoryFaultTolerantStepFactoryBeanTests.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/MapRepositoryFaultTolerantStepFactoryBeanTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.test.step; import static org.junit.Assert.assertEquals; @@ -27,6 +42,7 @@ import org.springframework.batch.item.ParseException; import org.springframework.batch.item.UnexpectedInputException; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; +import org.springframework.lang.Nullable; import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor; import org.springframework.transaction.PlatformTransactionManager; import org.springframework.util.Assert; @@ -63,7 +79,7 @@ public void setUp() throws Exception { writer = new SkipWriterStub(); processor = new SkipProcessorStub(); - factory = new FaultTolerantStepFactoryBean(); + factory = new FaultTolerantStepFactoryBean<>(); factory.setBeanName("stepName"); factory.setTransactionManager(transactionManager); @@ -98,7 +114,7 @@ public void testMultithreadedSunnyDay() throws Throwable { if (i%100==0) { logger.info("Starting step: "+i); - repository = new MapJobRepositoryFactoryBean(transactionManager).getJobRepository(); + repository = new MapJobRepositoryFactoryBean(transactionManager).getObject(); factory.setJobRepository(repository); jobExecution = repository.createJobExecution("vanillaJob", new JobParameters()); } @@ -113,17 +129,17 @@ public void testMultithreadedSunnyDay() throws Throwable { try { - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); stepExecution = jobExecution.createStepExecution(factory.getName()); repository.add(stepExecution); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); - List committed = new ArrayList(writer.getWritten()); + List committed = new ArrayList<>(writer.getWritten()); Collections.sort(committed); assertEquals("[1, 2, 3, 4, 5]", committed.toString()); - List processed = new ArrayList(processor.getProcessed()); + List processed = new ArrayList<>(processor.getProcessed()); Collections.sort(processed); assertEquals("[1, 2, 3, 4, 5]", processed.toString()); assertEquals(0, stepExecution.getSkipCount()); @@ -157,6 +173,8 @@ public void clear() { counter = -1; } + @Nullable + @Override public synchronized String read() throws Exception, UnexpectedInputException, ParseException { counter++; if (counter >= items.length) { @@ -169,7 +187,7 @@ public synchronized String read() throws Exception, UnexpectedInputException, Pa private static class SkipWriterStub implements ItemWriter { - private List written = new CopyOnWriteArrayList(); + private List written = new CopyOnWriteArrayList<>(); private Collection failures = Collections.emptySet(); @@ -181,6 +199,7 @@ public void clear() { written.clear(); } + @Override public void write(List items) throws Exception { for (String item : items) { written.add(item); @@ -199,7 +218,7 @@ private static class SkipProcessorStub implements ItemProcessor private final Log logger = LogFactory.getLog(getClass()); - private List processed = new CopyOnWriteArrayList(); + private List processed = new CopyOnWriteArrayList<>(); public List getProcessed() { return processed; @@ -209,6 +228,8 @@ public void clear() { processed.clear(); } + @Nullable + @Override public String process(String item) throws Exception { processed.add(item); logger.debug("Processed item: "+item); diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/SplitJobMapRepositoryIntegrationTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/SplitJobMapRepositoryIntegrationTests.java index 47a0e1dc49..7e59b9bdb4 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/SplitJobMapRepositoryIntegrationTests.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/SplitJobMapRepositoryIntegrationTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2009 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -33,6 +33,7 @@ import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; import org.springframework.context.support.ClassPathXmlApplicationContext; +import org.springframework.lang.Nullable; /** * @author Dave Syer @@ -45,12 +46,13 @@ public class SplitJobMapRepositoryIntegrationTests { /** Logger */ private final Log logger = LogFactory.getLog(getClass()); + @SuppressWarnings("resource") @Test public void testMultithreadedSplit() throws Throwable { JobLauncher jobLauncher = null; Job job = null; - + ClassPathXmlApplicationContext context = null; for (int i = 0; i < MAX_COUNT; i++) { @@ -62,8 +64,8 @@ public void testMultithreadedSplit() throws Throwable { logger.info("Starting job: " + i); context = new ClassPathXmlApplicationContext(getClass().getSimpleName() + "-context.xml", getClass()); - jobLauncher = (JobLauncher) context.getBean("jobLauncher", JobLauncher.class); - job = (Job) context.getBean("job", Job.class); + jobLauncher = context.getBean("jobLauncher", JobLauncher.class); + job = context.getBean("job", Job.class); } try { @@ -86,6 +88,8 @@ public static class CountingTasklet implements Tasklet { private AtomicInteger count = new AtomicInteger(0); + @Nullable + @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { contribution.incrementReadCount(); contribution.incrementWriteCount(1); diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/StepExecutionSerializationUtilsTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/StepExecutionSerializationUtilsTests.java index c2df1a748a..6d743b3421 100644 --- a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/StepExecutionSerializationUtilsTests.java +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/step/StepExecutionSerializationUtilsTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,7 +30,7 @@ import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.StepExecution; -import org.springframework.batch.support.SerializationUtils; +import org.springframework.util.SerializationUtils; /** * @author Dave Syer @@ -41,7 +41,7 @@ public class StepExecutionSerializationUtilsTests { @Test public void testCycle() throws Exception { StepExecution stepExecution = new StepExecution("step", new JobExecution(new JobInstance(123L, - "job"), 321L, new JobParameters()), 11L); + "job"), 321L, new JobParameters(), null), 11L); stepExecution.getExecutionContext().put("foo.bar.spam", 123); StepExecution result = getCopy(stepExecution); assertEquals(stepExecution, result); @@ -55,12 +55,13 @@ public void testMultipleCycles() throws Throwable { int threads = 10; Executor executor = Executors.newFixedThreadPool(threads); - CompletionService completionService = new ExecutorCompletionService(executor); + CompletionService completionService = new ExecutorCompletionService<>(executor); for (int i = 0; i < repeats; i++) { - final JobExecution jobExecution = new JobExecution(new JobInstance(123L, "job"), 321L, new JobParameters()); + final JobExecution jobExecution = new JobExecution(new JobInstance(123L, "job"), 321L, new JobParameters(), null); for (int j = 0; j < threads; j++) { completionService.submit(new Callable() { + @Override public StepExecution call() throws Exception { final StepExecution stepExecution = jobExecution.createStepExecution("step"); stepExecution.getExecutionContext().put("foo.bar.spam", 123); diff --git a/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/timeout/TimeoutJobIntegrationTests.java b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/timeout/TimeoutJobIntegrationTests.java new file mode 100644 index 0000000000..543055ac39 --- /dev/null +++ b/spring-batch-core-tests/src/test/java/org/springframework/batch/core/test/timeout/TimeoutJobIntegrationTests.java @@ -0,0 +1,75 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.test.timeout; + +import javax.sql.DataSource; + +import static org.junit.Assert.assertEquals; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.test.AbstractIntegrationTests; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/META-INF/batch/timeoutJob.xml" }) +public class TimeoutJobIntegrationTests extends AbstractIntegrationTests { + + /** Logger */ + @SuppressWarnings("unused") + private final Log logger = LogFactory.getLog(getClass()); + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + @Qualifier("chunkTimeoutJob") + private Job chunkTimeoutJob; + + @Autowired + @Qualifier("taskletTimeoutJob") + private Job taskletTimeoutJob; + + @Autowired + public void setDataSource(DataSource dataSource) { + this.dataSource = dataSource; + } + + @Test + public void testChunkTimeoutShouldFail() throws Exception { + JobExecution execution = jobLauncher.run(chunkTimeoutJob, new JobParametersBuilder().addLong("id", System.currentTimeMillis()) + .toJobParameters()); + assertEquals(BatchStatus.FAILED, execution.getStatus()); + } + + @Test + public void testTaskletTimeoutShouldFail() throws Exception { + JobExecution execution = jobLauncher.run(taskletTimeoutJob, new JobParametersBuilder().addLong("id", System.currentTimeMillis()) + .toJobParameters()); + assertEquals(BatchStatus.FAILED, execution.getStatus()); + } + +} diff --git a/spring-batch-core-tests/src/test/java/test/jdbc/datasource/DataSourceInitializer.java b/spring-batch-core-tests/src/test/java/test/jdbc/datasource/DataSourceInitializer.java index 5d4aad94cc..6a730f48c3 100644 --- a/spring-batch-core-tests/src/test/java/test/jdbc/datasource/DataSourceInitializer.java +++ b/spring-batch-core-tests/src/test/java/test/jdbc/datasource/DataSourceInitializer.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -56,6 +56,7 @@ public void setInitialize(boolean initialize) { this.initialize = initialize; } + @Override public void destroy() throws Exception { if (!initialized) { return; @@ -76,8 +77,9 @@ public void destroy() throws Exception { } } + @Override public void afterPropertiesSet() throws Exception { - Assert.notNull(dataSource); + Assert.notNull(dataSource, "A DataSource is required"); logger.info("Initializing with scripts: "+Arrays.asList(initScripts)); if (!initialized && initialize) { try { @@ -101,15 +103,16 @@ private void doExecuteScript(final Resource scriptResource) { if (scriptResource == null || !scriptResource.exists()) return; TransactionTemplate transactionTemplate = new TransactionTemplate(new DataSourceTransactionManager(dataSource)); - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { + @Override @SuppressWarnings("unchecked") - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); String[] scripts; try { scripts = StringUtils.delimitedListToStringArray(stripComments(IOUtils.readLines(scriptResource - .getInputStream())), ";"); + .getInputStream(), "UTF-8")), ";"); } catch (IOException e) { throw new BeanInitializationException("Cannot load script from [" + scriptResource + "]", e); @@ -134,10 +137,10 @@ public Object doInTransaction(TransactionStatus status) { } private String stripComments(List list) { - StringBuffer buffer = new StringBuffer(); + StringBuilder buffer = new StringBuilder(); for (String line : list) { if (!line.startsWith("//") && !line.startsWith("--")) { - buffer.append(line + "\n"); + buffer.append(line).append("\n"); } } return buffer.toString(); diff --git a/spring-batch-core-tests/src/test/java/test/jdbc/datasource/DerbyDataSourceFactoryBean.java b/spring-batch-core-tests/src/test/java/test/jdbc/datasource/DerbyDataSourceFactoryBean.java index 2a16eb3890..f58dca137b 100644 --- a/spring-batch-core-tests/src/test/java/test/jdbc/datasource/DerbyDataSourceFactoryBean.java +++ b/spring-batch-core-tests/src/test/java/test/jdbc/datasource/DerbyDataSourceFactoryBean.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package test.jdbc.datasource; import java.io.File; @@ -7,27 +22,29 @@ import org.apache.derby.jdbc.EmbeddedDataSource; import org.springframework.beans.factory.config.AbstractFactoryBean; -public class DerbyDataSourceFactoryBean extends AbstractFactoryBean { +public class DerbyDataSourceFactoryBean extends AbstractFactoryBean { - private String dataDirectory = "derby-home"; + private String dataDirectory = "build/derby-home"; public void setDataDirectory(String dataDirectory) { this.dataDirectory = dataDirectory; } - protected Object createInstance() throws Exception { + @Override + protected DataSource createInstance() throws Exception { File directory = new File(dataDirectory); System.setProperty("derby.system.home", directory.getCanonicalPath()); System.setProperty("derby.storage.fileSyncTransactionLog", "true"); System.setProperty("derby.storage.pageCacheSize", "100"); final EmbeddedDataSource ds = new EmbeddedDataSource(); - ds.setDatabaseName("derbydb"); + ds.setDatabaseName("build/derbydb"); ds.setCreateDatabase("create"); return ds; } + @Override public Class getObjectType() { return DataSource.class; } diff --git a/spring-batch-core-tests/src/test/resources/applicationContext-test1.xml b/spring-batch-core-tests/src/test/resources/applicationContext-test1.xml new file mode 100644 index 0000000000..65378b5c71 --- /dev/null +++ b/spring-batch-core-tests/src/test/resources/applicationContext-test1.xml @@ -0,0 +1,43 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core-tests/src/test/resources/applicationContext-test2.xml b/spring-batch-core-tests/src/test/resources/applicationContext-test2.xml new file mode 100644 index 0000000000..8ee9f5826f --- /dev/null +++ b/spring-batch-core-tests/src/test/resources/applicationContext-test2.xml @@ -0,0 +1,61 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core-tests/src/test/resources/batch-derby.properties b/spring-batch-core-tests/src/test/resources/batch-derby.properties index 380004cd1d..026209f3ef 100644 --- a/spring-batch-core-tests/src/test/resources/batch-derby.properties +++ b/spring-batch-core-tests/src/test/resources/batch-derby.properties @@ -1,7 +1,7 @@ # Placeholders batch.* # for Derby: batch.jdbc.driver=org.apache.derby.jdbc.EmbeddedDriver -batch.jdbc.url=jdbc:derby:derby-home/test;create=true +batch.jdbc.url=jdbc:derby:build/derby-home/test;create=true batch.jdbc.user=sa batch.jdbc.password= batch.jdbc.testWhileIdle=false @@ -13,4 +13,5 @@ batch.data.source.init=true batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.DerbyMaxValueIncrementer batch.database.incrementer.parent=columnIncrementerParent batch.grid.size=2 -batch.verify.cursor.position=false \ No newline at end of file +batch.verify.cursor.position=false +batch.jdbc.validationQuery=values 1 diff --git a/spring-batch-core-tests/src/test/resources/batch-hsql.properties b/spring-batch-core-tests/src/test/resources/batch-hsql.properties index 87d2d3919d..7af206ad17 100644 --- a/spring-batch-core-tests/src/test/resources/batch-hsql.properties +++ b/spring-batch-core-tests/src/test/resources/batch-hsql.properties @@ -1,7 +1,7 @@ # Placeholders batch.* # for HSQLDB: batch.jdbc.driver=org.hsqldb.jdbcDriver -batch.jdbc.url=jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true +batch.jdbc.url=jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true;hsqldb.tx=mvcc # use this one for a separate server process so you can inspect the results # (or add it to system properties with -D to override at run time). # batch.jdbc.url=jdbc:hsqldb:hsql://localhost:9005/samples @@ -16,3 +16,4 @@ batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.Hs batch.database.incrementer.parent=columnIncrementerParent batch.grid.size=2 batch.verify.cursor.position=true +batch.jdbc.validationQuery=SELECT 1 FROM INFORMATION_SCHEMA.SYSTEM_USERS diff --git a/spring-batch-core-tests/src/test/resources/batch-mysql.properties b/spring-batch-core-tests/src/test/resources/batch-mysql.properties index f7377931f1..1c157a6ed5 100644 --- a/spring-batch-core-tests/src/test/resources/batch-mysql.properties +++ b/spring-batch-core-tests/src/test/resources/batch-mysql.properties @@ -10,4 +10,5 @@ batch.data.source.init=false batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.MySQLMaxValueIncrementer batch.database.incrementer.parent=columnIncrementerParent batch.lob.handler.class=org.springframework.jdbc.support.lob.DefaultLobHandler -batch.verify.cursor.position=true \ No newline at end of file +batch.verify.cursor.position=true +batch.jdbc.validationQuery=SELECT 1 diff --git a/spring-batch-core-tests/src/test/resources/batch-postgres.properties b/spring-batch-core-tests/src/test/resources/batch-postgres.properties index 9e47c540f2..32c580e59b 100644 --- a/spring-batch-core-tests/src/test/resources/batch-postgres.properties +++ b/spring-batch-core-tests/src/test/resources/batch-postgres.properties @@ -10,7 +10,8 @@ batch.schema.script=classpath:/org/springframework/batch/core/schema-postgresql. batch.drop.script=classpath:/org/springframework/batch/core/schema-drop-postgresql.sql batch.business.schema.script=classpath:/business-schema-postgresql.sql batch.data.source.init=true -batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.PostgreSQLSequenceMaxValueIncrementer +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.PostgresSequenceMaxValueIncrementer batch.database.incrementer.parent=sequenceIncrementerParent batch.grid.size=2 batch.verify.cursor.position=true +batch.jdbc.validationQuery=SELECT 1 diff --git a/spring-batch-core-tests/src/test/resources/data-source-context.xml b/spring-batch-core-tests/src/test/resources/data-source-context.xml index 250993947a..dbc1c04a90 100644 --- a/spring-batch-core-tests/src/test/resources/data-source-context.xml +++ b/spring-batch-core-tests/src/test/resources/data-source-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd+http://www.springframework.org/schema/util%20https://www.springframework.org/schema/util/spring-util-3.1.xsd"> @@ -17,11 +17,12 @@ - + + diff --git a/spring-batch-core-tests/src/test/resources/expectedOutput.ldif b/spring-batch-core-tests/src/test/resources/expectedOutput.ldif new file mode 100644 index 0000000000..027fc24d05 --- /dev/null +++ b/spring-batch-core-tests/src/test/resources/expectedOutput.ldif @@ -0,0 +1,75 @@ +dn: cn=Bjorn Jensen,ou=Accounting,dc=airius,dc=com +telephonenumber: +1 408 555 1212 +objectclass: top +objectclass: person +objectclass: organizationalPerson +sn: Jensen +cn: Bjorn Jensen + +dn: cn=Barbara Jensen,ou=Product Development,dc=airius,dc=com +telephonenumber: +1 408 555 1212 +uid: bjensen +description: Babs is a big sailing fan, and travels extensively in search of perfect sailing conditions. +objectclass: top +objectclass: person +objectclass: organizationalPerson +title: Product Manager, Rod and Reel Division +sn: Jensen +cn: Barbara Jensen +cn: Barbara J Jensen +cn: Babs Jensen + +dn: cn=Gern Jensen,ou=Product Testing,dc=airius,dc=com +telephonenumber: +1 408 555 1212 +uid: gernj +description:: V2hhdCBhIGNhcmVmdWwgcmVhZGVyIHlvdSBhcmUhICBUaGlzIHZhbHVlIGlzIGJhc2UtNjQtZW5j + b2RlZCBiZWNhdXNlIGl0IGhhcyBhIGNvbnRyb2wgY2hhcmFjdGVyIGluIGl0IChhIENSKS4NICBC + eSB0aGUgd2F5LCB5b3Ugc2hvdWxkIHJlYWxseSBnZXQgb3V0IG1vcmUu +objectclass: top +objectclass: person +objectclass: organizationalPerson +sn: Jensen +cn: Gern Jensen +cn: Gern O Jensen + +dn:: b3U95Za25qWt6YOoLG89QWlyaXVz +ou:: 5Za25qWt6YOo +ou:: 44GI44GE44GO44KH44GG44G2 +ou: Sales +description: Japanese office +objectclass: top +objectclass: organizationalUnit + +dn:: dWlkPXJvZ2FzYXdhcmEsb3U95Za25qWt6YOoLG89QWlyaXVz +givenname:: 44Ot44OJ44OL44O8 +givenname:: 44KN44Gp44Gr44O8 +givenname: Rodney +sn:: 5bCP56yg5Y6f +sn:: 44GK44GM44GV44KP44KJ +sn: Ogasawara +userpassword: {SHA}O3HSv1MusyL4kTjP+HKI5uxuNoM= +mail: rogasawara@airius.co.jp +objectclass: top +objectclass: person +objectclass: organizationalPerson +objectclass: inetOrgPerson +uid: rogasawara +preferredlanguage: ja +cn:: 5bCP56yg5Y6fIOODreODieODi+ODvA== +cn:: 44GK44GM44GV44KP44KJIOOCjeOBqeOBq+ODvA== +cn: Rodney Ogasawara +title:: 5Za25qWt6YOoIOmDqOmVtw== +title:: 44GI44GE44GO44KH44GG44G2IOOBtuOBoeOCh+OBhg== +title: Sales, Director + +dn: cn=Horatio Jensen,ou=Product Testing,dc=airius,dc=com +telephonenumber: +1 408 555 1212 +uid: hjensen +jpegphoto:< file:///usr/local/directory/photos/hjensen.jpg +objectclass: top +objectclass: person +objectclass: organizationalPerson +sn: Jensen +cn: Horatio Jensen +cn: Horatio N Jensen + diff --git a/spring-batch-core-tests/src/test/resources/log4j.properties b/spring-batch-core-tests/src/test/resources/log4j.properties index 4f10f29e38..3bcb5f9452 100644 --- a/spring-batch-core-tests/src/test/resources/log4j.properties +++ b/spring-batch-core-tests/src/test/resources/log4j.properties @@ -1,7 +1,7 @@ log4j.rootCategory=WARN, stdout -log4j.appender.stdout=org.apache.log4j.ConsoleAppender -log4j.appender.stdout.layout=org.apache.log4j.PatternLayout +log4j.appender.stdout=org.apache.logging.log4j.core.appender.ConsoleAppender +log4j.appender.stdout.layout=org.apache.logging.log4j.core.layout.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %t %c{1}:%L - %m%n log4j.category.org.apache.activemq=ERROR diff --git a/spring-batch-core-tests/src/test/resources/org/springframework/batch/core/test/step/SplitJobMapRepositoryIntegrationTests-context.xml b/spring-batch-core-tests/src/test/resources/org/springframework/batch/core/test/step/SplitJobMapRepositoryIntegrationTests-context.xml index 60c6f57aec..2fd345bb21 100644 --- a/spring-batch-core-tests/src/test/resources/org/springframework/batch/core/test/step/SplitJobMapRepositoryIntegrationTests-context.xml +++ b/spring-batch-core-tests/src/test/resources/org/springframework/batch/core/test/step/SplitJobMapRepositoryIntegrationTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd+http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd"> diff --git a/spring-batch-core-tests/src/test/resources/simple-job-launcher-context.xml b/spring-batch-core-tests/src/test/resources/simple-job-launcher-context.xml index 3256fcc044..7946b246b0 100644 --- a/spring-batch-core-tests/src/test/resources/simple-job-launcher-context.xml +++ b/spring-batch-core-tests/src/test/resources/simple-job-launcher-context.xml @@ -1,11 +1,9 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + ,o=Airius +objectclass: top +objectclass: organizationalUnit +ou:: 5Za25qWt6YOo +# ou:: +ou;lang-ja:: 5Za25qWt6YOo +# ou;lang-ja:: +ou;lang-ja;phonetic:: 44GI44GE44GO44KH44GG44G2 +# ou;lang-ja:: +ou;lang-en: Sales +description: Japanese office + +dn:: dWlkPXJvZ2FzYXdhcmEsb3U95Za25qWt6YOoLG89QWlyaXVz +# dn:: uid=,ou=,o=Airius +userpassword: {SHA}O3HSv1MusyL4kTjP+HKI5uxuNoM= +objectclass: top +objectclass: person +objectclass: organizationalPerson +objectclass: inetOrgPerson +uid: rogasawara +mail: rogasawara@airius.co.jp +givenname;lang-ja:: 44Ot44OJ44OL44O8 +# givenname;lang-ja:: +sn;lang-ja:: 5bCP56yg5Y6f +# sn;lang-ja:: +cn;lang-ja:: 5bCP56yg5Y6fIOODreODieODi+ODvA== +# cn;lang-ja:: +title;lang-ja:: 5Za25qWt6YOoIOmDqOmVtw== +# title;lang-ja:: +preferredlanguage: ja +givenname:: 44Ot44OJ44OL44O8 +# givenname:: +sn:: 5bCP56yg5Y6f +# sn:: +cn:: 5bCP56yg5Y6fIOODreODieODi+ODvA== +# cn:: +title:: 5Za25qWt6YOoIOmDqOmVtw== +# title:: +givenname;lang-ja;phonetic:: 44KN44Gp44Gr44O8 +# givenname;lang-ja;phonetic:: +# +sn;lang-ja;phonetic:: 44GK44GM44GV44KP44KJ +# sn;lang-ja;phonetic:: +cn;lang-ja;phonetic:: 44GK44GM44GV44KP44KJIOOCjeOBqeOBq+ODvA== +# cn;lang-ja;phonetic:: +title;lang-ja;phonetic:: 44GI44GE44GO44KH44GG44G2IOOBtuOBoeOCh+OBhg== +# title;lang-ja;phonetic:: +# +givenname;lang-en: Rodney +sn;lang-en: Ogasawara +cn;lang-en: Rodney Ogasawara +title;lang-en: Sales, Director + + +# +# Example 5: An LDIF file containing an invalid attribute +# + +dn: cn=Harry Jacobs, ou=Product Development, dc=airius, dc=com +objectClass: top +objectClass: person +cn: Harry Jacobs +description: :A big sailing fan. + + +# +# Example 6: A file containing a reference to an external file +# + +dn: cn=Horatio Jensen, ou=Product Testing, dc=airius, dc=com +objectclass: top +objectclass: person +objectclass: organizationalPerson +cn: Horatio Jensen +cn: Horatio N Jensen +sn: Jensen +uid: hjensen +telephonenumber: +1 408 555 1212 +jpegphoto:< file:///usr/local/directory/photos/hjensen.jpg + + +# +# Example 7: A file containing a series of change records and comments +# + +# Add a new entry +dn: cn=Fiona Jensen, ou=Marketing, dc=airius, dc=com +changetype: add +objectclass: top +objectclass: person +objectclass: organizationalPerson +cn: Fiona Jensen +sn: Jensen +uid: fiona +telephonenumber: +1 408 555 1212 +jpegphoto:< file:///usr/local/directory/photos/fiona.jpg + +# Delete an existing entry +dn: cn=Robert Jensen, ou=Marketing, dc=airius, dc=com +changetype: delete + +# Modify an entry's relative distinguished name +dn: cn=Paul Jensen, ou=Product Development, dc=airius, dc=com +changetype: modrdn +newrdn: cn=Paula Jensen +deleteoldrdn: 1 + +# Rename an entry and move all of its children to a new location in +# the directory tree (only implemented by LDAPv3 servers). +dn: ou=PD Accountants, ou=Product Development, dc=airius, dc=com +changetype: modrdn +newrdn: ou=Product Development Accountants +deleteoldrdn: 0 +newsuperior: ou=Accounting, dc=airius, dc=com + + +# Modify an entry: add an additional value to the postaladdress +# attribute, completely delete the description attribute, replace +# the telephonenumber attribute with two values, and delete a specific +# value from the facsimiletelephonenumber attribute +dn: cn=Paula Jensen, ou=Product Development, dc=airius, dc=com +changetype: modify +add: postaladdress +postaladdress: 123 Anystreet $ Sunnyvale, CA $ 94086 +- + +delete: description +- +replace: telephonenumber +telephonenumber: +1 408 555 1234 +telephonenumber: +1 408 555 5678 +- +delete: facsimiletelephonenumber +facsimiletelephonenumber: +1 408 555 9876 +- + +# Modify an entry: replace the postaladdress attribute with an empty +# set of values (which will cause the attribute to be removed), and +# delete the entire description attribute. Note that the first will +# always succeed, while the second will only succeed if at least +# one value for the description attribute is present. +dn: cn=Ingrid Jensen, ou=Product Support, dc=airius, dc=com +changetype: modify +replace: postaladdress +- +delete: description +- + + +# +# Example 8: An LDIF file containing a change record with a control +# + +# Delete an entry. The operation will attach the LDAPv3 +# Tree Delete Control defined in [9]. The criticality +# field is "true" and the controlValue field is +# absent, as required by [9]. +dn: ou=Product Development, dc=airius, dc=com +control: 1.2.840.113556.1.4.805 true +changetype: delete diff --git a/spring-batch-core-tests/template.mf b/spring-batch-core-tests/template.mf deleted file mode 100755 index 610f374d6c..0000000000 --- a/spring-batch-core-tests/template.mf +++ /dev/null @@ -1,11 +0,0 @@ -Manifest-Version: 1.0 -Bundle-SymbolicName: org.springframework.batch.core.tests -Bundle-Name: Spring Batch Core Tests -Bundle-Vendor: Spring -Bundle-Version: ${version} -Bundle-ManifestVersion: 2 -Ignored-Existing-Headers: - Import-Package, - Export-Package -Import-Template: - *;version="0" diff --git a/spring-batch-core/.springBeans b/spring-batch-core/.springBeans index 2fe6e96d3a..b0422b610b 100644 --- a/spring-batch-core/.springBeans +++ b/spring-batch-core/.springBeans @@ -1,118 +1,209 @@ - - - 1 - - - - - - - src/test/resources/org/springframework/batch/core/configuration/support/test-context.xml - src/test/resources/org/springframework/batch/core/launch/support/job.xml - src/test/resources/org/springframework/batch/core/launch/support/test-environment.xml - src/test/resources/org/springframework/batch/core/launch/support/2jobs.xml - src/test/resources/org/springframework/batch/core/repository/dao/data-source-context.xml - src/test/resources/org/springframework/batch/core/launch/support/job2.xml - src/test/resources/org/springframework/batch/core/repository/dao/sql-dao-test.xml - src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-registry.xml - src/test/resources/org/springframework/batch/core/configuration/support/trivial-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/BranchStepJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/common-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/OneStepJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/TwoStepJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/scope/util/AsyncPlaceholderTargetSourceTests-context.xml - src/test/resources/org/springframework/batch/core/scope/AsyncStepScopeIntegrationTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/DecisionJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/JobExecutionListenerParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryParserTests-context.xml - src/test/resources/org/springframework/batch/core/partition/launch-context.xml - src/test/resources/org/springframework/batch/core/scope/util/MultipleContextPlaceholderTargetSourceTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/scope/util/PlaceholderTargetSourceTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/RepositoryJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/scope/util/SimplePlaceholderTargetSourceTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/SplitJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/scope/StepScopeDestructionCallbackIntegrationTests-context.xml - src/test/resources/org/springframework/batch/core/scope/StepScopeIntegrationTests-context.xml - src/test/resources/org/springframework/batch/core/scope/StepScopeNestedIntegrationTests-context.xml - src/test/resources/org/springframework/batch/core/scope/StepScopePlaceholderIntegrationTests-context.xml - src/test/resources/org/springframework/batch/core/scope/StepScopeStartupIntegrationTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepWithBasicProcessTaskJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepWithFaultTolerantProcessTaskJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepWithSimpleTaskJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StopJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/EndTransitionJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/DefaultFailureJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/DefaultSuccessJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/EndTransitionDefaultStatusJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/FailTransitionDefaultStatusJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/FailTransitionJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailFirstJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailSecondJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBeanNameTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepParserTaskletAttributesTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StopIncompleteJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForJobElementTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForStepElementTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StopAndRestartJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StopRestartOnCompletedStepJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StopRestartOnFailedStepJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/JobRegistryJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/launch/JobLauncherIntegrationTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StopCustomStatusJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/scope/util/PlaceholderTargetSourceCustomEditorTests-context.xml - src/test/resources/org/springframework/batch/core/scope/util/PlaceholderTargetSourceErrorTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/support/child-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementParentAttributeParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/DuplicateTransitionJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/launch/support/error.xml - src/test/resources/org/springframework/batch/core/step/item/FaultTolerantExceptionClassesTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/FlowJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/InlineItemHandlerParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/JobParserNextOutOfScopeTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/JobParserParentAttributeTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/JobParserUnreachableStepInFlowTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/JobParserUnreachableStepTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryParserReferenceTests-context.xml - src/test/resources/org/springframework/batch/core/launch/support/launcher-with-environment.xml - src/test/resources/org/springframework/batch/core/launch/support/launcher-with-locator.xml - src/test/resources/org/springframework/batch/core/resource/ListPreparedStatementSetterTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeMultipleFinalJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/support/parent-context.xml - src/test/resources/org/springframework/batch/core/repository/support/SimpleJobRepositoryProxyTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/SplitInterruptedJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBadRetryListenerTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBadStepListenerTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCommitIntervalCompletionPolicyTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCommitIntervalTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCompletionPolicyTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepParserNoCommitIntervalOrCompletionPolicyTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StepParserParentAttributeTests-context.xml - src/test/resources/org/springframework/batch/core/scope/StepScopePerformanceTests-context.xml - src/test/resources/org/springframework/batch/core/scope/StepScopeProxyTargetClassIntegrationTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/StopAndRestartFailedJobParserTests-context.xml - src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-registry-and-auto-register.xml - src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalAttributeParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalTransactionalAttributeParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSimpleAttributeParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementTransactionalAttributeParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/FlowStepParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/JobParserWrongSchemaInRootTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSkipPolicyParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/xml/JobStepParserTests-context.xml - src/test/resources/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarContextTests-context.xml - - - - - true - false - - src/test/resources/org/springframework/batch/core/launch/support/job.xml - src/test/resources/org/springframework/batch/core/launch/support/test-environment.xml - - - - + + + 1 + + + + + + + src/test/resources/org/springframework/batch/core/configuration/support/test-context.xml + src/test/resources/org/springframework/batch/core/launch/support/job.xml + src/test/resources/org/springframework/batch/core/launch/support/test-environment.xml + src/test/resources/org/springframework/batch/core/launch/support/2jobs.xml + src/test/resources/org/springframework/batch/core/repository/dao/data-source-context.xml + src/test/resources/org/springframework/batch/core/launch/support/job2.xml + src/test/resources/org/springframework/batch/core/repository/dao/sql-dao-test.xml + src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-registry.xml + src/test/resources/org/springframework/batch/core/configuration/support/trivial-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/BranchStepJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/common-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/OneStepJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/TwoStepJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/scope/AsyncStepScopeIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/DecisionJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/JobExecutionListenerParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryParserTests-context.xml + src/test/resources/org/springframework/batch/core/partition/launch-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/RepositoryJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/SplitJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/scope/StepScopeDestructionCallbackIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/scope/StepScopeIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/scope/StepScopeNestedIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/scope/StepScopePlaceholderIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/scope/StepScopeStartupIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StepWithBasicProcessTaskJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StepWithFaultTolerantProcessTaskJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StepWithSimpleTaskJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StopJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/EndTransitionJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/DefaultFailureJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/DefaultSuccessJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/EndTransitionDefaultStatusJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/FailTransitionDefaultStatusJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/FailTransitionJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailFirstJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailSecondJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBeanNameTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StepParserTaskletAttributesTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StopIncompleteJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForJobElementTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForStepElementTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StopAndRestartJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StopRestartOnCompletedStepJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StopRestartOnFailedStepJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/JobRegistryJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/launch/JobLauncherIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StopCustomStatusJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/support/child-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementParentAttributeParserTests-context.xml + src/test/resources/org/springframework/batch/core/launch/support/error.xml + src/test/resources/org/springframework/batch/core/step/item/FaultTolerantExceptionClassesTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/FlowJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/InlineItemHandlerParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/JobParserNextOutOfScopeTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/JobParserParentAttributeTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryParserReferenceTests-context.xml + src/test/resources/org/springframework/batch/core/launch/support/launcher-with-environment.xml + src/test/resources/org/springframework/batch/core/launch/support/launcher-with-locator.xml + src/test/resources/org/springframework/batch/core/resource/ListPreparedStatementSetterTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/support/parent-context.xml + src/test/resources/org/springframework/batch/core/repository/support/SimpleJobRepositoryProxyTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/SplitInterruptedJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCommitIntervalTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCompletionPolicyTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StepParserParentAttributeTests-context.xml + src/test/resources/org/springframework/batch/core/scope/StepScopePerformanceTests-context.xml + src/test/resources/org/springframework/batch/core/scope/StepScopeProxyTargetClassIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StopAndRestartFailedJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-registry-and-auto-register.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalAttributeParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalTransactionalAttributeParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSimpleAttributeParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementTransactionalAttributeParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/FlowStepParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSkipPolicyParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/JobStepParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarContextTests-context.xml + src/test/resources/org/springframework/batch/core/scope/AsyncJobScopeIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeForJobElementTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeForStepElementTests-context.xml + src/main/resources/baseContext.xml + src/main/resources/beanRefContext.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalSkipAndRetryAttributeParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementLateBindingParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementRetryPolicyParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSkipAndRetryAttributeParserTests-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/ChunkListenerParsingTests-context.xml + src/test/resources/META-INF/batch-jobs/contextClosingTests.xml + src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionAfterFlow-context.xml + src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionAfterSplit-context.xml + src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionAsFirstStep-context.xml + src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionCustomExitStatus-context.xml + src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionInvalidExitStatus-context.xml + src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionThrowsException-context.xml + src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionValidExitStatus-context.xml + src/test/resources/META-INF/batch-jobs/DecisionStepTests-restart-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/default-split-task-executor-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/DefaultUnknownJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/ExceptionHandlingParsingTests-context.xml + src/test/resources/META-INF/batch-jobs/FlowParserTests-context.xml + src/test/resources/META-INF/batch-jobs/FlowParserTestsStepGetsFailedTransitionWhenNextAttributePresent.xml + src/test/resources/META-INF/batch-jobs/FlowParserTestsStepNoOverrideWhenNextAndFailedTransitionElementExists.xml + src/test/resources/META-INF/batch-jobs/FlowParserTestsWildcardAndNextAttrJob.xml + src/test/resources/org/springframework/batch/core/configuration/xml/InlineItemHandlerWithStepScopeParserTests-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/ItemListenerParsingTests-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/ItemSkipParsingTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/support/job-context-with-separate-steps.xml + src/test/resources/org/springframework/batch/core/configuration/support/job-context-with-steps.xml + src/test/resources/org/springframework/batch/core/configuration/xml/JobExecutionListenerMethodAttributeParserTests-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/JobListenerParsingTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/JobParserValidatorTests-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/JobPropertySubstitutionTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/support/JobRegistryIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryDefaultParserTests-context.xml + src/test/resources/org/springframework/batch/core/scope/JobScopeDestructionCallbackIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/scope/JobScopeIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/scope/JobScopeNestedIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/scope/JobScopePlaceholderIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/scope/JobScopeProxyTargetClassIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/scope/JobScopeStartupIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/JsrDecisionParsingTests-context.xml + src/test/resources/META-INF/batch-jobs/JsrSplitParsingTests-context.xml + src/test/resources/META-INF/batch-jobs/jsrSpringInstanceTests.xml + src/test/resources/org/springframework/batch/core/configuration/xml/NamespacePrefixedJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeUnknownJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/override_batch.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ParentRetryableLateBindingStepFactoryBeanParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ParentRetryableStepFactoryBeanParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ParentSkippableLateBindingStepFactoryBeanParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ParentSkippableStepFactoryBeanParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/ParentStepFactoryBeanParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithFlowParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithLateBindingParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithNonDefaultTransactionManagerParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/support/placeholder-context.xml + src/test/resources/org/springframework/batch/core/configuration/support/profiles.xml + src/test/resources/org/springframework/batch/core/step/RestartInPriorStepTests-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryListenerTestBase-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerExhausted.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerListenerException.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerRetryOnce.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/SimpleItemBasedJobParsingTests-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/SimpleJobParsingTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/SplitNestedJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerInStepParserTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerMethodAttributeParserTests-context.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/StepListenerParsingTests-context.xml + src/test/resources/org/springframework/batch/core/scope/StepScopeClassIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInheritence-context.xml + src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInterface-context.xml + src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsProxyTargetClass-context.xml + src/test/resources/org/springframework/batch/core/scope/StepScopeProxyTargetClassOverrideIntegrationTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/StepWithPojoListenerJobParserTests-context.xml + src/test/resources/org/springframework/batch/core/repository/dao/TablePrefixTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/TaskletParserAdapterTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/TaskletParserBeanPropertiesTests-context.xml + src/test/resources/org/springframework/batch/core/configuration/xml/TaskletStepAllowStartIfCompleteTest-context.xml + src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-loader.xml + src/test/resources/org/springframework/batch/core/configuration/support/trivial-context-autoregister.xml + src/test/resources/org/springframework/batch/core/jsr/configuration/xml/user-specified-split-task-executor-context.xml + java:org.springframework.batch.core.configuration.annotation.AbstractBatchConfiguration + java:org.springframework.batch.core.configuration.annotation.JobBuilderConfigurationTests$AnotherConfiguration + java:org.springframework.batch.core.jsr.configuration.xml.BatchParserTests$BaseConfiguration + java:org.springframework.batch.core.configuration.annotation.JobBuilderConfigurationTests$BeansConfigurer + java:org.springframework.batch.core.explore.support.SimpleJobExplorerIntegrationTests$Config + java:org.springframework.batch.core.configuration.annotation.DataSourceConfiguration + java:org.springframework.batch.core.configuration.annotation.JobLoaderConfigurationTests$LoaderFactoryConfiguration + java:org.springframework.batch.core.configuration.annotation.JobLoaderConfigurationTests$LoaderRegistrarConfiguration + java:org.springframework.batch.core.configuration.annotation.ModularBatchConfiguration + java:org.springframework.batch.core.step.builder.RegisterMultiListenerTests$MultiListenerFaultTolerantTestConfiguration + java:org.springframework.batch.core.step.builder.RegisterMultiListenerTests$MultiListenerTestConfiguration + java:org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration + java:org.springframework.batch.core.configuration.annotation.StepScopeConfigurationTests$StepScopeConfigurationForcingInterfaceProxy + java:org.springframework.batch.core.configuration.annotation.StepScopeConfigurationTests$StepScopeConfigurationInjectingProxy + java:org.springframework.batch.core.configuration.annotation.StepScopeConfigurationTests$StepScopeConfigurationRequiringProxyTargetClass + java:org.springframework.batch.core.configuration.annotation.StepScopeConfigurationTests$StepScopeConfigurationWithDefaults + java:org.springframework.batch.core.configuration.annotation.JobLoaderConfigurationTests$TestConfiguration + java:org.springframework.batch.core.configuration.annotation.JobBuilderConfigurationTests$TestConfiguration + java:org.springframework.batch.core.configuration.annotation.JobBuilderConfigurationTests$TestConfigurer + java:org.springframework.batch.core.configuration.annotation.JobLoaderConfigurationTests$VanillaConfiguration + java:org.springframework.batch.core.configuration.annotation.StepScopeConfiguration + + + + + true + false + + src/test/resources/org/springframework/batch/core/launch/support/job.xml + src/test/resources/org/springframework/batch/core/launch/support/test-environment.xml + + + + + + diff --git a/spring-batch-core/build.gradle b/spring-batch-core/build.gradle new file mode 100644 index 0000000000..3e7271a2d8 --- /dev/null +++ b/spring-batch-core/build.gradle @@ -0,0 +1,45 @@ +/** + * Generate schema creation and drop scripts for various databases + * supported by Spring Batch. + * + * @author David Syer (original Ant/Maven work) + * @author Chris Beams (port to Gradle) + */ +task generateSql { + group = "Build" + description = "Generates schema creation and drop scripts for supported databases." + + configurations { vpp } + dependencies { vpp 'foundrylogic.vpp:vpp:2.2.1' } + + def generatedResourcesDir = new File('src/main/resources/org/springframework/batch/core') + + outputs.dir generatedResourcesDir + + ant.typedef(resource: 'foundrylogic/vpp/typedef.properties', + classpath: configurations.vpp.asPath) + ant.taskdef(resource: 'foundrylogic/vpp/taskdef.properties', + classpath: configurations.vpp.asPath) + + doLast { + ['db2', 'derby', 'h2', 'hsqldb', 'mysql', + 'oracle10g', 'postgresql', 'sqlf', 'sqlserver', 'sybase'].each { dbType -> + ant.vppcopy(todir: generatedResourcesDir, overwrite: 'true') { + config { + context { + property key: 'includes', value: 'src/main/sql' + property file: "src/main/sql/${dbType}.properties" + } + engine { + property key: 'velocimacro.library', value: "src/main/sql/${dbType}.vpp" + } + } + fileset dir: 'src/main/sql', includes: 'schema*.sql.vpp' + mapper type: 'glob', from: '*.sql.vpp', to: "*-${dbType}.sql" + } + } + } +} + +// tie schema generation to the build lifecycle +//compileJava.dependsOn generateSql diff --git a/spring-batch-core/pom.xml b/spring-batch-core/pom.xml deleted file mode 100644 index 32fb9dce7c..0000000000 --- a/spring-batch-core/pom.xml +++ /dev/null @@ -1,302 +0,0 @@ - - - 4.0.0 - spring-batch-core - jar - Core - Core domain for batch processing, expressing a domain of Jobs, Steps, Chunks, etc. - http://static.springframework.org/spring-batch/${project.artifactId} - - org.springframework.batch - spring-batch-parent - 2.2.2.BUILD-SNAPSHOT - ../spring-batch-parent - - - - org.springframework.batch - spring-batch-infrastructure - ${project.version} - - - org.hsqldb - hsqldb - - - com.h2database - h2 - 1.2.132 - - - commons-io - commons-io - test - - - commons-dbcp - commons-dbcp - true - test - - - - - - - junit - junit - - - org.aspectj - aspectjrt - true - - - org.aspectj - aspectjweaver - true - - - cglib - cglib-nodep - test - - - com.thoughtworks.xstream - xstream - - - org.codehaus.jettison - jettison - - - org.osgi - osgi_R4_core - true - - - org.springframework - spring-aop - - - org.springframework - spring-beans - - - org.springframework - spring-context - - - org.springframework - spring-core - - - org.springframework - spring-jdbc - true - - - org.springframework.osgi - spring-osgi-core - true - - - org.springframework - spring-test - test - - - org.springframework - spring-tx - - - org.slf4j - slf4j-log4j12 - true - - - log4j - log4j - true - - - org.mockito - mockito-all - test - - - - - - org.apache.maven.plugins - maven-antrun-plugin - - - generate-sql - generate-sources - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - run - - - - - - - - - org.apache.maven.plugins - maven-surefire-plugin - - junit:junit - - - - - - - - - org.codehaus.mojo - emma-maven-plugin - 1.0-alpha-1 - - - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/BatchStatus.java b/spring-batch-core/src/main/java/org/springframework/batch/core/BatchStatus.java index 15e2414dcf..fab23ada7e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/BatchStatus.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/BatchStatus.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,10 +17,12 @@ package org.springframework.batch.core; /** - * Enumeration representing the status of a an Execution. + * Enumeration representing the status of an Execution. * * @author Lucas Ward * @author Dave Syer + * @author Michael Minella + * @author Mahmoud Ben Hassine */ public enum BatchStatus { @@ -76,8 +78,9 @@ public BatchStatus upgradeTo(BatchStatus other) { return max(this, other); } // Both less than or equal to STARTED - if (this == COMPLETED || other == COMPLETED) + if (this == COMPLETED || other == COMPLETED) { return COMPLETED; + } return max(this, other); } @@ -105,6 +108,29 @@ public boolean isLessThanOrEqualTo(BatchStatus other) { return this.compareTo(other) <= 0; } + /** + * Converts the current status to the JSR-352 equivalent + * + * @return JSR-352 equivalent to the current status + */ + public javax.batch.runtime.BatchStatus getBatchStatus() { + if(this == ABANDONED) { + return javax.batch.runtime.BatchStatus.ABANDONED; + } else if(this == COMPLETED) { + return javax.batch.runtime.BatchStatus.COMPLETED; + } else if(this == STARTED) { + return javax.batch.runtime.BatchStatus.STARTED; + } else if(this == STARTING) { + return javax.batch.runtime.BatchStatus.STARTING; + } else if(this == STOPPED) { + return javax.batch.runtime.BatchStatus.STOPPED; + } else if(this == STOPPING) { + return javax.batch.runtime.BatchStatus.STOPPING; + } else { + return javax.batch.runtime.BatchStatus.FAILED; + } + } + /** * Find a BatchStatus that matches the beginning of the given value. If no * match is found, return COMPLETED as the default because has is low diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/ChunkListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/ChunkListener.java index 8596bd4d37..b9ab2f6051 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/ChunkListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/ChunkListener.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,11 +19,12 @@ /** * Listener interface for the lifecycle of a chunk. A chunk - * can be through of as a collection of items that will be + * can be thought of as a collection of items that will be * committed together. * * @author Lucas Ward * @author Michael Minella + * @author Mahmoud Ben Hassine * */ public interface ChunkListener extends StepListener { @@ -49,7 +50,7 @@ public interface ChunkListener extends StepListener { * after transaction rollback. While the rollback will have occurred, * transactional resources might still be active and accessible. Due to * this, data access code within this callback will still "participate" in - * the original transaction unless it declares that it run in its own + * the original transaction unless it declares that it runs in its own * transaction. Hence: Use PROPAGATION_REQUIRES_NEW for any * transactional operation that is called from here. * diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/DefaultJobKeyGenerator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/DefaultJobKeyGenerator.java index 410037d263..e464f16609 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/DefaultJobKeyGenerator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/DefaultJobKeyGenerator.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,6 +24,8 @@ import java.util.List; import java.util.Map; +import org.springframework.util.Assert; + /** * Default implementation of the {@link JobKeyGenerator} interface. * This implementation provides a single hash value based on the JobParameters @@ -31,6 +33,7 @@ * are used in the calculation of the key. * * @author Michael Minella + * @author Mahmoud Ben Hassine * @since 2.2 */ public class DefaultJobKeyGenerator implements JobKeyGenerator { @@ -42,15 +45,16 @@ public class DefaultJobKeyGenerator implements JobKeyGenerator { @Override public String generateKey(JobParameters source) { + Assert.notNull(source, "source must not be null"); Map props = source.getParameters(); - StringBuffer stringBuffer = new StringBuffer(); - List keys = new ArrayList(props.keySet()); + StringBuilder stringBuffer = new StringBuilder(); + List keys = new ArrayList<>(props.keySet()); Collections.sort(keys); for (String key : keys) { JobParameter jobParameter = props.get(key); if(jobParameter.isIdentifying()) { String value = jobParameter.getValue()==null ? "" : jobParameter.toString(); - stringBuffer.append(key + "=" + value + ";"); + stringBuffer.append(key).append("=").append(value).append(";"); } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/Entity.java b/spring-batch-core/src/main/java/org/springframework/batch/core/Entity.java index 53cd33e1be..47f4032f90 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/Entity.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/Entity.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/ExitStatus.java b/spring-batch-core/src/main/java/org/springframework/batch/core/ExitStatus.java index 66225cdef2..be000c5251 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/ExitStatus.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/ExitStatus.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,12 +15,12 @@ */ package org.springframework.batch.core; +import org.springframework.util.StringUtils; + import java.io.PrintWriter; import java.io.Serializable; import java.io.StringWriter; -import org.springframework.util.StringUtils; - /** * Value object used to carry information about the status of a * job or step execution. @@ -95,6 +95,8 @@ public String getExitCode() { /** * Getter for the exit description (defaults to blank) + * + * @return {@link String} containing the exit description. */ public String getExitDescription() { return exitDescription; @@ -105,8 +107,8 @@ public String getExitDescription() { * code, and a concatenation of the descriptions. If either value has a * higher severity then its exit code will be used in the result. In the * case of equal severity, the exit code is replaced if the new value is - * alphabetically greater.
      - *
      + * alphabetically greater.
      + *
      * * Severity is defined by the exit code: *
        @@ -117,7 +119,7 @@ public String getExitDescription() { *
      • Codes beginning with FAILED have severity 5
      • *
      • Codes beginning with UNKNOWN have severity 6
      • *
      - * Others have severity 7, so custom exit codes always win.
      + * Others have severity 7, so custom exit codes always win.
      * * If the input is null just return this. * @@ -138,7 +140,8 @@ public ExitStatus and(ExitStatus status) { /** * @param status an {@link ExitStatus} to compare - * @return 1,0,-1 according to the severity and exit code + * @return greater than zero, 0, less than zero according to the severity and exit code + * @see java.lang.Comparable */ @Override public int compareTo(ExitStatus status) { @@ -225,10 +228,10 @@ public ExitStatus replaceExitCode(String code) { /** * Check if this status represents a running process. * - * @return true if the exit code is "RUNNING" or "UNKNOWN" + * @return true if the exit code is "EXECUTING" or "UNKNOWN" */ public boolean isRunning() { - return "RUNNING".equals(this.exitCode) || "UNKNOWN".equals(this.exitCode); + return "EXECUTING".equals(this.exitCode) || "UNKNOWN".equals(this.exitCode); } /** @@ -241,7 +244,7 @@ public boolean isRunning() { * description */ public ExitStatus addExitDescription(String description) { - StringBuffer buffer = new StringBuffer(); + StringBuilder buffer = new StringBuilder(); boolean changed = StringUtils.hasText(description) && !exitDescription.equals(description); if (StringUtils.hasText(exitDescription)) { buffer.append(exitDescription); @@ -259,7 +262,7 @@ public ExitStatus addExitDescription(String description) { * Extract the stack trace from the throwable provided and append it to * the exist description. * - * @param throwable + * @param throwable {@link Throwable} instance containing the stack trace. * @return a new ExitStatus with the stack trace appended */ public ExitStatus addExitDescription(Throwable throwable) { @@ -269,4 +272,17 @@ public ExitStatus addExitDescription(Throwable throwable) { return addExitDescription(message); } + /** + * @param status the exit code to be evaluated + * @return true if the value matches a known exit code + */ + public static boolean isNonDefaultExitStatus(ExitStatus status) { + return status == null || status.getExitCode() == null || + status.getExitCode().equals(ExitStatus.COMPLETED.getExitCode()) || + status.getExitCode().equals(ExitStatus.EXECUTING.getExitCode()) || + status.getExitCode().equals(ExitStatus.FAILED.getExitCode()) || + status.getExitCode().equals(ExitStatus.NOOP.getExitCode()) || + status.getExitCode().equals(ExitStatus.STOPPED.getExitCode()) || + status.getExitCode().equals(ExitStatus.UNKNOWN.getExitCode()); + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/ItemProcessListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/ItemProcessListener.java index 544ec5e5f8..8000ef17e7 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/ItemProcessListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/ItemProcessListener.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,6 +16,7 @@ package org.springframework.batch.core; import org.springframework.batch.item.ItemProcessor; +import org.springframework.lang.Nullable; /** * Listener interface for the processing of an item. Implementations @@ -24,6 +25,7 @@ * exceptions thrown by the processor. * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public interface ItemProcessListener extends StepListener { @@ -37,13 +39,13 @@ public interface ItemProcessListener extends StepListener { /** * Called after {@link ItemProcessor#process(Object)} returns. If the - * processor returns null, this method will still be called, with - * a null result, allowing for notification of 'filtered' items. + * processor returns {@code null}, this method will still be called, with + * a {code null} result, allowing for notification of 'filtered' items. * * @param item to be processed * @param result of processing */ - void afterProcess(T item, S result); + void afterProcess(T item, @Nullable S result); /** * Called if an exception was thrown from {@link ItemProcessor#process(Object)}. diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/ItemReadListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/ItemReadListener.java index 72ff04c874..8495d3250c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/ItemReadListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/ItemReadListener.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ * Listener interface around the reading of an item. * * @author Lucas Ward + * @author Mahmoud Ben Hassine * */ public interface ItemReadListener extends StepListener { @@ -32,7 +33,9 @@ public interface ItemReadListener extends StepListener { void beforeRead(); /** - * Called after {@link ItemReader#read()} + * Called after {@link ItemReader#read()}. + * This method is called only for actual items (ie it is not called when the + * reader returns null). * * @param item returned from read() */ diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/ItemWriteListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/ItemWriteListener.java index 755ee66599..7fda29adcf 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/ItemWriteListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/ItemWriteListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,9 +21,21 @@ import org.springframework.batch.item.ItemWriter; /** + *

      * Listener interface for the writing of items. Implementations * of this interface will be notified before, after, and in case * of any exception thrown while writing a list of items. + *

      + * + *

      + * Note: This listener is designed to work around the + * lifecycle of an item. This means that each method should be + * called once within the lifecycle of an item and in fault + * tolerant scenarios, any transactional work that is done in + * one of these methods would be rolled back and not re-applied. + * Because of this, it is recommended to not perform any logic + * using this listener that participates in a transaction. + *

      * * @author Lucas Ward * diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/Job.java b/spring-batch-core/src/main/java/org/springframework/batch/core/Job.java index f6c5cafaf7..3f57f02136 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/Job.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/Job.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,6 +15,8 @@ */ package org.springframework.batch.core; +import org.springframework.lang.Nullable; + /** * Batch domain object representing a job. Job is an explicit abstraction * representing the configuration of a job specified by a developer. It should @@ -22,7 +24,7 @@ * step. * * @author Dave Syer - * + * @author Mahmoud Ben Hassine */ public interface Job { @@ -47,11 +49,12 @@ public interface Job { /** * If clients need to generate new parameters for the next execution in a - * sequence they can use this incrementer. The return value may be null, in - * the case that this job does not have a natural sequence. + * sequence they can use this incrementer. The return value may be {@code null}, + * in the case that this job does not have a natural sequence. * * @return in incrementer to be used for creating new parameters */ + @Nullable JobParametersIncrementer getJobParametersIncrementer(); /** @@ -60,8 +63,8 @@ public interface Job { * the execution. * * @return a validator that can be used to check parameter values (never - * null) + * {@code null}) */ JobParametersValidator getJobParametersValidator(); -} \ No newline at end of file +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/JobExecution.java b/spring-batch-core/src/main/java/org/springframework/batch/core/JobExecution.java index ba50b838fe..0618edef0c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/JobExecution.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/JobExecution.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,18 +23,21 @@ import java.util.Collections; import java.util.Date; import java.util.HashSet; +import java.util.LinkedHashSet; import java.util.List; import java.util.Set; import java.util.concurrent.CopyOnWriteArrayList; -import java.util.concurrent.CopyOnWriteArraySet; import org.springframework.batch.item.ExecutionContext; +import org.springframework.lang.Nullable; /** * Batch domain object representing the execution of a job. * * @author Lucas Ward * @author Michael Minella + * @author Mahmoud Ben Hassine + * @author Dimitrios Liapis * */ @SuppressWarnings("serial") @@ -44,7 +47,7 @@ public class JobExecution extends Entity { private JobInstance jobInstance; - private volatile Collection stepExecutions = new CopyOnWriteArraySet(); + private volatile Collection stepExecutions = Collections.synchronizedSet(new LinkedHashSet<>()); private volatile BatchStatus status = BatchStatus.STARTING; @@ -60,35 +63,68 @@ public class JobExecution extends Entity { private volatile ExecutionContext executionContext = new ExecutionContext(); - private transient volatile List failureExceptions = new CopyOnWriteArrayList(); + private transient volatile List failureExceptions = new CopyOnWriteArrayList<>(); + + private final String jobConfigurationName; + + public JobExecution(JobExecution original) { + this.jobParameters = original.getJobParameters(); + this.jobInstance = original.getJobInstance(); + this.stepExecutions = original.getStepExecutions(); + this.status = original.getStatus(); + this.startTime = original.getStartTime(); + this.createTime = original.getCreateTime(); + this.endTime = original.getEndTime(); + this.lastUpdated = original.getLastUpdated(); + this.exitStatus = original.getExitStatus(); + this.executionContext = original.getExecutionContext(); + this.failureExceptions = original.getFailureExceptions(); + this.jobConfigurationName = original.getJobConfigurationName(); + this.setId(original.getId()); + this.setVersion(original.getVersion()); + } /** * Because a JobExecution isn't valid unless the job is set, this * constructor is the only valid one from a modeling point of view. * * @param job the job of which this execution is a part + * @param id {@link Long} that represents the id for the JobExecution. + * @param jobParameters {@link JobParameters} instance for this JobExecution. + * @param jobConfigurationName {@link String} instance that represents the + * job configuration name (used with JSR-352). */ - public JobExecution(JobInstance job, Long id, JobParameters jobParameters) { + public JobExecution(JobInstance job, Long id, @Nullable JobParameters jobParameters, String jobConfigurationName) { super(id); this.jobInstance = job; this.jobParameters = jobParameters == null ? new JobParameters() : jobParameters; + this.jobConfigurationName = jobConfigurationName; + } + + public JobExecution(JobInstance job, JobParameters jobParameters, String jobConfigurationName) { + this(job, null, jobParameters, jobConfigurationName); + } + + public JobExecution(Long id, JobParameters jobParameters, String jobConfigurationName) { + this(null, id, jobParameters, jobConfigurationName); } /** * Constructor for transient (unsaved) instances. * * @param job the enclosing {@link JobInstance} + * @param jobParameters {@link JobParameters} instance for this JobExecution. */ public JobExecution(JobInstance job, JobParameters jobParameters) { - this(job, null, jobParameters); + this(job, null, jobParameters, null); } public JobExecution(Long id, JobParameters jobParameters) { - this(null, id, jobParameters); + this(null, id, jobParameters, null); } public JobExecution(Long id) { - this(null, id, null); + this(null, id, null, null); } public JobParameters getJobParameters() { @@ -153,7 +189,7 @@ public Long getJobId() { } /** - * @param exitStatus + * @param exitStatus {@link ExitStatus} instance to be used for job execution. */ public void setExitStatus(ExitStatus exitStatus) { this.exitStatus = exitStatus; @@ -179,12 +215,14 @@ public JobInstance getJobInstance() { * @return the step executions that were registered */ public Collection getStepExecutions() { - return Collections.unmodifiableList(new ArrayList(stepExecutions)); + return Collections.unmodifiableList(new ArrayList<>(stepExecutions)); } /** * Register a step execution with the current job execution. * @param stepName the name of the step the new execution is associated with + * @return {@link StepExecution} an empty {@code StepExecution} associated with this + * {@code JobExecution}. */ public StepExecution createStepExecution(String stepName) { StepExecution stepExecution = new StepExecution(stepName, this); @@ -196,10 +234,11 @@ public StepExecution createStepExecution(String stepName) { * Test if this {@link JobExecution} indicates that it is running. It should * be noted that this does not necessarily mean that it has been persisted * as such yet. - * @return true if the end time is null + * + * @return true if the end time is null and the start time is not null */ public boolean isRunning() { - return endTime == null; + return startTime != null && endTime == null; } /** @@ -256,10 +295,14 @@ public void setCreateTime(Date createTime) { this.createTime = createTime; } + public String getJobConfigurationName() { + return this.jobConfigurationName; + } + /** * Package private method for re-constituting the step executions from * existing instances. - * @param stepExecution + * @param stepExecution execution to be added */ void addStepExecution(StepExecution stepExecution) { stepExecutions.add(stepExecution); @@ -278,7 +321,7 @@ public Date getLastUpdated() { /** * Set the last time this JobExecution was updated. * - * @param lastUpdated + * @param lastUpdated {@link Date} instance to mark job execution's lastUpdated attribute. */ public void setLastUpdated(Date lastUpdated) { this.lastUpdated = lastUpdated; @@ -291,7 +334,7 @@ public List getFailureExceptions() { /** * Add the provided throwable to the failure exception list. * - * @param t + * @param t {@link Throwable} instance to be added failure exception list. */ public synchronized void addFailureException(Throwable t) { this.failureExceptions.add(t); @@ -301,26 +344,31 @@ public synchronized void addFailureException(Throwable t) { * Return all failure causing exceptions for this JobExecution, including * step executions. * - * @return List containing all exceptions causing failure for + * @return List<Throwable> containing all exceptions causing failure for * this JobExecution. */ public synchronized List getAllFailureExceptions() { - Set allExceptions = new HashSet(failureExceptions); + Set allExceptions = new HashSet<>(failureExceptions); for (StepExecution stepExecution : stepExecutions) { allExceptions.addAll(stepExecution.getFailureExceptions()); } - return new ArrayList(allExceptions); + return new ArrayList<>(allExceptions); } /** - * Deserialise and ensure transient fields are re-instantiated when read - * back + * Deserialize and ensure transient fields are re-instantiated when read + * back. + * + * @param stream instance of {@link ObjectInputStream}. + * + * @throws IOException thrown if error occurs during read. + * @throws ClassNotFoundException thrown if class is not found. */ private void readObject(ObjectInputStream stream) throws IOException, ClassNotFoundException { stream.defaultReadObject(); - failureExceptions = new ArrayList(); + failureExceptions = new ArrayList<>(); } /* @@ -345,5 +393,4 @@ public void addStepExecutions(List stepExecutions) { this.stepExecutions.addAll(stepExecutions); } } - } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/JobExecutionException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/JobExecutionException.java index 5e87b63049..471d1500cc 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/JobExecutionException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/JobExecutionException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,6 +24,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class JobExecutionException extends Exception { /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/JobExecutionListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/JobExecutionListener.java index 1cd2fd2b4a..52a24f1227 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/JobExecutionListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/JobExecutionListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/JobInstance.java b/spring-batch-core/src/main/java/org/springframework/batch/core/JobInstance.java index dfe83d5fee..fb64606968 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/JobInstance.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/JobInstance.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,34 +23,37 @@ * JobInstance can be restarted multiple times in case of execution failure and * it's lifecycle ends with first successful execution. * - * Trying to execute an existing JobIntance that has already completed + * Trying to execute an existing JobInstance that has already completed * successfully will result in error. Error will be raised also for an attempt * to restart a failed JobInstance if the Job is not restartable. * * @see Job * @see JobParameters * @see JobExecution + * @see javax.batch.runtime.JobInstance * * @author Lucas Ward * @author Dave Syer * @author Robert Kasanicky * @author Michael Minella + * @author Mahmoud Ben Hassine * */ @SuppressWarnings("serial") -public class JobInstance extends Entity { +public class JobInstance extends Entity implements javax.batch.runtime.JobInstance{ private final String jobName; public JobInstance(Long id, String jobName) { super(id); - Assert.hasLength(jobName); + Assert.hasLength(jobName, "A jobName is required"); this.jobName = jobName; } /** * @return the job name. (Equivalent to getJob().getName()) */ + @Override public String getJobName() { return jobName; } @@ -60,4 +63,8 @@ public String toString() { return super.toString() + ", Job=[" + jobName + "]"; } + @Override + public long getInstanceId() { + return super.getId(); + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/JobInterruptedException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/JobInterruptedException.java index 8d2ba0d51b..8386de9dde 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/JobInterruptedException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/JobInterruptedException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -28,6 +28,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class JobInterruptedException extends JobExecutionException { private BatchStatus status = BatchStatus.STOPPED; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/JobKeyGenerator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/JobKeyGenerator.java index 54e0ba200c..e6008b3b12 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/JobKeyGenerator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/JobKeyGenerator.java @@ -1,11 +1,11 @@ /* - * Copyright 2013-2013 the original author or authors. + * Copyright 2013-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ * unique {@link JobInstance}. * * @author Michael Minella + * @author Mahmoud Ben Hassine * * @param The type of the source data used to calculate the key. * @since 2.2 @@ -29,7 +30,7 @@ public interface JobKeyGenerator { /** * Method to generate the unique key used to identify a job instance. * - * @param source Source information used to generate the key + * @param source Source information used to generate the key (must not be {@code null}) * * @return a unique string identifying the job based on the information * supplied diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/JobParameter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/JobParameter.java index 8938876741..199e64c8bd 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/JobParameter.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/JobParameter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -42,6 +42,8 @@ public class JobParameter implements Serializable { /** * Construct a new JobParameter as a String. + * @param parameter {@link String} instance. + * @param identifying true if JobParameter should be identifying. */ public JobParameter(String parameter, boolean identifying) { this.parameter = parameter; @@ -52,7 +54,8 @@ public JobParameter(String parameter, boolean identifying) { /** * Construct a new JobParameter as a Long. * - * @param parameter + * @param parameter {@link Long} instance. + * @param identifying true if JobParameter should be identifying. */ public JobParameter(Long parameter, boolean identifying) { this.parameter = parameter; @@ -63,7 +66,8 @@ public JobParameter(Long parameter, boolean identifying) { /** * Construct a new JobParameter as a Date. * - * @param parameter + * @param parameter {@link Date} instance. + * @param identifying true if JobParameter should be identifying. */ public JobParameter(Date parameter, boolean identifying) { this.parameter = parameter; @@ -74,7 +78,8 @@ public JobParameter(Date parameter, boolean identifying) { /** * Construct a new JobParameter as a Double. * - * @param parameter + * @param parameter {@link Double} instance. + * @param identifying true if JobParameter should be identifying. */ public JobParameter(Double parameter, boolean identifying) { this.parameter = parameter; @@ -85,6 +90,8 @@ public JobParameter(Double parameter, boolean identifying) { /** * Construct a new JobParameter as a String. + * + * @param parameter {@link String} instance. */ public JobParameter(String parameter) { this.parameter = parameter; @@ -95,7 +102,7 @@ public JobParameter(String parameter) { /** * Construct a new JobParameter as a Long. * - * @param parameter + * @param parameter {@link Long} instance. */ public JobParameter(Long parameter) { this.parameter = parameter; @@ -106,7 +113,7 @@ public JobParameter(Long parameter) { /** * Construct a new JobParameter as a Date. * - * @param parameter + * @param parameter {@link Date} instance. */ public JobParameter(Date parameter) { this.parameter = parameter; @@ -117,7 +124,7 @@ public JobParameter(Date parameter) { /** * Construct a new JobParameter as a Double. * - * @param parameter + * @param parameter {@link Double} instance. */ public JobParameter(Double parameter) { this.parameter = parameter; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/JobParameters.java b/spring-batch-core/src/main/java/org/springframework/batch/core/JobParameters.java index e62ba71c10..74502ef669 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/JobParameters.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/JobParameters.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,9 @@ import java.util.Date; import java.util.LinkedHashMap; import java.util.Map; +import java.util.Properties; + +import org.springframework.lang.Nullable; /** * Value object representing runtime parameters to a batch job. Because the @@ -34,6 +37,7 @@ * * @author Lucas Ward * @author Michael Minella + * @author Mahmoud Ben Hassine * @since 1.0 */ @SuppressWarnings("serial") @@ -42,25 +46,26 @@ public class JobParameters implements Serializable { private final Map parameters; public JobParameters() { - this.parameters = new LinkedHashMap(); + this.parameters = new LinkedHashMap<>(); } public JobParameters(Map parameters) { - this.parameters = new LinkedHashMap(parameters); + this.parameters = new LinkedHashMap<>(parameters); } /** * Typesafe Getter for the Long represented by the provided key. * * @param key The key to get a value for - * @return The Long value + * @return The Long value or {@code null} if the key is absent */ + @Nullable public Long getLong(String key){ if (!parameters.containsKey(key)) { - return 0L; + return null; } Object value = parameters.get(key).getValue(); - return value==null ? 0L : ((Long)value).longValue(); + return value==null ? null : ((Long)value).longValue(); } /** @@ -71,7 +76,10 @@ public Long getLong(String key){ * @param defaultValue to return if the value doesn't exist * @return the parameter represented by the provided key, defaultValue * otherwise. + * @deprecated Use {@link JobParameters#getLong(java.lang.String, java.lang.Long)} + * instead. This method will be removed in a future release. */ + @Deprecated public Long getLong(String key, long defaultValue){ if(parameters.containsKey(key)){ return getLong(key); @@ -81,12 +89,32 @@ public Long getLong(String key, long defaultValue){ } } + /** + * Typesafe Getter for the Long represented by the provided key. If the + * key does not exist, the default value will be returned. + * + * @param key to return the value for + * @param defaultValue to return if the value doesn't exist + * @return the parameter represented by the provided key, defaultValue + * otherwise. + */ + @Nullable + public Long getLong(String key, @Nullable Long defaultValue){ + if(parameters.containsKey(key)){ + return getLong(key); + } + else{ + return defaultValue; + } + } + /** * Typesafe Getter for the String represented by the provided key. * * @param key The key to get a value for - * @return The String value + * @return The String value or {@code null} if the key is absent */ + @Nullable public String getString(String key){ JobParameter value = parameters.get(key); return value==null ? null : value.toString(); @@ -101,7 +129,8 @@ public String getString(String key){ * @return the parameter represented by the provided key, defaultValue * otherwise. */ - public String getString(String key, String defaultValue){ + @Nullable + public String getString(String key, @Nullable String defaultValue){ if(parameters.containsKey(key)){ return getString(key); } @@ -114,14 +143,15 @@ public String getString(String key, String defaultValue){ * Typesafe Getter for the Long represented by the provided key. * * @param key The key to get a value for - * @return The Double value + * @return The Double value or {@code null} if the key is absent */ + @Nullable public Double getDouble(String key){ if (!parameters.containsKey(key)) { - return 0.0; + return null; } Double value = (Double)parameters.get(key).getValue(); - return value==null ? 0.0 : value.doubleValue(); + return value==null ? null : value.doubleValue(); } /** @@ -132,7 +162,10 @@ public Double getDouble(String key){ * @param defaultValue to return if the value doesn't exist * @return the parameter represented by the provided key, defaultValue * otherwise. + * @deprecated Use {@link JobParameters#getDouble(java.lang.String, java.lang.Double)} + * instead. This method will be removed in a future release. */ + @Deprecated public Double getDouble(String key, double defaultValue){ if(parameters.containsKey(key)){ return getDouble(key); @@ -142,12 +175,33 @@ public Double getDouble(String key, double defaultValue){ } } + /** + * Typesafe Getter for the Double represented by the provided key. If the + * key does not exist, the default value will be returned. + * + * @param key to return the value for + * @param defaultValue to return if the value doesn't exist + * @return the parameter represented by the provided key, defaultValue + * otherwise. + */ + @Nullable + public Double getDouble(String key, @Nullable Double defaultValue){ + if(parameters.containsKey(key)){ + return getDouble(key); + } + else{ + return defaultValue; + } + } + /** * Typesafe Getter for the Date represented by the provided key. * * @param key The key to get a value for - * @return The java.util.Date value + * @return The java.util.Date value or {@code null} if the key + * is absent */ + @Nullable public Date getDate(String key){ return this.getDate(key,null); } @@ -161,7 +215,8 @@ public Date getDate(String key){ * @return the parameter represented by the provided key, defaultValue * otherwise. */ - public Date getDate(String key, Date defaultValue){ + @Nullable + public Date getDate(String key, @Nullable Date defaultValue){ if(parameters.containsKey(key)){ return (Date)parameters.get(key).getValue(); } @@ -176,7 +231,7 @@ public Date getDate(String key, Date defaultValue){ * @return an unmodifiable map containing all parameters. */ public Map getParameters(){ - return new LinkedHashMap(parameters); + return new LinkedHashMap<>(parameters); } /** @@ -209,4 +264,16 @@ public int hashCode() { public String toString() { return parameters.toString(); } + + public Properties toProperties() { + Properties props = new Properties(); + + for (Map.Entry param : parameters.entrySet()) { + if(param.getValue() != null) { + props.put(param.getKey(), param.getValue().toString()); + } + } + + return props; + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersBuilder.java index e4e5de7a54..6b194fb645 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersBuilder.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,45 +17,86 @@ package org.springframework.batch.core; import java.util.Date; +import java.util.HashMap; import java.util.LinkedHashMap; import java.util.Map; +import java.util.Properties; +import org.springframework.batch.core.explore.JobExplorer; import org.springframework.util.Assert; /** * Helper class for creating {@link JobParameters}. Useful because all * {@link JobParameter} objects are immutable, and must be instantiated separately - * to ensure typesafety. Once created, it can be used in the + * to ensure type safety. Once created, it can be used in the * same was a java.lang.StringBuilder (except, order is irrelevant), by adding * various parameter types and creating a valid {@link JobParameters} once - * finished.
      - *
      + * finished.
      + *
      * Using the identifying flag indicates if the parameter will be used * in the identification of a JobInstance. That flag defaults to true. * * @author Lucas Ward * @author Michael Minella + * @author Glenn Renfro + * @author Mahmoud Ben Hassine * @since 1.0 * @see JobParameters * @see JobParameter */ public class JobParametersBuilder { - private final Map parameterMap; + private Map parameterMap; + + private JobExplorer jobExplorer; /** * Default constructor. Initializes the builder with empty parameters. */ public JobParametersBuilder() { + this.parameterMap = new LinkedHashMap<>(); + } - this.parameterMap = new LinkedHashMap(); + /** + * @param jobExplorer {@link JobExplorer} used for looking up previous job parameter information + */ + public JobParametersBuilder(JobExplorer jobExplorer) { + this.jobExplorer = jobExplorer; + this.parameterMap = new LinkedHashMap<>(); } /** * Copy constructor. Initializes the builder with the supplied parameters. + * @param jobParameters {@link JobParameters} instance used to initialize the builder. */ public JobParametersBuilder(JobParameters jobParameters) { - this.parameterMap = new LinkedHashMap(jobParameters.getParameters()); + this(jobParameters, null); + } + + /** + * Constructor to add conversion capabilities to support JSR-352. Per the spec, it is expected that all + * keys and values in the provided {@link Properties} instance are Strings + * + * @param properties the job parameters to be used + */ + public JobParametersBuilder(Properties properties) { + this.parameterMap = new LinkedHashMap<>(); + + if(properties != null) { + for (Map.Entry curProperty : properties.entrySet()) { + this.parameterMap.put((String) curProperty.getKey(), new JobParameter((String) curProperty.getValue(), false)); + } + } + } + + /** + * Copy constructor. Initializes the builder with the supplied parameters. + * @param jobParameters {@link JobParameters} instance used to initialize the builder. + * @param jobExplorer {@link JobExplorer} used for looking up previous job parameter information + */ + public JobParametersBuilder(JobParameters jobParameters, JobExplorer jobExplorer) { + this.jobExplorer = jobExplorer; + this.parameterMap = new LinkedHashMap<>(jobParameters.getParameters()); } /** @@ -66,7 +107,7 @@ public JobParametersBuilder(JobParameters jobParameters) { * @return a reference to this object. */ public JobParametersBuilder addString(String key, String parameter) { - parameterMap.put(key, new JobParameter(parameter, true)); + this.parameterMap.put(key, new JobParameter(parameter, true)); return this; } @@ -79,7 +120,7 @@ public JobParametersBuilder addString(String key, String parameter) { * @return a reference to this object. */ public JobParametersBuilder addString(String key, String parameter, boolean identifying) { - parameterMap.put(key, new JobParameter(parameter, identifying)); + this.parameterMap.put(key, new JobParameter(parameter, identifying)); return this; } @@ -91,7 +132,7 @@ public JobParametersBuilder addString(String key, String parameter, boolean iden * @return a reference to this object. */ public JobParametersBuilder addDate(String key, Date parameter) { - parameterMap.put(key, new JobParameter(parameter, true)); + this.parameterMap.put(key, new JobParameter(parameter, true)); return this; } @@ -104,7 +145,7 @@ public JobParametersBuilder addDate(String key, Date parameter) { * @return a reference to this object. */ public JobParametersBuilder addDate(String key, Date parameter, boolean identifying) { - parameterMap.put(key, new JobParameter(parameter, identifying)); + this.parameterMap.put(key, new JobParameter(parameter, identifying)); return this; } @@ -116,7 +157,7 @@ public JobParametersBuilder addDate(String key, Date parameter, boolean identify * @return a reference to this object. */ public JobParametersBuilder addLong(String key, Long parameter) { - parameterMap.put(key, new JobParameter(parameter, true)); + this.parameterMap.put(key, new JobParameter(parameter, true)); return this; } @@ -129,7 +170,7 @@ public JobParametersBuilder addLong(String key, Long parameter) { * @return a reference to this object. */ public JobParametersBuilder addLong(String key, Long parameter, boolean identifying) { - parameterMap.put(key, new JobParameter(parameter, identifying)); + this.parameterMap.put(key, new JobParameter(parameter, identifying)); return this; } @@ -141,7 +182,7 @@ public JobParametersBuilder addLong(String key, Long parameter, boolean identify * @return a reference to this object. */ public JobParametersBuilder addDouble(String key, Double parameter) { - parameterMap.put(key, new JobParameter(parameter, true)); + this.parameterMap.put(key, new JobParameter(parameter, true)); return this; } @@ -154,18 +195,18 @@ public JobParametersBuilder addDouble(String key, Double parameter) { * @return a reference to this object. */ public JobParametersBuilder addDouble(String key, Double parameter, boolean identifying) { - parameterMap.put(key, new JobParameter(parameter, identifying)); + this.parameterMap.put(key, new JobParameter(parameter, identifying)); return this; } /** * Conversion method that takes the current state of this builder and - * returns it as a JobruntimeParameters object. + * returns it as a JobParameters object. * * @return a valid {@link JobParameters} object. */ public JobParameters toJobParameters() { - return new JobParameters(parameterMap); + return new JobParameters(this.parameterMap); } /** @@ -177,7 +218,65 @@ public JobParameters toJobParameters() { */ public JobParametersBuilder addParameter(String key, JobParameter jobParameter) { Assert.notNull(jobParameter, "JobParameter must not be null"); - parameterMap.put(key, jobParameter); + this.parameterMap.put(key, jobParameter); + return this; + } + + /** + * Copy job parameters into the current state. + * @param jobParameters parameters to copy in + * @return a reference to this object. + */ + public JobParametersBuilder addJobParameters(JobParameters jobParameters) { + Assert.notNull(jobParameters, "jobParameters must not be null"); + + this.parameterMap.putAll(jobParameters.getParameters()); + + return this; + } + + /** + * Initializes the {@link JobParameters} based on the state of the {@link Job}. This + * should be called after all parameters have been entered into the builder. + * All parameters already set on this builder instance will be appended to + * those retrieved from the job incrementer, overriding any with the same key (Same + * behaviour as {@link org.springframework.batch.core.launch.support.CommandLineJobRunner} + * with "-next" option and {@link org.springframework.batch.core.launch.JobOperator#startNextInstance(String)}) + * + * @param job the job for which the {@link JobParameters} are being constructed. + * @return a reference to this object. + * + * @since 4.0 + */ + public JobParametersBuilder getNextJobParameters(Job job) { + Assert.state(this.jobExplorer != null, "A JobExplorer is required to get next job parameters"); + Assert.notNull(job, "Job must not be null"); + Assert.notNull(job.getJobParametersIncrementer(), "No job parameters incrementer found for job=" + job.getName()); + + String name = job.getName(); + JobParameters nextParameters; + JobInstance lastInstance = this.jobExplorer.getLastJobInstance(name); + JobParametersIncrementer incrementer = job.getJobParametersIncrementer(); + if (lastInstance == null) { + // Start from a completely clean sheet + nextParameters = incrementer.getNext(new JobParameters()); + } + else { + JobExecution previousExecution = this.jobExplorer.getLastJobExecution(lastInstance); + if (previousExecution == null) { + // Normally this will not happen - an instance exists with no executions + nextParameters = incrementer.getNext(new JobParameters()); + } + else { + nextParameters = incrementer.getNext(previousExecution.getJobParameters()); + } + } + + // start with parameters from the incrementer + Map nextParametersMap = new HashMap<>(nextParameters.getParameters()); + // append new parameters (overriding those with the same key) + nextParametersMap.putAll(this.parameterMap); + this.parameterMap = nextParametersMap; return this; } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersIncrementer.java b/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersIncrementer.java index be86243bc2..e07ee14ac0 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersIncrementer.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersIncrementer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,11 +15,14 @@ */ package org.springframework.batch.core; +import org.springframework.lang.Nullable; + /** * Interface for obtaining the next {@link JobParameters} in a sequence. * * @author Dave Syer * @author Lucas Ward + * @author Mahmoud Ben Hassine * @since 2.0 */ public interface JobParametersIncrementer { @@ -30,8 +33,8 @@ public interface JobParametersIncrementer { * instance of a job. * * @param parameters the last value used - * @return the next value to use + * @return the next value to use (never {@code null}) */ - JobParameters getNext(JobParameters parameters); + JobParameters getNext(@Nullable JobParameters parameters); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersInvalidException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersInvalidException.java index cbb0eab7ed..ad0a2e091d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersInvalidException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersInvalidException.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core; /** @@ -7,6 +22,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class JobParametersInvalidException extends JobExecutionException { public JobParametersInvalidException(String msg) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersValidator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersValidator.java index 8e21e2ada1..15eecabadf 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersValidator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/JobParametersValidator.java @@ -1,12 +1,29 @@ +/* + * Copyright 2010-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core; +import org.springframework.lang.Nullable; /** * Strategy interface for a {@link Job} to use in validating its parameters for * an execution. * * @author Dave Syer - * + * @author Mahmoud Ben Hassine + * */ public interface JobParametersValidator { @@ -14,9 +31,9 @@ public interface JobParametersValidator { * Check the parameters meet whatever requirements are appropriate, and * throw an exception if not. * - * @param parameters some {@link JobParameters} + * @param parameters some {@link JobParameters} (can be {@code null}) * @throws JobParametersInvalidException if the parameters are invalid */ - void validate(JobParameters parameters) throws JobParametersInvalidException; + void validate(@Nullable JobParameters parameters) throws JobParametersInvalidException; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/SkipListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/SkipListener.java index feac11c37a..a3e7afd08d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/SkipListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/SkipListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/StartLimitExceededException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/StartLimitExceededException.java index 7776b2b71c..2b78792245 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/StartLimitExceededException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/StartLimitExceededException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,6 +19,7 @@ /** * Indicates the step's start limit has been exceeded. */ +@SuppressWarnings("serial") public class StartLimitExceededException extends RuntimeException { public StartLimitExceededException(String message) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/Step.java b/spring-batch-core/src/main/java/org/springframework/batch/core/Step.java index df0d610961..3f7f2eae74 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/Step.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/Step.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,6 +24,7 @@ */ public interface Step { + static final String STEP_TYPE_KEY = "batch.stepType"; /** * @return the name of this step. */ @@ -42,7 +43,7 @@ public interface Step { /** * Process the step and assign progress and status meta information to the {@link StepExecution} provided. The * {@link Step} is responsible for setting the meta information and also saving it if required by the - * implementation.
      + * implementation.
      * * It is not safe to re-use an instance of {@link Step} to process multiple concurrent executions. * diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/StepContribution.java b/spring-batch-core/src/main/java/org/springframework/batch/core/StepContribution.java index 6bffd54f39..28bde4bcf2 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/StepContribution.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/StepContribution.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ * they can be applied at a chunk boundary. * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ @SuppressWarnings("serial") @@ -43,17 +44,21 @@ public class StepContribution implements Serializable { private ExitStatus exitStatus = ExitStatus.EXECUTING; + private volatile StepExecution stepExecution; + /** - * @param execution + * @param execution {@link StepExecution} the stepExecution used to initialize + * {@code skipCount}. */ public StepContribution(StepExecution execution) { + this.stepExecution = execution; this.parentSkipCount = execution.getSkipCount(); } /** * Set the {@link ExitStatus} for this contribution. * - * @param status + * @param status {@link ExitStatus} instance to be used to set the exit status. */ public void setExitStatus(ExitStatus status) { this.exitStatus = status; @@ -70,6 +75,8 @@ public ExitStatus getExitStatus() { /** * Increment the counter for the number of items processed. + * + * @param count int amount to increment by. */ public void incrementFilterCount(int count) { filterCount += count; @@ -84,6 +91,8 @@ public void incrementReadCount() { /** * Increment the counter for the number of items written. + * + * @param count int amount to increment by. */ public void incrementWriteCount(int count) { writeCount += count; @@ -109,6 +118,7 @@ public int getWriteCount() { /** * Public getter for the filter counter. + * * @return the filter counter */ public int getFilterCount() { @@ -141,6 +151,8 @@ public void incrementReadSkipCount() { /** * Increment the read skip count for this contribution + * + * @param count int amount to increment by. */ public void incrementReadSkipCount(int count) { readSkipCount += count; @@ -176,12 +188,21 @@ public int getWriteSkipCount() { /** * Public getter for the process skip count. + * * @return the process skip count */ public int getProcessSkipCount() { return processSkipCount; } + /** + * Public getter for the parent step execution of this contribution. + * @return parent step execution of this contribution + */ + public StepExecution getStepExecution() { + return stepExecution; + } + /* * (non-Javadoc) * diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/StepExecution.java b/spring-batch-core/src/main/java/org/springframework/batch/core/StepExecution.java index 96fa8a4fc5..a51e2273f8 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/StepExecution.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/StepExecution.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -72,7 +72,7 @@ public class StepExecution extends Entity { private volatile int filterCount; - private transient volatile List failureExceptions = new CopyOnWriteArrayList(); + private transient volatile List failureExceptions = new CopyOnWriteArrayList<>(); /** * Constructor with mandatory properties. @@ -97,11 +97,26 @@ public StepExecution(String stepName, JobExecution jobExecution, Long id) { */ public StepExecution(String stepName, JobExecution jobExecution) { super(); - Assert.hasLength(stepName); + Assert.hasLength(stepName, "A stepName is required"); this.stepName = stepName; this.jobExecution = jobExecution; } + /** + * Constructor that requires only a stepName. Intended only to be + * used via serialization libraries to address the circular + * reference between {@link JobExecution} and StepExecution. + * + * @param stepName the name of the executed step + */ + @SuppressWarnings("unused") + private StepExecution(String stepName) { + super(); + Assert.hasLength(stepName, "A stepName is required"); + this.stepName = stepName; + this.jobExecution = null; + } + /** * Returns the {@link ExecutionContext} for this execution * @@ -221,6 +236,7 @@ public void setFilterCount(int filterCount) { /** * Setter for number of rollbacks for this execution + * @param rollbackCount int the number of rollbacks. */ public void setRollbackCount(int rollbackCount) { this.rollbackCount = rollbackCount; @@ -293,7 +309,7 @@ public Long getJobExecutionId() { } /** - * @param exitStatus + * @param exitStatus {@link ExitStatus} instance used to establish the exit status. */ public void setExitStatus(ExitStatus exitStatus) { this.exitStatus = exitStatus; @@ -330,7 +346,7 @@ public StepContribution createStepContribution() { * called. Synchronizes access to the {@link StepExecution} so that changes * are atomic. * - * @param contribution + * @param contribution {@link StepContribution} instance used to update the StepExecution state. */ public synchronized void apply(StepContribution contribution) { readSkipCount += contribution.getReadSkipCount(); @@ -408,7 +424,7 @@ public int getWriteSkipCount() { /** * Set the number of records skipped on read * - * @param readSkipCount + * @param readSkipCount int containing read skip count to be used for the step execution. */ public void setReadSkipCount(int readSkipCount) { this.readSkipCount = readSkipCount; @@ -417,7 +433,7 @@ public void setReadSkipCount(int readSkipCount) { /** * Set the number of records skipped on write * - * @param writeSkipCount + * @param writeSkipCount int containing write skip count to be used for the step execution. */ public void setWriteSkipCount(int writeSkipCount) { this.writeSkipCount = writeSkipCount; @@ -433,7 +449,7 @@ public int getProcessSkipCount() { /** * Set the number of records skipped during processing. * - * @param processSkipCount + * @param processSkipCount int containing process skip count to be used for the step execution. */ public void setProcessSkipCount(int processSkipCount) { this.processSkipCount = processSkipCount; @@ -449,7 +465,8 @@ public Date getLastUpdated() { /** * Set the time when the StepExecution was last updated before persisting * - * @param lastUpdated + * @param lastUpdated {@link Date} instance used to establish the last + * updated date for the Step Execution. */ public void setLastUpdated(Date lastUpdated) { this.lastUpdated = lastUpdated; @@ -484,12 +501,17 @@ public boolean equals(Object obj) { } /** - * Deserialise and ensure transient fields are re-instantiated when read - * back + * Deserialize and ensure transient fields are re-instantiated when read + * back. + * + * @param stream instance of {@link ObjectInputStream}. + * + * @throws IOException thrown if error occurs during read. + * @throws ClassNotFoundException thrown if class is not found. */ private void readObject(ObjectInputStream stream) throws IOException, ClassNotFoundException { stream.defaultReadObject(); - failureExceptions = new ArrayList(); + failureExceptions = new ArrayList<>(); } /* diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/StepExecutionListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/StepExecutionListener.java index 04a668e6a0..0bb362df88 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/StepExecutionListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/StepExecutionListener.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,12 +15,14 @@ */ package org.springframework.batch.core; +import org.springframework.lang.Nullable; /** * Listener interface for the lifecycle of a {@link Step}. * * @author Lucas Ward * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public interface StepExecutionListener extends StepListener { @@ -29,7 +31,7 @@ public interface StepExecutionListener extends StepListener { * Initialize the state of the listener with the {@link StepExecution} from * the current scope. * - * @param stepExecution + * @param stepExecution instance of {@link StepExecution}. */ void beforeStep(StepExecution stepExecution); @@ -42,8 +44,10 @@ public interface StepExecutionListener extends StepListener { * failed). Throwing exception in this method has no effect, it will only be * logged. * + * @param stepExecution {@link StepExecution} instance. * @return an {@link ExitStatus} to combine with the normal value. Return - * null to leave the old value unchanged. + * {@code null} to leave the old value unchanged. */ + @Nullable ExitStatus afterStep(StepExecution stepExecution); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/StepListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/StepListener.java index 0d970a7848..005bab04b9 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/StepListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/StepListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/UnexpectedJobExecutionException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/UnexpectedJobExecutionException.java index bbf4f23975..ec5aab2831 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/UnexpectedJobExecutionException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/UnexpectedJobExecutionException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,6 +40,7 @@ public UnexpectedJobExecutionException(String msg) { * Constructs a new instance with a message. * * @param msg the exception message. + * @param nested instance of {@link Throwable} that is the cause of the exception. * */ public UnexpectedJobExecutionException(String msg, Throwable nested) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterChunk.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterChunk.java index 2a0366f598..dd4404636f 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterChunk.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterChunk.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,17 +15,17 @@ */ package org.springframework.batch.core.annotation; +import org.springframework.batch.core.ChunkListener; +import org.springframework.batch.core.scope.context.ChunkContext; + import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; -import org.springframework.batch.core.ChunkListener; -import org.springframework.batch.core.scope.context.ChunkContext; - /** * Marks a method to be called after a chunk is executed.
      - *
      + *
      * Expected signature: void afterChunk(ChunkContext context) * * @author Lucas Ward diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterChunkError.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterChunkError.java index eba8a66a23..e4262746b0 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterChunkError.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterChunkError.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,18 +15,18 @@ */ package org.springframework.batch.core.annotation; +import org.springframework.batch.core.ChunkListener; +import org.springframework.batch.core.scope.context.ChunkContext; + import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; -import org.springframework.batch.core.ChunkListener; -import org.springframework.batch.core.scope.context.ChunkContext; - /** * Marks a method to be called after a has failed and been * marked for rollback.
      - *
      + *
      * Expected signature: void afterFailedChunk(ChunkContext context) * * @author Michael Minella diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterJob.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterJob.java index 94ecf5b343..e2aa3cdf05 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterJob.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterJob.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterProcess.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterProcess.java index 652040c0f8..107139653e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterProcess.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterProcess.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterRead.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterRead.java index 2d99719f45..6c2562af0b 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterRead.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterRead.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterStep.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterStep.java index 2e30bc310a..bdbb376ecc 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterStep.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterStep.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterWrite.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterWrite.java index d253fbb293..516afb11a1 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterWrite.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/AfterWrite.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeChunk.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeChunk.java index fa86e9ab08..6068726e99 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeChunk.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeChunk.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeJob.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeJob.java index 318320953b..ff2c077f42 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeJob.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeJob.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeProcess.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeProcess.java index 585bb26c9e..41e11ff467 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeProcess.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeProcess.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeRead.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeRead.java index 5d03df98b0..efe6f2fa65 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeRead.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeRead.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeStep.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeStep.java index a32afccd6d..d6daaf65ca 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeStep.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeStep.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeWrite.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeWrite.java index db025cb663..9e558ad73d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeWrite.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/BeforeWrite.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnProcessError.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnProcessError.java index ea4be526a1..828793ad75 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnProcessError.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnProcessError.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnReadError.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnReadError.java index 00c69d2940..971e4b7dae 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnReadError.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnReadError.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnSkipInProcess.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnSkipInProcess.java index 92468c0007..71f49852f4 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnSkipInProcess.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnSkipInProcess.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnSkipInRead.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnSkipInRead.java index 3387795b75..b733461fa0 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnSkipInRead.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnSkipInRead.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnSkipInWrite.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnSkipInWrite.java index c4798267be..2f8ffca343 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnSkipInWrite.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnSkipInWrite.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnWriteError.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnWriteError.java index 466a8459f4..c4c9f7d876 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnWriteError.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/OnWriteError.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/package-info.java new file mode 100644 index 0000000000..bb88ba4d60 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/annotation/package-info.java @@ -0,0 +1,6 @@ +/** + * Annotations for java based configuration of listeners. + * + * @author Michael Minella + */ +package org.springframework.batch.core.annotation; \ No newline at end of file diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/BatchConfigurationException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/BatchConfigurationException.java new file mode 100644 index 0000000000..c3b2f0dbd4 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/BatchConfigurationException.java @@ -0,0 +1,37 @@ +/* + * Copyright 2014-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.configuration; + +/** + * Represents an error has occurred in the configuration of base batch + * infrastructure (creation of a {@link org.springframework.batch.core.repository.JobRepository} + * for example. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @since 2.2.6 + */ +public class BatchConfigurationException extends RuntimeException { + + private static final long serialVersionUID = 1L; + + /** + * @param t an exception to be wrapped + */ + public BatchConfigurationException(Throwable t) { + super(t); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/DuplicateJobException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/DuplicateJobException.java index 777ead7435..993c3e9757 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/DuplicateJobException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/DuplicateJobException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,18 +25,21 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class DuplicateJobException extends JobExecutionException { /** * Create an exception with the given message. + * + * @param msg error message. */ public DuplicateJobException(String msg) { super(msg); } /** - * @param msg The message to send to caller - * @param e the cause of the exception + * @param msg error message. + * @param e instance of {@link Throwable} that is the cause of the exception. */ public DuplicateJobException(String msg, Throwable e) { super(msg, e); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/JobFactory.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/JobFactory.java index 95c7d35def..be4ce491cf 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/JobFactory.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/JobFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/JobLocator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/JobLocator.java index a7e6f9f836..a22c4494fd 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/JobLocator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/JobLocator.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,12 +17,14 @@ import org.springframework.batch.core.Job; import org.springframework.batch.core.launch.NoSuchJobException; +import org.springframework.lang.Nullable; /** * A runtime service locator interface for retrieving job configurations by * name. * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public interface JobLocator { @@ -37,5 +39,5 @@ public interface JobLocator { * @throws NoSuchJobException if the required configuration can * not be found. */ - Job getJob(String name) throws NoSuchJobException; + Job getJob(@Nullable String name) throws NoSuchJobException; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/JobRegistry.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/JobRegistry.java index 3415750645..ccb99e9fd1 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/JobRegistry.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/JobRegistry.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/ListableJobLocator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/ListableJobLocator.java index 1a017a448f..a3ca54a075 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/ListableJobLocator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/ListableJobLocator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/StepRegistry.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/StepRegistry.java index c3e857c38f..5a3d7384a9 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/StepRegistry.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/StepRegistry.java @@ -1,3 +1,18 @@ +/* + * Copyright 2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration; import org.springframework.batch.core.Step; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/AbstractBatchConfiguration.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/AbstractBatchConfiguration.java index 0967c2e8bb..6319bb41e5 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/AbstractBatchConfiguration.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/AbstractBatchConfiguration.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,17 +15,14 @@ */ package org.springframework.batch.core.configuration.annotation; -import java.util.Collection; - -import javax.sql.DataSource; - import org.springframework.batch.core.configuration.JobRegistry; import org.springframework.batch.core.configuration.support.MapJobRegistry; +import org.springframework.batch.core.explore.JobExplorer; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.scope.JobScope; import org.springframework.batch.core.scope.StepScope; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.context.ApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Import; @@ -35,23 +32,25 @@ import org.springframework.transaction.PlatformTransactionManager; import org.springframework.util.Assert; +import javax.sql.DataSource; +import java.util.Collection; + /** * Base {@code Configuration} class providing common structure for enabling and using Spring Batch. Customization is * available by implementing the {@link BatchConfigurer} interface. {@link BatchConfigurer}. * * @author Dave Syer + * @author Michael Minella + * @author Mahmoud Ben Hassine * @since 2.2 * @see EnableBatchProcessing */ @Configuration -@Import(StepScopeConfiguration.class) +@Import(ScopeConfiguration.class) public abstract class AbstractBatchConfiguration implements ImportAware { - @Autowired - private ApplicationContext context; - @Autowired(required = false) - private Collection dataSources; + private DataSource dataSource; private BatchConfigurer configurer; @@ -71,6 +70,9 @@ public StepBuilderFactory stepBuilders() throws Exception { @Bean public abstract JobLauncher jobLauncher() throws Exception; + @Bean + public abstract JobExplorer jobExplorer() throws Exception; + @Bean public JobRegistry jobRegistry() throws Exception { return new MapJobRegistry(); @@ -92,16 +94,17 @@ protected BatchConfigurer getConfigurer(Collection configurers) return this.configurer; } if (configurers == null || configurers.isEmpty()) { - if (dataSources == null || dataSources.isEmpty() || dataSources.size() > 1) { - throw new IllegalStateException( - "To use the default BatchConfigurer the context must contain precisely one DataSource, found " - + (dataSources == null ? 0 : dataSources.size())); + if (dataSource == null) { + DefaultBatchConfigurer configurer = new DefaultBatchConfigurer(); + configurer.initialize(); + this.configurer = configurer; + return configurer; + } else { + DefaultBatchConfigurer configurer = new DefaultBatchConfigurer(dataSource); + configurer.initialize(); + this.configurer = configurer; + return configurer; } - DataSource dataSource = dataSources.iterator().next(); - DefaultBatchConfigurer configurer = new DefaultBatchConfigurer(dataSource); - configurer.initialize(); - this.configurer = configurer; - return configurer; } if (configurers.size() > 1) { throw new IllegalStateException( @@ -115,20 +118,26 @@ protected BatchConfigurer getConfigurer(Collection configurers) } /** - * Extract step scope configuration into a separate unit so that it can be non-static. + * Extract job/step scope configuration into a separate unit. * * @author Dave Syer * */ @Configuration -class StepScopeConfiguration { - - private StepScope stepScope = new StepScope(); +class ScopeConfiguration { @Bean - public StepScope stepScope() { + public static StepScope stepScope() { + StepScope stepScope = new StepScope(); stepScope.setAutoProxy(false); return stepScope; } -} \ No newline at end of file + @Bean + public static JobScope jobScope() { + JobScope jobScope = new JobScope(); + jobScope.setAutoProxy(false); + return jobScope; + } + +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/BatchConfigurationSelector.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/BatchConfigurationSelector.java index 19a93a81c6..609d9fb12f 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/BatchConfigurationSelector.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/BatchConfigurationSelector.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/BatchConfigurer.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/BatchConfigurer.java index e5064d5ab7..95befd9a47 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/BatchConfigurer.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/BatchConfigurer.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,6 +15,7 @@ */ package org.springframework.batch.core.configuration.annotation; +import org.springframework.batch.core.explore.JobExplorer; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.repository.JobRepository; import org.springframework.transaction.PlatformTransactionManager; @@ -33,4 +34,5 @@ public interface BatchConfigurer { JobLauncher getJobLauncher() throws Exception; + JobExplorer getJobExplorer() throws Exception; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/DefaultBatchConfigurer.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/DefaultBatchConfigurer.java index 8ad30f99df..4d99ddf1e5 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/DefaultBatchConfigurer.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/DefaultBatchConfigurer.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,10 +18,19 @@ import javax.annotation.PostConstruct; import javax.sql.DataSource; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import org.springframework.batch.core.configuration.BatchConfigurationException; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.explore.support.JobExplorerFactoryBean; +import org.springframework.batch.core.explore.support.MapJobExplorerFactoryBean; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.launch.support.SimpleJobLauncher; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.support.JobRepositoryFactoryBean; +import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; +import org.springframework.batch.support.transaction.ResourcelessTransactionManager; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.datasource.DataSourceTransactionManager; import org.springframework.stereotype.Component; @@ -29,16 +38,31 @@ @Component public class DefaultBatchConfigurer implements BatchConfigurer { + private static final Log logger = LogFactory.getLog(DefaultBatchConfigurer.class); private DataSource dataSource; private PlatformTransactionManager transactionManager; private JobRepository jobRepository; private JobLauncher jobLauncher; + private JobExplorer jobExplorer; - @Autowired + /** + * Sets the dataSource. If the {@link DataSource} has been set once, all future + * values are passed are ignored (to prevent {@code}@Autowired{@code} from overwriting + * the value). + * + * @param dataSource The data source to use + */ + @Autowired(required = false) public void setDataSource(DataSource dataSource) { - this.dataSource = dataSource; - this.transactionManager = new DataSourceTransactionManager(dataSource); + if(this.dataSource == null) { + this.dataSource = dataSource; + } + + if(getTransactionManager() == null) { + logger.warn("No transaction manager was provided, using a DataSourceTransactionManager"); + this.transactionManager = new DataSourceTransactionManager(this.dataSource); + } } protected DefaultBatchConfigurer() {} @@ -62,13 +86,41 @@ public JobLauncher getJobLauncher() { return jobLauncher; } + @Override + public JobExplorer getJobExplorer() { + return jobExplorer; + } + @PostConstruct - public void initialize() throws Exception { - this.jobRepository = createJobRepository(); - this.jobLauncher = createJobLauncher(); + public void initialize() { + try { + if(dataSource == null) { + logger.warn("No datasource was provided...using a Map based JobRepository"); + + if(getTransactionManager() == null) { + logger.warn("No transaction manager was provided, using a ResourcelessTransactionManager"); + this.transactionManager = new ResourcelessTransactionManager(); + } + + MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean(getTransactionManager()); + jobRepositoryFactory.afterPropertiesSet(); + this.jobRepository = jobRepositoryFactory.getObject(); + + MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory); + jobExplorerFactory.afterPropertiesSet(); + this.jobExplorer = jobExplorerFactory.getObject(); + } else { + this.jobRepository = createJobRepository(); + this.jobExplorer = createJobExplorer(); + } + + this.jobLauncher = createJobLauncher(); + } catch (Exception e) { + throw new BatchConfigurationException(e); + } } - private JobLauncher createJobLauncher() throws Exception { + protected JobLauncher createJobLauncher() throws Exception { SimpleJobLauncher jobLauncher = new SimpleJobLauncher(); jobLauncher.setJobRepository(jobRepository); jobLauncher.afterPropertiesSet(); @@ -78,9 +130,15 @@ private JobLauncher createJobLauncher() throws Exception { protected JobRepository createJobRepository() throws Exception { JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean(); factory.setDataSource(dataSource); - factory.setTransactionManager(transactionManager); + factory.setTransactionManager(getTransactionManager()); factory.afterPropertiesSet(); - return (JobRepository) factory.getObject(); + return factory.getObject(); } + protected JobExplorer createJobExplorer() throws Exception { + JobExplorerFactoryBean jobExplorerFactoryBean = new JobExplorerFactoryBean(); + jobExplorerFactoryBean.setDataSource(this.dataSource); + jobExplorerFactoryBean.afterPropertiesSet(); + return jobExplorerFactoryBean.getObject(); + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/EnableBatchProcessing.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/EnableBatchProcessing.java index 177d58d6e0..91b06ef660 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/EnableBatchProcessing.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/EnableBatchProcessing.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,19 +20,21 @@ import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; +import javax.sql.DataSource; import org.springframework.batch.core.configuration.JobRegistry; import org.springframework.batch.core.configuration.support.ApplicationContextFactory; import org.springframework.batch.core.configuration.support.AutomaticJobRegistrar; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.repository.JobRepository; +import org.springframework.beans.factory.UnsatisfiedDependencyException; import org.springframework.context.annotation.Import; import org.springframework.transaction.PlatformTransactionManager; /** *

      * Enable Spring Batch features and provide a base configuration for setting up batch jobs in an @Configuration - * class, roughly equivalent to using the {@code } XML namespace. + * class, roughly equivalent to using the {@code } XML namespace.

      * *
        * @Configuration
      @@ -60,7 +62,7 @@
        * }
        * 
      * - * The user has to provide a {@link DataSource} as a bean in the context, or else implement {@link BatchConfigurer} in + * The user should to provide a {@link DataSource} as a bean in the context, or else implement {@link BatchConfigurer} in * the configuration class itself, e.g. * *
      @@ -83,15 +85,25 @@
        * }
        * 
      * + * If a user does not provide a {@link javax.sql.DataSource} within the context, a Map based + * {@link org.springframework.batch.core.repository.JobRepository} will be used. If multiple + * {@link javax.sql.DataSource}s are defined in the context, the one annotated with + * {@link org.springframework.context.annotation.Primary} will be used (Note that if none + * of them is annotated with {@link org.springframework.context.annotation.Primary}, the one + * named dataSource will be used if any, otherwise a {@link UnsatisfiedDependencyException} + * will be thrown). + * * Note that only one of your configuration classes needs to have the @EnableBatchProcessing * annotation. Once you have an @EnableBatchProcessing class in your configuration you will have an - * instance of {@link StepScope} so your beans inside steps can have @Scope("step"). You will also be + * instance of {@link StepScope} and {@link org.springframework.batch.core.scope.JobScope} so your beans inside steps + * can have @Scope("step") and @Scope("job") respectively. You will also be * able to @Autowired some useful stuff into your context: * *
        *
      • a {@link JobRepository} (bean name "jobRepository")
      • *
      • a {@link JobLauncher} (bean name "jobLauncher")
      • *
      • a {@link JobRegistry} (bean name "jobRegistry")
      • + *
      • a {@link org.springframework.batch.core.explore.JobExplorer} (bean name "jobExplorer")
      • *
      • a {@link PlatformTransactionManager} (bean name "transactionManager")
      • *
      • a {@link JobBuilderFactory} (bean name "jobBuilders") as a convenience to prevent you from having to inject the * job repository into every job, as in the examples above
      • @@ -99,6 +111,37 @@ * job repository and transaction manager into every step *
      * + * The transaction manager provided by this annotation will be of type: + * + *
        + *
      • {@link org.springframework.batch.support.transaction.ResourcelessTransactionManager} + * if no {@link javax.sql.DataSource} is provided within the context
      • + *
      • {@link org.springframework.jdbc.datasource.DataSourceTransactionManager} + * if a {@link javax.sql.DataSource} is provided within the context
      • + *
      + * + * In order to use a custom transaction manager, a custom {@link BatchConfigurer} should be provided. For example: + * + *
      + * @Configuration
      + * @EnableBatchProcessing
      + * public class AppConfig extends DefaultBatchConfigurer {
      + *
      + *    @Bean
      + *    public Job job() {
      + *       ...
      + *    }
      + *
      + *    @Override
      + *    public PlatformTransactionManager getTransactionManager() {
      + *       return new MyTransactionManager();
      + *    }
      + *
      + *  ...
      + *
      + * }
      + * 
      + * * If the configuration is specified as modular=true then the context will also contain an * {@link AutomaticJobRegistrar}. The job registrar is useful for modularizing your configuration if there are multiple * jobs. It works by creating separate child application contexts containing job configurations and registering those @@ -130,8 +173,6 @@ * especially if a {@link BatchConfigurer} is provided, because cyclic configuration dependencies are otherwise likely * to develop. * - *

      - * *

      * For reference, the first example above can be compared to the following Spring XML configuration: * @@ -152,6 +193,7 @@ * * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ @Target(ElementType.TYPE) @@ -164,6 +206,9 @@ * Indicate whether the configuration is going to be modularized into multiple application contexts. If true then * you should not create any @Bean Job definitions in this context, but rather supply them in separate (child) * contexts through an {@link ApplicationContextFactory}. + * + * @return boolean indicating whether the configuration is going to be + * modularized into multiple application contexts. Defaults to false. */ boolean modular() default false; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/JobBuilderFactory.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/JobBuilderFactory.java index b6414d3f51..8aba961b02 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/JobBuilderFactory.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/JobBuilderFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/JobScope.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/JobScope.java new file mode 100644 index 0000000000..ab6507f982 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/JobScope.java @@ -0,0 +1,53 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.configuration.annotation; + +import org.springframework.context.annotation.Scope; +import org.springframework.context.annotation.ScopedProxyMode; + +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; + +/** + *

      + * Convenient annotation for job scoped beans that defaults the proxy mode, so that it doesn't have to be specified + * explicitly on every bean definition. Use this on any @Bean that needs to inject @Values from the job + * context, and any bean that needs to share a lifecycle with a job execution (e.g. an JobExecutionListener). E.g. + *

      + * + *
      + * @Bean
      + * @JobScope
      + * protected Callable<String> value(@Value("#{jobExecution.jobInstance.jobName}")
      + * final String value) {
      + * 	return new SimpleCallable(value);
      + * }
      + * 
      + * + *

      Marking a @Bean as @JobScope is equivalent to marking it as @Scope(value="job", proxyMode=TARGET_CLASS)

      + * + * @author Michael Minella + * + * @since 3.0.1 + * + */ +@Scope(value = "job", proxyMode = ScopedProxyMode.TARGET_CLASS) +@Retention(RetentionPolicy.RUNTIME) +@Documented +public @interface JobScope { + +} \ No newline at end of file diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/ModularBatchConfiguration.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/ModularBatchConfiguration.java index 1d52179467..7d5ea1709c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/ModularBatchConfiguration.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/ModularBatchConfiguration.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,11 +15,10 @@ */ package org.springframework.batch.core.configuration.annotation; -import java.util.Collection; - import org.springframework.batch.core.configuration.support.ApplicationContextFactory; import org.springframework.batch.core.configuration.support.AutomaticJobRegistrar; import org.springframework.batch.core.configuration.support.DefaultJobLoader; +import org.springframework.batch.core.explore.JobExplorer; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.repository.JobRepository; import org.springframework.beans.factory.annotation.Autowired; @@ -28,6 +27,8 @@ import org.springframework.context.annotation.Configuration; import org.springframework.transaction.PlatformTransactionManager; +import java.util.Collection; + /** * Base {@code Configuration} class providing common structure for enabling and using Spring Batch. Customization is * available by implementing the {@link BatchConfigurer} interface. @@ -65,6 +66,12 @@ public PlatformTransactionManager transactionManager() throws Exception { return getConfigurer(configurers).getTransactionManager(); } + @Override + @Bean + public JobExplorer jobExplorer() throws Exception { + return getConfigurer(configurers).getJobExplorer(); + } + @Bean public AutomaticJobRegistrar jobRegistrar() throws Exception { registrar.setJobLoader(new DefaultJobLoader(jobRegistry())); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/SimpleBatchConfiguration.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/SimpleBatchConfiguration.java index b3310a2679..2d0f3f6311 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/SimpleBatchConfiguration.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/SimpleBatchConfiguration.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,14 +15,13 @@ */ package org.springframework.batch.core.configuration.annotation; -import java.util.concurrent.atomic.AtomicReference; - import org.aopalliance.intercept.MethodInterceptor; import org.aopalliance.intercept.MethodInvocation; import org.springframework.aop.framework.ProxyFactory; import org.springframework.aop.target.AbstractLazyCreationTargetSource; import org.springframework.batch.core.configuration.JobRegistry; import org.springframework.batch.core.configuration.support.MapJobRegistry; +import org.springframework.batch.core.explore.JobExplorer; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.repository.JobRepository; import org.springframework.beans.factory.annotation.Autowired; @@ -31,6 +30,8 @@ import org.springframework.context.annotation.Configuration; import org.springframework.transaction.PlatformTransactionManager; +import java.util.concurrent.atomic.AtomicReference; + /** * Base {@code Configuration} class providing common structure for enabling and using Spring Batch. Customization is * available by implementing the {@link BatchConfigurer} interface. The main components are created as lazy proxies that @@ -50,13 +51,15 @@ public class SimpleBatchConfiguration extends AbstractBatchConfiguration { private boolean initialized = false; - private AtomicReference jobRepository = new AtomicReference(); + private AtomicReference jobRepository = new AtomicReference<>(); + + private AtomicReference jobLauncher = new AtomicReference<>(); - private AtomicReference jobLauncher = new AtomicReference(); + private AtomicReference jobRegistry = new AtomicReference<>(); - private AtomicReference jobRegistry = new AtomicReference(); + private AtomicReference transactionManager = new AtomicReference<>(); - private AtomicReference transactionManager = new AtomicReference(); + private AtomicReference jobExplorer = new AtomicReference<>(); @Override @Bean @@ -76,6 +79,12 @@ public JobRegistry jobRegistry() throws Exception { return createLazyProxy(jobRegistry, JobRegistry.class); } + @Override + @Bean + public JobExplorer jobExplorer() { + return createLazyProxy(jobExplorer, JobExplorer.class); + } + @Override @Bean public PlatformTransactionManager transactionManager() throws Exception { @@ -84,7 +93,7 @@ public PlatformTransactionManager transactionManager() throws Exception { private T createLazyProxy(AtomicReference reference, Class type) { ProxyFactory factory = new ProxyFactory(); - factory.setTargetSource(new ReferenceTargetSource(reference)); + factory.setTargetSource(new ReferenceTargetSource<>(reference)); factory.addAdvice(new PassthruAdvice()); factory.setInterfaces(new Class[] { type }); @SuppressWarnings("unchecked") @@ -107,6 +116,7 @@ protected void initialize() throws Exception { jobLauncher.set(configurer.getJobLauncher()); transactionManager.set(configurer.getTransactionManager()); jobRegistry.set(new MapJobRegistry()); + jobExplorer.set(configurer.getJobExplorer()); initialized = true; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/StepBuilderFactory.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/StepBuilderFactory.java index fecca1672f..46fcaccb94 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/StepBuilderFactory.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/StepBuilderFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -45,7 +45,7 @@ public StepBuilderFactory(JobRepository jobRepository, PlatformTransactionManage * @return a step builder */ public StepBuilder get(String name) { - StepBuilder builder = (StepBuilder) new StepBuilder(name).repository(jobRepository).transactionManager( + StepBuilder builder = new StepBuilder(name).repository(jobRepository).transactionManager( transactionManager); return builder; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/StepScope.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/StepScope.java index cee43408f2..893e82ee28 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/StepScope.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/StepScope.java @@ -1,12 +1,27 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.annotation; +import org.springframework.context.annotation.Scope; +import org.springframework.context.annotation.ScopedProxyMode; + import java.lang.annotation.Documented; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; -import org.springframework.context.annotation.Scope; -import org.springframework.context.annotation.ScopedProxyMode; - /** *

      * Convenient annotation for step scoped beans that defaults the proxy mode, so that it doesn't have to be specified @@ -27,7 +42,7 @@ * * @author Dave Syer * - * @Since 2.2 + * @since 2.2 * */ @Scope(value = "step", proxyMode = ScopedProxyMode.TARGET_CLASS) diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/package-info.java new file mode 100644 index 0000000000..6b876be28c --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/package-info.java @@ -0,0 +1,10 @@ +/** + * Annotations and builder factories for java based configuration + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.configuration.annotation; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/package-info.java new file mode 100644 index 0000000000..95c8411bae --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/package-info.java @@ -0,0 +1,10 @@ +/** + * Interfaces for registration and location of job configurations. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.configuration; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/package.html deleted file mode 100644 index 19c3ad676c..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Interfaces for registration and location of job configurations. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/AbstractApplicationContextFactory.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/AbstractApplicationContextFactory.java index 5c977b7436..1b164844df 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/AbstractApplicationContextFactory.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/AbstractApplicationContextFactory.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,11 +17,13 @@ package org.springframework.batch.core.configuration.support; import java.util.ArrayList; +import java.util.Arrays; import java.util.Collection; import java.util.List; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.beans.BeansException; import org.springframework.beans.factory.BeanFactoryAware; import org.springframework.beans.factory.config.BeanFactoryPostProcessor; @@ -37,6 +39,7 @@ import org.springframework.context.ConfigurableApplicationContext; import org.springframework.context.support.PropertySourcesPlaceholderConfigurer; import org.springframework.util.Assert; +import org.springframework.util.ClassUtils; /** * {@link ApplicationContextFactory} implementation that takes a parent context and a path to the context to create. @@ -49,7 +52,7 @@ public abstract class AbstractApplicationContextFactory implements ApplicationCo private static final Log logger = LogFactory.getLog(AbstractApplicationContextFactory.class); - private Object resource; + private Object[] resources; private ConfigurableApplicationContext parent; @@ -60,17 +63,19 @@ public abstract class AbstractApplicationContextFactory implements ApplicationCo private Collection> beanPostProcessorExcludeClasses; /** - * Create a factory instance with the resource specified. The resource is a Spring configuration file or java - * package containing configuration files. + * Create a factory instance with the resource specified. The resources are Spring configuration files or java + * packages containing configuration files. + * + * @param resource resource to be used in the creation of the ApplicationContext. */ - public AbstractApplicationContextFactory(Object resource) { + public AbstractApplicationContextFactory(Object... resource) { - this.resource = resource; - beanFactoryPostProcessorClasses = new ArrayList>(); + this.resources = resource; + beanFactoryPostProcessorClasses = new ArrayList<>(); beanFactoryPostProcessorClasses.add(PropertyPlaceholderConfigurer.class); beanFactoryPostProcessorClasses.add(PropertySourcesPlaceholderConfigurer.class); beanFactoryPostProcessorClasses.add(CustomEditorConfigurer.class); - beanPostProcessorExcludeClasses = new ArrayList>(); + beanPostProcessorExcludeClasses = new ArrayList<>(); /* * Assume that a BeanPostProcessor that is BeanFactoryAware must be specific to the parent and remove it from * the child (e.g. an AutoProxyCreator will not work properly). Unfortunately there might still be a a @@ -108,7 +113,7 @@ protected final boolean isCopyConfiguration() { public void setBeanFactoryPostProcessorClasses( Class[] beanFactoryPostProcessorClasses) { - this.beanFactoryPostProcessorClasses = new ArrayList>(); + this.beanFactoryPostProcessorClasses = new ArrayList<>(); for (int i = 0; i < beanFactoryPostProcessorClasses.length; i++) { this.beanFactoryPostProcessorClasses.add(beanFactoryPostProcessorClasses[i]); } @@ -123,7 +128,7 @@ public void setBeanFactoryPostProcessorClasses( * @param beanPostProcessorExcludeClasses the classes to set */ public void setBeanPostProcessorExcludeClasses(Class[] beanPostProcessorExcludeClasses) { - this.beanPostProcessorExcludeClasses = new ArrayList>(); + this.beanPostProcessorExcludeClasses = new ArrayList<>(); for (int i = 0; i < beanPostProcessorExcludeClasses.length; i++) { this.beanPostProcessorExcludeClasses.add(beanPostProcessorExcludeClasses[i]); } @@ -162,16 +167,16 @@ public void setApplicationContext(ApplicationContext applicationContext) throws @Override public ConfigurableApplicationContext createApplicationContext() { - if (resource == null) { + if (resources == null || resources.length == 0) { return parent; } - return createApplicationContext(parent, resource); + return createApplicationContext(parent, resources); } protected abstract ConfigurableApplicationContext createApplicationContext(ConfigurableApplicationContext parent, - Object resource); + Object... resources); /** * Extension point for special subclasses that want to do more complex things with the context prior to refresh. The @@ -199,23 +204,58 @@ protected void prepareContext(ConfigurableApplicationContext parent, Configurabl protected void prepareBeanFactory(ConfigurableListableBeanFactory parent, ConfigurableListableBeanFactory beanFactory) { if (copyConfiguration && parent != null) { - beanFactory.copyConfigurationFrom(parent); - List beanPostProcessors = beanFactory instanceof AbstractBeanFactory ? ((AbstractBeanFactory) beanFactory) - .getBeanPostProcessors() : new ArrayList(); - for (BeanPostProcessor beanPostProcessor : new ArrayList(beanPostProcessors)) { - for (Class cls : beanPostProcessorExcludeClasses) { - if (cls.isAssignableFrom(beanPostProcessor.getClass())) { - logger.debug("Removing bean post processor: " + beanPostProcessor + " of type " + cls); - beanPostProcessors.remove(beanPostProcessor); - } + List parentPostProcessors = new ArrayList<>(); + List childPostProcessors = new ArrayList<>(); + + childPostProcessors.addAll(beanFactory instanceof AbstractBeanFactory ? ((AbstractBeanFactory) beanFactory) + .getBeanPostProcessors() : new ArrayList<>()); + parentPostProcessors.addAll(parent instanceof AbstractBeanFactory ? ((AbstractBeanFactory) parent) + .getBeanPostProcessors() : new ArrayList<>()); + + try { + Class applicationContextAwareProcessorClass = + ClassUtils.forName("org.springframework.context.support.ApplicationContextAwareProcessor", + parent.getBeanClassLoader()); + + for (BeanPostProcessor beanPostProcessor : new ArrayList<>(parentPostProcessors)) { + if (applicationContextAwareProcessorClass.isAssignableFrom(beanPostProcessor.getClass())) { + logger.debug("Removing parent ApplicationContextAwareProcessor"); + parentPostProcessors.remove(beanPostProcessor); + } + } + } + catch (ClassNotFoundException e) { + throw new IllegalStateException(e); + } + + List aggregatedPostProcessors = new ArrayList<>(); + aggregatedPostProcessors.addAll(childPostProcessors); + aggregatedPostProcessors.addAll(parentPostProcessors); + + for (BeanPostProcessor beanPostProcessor : new ArrayList<>(aggregatedPostProcessors)) { + for (Class cls : beanPostProcessorExcludeClasses) { + if (cls.isAssignableFrom(beanPostProcessor.getClass())) { + if (logger.isDebugEnabled()) { + logger.debug("Removing bean post processor: " + beanPostProcessor + " of type " + cls); } + aggregatedPostProcessors.remove(beanPostProcessor); } + } + } + + beanFactory.copyConfigurationFrom(parent); + + List beanPostProcessors = beanFactory instanceof AbstractBeanFactory ? ((AbstractBeanFactory) beanFactory) + .getBeanPostProcessors() : new ArrayList<>(); + + beanPostProcessors.clear(); + beanPostProcessors.addAll(aggregatedPostProcessors); } } @Override public String toString() { - return "ApplicationContextFactory [resource=" + resource + "]"; + return "ApplicationContextFactory [resources=" + Arrays.toString(resources) + "]"; } @Override diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ApplicationContextFactory.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ApplicationContextFactory.java index 1f7506439e..af862552b0 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ApplicationContextFactory.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ApplicationContextFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ApplicationContextJobFactory.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ApplicationContextJobFactory.java index 125e3daa9d..0efb7189ec 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ApplicationContextJobFactory.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ApplicationContextJobFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -38,6 +38,7 @@ public class ApplicationContextJobFactory implements JobFactory { * containing a job with the job name provided */ public ApplicationContextJobFactory(String jobName, ApplicationContextFactory applicationContextFactory) { + @SuppressWarnings("resource") ConfigurableApplicationContext context = applicationContextFactory.createApplicationContext(); this.job = context.getBean(jobName, Job.class); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/AutomaticJobRegistrar.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/AutomaticJobRegistrar.java index 293108b749..2c64127559 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/AutomaticJobRegistrar.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/AutomaticJobRegistrar.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,11 +25,9 @@ import org.springframework.beans.factory.InitializingBean; import org.springframework.context.ApplicationContext; import org.springframework.context.ApplicationContextAware; -import org.springframework.context.ApplicationEvent; -import org.springframework.context.ApplicationListener; import org.springframework.context.Lifecycle; -import org.springframework.context.event.ContextClosedEvent; -import org.springframework.context.event.ContextRefreshedEvent; +import org.springframework.context.SmartLifecycle; +import org.springframework.context.event.ApplicationContextEvent; import org.springframework.core.Ordered; import org.springframework.util.Assert; @@ -40,13 +38,14 @@ * * @author Lucas Ward * @author Dave Syer + * @author Mahmoud Ben Hassine * * @since 2.1 */ -public class AutomaticJobRegistrar implements Ordered, Lifecycle, ApplicationListener, ApplicationContextAware, +public class AutomaticJobRegistrar implements Ordered, SmartLifecycle, ApplicationContextAware, InitializingBean { - private Collection applicationContextFactories = new ArrayList(); + private Collection applicationContextFactories = new ArrayList<>(); private JobLoader jobLoader; @@ -54,13 +53,17 @@ public class AutomaticJobRegistrar implements Ordered, Lifecycle, ApplicationLis private volatile boolean running = false; + private int phase = Integer.MIN_VALUE + 1000; + + private boolean autoStartup = true; + private Object lifecycleMonitor = new Object(); private int order = Ordered.LOWEST_PRECEDENCE; /** - * The enclosing application context, which can be used to check if {@link ApplicationEvent events} come from the - * expected source. + * The enclosing application context, which can be used to check if {@link ApplicationContextEvent events} come + * from the expected source. * * @param applicationContext the enclosing application context if there is one * @see ApplicationContextAware#setApplicationContext(ApplicationContext) @@ -125,25 +128,6 @@ public void afterPropertiesSet() { } - /** - * Creates all the application contexts required and set up job registry entries with all the instances of - * {@link Job} found therein. Also closes the contexts when the enclosing context is closed. - * - * @see InitializingBean#afterPropertiesSet() - */ - @Override - public final void onApplicationEvent(ApplicationEvent event) { - // TODO: With Spring 3 a SmartLifecycle is started automatically - if (event.getSource() == applicationContext) { - if (event instanceof ContextRefreshedEvent) { - start(); - } - else if (event instanceof ContextClosedEvent) { - stop(); - } - } - } - /** * Delegates to {@link JobLoader#clear()}. * @@ -193,4 +177,36 @@ public boolean isRunning() { } } + @Override + public boolean isAutoStartup() { + return autoStartup; + } + + /** + * @param autoStartup true for auto start. + * @see #isAutoStartup() + */ + public void setAutoStartup(boolean autoStartup) { + this.autoStartup = autoStartup; + } + + @Override + public int getPhase() { + return phase; + } + + /** + * @param phase the phase. + * @see #getPhase() + */ + public void setPhase(int phase) { + this.phase = phase; + } + + @Override + public void stop(Runnable callback) { + stop(); + callback.run(); + } + } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ClassPathXmlApplicationContextFactory.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ClassPathXmlApplicationContextFactory.java index cda0ca50e9..1c60796dbe 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ClassPathXmlApplicationContextFactory.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ClassPathXmlApplicationContextFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ * * @deprecated use {@link GenericApplicationContextFactory} instead */ +@Deprecated public class ClassPathXmlApplicationContextFactory extends GenericApplicationContextFactory { /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ClassPathXmlJobRegistry.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ClassPathXmlJobRegistry.java index 728c7a8f45..6e6befb601 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ClassPathXmlJobRegistry.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ClassPathXmlJobRegistry.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.support; /** @@ -7,6 +22,7 @@ * * @deprecated in version 2.1, please us {@link AutomaticJobRegistrar} instead */ +@Deprecated public abstract class ClassPathXmlJobRegistry { } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ClasspathXmlApplicationContextsFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ClasspathXmlApplicationContextsFactoryBean.java index ba8b9d2630..b3cafa638e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ClasspathXmlApplicationContextsFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ClasspathXmlApplicationContextsFactoryBean.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,7 +19,6 @@ import java.util.Arrays; import java.util.List; -import org.springframework.beans.BeansException; import org.springframework.beans.factory.BeanFactoryAware; import org.springframework.beans.factory.FactoryBean; import org.springframework.beans.factory.config.BeanFactoryPostProcessor; @@ -37,9 +36,9 @@ * @author Dave Syer * */ -public class ClasspathXmlApplicationContextsFactoryBean implements FactoryBean, ApplicationContextAware { +public class ClasspathXmlApplicationContextsFactoryBean implements FactoryBean, ApplicationContextAware { - private List resources = new ArrayList(); + private List resources = new ArrayList<>(); private boolean copyConfiguration = true; @@ -57,7 +56,7 @@ public class ClasspathXmlApplicationContextsFactoryBean implements FactoryBean, * resources can be given as a pattern (e.g. * classpath*:/config/*-context.xml). * - * @param resources + * @param resources array of resources to use */ public void setResources(Resource[] resources) { this.resources = Arrays.asList(resources); @@ -105,17 +104,16 @@ public void setBeanPostProcessorExcludeClasses(Class[] beanPostProcessorExclu * in {@link #setResources(Resource[])}. * * @return an array of {@link ApplicationContextFactory} - * @throws Exception * @see org.springframework.beans.factory.FactoryBean#getObject() */ @Override - public Object getObject() throws Exception { + public ApplicationContextFactory[] getObject() throws Exception { if (resources == null) { return new ApplicationContextFactory[0]; } - List applicationContextFactories = new ArrayList(); + List applicationContextFactories = new ArrayList<>(); for (Resource resource : resources) { GenericApplicationContextFactory factory = new GenericApplicationContextFactory(resource); factory.setCopyConfiguration(copyConfiguration); @@ -158,7 +156,6 @@ public boolean isSingleton() { * factories. * * @param applicationContext the {@link ApplicationContext} to set - * @throws BeansException * @see ApplicationContextAware#setApplicationContext(org.springframework.context.ApplicationContext) */ @Override diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/DefaultJobLoader.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/DefaultJobLoader.java index cf3bc97db7..562c1f6be9 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/DefaultJobLoader.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/DefaultJobLoader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,6 +23,7 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.batch.core.Job; import org.springframework.batch.core.Step; import org.springframework.batch.core.configuration.DuplicateJobException; @@ -34,6 +35,7 @@ import org.springframework.beans.factory.InitializingBean; import org.springframework.context.ApplicationContext; import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -44,6 +46,7 @@ * * @author Dave Syer * @author Stephane Nicoll + * @author Mahmoud Ben Hassine */ public class DefaultJobLoader implements JobLoader, InitializingBean { @@ -52,9 +55,9 @@ public class DefaultJobLoader implements JobLoader, InitializingBean { private JobRegistry jobRegistry; private StepRegistry stepRegistry; - private Map contexts = new ConcurrentHashMap(); + private Map contexts = new ConcurrentHashMap<>(); - private Map> contextToJobNames = new ConcurrentHashMap>(); + private Map> contextToJobNames = new ConcurrentHashMap<>(); /** * Default constructor useful for declarative configuration. @@ -76,9 +79,9 @@ public DefaultJobLoader(JobRegistry jobRegistry) { * Creates a job loader with the job and step registries provided. * * @param jobRegistry a {@link JobRegistry} - * @param stepRegistry a {@link StepRegistry} + * @param stepRegistry a {@link StepRegistry} (can be {@code null}) */ - public DefaultJobLoader(JobRegistry jobRegistry, StepRegistry stepRegistry) { + public DefaultJobLoader(JobRegistry jobRegistry, @Nullable StepRegistry stepRegistry) { this.jobRegistry = jobRegistry; this.stepRegistry = stepRegistry; } @@ -118,6 +121,7 @@ public void clear() { doUnregister(jobName); } contexts.clear(); + contextToJobNames.clear(); } @Override @@ -127,7 +131,9 @@ public Collection reload(ApplicationContextFactory factory) { if (contexts.containsKey(factory)) { ConfigurableApplicationContext context = contexts.get(factory); for (String name : contextToJobNames.get(context)) { - logger.debug("Unregistering job: " + name + " from context: " + context.getDisplayName()); + if (logger.isDebugEnabled()) { + logger.debug("Unregistering job: " + name + " from context: " + context.getDisplayName()); + } doUnregister(name); } context.close(); @@ -147,6 +153,7 @@ public Collection load(ApplicationContextFactory factory) throws DuplicateJ return doLoad(factory, false); } + @SuppressWarnings("resource") private Collection doLoad(ApplicationContextFactory factory, boolean unregister) throws DuplicateJobException { Collection jobNamesBefore = jobRegistry.getJobNames(); @@ -155,7 +162,7 @@ private Collection doLoad(ApplicationContextFactory factory, boolean unregi // Try to detect auto-registration (e.g. through a bean post processor) boolean autoRegistrationDetected = jobNamesAfter.size() > jobNamesBefore.size(); - Collection jobsRegistered = new HashSet(); + Collection jobsRegistered = new HashSet<>(); if (autoRegistrationDetected) { for (String name : jobNamesAfter) { if (!jobNamesBefore.contains(name)) { @@ -176,18 +183,22 @@ private Collection doLoad(ApplicationContextFactory factory, boolean unregi // On reload try to unregister first if (unregister) { - logger.debug("Unregistering job: " + jobName + " from context: " + context.getDisplayName()); + if (logger.isDebugEnabled()) { + logger.debug("Unregistering job: " + jobName + " from context: " + context.getDisplayName()); + } doUnregister(jobName); } - logger.debug("Registering job: " + jobName + " from context: " + context.getDisplayName()); + if (logger.isDebugEnabled()) { + logger.debug("Registering job: " + jobName + " from context: " + context.getDisplayName()); + } doRegister(context, job); jobsRegistered.add(jobName); } } - Collection result = new ArrayList(); + Collection result = new ArrayList<>(); for (String name : jobsRegistered) { try { result.add(jobRegistry.getJob(name)); @@ -207,7 +218,7 @@ private Collection doLoad(ApplicationContextFactory factory, boolean unregi /** * Returns all the {@link Step} instances defined by the specified {@link StepLocator}. - *

      + *
      * The specified jobApplicationContext is used to collect additional steps that * are not exposed by the step locator * @@ -218,7 +229,7 @@ private Collection doLoad(ApplicationContextFactory factory, boolean unregi */ private Collection getSteps(final StepLocator stepLocator, final ApplicationContext jobApplicationContext) { final Collection stepNames = stepLocator.getStepNames(); - final Collection result = new ArrayList(); + final Collection result = new ArrayList<>(); for (String stepName : stepNames) { result.add(stepLocator.getStep(stepName)); } @@ -226,7 +237,6 @@ private Collection getSteps(final StepLocator stepLocator, final Applicati // Because some steps are referenced by name, we need to look in the context to see if there // are more Step instances defined. Right now they are registered as being available in the // context of the job but we have no idea if they are linked to that Job or not. - @SuppressWarnings("unchecked") final Map allSteps = jobApplicationContext.getBeansOfType(Step.class); for (Map.Entry entry : allSteps.entrySet()) { if (!stepNames.contains(entry.getKey())) { @@ -238,7 +248,7 @@ private Collection getSteps(final StepLocator stepLocator, final Applicati /** * Registers the specified {@link Job} defined in the specified {@link ConfigurableApplicationContext}. - *

      + *
      * Makes sure to update the {@link StepRegistry} if it is available. * * @param context the context in which the job is defined diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/GenericApplicationContextFactory.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/GenericApplicationContextFactory.java index 34e92ce864..4fd69ea6ad 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/GenericApplicationContextFactory.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/GenericApplicationContextFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,8 +16,6 @@ package org.springframework.batch.core.configuration.support; -import java.io.IOException; - import org.springframework.beans.factory.config.BeanFactoryPostProcessor; import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; import org.springframework.beans.factory.support.DefaultListableBeanFactory; @@ -28,6 +26,12 @@ import org.springframework.context.support.GenericXmlApplicationContext; import org.springframework.core.io.Resource; import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; /** * {@link ApplicationContextFactory} implementation that takes a parent context and a path to the context to create. @@ -41,30 +45,48 @@ public class GenericApplicationContextFactory extends AbstractApplicationContext /** * Create an application context factory for the resource specified. The resource can be an actual {@link Resource}, * in which case it will be interpreted as an XML file, or it can be a @Configuration class, or a package name. + * All types must be the same (mixing XML with a java package for example is not allowed and will result in an + * {@link java.lang.IllegalArgumentException}). * - * @param resource a resource (XML configuration file, @Configuration class or java package to scan) + * @param resources some resources (XML configuration files, @Configuration classes or java packages to scan) */ - public GenericApplicationContextFactory(Object resource) { - super(resource); + public GenericApplicationContextFactory(Object... resources) { + super(resources); } /** - * @see AbstractApplicationContextFactory#createApplicationContext(ConfigurableApplicationContext, Object) + * @see AbstractApplicationContextFactory#createApplicationContext(ConfigurableApplicationContext, Object...) */ @Override protected ConfigurableApplicationContext createApplicationContext(ConfigurableApplicationContext parent, - Object resource) { - if (resource instanceof Resource) { - return new ResourceXmlApplicationContext(parent, (Resource) resource); - } - if (resource instanceof Class) { - return new ResourceAnnotationApplicationContext(parent, resource); + Object... resources) { + ConfigurableApplicationContext context; + + if (allObjectsOfType(resources, Resource.class)) { + context = new ResourceXmlApplicationContext(parent, resources); + } else if (allObjectsOfType(resources, Class.class)) { + context = new ResourceAnnotationApplicationContext(parent, resources); + } else if (allObjectsOfType(resources, String.class)) { + context = new ResourceAnnotationApplicationContext(parent, resources); + } else { + List> types = new ArrayList<>(); + for (Object resource : resources) { + types.add(resource.getClass()); + } + throw new IllegalArgumentException("No application context could be created for resource types: " + + Arrays.toString(types.toArray())); } - if (resource instanceof String) { - return new ResourceAnnotationApplicationContext(parent, resource); + + return context; + } + + private boolean allObjectsOfType(Object[] objects, Class type) { + for (Object object : objects) { + if (!type.isInstance(object)) { + return false; + } } - throw new IllegalArgumentException("No application context could be created for resource type: " - + resource.getClass()); + return true; } private abstract class ApplicationContextHelper { @@ -74,7 +96,7 @@ private abstract class ApplicationContextHelper { private final ConfigurableApplicationContext parent; public ApplicationContextHelper(ConfigurableApplicationContext parent, GenericApplicationContext context, - Object config) { + Object... config) { this.parent = parent; if (parent != null) { Assert.isTrue(parent.getBeanFactory() instanceof DefaultListableBeanFactory, @@ -90,9 +112,9 @@ public ApplicationContextHelper(ConfigurableApplicationContext parent, GenericAp prepareContext(parent, context); } - protected abstract String generateId(Object config); + protected abstract String generateId(Object... configs); - protected abstract void loadConfiguration(Object config); + protected abstract void loadConfiguration(Object... configs); protected void prepareBeanFactory(ConfigurableListableBeanFactory beanFactory) { if (parentBeanFactory != null) { @@ -114,24 +136,35 @@ private final class ResourceXmlApplicationContext extends GenericXmlApplicationC /** * @param parent */ - public ResourceXmlApplicationContext(ConfigurableApplicationContext parent, Resource resource) { - helper = new ApplicationContextHelper(parent, this, resource) { + public ResourceXmlApplicationContext(ConfigurableApplicationContext parent, Object... resources) { + + class ResourceXmlApplicationContextHelper extends ApplicationContextHelper { + + ResourceXmlApplicationContextHelper(ConfigurableApplicationContext parent, GenericApplicationContext context, Object... config) { + super(parent, context, config); + } + @Override - protected String generateId(Object config) { - Resource resource = (Resource) config; - try { - return resource.getURI().toString(); - } - catch (IOException e) { - return resource.toString(); - } + protected String generateId(Object... configs) { + Resource[] resources = Arrays.copyOfRange(configs, 0, configs.length, Resource[].class); + try { + List uris = new ArrayList<>(); + for (Resource resource : resources) { + uris.add(resource.getURI().toString()); + } + return StringUtils.collectionToCommaDelimitedString(uris); + } + catch (IOException e) { + return Arrays.toString(resources); + } } @Override - protected void loadConfiguration(Object config) { - Resource resource = (Resource) config; - load(resource); + protected void loadConfiguration(Object... configs) { + Resource[] resources = Arrays.copyOfRange(configs, 0, configs.length, Resource[].class); + load(resources); } - }; + } + helper = new ResourceXmlApplicationContextHelper(parent, this, resources); refresh(); } @@ -152,30 +185,41 @@ private final class ResourceAnnotationApplicationContext extends AnnotationConfi private final ApplicationContextHelper helper; - public ResourceAnnotationApplicationContext(ConfigurableApplicationContext parent, Object resource) { - helper = new ApplicationContextHelper(parent, this, resource) { + public ResourceAnnotationApplicationContext(ConfigurableApplicationContext parent, Object... resources) { + + class ResourceAnnotationApplicationContextHelper extends ApplicationContextHelper { + + public ResourceAnnotationApplicationContextHelper(ConfigurableApplicationContext parent, GenericApplicationContext context, Object... config) { + super(parent, context, config); + } + @Override - protected String generateId(Object config) { - if (config instanceof Class) { - Class type = (Class) config; - return type.getName(); + protected String generateId(Object... configs) { + if (allObjectsOfType(configs, Class.class)) { + Class[] types = Arrays.copyOfRange(configs, 0, configs.length, Class[].class); + List names = new ArrayList<>(); + for (Class type : types) { + names.add(type.getName()); + } + return StringUtils.collectionToCommaDelimitedString(names); } else { - return config.toString(); + return Arrays.toString(configs); } } @Override - protected void loadConfiguration(Object config) { - if (config instanceof Class) { - Class type = (Class) config; - register(type); + protected void loadConfiguration(Object... configs) { + if (allObjectsOfType(configs, Class.class)) { + Class[] types = Arrays.copyOfRange(configs, 0, configs.length, Class[].class); + register(types); } else { - String pkg = (String) config; - scan(pkg); + String[] pkgs = Arrays.copyOfRange(configs, 0, configs.length, String[].class); + scan(pkgs); } } - }; + } + helper = new ResourceAnnotationApplicationContextHelper(parent, this, resources); refresh(); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/GroupAwareJob.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/GroupAwareJob.java index be40fc9859..3ceb7e157c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/GroupAwareJob.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/GroupAwareJob.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,6 +19,7 @@ import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParametersIncrementer; import org.springframework.batch.core.JobParametersValidator; +import org.springframework.lang.Nullable; import org.springframework.util.ClassUtils; /** @@ -28,10 +29,11 @@ * financeDepartment, which would result in a {@link Job} with * identical functionality but named financeDepartment.overnightJob * . The use of a "." separator for elements is deliberate, since it is a "safe" - * character in a URL. + * character in a URL. * * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public class GroupAwareJob implements Job { @@ -58,10 +60,10 @@ public GroupAwareJob(Job delegate) { /** * Create a new {@link Job} with the given group name and delegate. * - * @param groupName the group name to prepend + * @param groupName the group name to prepend (can be {@code null}) * @param delegate a delegate for the features of a regular Job */ - public GroupAwareJob(String groupName, Job delegate) { + public GroupAwareJob(@Nullable String groupName, Job delegate) { super(); this.groupName = groupName; this.delegate = delegate; @@ -89,6 +91,7 @@ public boolean isRestartable() { } @Override + @Nullable public JobParametersIncrementer getJobParametersIncrementer() { return delegate.getJobParametersIncrementer(); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/JobFactoryRegistrationListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/JobFactoryRegistrationListener.java index 1521a15afd..db246f5dfa 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/JobFactoryRegistrationListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/JobFactoryRegistrationListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/JobLoader.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/JobLoader.java index 7352ed7027..b997e62378 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/JobLoader.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/JobLoader.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/JobRegistryBeanPostProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/JobRegistryBeanPostProcessor.java index 643e9c439c..4eb2a26251 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/JobRegistryBeanPostProcessor.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/JobRegistryBeanPostProcessor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -52,7 +52,7 @@ public class JobRegistryBeanPostProcessor implements BeanPostProcessor, BeanFact // It doesn't make sense for this to have a default value... private JobRegistry jobRegistry = null; - private Collection jobNames = new HashSet(); + private Collection jobNames = new HashSet<>(); private String groupName = null; @@ -106,14 +106,16 @@ public void afterPropertiesSet() throws Exception { } /** - * De-register all the {@link Job} instances that were regsistered by this + * Unregister all the {@link Job} instances that were registered by this * post processor. * @see org.springframework.beans.factory.DisposableBean#destroy() */ @Override public void destroy() throws Exception { for (String name : jobNames) { - logger.debug("Unregistering job: " + name); + if (logger.isDebugEnabled()) { + logger.debug("Unregistering job: " + name); + } jobRegistry.unregister(name); } jobNames.clear(); @@ -138,7 +140,9 @@ public Object postProcessAfterInitialization(Object bean, String beanName) throw job = groupName==null ? job : new GroupAwareJob(groupName, job); ReferenceJobFactory jobFactory = new ReferenceJobFactory(job); String name = jobFactory.getJobName(); - logger.debug("Registering job: " + name); + if (logger.isDebugEnabled()) { + logger.debug("Registering job: " + name); + } jobRegistry.register(jobFactory); jobNames.add(name); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/MapJobRegistry.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/MapJobRegistry.java index 9cd93cb541..590aaa463a 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/MapJobRegistry.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/MapJobRegistry.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,6 +25,7 @@ import org.springframework.batch.core.configuration.JobFactory; import org.springframework.batch.core.configuration.JobRegistry; import org.springframework.batch.core.launch.NoSuchJobException; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -32,6 +33,7 @@ * * @author Dave Syer * @author Robert Fischer + * @author Mahmoud Ben Hassine */ public class MapJobRegistry implements JobRegistry { @@ -39,11 +41,11 @@ public class MapJobRegistry implements JobRegistry { * The map holding the registered job factories. */ // The "final" ensures that it is visible and initialized when the constructor resolves. - private final ConcurrentMap map = new ConcurrentHashMap(); + private final ConcurrentMap map = new ConcurrentHashMap<>(); @Override public void register(JobFactory jobFactory) throws DuplicateJobException { - Assert.notNull(jobFactory); + Assert.notNull(jobFactory, "jobFactory is null"); String name = jobFactory.getJobName(); Assert.notNull(name, "Job configuration must have a name."); JobFactory previousValue = map.putIfAbsent(name, jobFactory); @@ -60,7 +62,7 @@ public void unregister(String name) { } @Override - public Job getJob(String name) throws NoSuchJobException { + public Job getJob(@Nullable String name) throws NoSuchJobException { JobFactory factory = map.get(name); if (factory == null) { throw new NoSuchJobException("No job configuration with the name [" + name + "] was registered"); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/MapStepRegistry.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/MapStepRegistry.java index fea8816e7b..b54e4ca507 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/MapStepRegistry.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/MapStepRegistry.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -37,7 +37,7 @@ */ public class MapStepRegistry implements StepRegistry { - private final ConcurrentMap> map = new ConcurrentHashMap>(); + private final ConcurrentMap> map = new ConcurrentHashMap<>(); @Override public void register(String jobName, Collection steps) throws DuplicateJobException { @@ -45,7 +45,7 @@ public void register(String jobName, Collection steps) throws DuplicateJob Assert.notNull(steps, "The job steps cannot be null."); - final Map jobSteps = new HashMap(); + final Map jobSteps = new HashMap<>(); for (Step step : steps) { jobSteps.put(step.getName(), step); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/OsgiBundleXmlApplicationContextFactory.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/OsgiBundleXmlApplicationContextFactory.java deleted file mode 100644 index 82c2942faf..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/OsgiBundleXmlApplicationContextFactory.java +++ /dev/null @@ -1,120 +0,0 @@ -/* - * Copyright 2006-2013 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.core.configuration.support; - -import org.osgi.framework.BundleContext; -import org.springframework.beans.BeansException; -import org.springframework.context.ApplicationContext; -import org.springframework.context.ApplicationContextAware; -import org.springframework.context.ConfigurableApplicationContext; -import org.springframework.osgi.context.BundleContextAware; -import org.springframework.osgi.context.support.OsgiBundleXmlApplicationContext; - -/** - * {@link ApplicationContextFactory} that can be used to load a context from an - * XML location in a bundle. - * - * @author Dave Syer - * - */ -public class OsgiBundleXmlApplicationContextFactory implements BundleContextAware, ApplicationContextFactory, -ApplicationContextAware { - - private BundleContext bundleContext; - - private ApplicationContext parent; - - private String path; - - private String displayName; - - /** - * @param path the resource path to the xml to load for the child context. - */ - public void setPath(String path) { - this.path = path; - } - - /** - * @param displayName the display name for the application context created. - */ - public void setDisplayName(String displayName) { - this.displayName = displayName; - } - - /** - * Setter for the parent application context. - * - * @see org.springframework.context.ApplicationContextAware#setApplicationContext(org.springframework.context.ApplicationContext) - */ - @Override - public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { - parent = applicationContext; - } - - /** - * Stash the {@link BundleContext} for creating a job application context - * later. - * - * @see org.springframework.osgi.context.BundleContextAware#setBundleContext(org.osgi.framework.BundleContext) - */ - @Override - public void setBundleContext(BundleContext context) { - this.bundleContext = context; - } - - /** - * Create an application context from the provided path, using the current - * OSGi {@link BundleContext} and the enclosing Spring - * {@link ApplicationContext} as a parent context. - * - * @see ApplicationContextFactory#createApplicationContext() - */ - @Override - public ConfigurableApplicationContext createApplicationContext() { - OsgiBundleXmlApplicationContext context = new OsgiBundleXmlApplicationContext(new String[] { path }, parent); - String displayName = bundleContext.getBundle().getSymbolicName() + ":" + this.displayName; - context.setDisplayName(displayName); - context.setBundleContext(bundleContext); - context.refresh(); - return context; - } - - @Override - public String toString() { - String bundleId = bundleContext == null ? null : (bundleContext.getBundle() == null ? bundleContext.toString() - : "" + bundleContext.getBundle().getBundleId()); - return "OsgiBundleXmlApplicationContext [path=" + path + ", bundle=" + bundleId + "]"; - } - - @Override - public int hashCode() { - return toString().hashCode(); - } - - @Override - public boolean equals(Object obj) { - if (this == obj) { - return true; - } - if (obj == null) { - return false; - } - return toString().equals(obj.toString()); - } - -} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ReferenceJobFactory.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ReferenceJobFactory.java index 0427aebe3e..7d8ab4fc05 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ReferenceJobFactory.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/ReferenceJobFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/package-info.java new file mode 100644 index 0000000000..ba3ed75998 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/package-info.java @@ -0,0 +1,10 @@ +/** + * Specific implementations of configuration concerns. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.configuration.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/package.html deleted file mode 100644 index f0c19561d4..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/support/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Specific implementations of configuration concerns. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/AbstractFlowParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/AbstractFlowParser.java index 9f91af2f16..5b4c3e3d26 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/AbstractFlowParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/AbstractFlowParser.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,12 +18,16 @@ import java.util.ArrayList; import java.util.Arrays; import java.util.Collection; -import java.util.HashMap; import java.util.HashSet; +import java.util.LinkedHashMap; import java.util.List; import java.util.Map; import java.util.Set; +import org.w3c.dom.Element; +import org.w3c.dom.Node; +import org.w3c.dom.NodeList; + import org.springframework.batch.core.job.flow.FlowExecutionStatus; import org.springframework.beans.factory.config.BeanDefinition; import org.springframework.beans.factory.parsing.CompositeComponentDefinition; @@ -33,43 +37,42 @@ import org.springframework.beans.factory.xml.ParserContext; import org.springframework.util.StringUtils; import org.springframework.util.xml.DomUtils; -import org.w3c.dom.Element; -import org.w3c.dom.Node; -import org.w3c.dom.NodeList; /** * @author Dave Syer - * + * @author Michael Minella + * @author Chris Schaefer + * */ public abstract class AbstractFlowParser extends AbstractSingleBeanDefinitionParser { - private static final String ID_ATTR = "id"; + protected static final String ID_ATTR = "id"; - private static final String STEP_ELE = "step"; + protected static final String STEP_ELE = "step"; - private static final String FLOW_ELE = "flow"; + protected static final String FLOW_ELE = "flow"; - private static final String DECISION_ELE = "decision"; + protected static final String DECISION_ELE = "decision"; - private static final String SPLIT_ELE = "split"; + protected static final String SPLIT_ELE = "split"; - private static final String NEXT_ATTR = "next"; + protected static final String NEXT_ATTR = "next"; - private static final String NEXT_ELE = "next"; + protected static final String NEXT_ELE = "next"; - private static final String END_ELE = "end"; + protected static final String END_ELE = "end"; - private static final String FAIL_ELE = "fail"; + protected static final String FAIL_ELE = "fail"; - private static final String STOP_ELE = "stop"; + protected static final String STOP_ELE = "stop"; - private static final String ON_ATTR = "on"; + protected static final String ON_ATTR = "on"; - private static final String TO_ATTR = "to"; + protected static final String TO_ATTR = "to"; - private static final String RESTART_ATTR = "restart"; + protected static final String RESTART_ATTR = "restart"; - private static final String EXIT_CODE_ATTR = "exit-code"; + protected static final String EXIT_CODE_ATTR = "exit-code"; private static final InlineStepParser stepParser = new InlineStepParser(); @@ -78,7 +81,7 @@ public abstract class AbstractFlowParser extends AbstractSingleBeanDefinitionPar private static final DecisionParser decisionParser = new DecisionParser(); // For generating unique state names for end transitions - private static int endCounter = 0; + protected static int endCounter = 0; private String jobFactoryRef; @@ -86,8 +89,8 @@ public abstract class AbstractFlowParser extends AbstractSingleBeanDefinitionPar * Convenience method for subclasses to set the job factory reference if it * is available (null is fine, but the quality of error reports is better if * it is available). - * - * @param jobFactoryRef + * + * @param jobFactoryRef name of the ref */ protected void setJobFactoryRef(String jobFactoryRef) { this.jobFactoryRef = jobFactoryRef; @@ -95,7 +98,7 @@ protected void setJobFactoryRef(String jobFactoryRef) { /* * (non-Javadoc) - * + * * @see AbstractSingleBeanDefinitionParser#getBeanClass(Element) */ @Override @@ -110,7 +113,7 @@ protected Class getBeanClass(Element element) { @Override protected void doParse(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { - List stateTransitions = new ArrayList(); + List stateTransitions = new ArrayList<>(); SplitParser splitParser = new SplitParser(jobFactoryRef); CompositeComponentDefinition compositeDef = new CompositeComponentDefinition(element.getTagName(), @@ -118,7 +121,7 @@ protected void doParse(Element element, ParserContext parserContext, BeanDefinit parserContext.pushContainingComponent(compositeDef); boolean stepExists = false; - Map> reachableElementMap = new HashMap>(); + Map> reachableElementMap = new LinkedHashMap<>(); String startElement = null; NodeList children = element.getChildNodes(); for (int i = 0; i < children.getLength(); i++) { @@ -160,7 +163,7 @@ else if (nodeName.equals(SPLIT_ELE)) { } // Ensure that all elements are reachable - Set allReachableElements = new HashSet(); + Set allReachableElements = new HashSet<>(); findAllReachableElements(startElement, reachableElementMap, allReachableElements); for (String elementId : reachableElementMap.keySet()) { if (!allReachableElements.contains(elementId)) { @@ -168,21 +171,20 @@ else if (nodeName.equals(SPLIT_ELE)) { } } - ManagedList managedList = new ManagedList(); - @SuppressWarnings( { "unchecked", "unused" }) - boolean dummy = managedList.addAll(stateTransitions); + ManagedList managedList = new ManagedList<>(); + managedList.addAll(stateTransitions); builder.addPropertyValue("stateTransitions", managedList); } /** * Find all of the elements that are pointed to by this element. - * - * @param element + * + * @param element The parent element * @return a collection of reachable element names */ private Set findReachableElements(Element element) { - Set reachableElements = new HashSet(); + Set reachableElements = new HashSet<>(); String nextAttribute = element.getAttribute(NEXT_ATTR); if (StringUtils.hasText(nextAttribute)) { @@ -206,12 +208,12 @@ private Set findReachableElements(Element element) { /** * Find all of the elements reachable from the startElement. - * - * @param startElement - * @param reachableElementMap + * + * @param startElement name of the element to start from + * @param reachableElementMap Map of elements that can be reached from the startElement * @param accumulator a collection of reachable element names */ - private void findAllReachableElements(String startElement, Map> reachableElementMap, + protected void findAllReachableElements(String startElement, Map> reachableElementMap, Set accumulator) { Set reachableIds = reachableElementMap.get(startElement); accumulator.add(startElement); @@ -233,7 +235,7 @@ private void findAllReachableElements(String startElement, Map getNextElements(ParserContext parserContext, BeanDefinition stateDef, + public static Collection getNextElements(ParserContext parserContext, BeanDefinition stateDef, Element element) { return getNextElements(parserContext, null, stateDef, element); } @@ -248,10 +250,10 @@ protected static Collection getNextElements(ParserContext parser * {@link org.springframework.batch.core.job.flow.support.StateTransition} * references */ - protected static Collection getNextElements(ParserContext parserContext, String stepId, + public static Collection getNextElements(ParserContext parserContext, String stepId, BeanDefinition stateDef, Element element) { - Collection list = new ArrayList(); + Collection list = new ArrayList<>(); String shortNextAttribute = element.getAttribute(NEXT_ATTR); boolean hasNextAttribute = StringUtils.hasText(shortNextAttribute); @@ -260,7 +262,7 @@ protected static Collection getNextElements(ParserContext parser } boolean transitionElementExists = false; - List patterns = new ArrayList(); + List patterns = new ArrayList<>(); for (String transitionName : new String[] { NEXT_ELE, STOP_ELE, END_ELE, FAIL_ELE }) { List transitionElements = DomUtils.getChildElementsByTagName(element, transitionName); for (Element transitionElement : transitionElements) { @@ -283,7 +285,7 @@ protected static Collection getNextElements(ParserContext parser else if (hasNextAttribute) { parserContext.getReaderContext().error( "The <" + element.getNodeName() + "/> may not contain a '" + NEXT_ATTR - + "' attribute and a transition element", element); + + "' attribute and a transition element", element); } return list; @@ -292,10 +294,10 @@ else if (hasNextAttribute) { /** * @param transitionElement The element to parse * @param patterns a list of patterns on state transitions for this element - * @param element + * @param element {@link Element} representing the source. * @param parserContext the parser context for the bean factory */ - private static void verifyUniquePattern(Element transitionElement, List patterns, Element element, + protected static void verifyUniquePattern(Element transitionElement, List patterns, Element element, ParserContext parserContext) { String onAttribute = transitionElement.getAttribute(ON_ATTR); if (patterns.contains(onAttribute)) { @@ -309,7 +311,7 @@ private static void verifyUniquePattern(Element transitionElement, List * @param transitionElement The element to parse * @param stateDef The bean definition for the current state * @param parserContext the parser context for the bean factory - * @param a collection of + * @return a collection of * {@link org.springframework.batch.core.job.flow.support.StateTransition} * references */ @@ -340,11 +342,12 @@ private static Collection parseTransitionElement(Element transit * default to batchStatus. * @param stateDef The bean definition for the current state * @param parserContext the parser context for the bean factory - * @param a collection of + * @param abandon the abandon flag to be used by the transition. + * @return a collection of * {@link org.springframework.batch.core.job.flow.support.StateTransition} * references */ - private static Collection createTransition(FlowExecutionStatus status, String on, String next, + protected static Collection createTransition(FlowExecutionStatus status, String on, String next, String exitCode, BeanDefinition stateDef, ParserContext parserContext, boolean abandon) { BeanDefinition endState = null; @@ -373,7 +376,7 @@ private static Collection createTransition(FlowExecutionStatus s } - Collection list = new ArrayList(); + Collection list = new ArrayList<>(); list.add(getStateTransitionReference(parserContext, stateDef, on, next)); if (endState != null) { // @@ -389,8 +392,8 @@ private static Collection createTransition(FlowExecutionStatus s * @param elementName An end transition element name * @return the BatchStatus corresponding to the transition name */ - private static FlowExecutionStatus getBatchStatusFromEndTransitionName(String elementName) { - elementName = stripNamespace(elementName); + protected static FlowExecutionStatus getBatchStatusFromEndTransitionName(String elementName) { + elementName = stripNamespace(elementName); if (STOP_ELE.equals(elementName)) { return FlowExecutionStatus.STOPPED; } @@ -405,17 +408,17 @@ else if (FAIL_ELE.equals(elementName)) { } } - /** - * Strip the namespace from the element name if it exists. - */ - private static String stripNamespace(String elementName){ - if(elementName.startsWith("batch:")){ - return elementName.substring(6); - } - else{ - return elementName; - } - } + /** + * Strip the namespace from the element name if it exists. + */ + private static String stripNamespace(String elementName){ + if(elementName.startsWith("batch:")){ + return elementName.substring(6); + } + else{ + return elementName; + } + } /** * @param parserContext the parser context diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/AbstractListenerParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/AbstractListenerParser.java index 870195a492..6331381cc1 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/AbstractListenerParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/AbstractListenerParser.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import java.util.ArrayList; @@ -39,11 +54,10 @@ public AbstractBeanDefinition parse(Element element, ParserContext parserContext return builder.getBeanDefinition(); } - @SuppressWarnings("unchecked") public void doParse(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { builder.addPropertyValue("delegate", parseListenerElement(element, parserContext, builder.getRawBeanDefinition())); - ManagedMap metaDataMap = new ManagedMap(); + ManagedMap metaDataMap = new ManagedMap<>(); for (String metaDataPropertyName : getMethodNameAttributes()) { String listenerMethod = element.getAttribute(metaDataPropertyName); if (StringUtils.hasText(listenerMethod)) { @@ -53,7 +67,6 @@ public void doParse(Element element, ParserContext parserContext, BeanDefinition builder.addPropertyValue("metaDataMap", metaDataMap); } - @SuppressWarnings("unchecked") public static BeanMetadataElement parseListenerElement(Element element, ParserContext parserContext, BeanDefinition enclosing) { String listenerRef = element.getAttribute(REF_ATTR); List beanElements = DomUtils.getChildElementsByTagName(element, BEAN_ELE); @@ -112,14 +125,14 @@ else if (refElements.size() > 1) { } private List getMethodNameAttributes() { - List methodNameAttributes = new ArrayList(); + List methodNameAttributes = new ArrayList<>(); for (ListenerMetaData metaData : getMetaDataValues()) { methodNameAttributes.add(metaData.getPropertyName()); } return methodNameAttributes; } - protected abstract Class getBeanClass(); + protected abstract Class> getBeanClass(); protected abstract ListenerMetaData[] getMetaDataValues(); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/AbstractStepParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/AbstractStepParser.java index 8d33cb0b1e..4c5527698e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/AbstractStepParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/AbstractStepParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,6 +15,10 @@ */ package org.springframework.batch.core.configuration.xml; +import org.w3c.dom.Element; +import org.w3c.dom.Node; +import org.w3c.dom.NodeList; + import org.springframework.batch.core.listener.StepListenerMetaData; import org.springframework.beans.MutablePropertyValues; import org.springframework.beans.factory.config.BeanDefinition; @@ -28,9 +32,6 @@ import org.springframework.beans.factory.xml.ParserContext; import org.springframework.util.StringUtils; import org.springframework.util.xml.DomUtils; -import org.w3c.dom.Element; -import org.w3c.dom.Node; -import org.w3c.dom.NodeList; /** * Internal parser for the <step/> elements inside a job. A step element @@ -89,9 +90,10 @@ public abstract class AbstractStepParser { /** * @param stepElement The <step/> element - * @param parserContext + * @param parserContext instance of {@link ParserContext}. * @param jobFactoryRef the reference to the {@link JobParserJobFactoryBean} * from the enclosing tag. Use 'null' if unknown. + * @return {@link AbstractBeanDefinition} for the stepElement. */ protected AbstractBeanDefinition parseStep(Element stepElement, ParserContext parserContext, String jobFactoryRef) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/BeanDefinitionUtils.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/BeanDefinitionUtils.java index b864f253f5..33efd79037 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/BeanDefinitionUtils.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/BeanDefinitionUtils.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/ChunkElementParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/ChunkElementParser.java index f9b2ef44b1..ff9327cde9 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/ChunkElementParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/ChunkElementParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,9 +15,13 @@ */ package org.springframework.batch.core.configuration.xml; +import java.util.List; + +import org.w3c.dom.Element; + import org.springframework.batch.core.listener.StepListenerMetaData; -import org.springframework.batch.core.step.item.ForceRollbackForWriteSkipException; import org.springframework.batch.repeat.policy.SimpleCompletionPolicy; +import org.springframework.beans.BeanMetadataElement; import org.springframework.beans.MutablePropertyValues; import org.springframework.beans.factory.config.BeanDefinition; import org.springframework.beans.factory.config.BeanDefinitionHolder; @@ -30,11 +34,9 @@ import org.springframework.beans.factory.support.ManagedList; import org.springframework.beans.factory.support.ManagedMap; import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.CollectionUtils; import org.springframework.util.StringUtils; import org.springframework.util.xml.DomUtils; -import org.w3c.dom.Element; - -import java.util.List; /** * Internal parser for the <chunk/> element inside a step. @@ -66,8 +68,11 @@ public class ChunkElementParser { StepListenerMetaData.itemListenerMetaData()); /** - * @param element - * @param parserContext + * @param bd {@link AbstractBeanDefinition} instance of the containing bean. + * @param element the element to parse + * @param parserContext the context to use + * @param underspecified if true, a fatal error will not be raised if attribute + * or element is missing. */ protected void parse(Element element, AbstractBeanDefinition bd, ParserContext parserContext, boolean underspecified) { @@ -108,26 +113,33 @@ protected void parse(Element element, AbstractBeanDefinition bd, ParserContext p if (propertyValues.contains("commitInterval")) { parserContext.getReaderContext().error( "The <" + element.getNodeName() + "/> element must contain either '" + COMMIT_INTERVAL_ATTR - + "' " + "or '" + CHUNK_COMPLETION_POLICY_ATTR + "', but not both.", element); + + "' " + "or '" + CHUNK_COMPLETION_POLICY_ATTR + "', but not both.", element); } else { parserContext.getReaderContext().error( "The <" + element.getNodeName() + "/> element must contain either '" + COMMIT_INTERVAL_ATTR - + "' " + "or '" + CHUNK_COMPLETION_POLICY_ATTR + "'.", element); + + "' " + "or '" + CHUNK_COMPLETION_POLICY_ATTR + "'.", element); } } String skipLimit = element.getAttribute("skip-limit"); - ManagedMap skippableExceptions = handleExceptionElement(element, parserContext, "skippable-exception-classes"); + ManagedMap skippableExceptions = + new ExceptionElementParser().parse(element, parserContext, "skippable-exception-classes"); if (StringUtils.hasText(skipLimit)) { if (skippableExceptions == null) { - skippableExceptions = new ManagedMap(); + skippableExceptions = new ManagedMap<>(); skippableExceptions.setMergeEnabled(true); } propertyValues.addPropertyValue("skipLimit", skipLimit); } if (skippableExceptions != null) { + List exceptionClassElements = DomUtils.getChildElementsByTagName(element, "skippable-exception-classes"); + + if(!CollectionUtils.isEmpty(exceptionClassElements)) { + skippableExceptions.setMergeEnabled(exceptionClassElements.get(0).hasAttribute(MERGE_ATTR) + && Boolean.valueOf(exceptionClassElements.get(0).getAttribute(MERGE_ATTR))); + } // Even if there is no retryLimit, we can still accept exception // classes for an abstract parent bean definition propertyValues.addPropertyValue("skippableExceptionClasses", skippableExceptions); @@ -137,15 +149,22 @@ protected void parse(Element element, AbstractBeanDefinition bd, ParserContext p underspecified); String retryLimit = element.getAttribute("retry-limit"); - ManagedMap retryableExceptions = handleExceptionElement(element, parserContext, "retryable-exception-classes"); + ManagedMap retryableExceptions = + new ExceptionElementParser().parse(element, parserContext, "retryable-exception-classes"); if (StringUtils.hasText(retryLimit)) { if (retryableExceptions == null) { - retryableExceptions = new ManagedMap(); + retryableExceptions = new ManagedMap<>(); retryableExceptions.setMergeEnabled(true); } propertyValues.addPropertyValue("retryLimit", retryLimit); } if (retryableExceptions != null) { + List exceptionClassElements = DomUtils.getChildElementsByTagName(element, "retryable-exception-classes"); + + if(!CollectionUtils.isEmpty(exceptionClassElements)) { + retryableExceptions.setMergeEnabled(exceptionClassElements.get(0).hasAttribute(MERGE_ATTR) + && Boolean.valueOf(exceptionClassElements.get(0).getAttribute(MERGE_ATTR))); + } // Even if there is no retryLimit, we can still accept exception // classes for an abstract parent bean definition propertyValues.addPropertyValue("retryableExceptionClasses", retryableExceptions); @@ -183,13 +202,12 @@ protected void parse(Element element, AbstractBeanDefinition bd, ParserContext p private void handleItemHandler(AbstractBeanDefinition enclosing, String handlerName, String propertyName, String adapterClassName, boolean required, Element element, ParserContext parserContext, MutablePropertyValues propertyValues, boolean underspecified) { String refName = element.getAttribute(handlerName); - @SuppressWarnings("unchecked") List children = DomUtils.getChildElementsByTagName(element, handlerName); if (children.size() == 1) { if (StringUtils.hasText(refName)) { parserContext.getReaderContext().error( "The <" + element.getNodeName() + "/> element may not have both a '" + handlerName - + "' attribute and a <" + handlerName + "/> element.", element); + + "' attribute and a <" + handlerName + "/> element.", element); } handleItemHandlerElement(enclosing, propertyName, adapterClassName, propertyValues, children.get(0), parserContext); } @@ -204,7 +222,7 @@ else if (StringUtils.hasText(refName)) { else if (required && !underspecified) { parserContext.getReaderContext().error( "The <" + element.getNodeName() + "/> element has neither a '" + handlerName - + "' attribute nor a <" + handlerName + "/> element.", element); + + "' attribute nor a <" + handlerName + "/> element.", element); } } @@ -212,7 +230,6 @@ else if (required && !underspecified) { * Handle the <reader/>, <processor/>, or <writer/> that * is defined within the item handler. */ - @SuppressWarnings("unchecked") private void handleItemHandlerElement(AbstractBeanDefinition enclosing, String propertyName, String adapterClassName, MutablePropertyValues propertyValues, Element element, ParserContext parserContext) { List beanElements = DomUtils.getChildElementsByTagName(element, BEAN_ELE); @@ -220,7 +237,7 @@ private void handleItemHandlerElement(AbstractBeanDefinition enclosing, String p if (beanElements.size() + refElements.size() != 1) { parserContext.getReaderContext().error( "The <" + element.getNodeName() + "/> must have exactly one of either a <" + BEAN_ELE - + "/> element or a <" + REF_ELE + "/> element.", element); + + "/> element or a <" + REF_ELE + "/> element.", element); } else if (beanElements.size() == 1) { Element beanElement = beanElements.get(0); @@ -270,7 +287,7 @@ private void handleRetryListenersElement(Element element, MutablePropertyValues CompositeComponentDefinition compositeDef = new CompositeComponentDefinition(listenersElement.getTagName(), parserContext.extractSource(element)); parserContext.pushContainingComponent(compositeDef); - ManagedList retryListenerBeans = new ManagedList(); + ManagedList retryListenerBeans = new ManagedList<>(); retryListenerBeans.setMergeEnabled(listenersElement.hasAttribute(MERGE_ATTR) && Boolean.valueOf(listenersElement.getAttribute(MERGE_ATTR))); handleRetryListenerElements(parserContext, listenersElement, retryListenerBeans, enclosing); @@ -279,8 +296,7 @@ private void handleRetryListenersElement(Element element, MutablePropertyValues } } - @SuppressWarnings("unchecked") - private void handleRetryListenerElements(ParserContext parserContext, Element element, ManagedList beans, + private void handleRetryListenerElements(ParserContext parserContext, Element element, ManagedList beans, BeanDefinition enclosing) { List listenerElements = DomUtils.getChildElementsByTagName(element, "listener"); if (listenerElements != null) { @@ -290,11 +306,10 @@ private void handleRetryListenerElements(ParserContext parserContext, Element el } } - @SuppressWarnings("unchecked") private void handleStreamsElement(Element element, MutablePropertyValues propertyValues, ParserContext parserContext) { Element streamsElement = DomUtils.getChildElementByTagName(element, "streams"); if (streamsElement != null) { - ManagedList streamBeans = new ManagedList(); + ManagedList streamBeans = new ManagedList<>(); streamBeans.setMergeEnabled(streamsElement.hasAttribute(MERGE_ATTR) && Boolean.valueOf(streamsElement.getAttribute(MERGE_ATTR))); List streamElements = DomUtils.getChildElementsByTagName(streamsElement, "stream"); @@ -314,34 +329,4 @@ private void handleStreamsElement(Element element, MutablePropertyValues propert } } - @SuppressWarnings("unchecked") - private ManagedMap handleExceptionElement(Element element, ParserContext parserContext, String exceptionListName) { - List children = DomUtils.getChildElementsByTagName(element, exceptionListName); - if (children.size() == 1) { - ManagedMap map = new ManagedMap(); - Element exceptionClassesElement = children.get(0); - map.setMergeEnabled(exceptionClassesElement.hasAttribute(MERGE_ATTR) - && Boolean.valueOf(exceptionClassesElement.getAttribute(MERGE_ATTR))); - addExceptionClasses("include", true, exceptionClassesElement, map, parserContext); - addExceptionClasses("exclude", false, exceptionClassesElement, map, parserContext); - map.put(ForceRollbackForWriteSkipException.class, true); - return map; - } - else if (children.size() > 1) { - parserContext.getReaderContext().error( - "The <" + exceptionListName + "/> element may not appear more than once in a single <" - + element.getNodeName() + "/>.", element); - } - return null; - } - - @SuppressWarnings("unchecked") - private void addExceptionClasses(String elementName, boolean include, Element exceptionClassesElement, - ManagedMap map, ParserContext parserContext) { - for (Element child : (List) DomUtils.getChildElementsByTagName(exceptionClassesElement, elementName)) { - String className = child.getAttribute("class"); - map.put(new TypedStringValue(className, Class.class), include); - } - } - } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/CoreNamespaceHandler.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/CoreNamespaceHandler.java index 53890ac24c..5c461f9a27 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/CoreNamespaceHandler.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/CoreNamespaceHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/CoreNamespacePostProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/CoreNamespacePostProcessor.java index e7223c8fae..d8c71d9026 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/CoreNamespacePostProcessor.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/CoreNamespacePostProcessor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -126,8 +126,7 @@ private Object injectDefaults(Object bean) { if (jobRepository == null) { fb.setJobRepository((JobRepository) applicationContext.getBean(DEFAULT_JOB_REPOSITORY_NAME)); } - } - else if (bean instanceof StepParserStepFactoryBean) { + } else if (bean instanceof StepParserStepFactoryBean) { StepParserStepFactoryBean fb = (StepParserStepFactoryBean) bean; JobRepository jobRepository = fb.getJobRepository(); if (jobRepository == null) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/CoreNamespaceUtils.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/CoreNamespaceUtils.java index 426b038ba6..c72bec0d57 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/CoreNamespaceUtils.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/CoreNamespaceUtils.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2009 the original author or authors. + * Copyright 2006-2013 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,8 +15,8 @@ */ package org.springframework.batch.core.configuration.xml; -import java.util.Map; - +import org.springframework.batch.core.job.flow.support.DefaultStateTransitionComparator; +import org.springframework.batch.core.job.flow.support.StateTransition; import org.springframework.beans.PropertyValue; import org.springframework.beans.factory.config.BeanDefinition; import org.springframework.beans.factory.config.TypedStringValue; @@ -28,16 +28,26 @@ import org.springframework.util.StringUtils; import org.w3c.dom.Element; +import java.util.Comparator; +import java.util.Map; + /** * Utility methods used in parsing of the batch core namespace * * @author Thomas Risberg + * @author Michael Minella */ public class CoreNamespaceUtils { private static final String STEP_SCOPE_PROCESSOR_BEAN_NAME = "org.springframework.batch.core.scope.internalStepScope"; - private static final String STEP_SCOPE_PROCESSOR_CLASS_NAME = "org.springframework.batch.core.scope.StepScope"; + private static final String XML_CONFIG_STEP_SCOPE_PROCESSOR_CLASS_NAME = "org.springframework.batch.core.scope.StepScope"; + + private static final String JAVA_CONFIG_SCOPE_CLASS_NAME = "org.springframework.batch.core.configuration.annotation.ScopeConfiguration"; + + private static final String JOB_SCOPE_PROCESSOR_BEAN_NAME = "org.springframework.batch.core.scope.internalJobScope"; + + private static final String JOB_SCOPE_PROCESSOR_CLASS_NAME = "org.springframework.batch.core.scope.JobScope"; private static final String CUSTOM_EDITOR_CONFIGURER_CLASS_NAME = "org.springframework.beans.factory.config.CustomEditorConfigurer"; @@ -47,45 +57,73 @@ public class CoreNamespaceUtils { private static final String CORE_NAMESPACE_POST_PROCESSOR_CLASS_NAME = "org.springframework.batch.core.configuration.xml.CoreNamespacePostProcessor"; - protected static void autoregisterBeansForNamespace(ParserContext parserContext, Object source) { + public static void autoregisterBeansForNamespace(ParserContext parserContext, Object source) { checkForStepScope(parserContext, source); + checkForJobScope(parserContext, source); addRangePropertyEditor(parserContext); addCoreNamespacePostProcessor(parserContext); + addStateTransitionComparator(parserContext); } private static void checkForStepScope(ParserContext parserContext, Object source) { - boolean foundStepScope = false; + checkForScope(parserContext, source, XML_CONFIG_STEP_SCOPE_PROCESSOR_CLASS_NAME, STEP_SCOPE_PROCESSOR_BEAN_NAME); + } + + private static void checkForJobScope(ParserContext parserContext, Object source) { + checkForScope(parserContext, source, JOB_SCOPE_PROCESSOR_CLASS_NAME, JOB_SCOPE_PROCESSOR_BEAN_NAME); + } + + private static void checkForScope(ParserContext parserContext, Object source, String scopeClassName, + String scopeBeanName) { + boolean foundScope = false; String[] beanNames = parserContext.getRegistry().getBeanDefinitionNames(); for (String beanName : beanNames) { BeanDefinition bd = parserContext.getRegistry().getBeanDefinition(beanName); - if (STEP_SCOPE_PROCESSOR_CLASS_NAME.equals(bd.getBeanClassName())) { - foundStepScope = true; + if (scopeClassName.equals(bd.getBeanClassName()) || JAVA_CONFIG_SCOPE_CLASS_NAME.equals(bd.getBeanClassName())) { + foundScope = true; break; } } - if (!foundStepScope) { + if (!foundScope) { BeanDefinitionBuilder stepScopeBuilder = BeanDefinitionBuilder - .genericBeanDefinition(STEP_SCOPE_PROCESSOR_CLASS_NAME); + .genericBeanDefinition(scopeClassName); AbstractBeanDefinition abd = stepScopeBuilder.getBeanDefinition(); abd.setRole(BeanDefinition.ROLE_INFRASTRUCTURE); abd.setSource(source); - parserContext.getRegistry().registerBeanDefinition(STEP_SCOPE_PROCESSOR_BEAN_NAME, abd); + parserContext.getRegistry().registerBeanDefinition(scopeBeanName, abd); } } + /** + * Register a {@link Comparator} to be used to sort {@link StateTransition}s + * + * @param parserContext + */ + private static void addStateTransitionComparator(ParserContext parserContext) { + BeanDefinitionRegistry registry = parserContext.getRegistry(); + if (!stateTransitionComparatorAlreadyDefined(registry)) { + AbstractBeanDefinition defaultStateTransitionComparator = BeanDefinitionBuilder.genericBeanDefinition( + DefaultStateTransitionComparator.class).getBeanDefinition(); + registry.registerBeanDefinition(DefaultStateTransitionComparator.STATE_TRANSITION_COMPARATOR, defaultStateTransitionComparator); + } + } + + private static boolean stateTransitionComparatorAlreadyDefined(BeanDefinitionRegistry registry) { + return registry.containsBeanDefinition(DefaultStateTransitionComparator.STATE_TRANSITION_COMPARATOR); + } + /** * Register a RangePropertyEditor if one does not already exist. * * @param parserContext */ - @SuppressWarnings("unchecked") private static void addRangePropertyEditor(ParserContext parserContext) { BeanDefinitionRegistry registry = parserContext.getRegistry(); if (!rangeArrayEditorAlreadyDefined(registry)) { AbstractBeanDefinition customEditorConfigurer = BeanDefinitionBuilder.genericBeanDefinition( CUSTOM_EDITOR_CONFIGURER_CLASS_NAME).getBeanDefinition(); customEditorConfigurer.setRole(BeanDefinition.ROLE_INFRASTRUCTURE); - ManagedMap editors = new ManagedMap(); + ManagedMap editors = new ManagedMap<>(); editors.put(RANGE_ARRAY_CLASS_NAME, RANGE_ARRAY_EDITOR_CLASS_NAME); customEditorConfigurer.getPropertyValues().addPropertyValue("customEditors", editors); registry.registerBeanDefinition(CUSTOM_EDITOR_CONFIGURER_CLASS_NAME, customEditorConfigurer); @@ -143,7 +181,7 @@ private static boolean coreNamespaceBeanPostProcessorAlreadyDefined(BeanDefiniti * Should this element be treated as incomplete? If it has a parent or is * abstract, then it may not have all properties. * - * @param element + * @param element to be evaluated. * @return TRUE if the element is abstract or has a parent */ public static boolean isUnderspecified(Element element) { @@ -151,7 +189,7 @@ public static boolean isUnderspecified(Element element) { } /** - * @param element + * @param element to be evaluated. * @return TRUE if the element is abstract */ public static boolean isAbstract(Element element) { @@ -176,7 +214,8 @@ public static boolean namespaceMatchesVersion(Element element) { private static boolean matchesVersionInternal(Element element) { String schemaLocation = element.getAttributeNS("http://www.w3.org/2001/XMLSchema-instance", "schemaLocation"); - return schemaLocation.matches("(?m).*spring-batch-2.2.xsd.*") + return schemaLocation.matches("(?m).*spring-batch-3.0.xsd.*") + || schemaLocation.matches("(?m).*spring-batch-2.2.xsd.*") || schemaLocation.matches("(?m).*spring-batch.xsd.*") || !schemaLocation.matches("(?m).*spring-batch.*"); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/DecisionParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/DecisionParser.java index 5fa21668c3..4e2e1c3085 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/DecisionParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/DecisionParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/ExceptionElementParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/ExceptionElementParser.java new file mode 100644 index 0000000000..aba04df02d --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/ExceptionElementParser.java @@ -0,0 +1,54 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.configuration.xml; + +import java.util.List; + +import org.springframework.batch.core.step.item.ForceRollbackForWriteSkipException; +import org.springframework.beans.factory.config.TypedStringValue; +import org.springframework.beans.factory.support.ManagedMap; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.xml.DomUtils; +import org.w3c.dom.Element; + +public class ExceptionElementParser { + + public ManagedMap parse(Element element, ParserContext parserContext, String exceptionListName) { + List children = DomUtils.getChildElementsByTagName(element, exceptionListName); + if (children.size() == 1) { + ManagedMap map = new ManagedMap<>(); + Element exceptionClassesElement = children.get(0); + addExceptionClasses("include", true, exceptionClassesElement, map, parserContext); + addExceptionClasses("exclude", false, exceptionClassesElement, map, parserContext); + map.put(new TypedStringValue(ForceRollbackForWriteSkipException.class.getName(), Class.class), true); + return map; + } + else if (children.size() > 1) { + parserContext.getReaderContext().error( + "The <" + exceptionListName + "/> element may not appear more than once in a single <" + + element.getNodeName() + "/>.", element); + } + return null; + } + + private void addExceptionClasses(String elementName, boolean include, Element exceptionClassesElement, + ManagedMap map, ParserContext parserContext) { + for (Element child : DomUtils.getChildElementsByTagName(exceptionClassesElement, elementName)) { + String className = child.getAttribute("class"); + map.put(new TypedStringValue(className, Class.class), include); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/FlowElementParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/FlowElementParser.java index c13e75d2a6..06fb7176da 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/FlowElementParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/FlowElementParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,7 +26,7 @@ import org.w3c.dom.Element; /** - * Internal parser for the <flow/> elements inside a job.. + * Internal parser for the <flow/> elements inside a job. * * @see JobParser * diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/InlineFlowParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/InlineFlowParser.java index 49a88311c2..8d15602302 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/InlineFlowParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/InlineFlowParser.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,23 +15,25 @@ */ package org.springframework.batch.core.configuration.xml; +import org.springframework.batch.core.job.flow.support.DefaultStateTransitionComparator; import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.config.RuntimeBeanReference; import org.springframework.beans.factory.support.BeanDefinitionBuilder; import org.springframework.beans.factory.xml.ParserContext; import org.w3c.dom.Element; /** * @author Dave Syer - * + * @author Michael Minella + * */ public class InlineFlowParser extends AbstractFlowParser { - private final String flowName; /** * Construct a {@link InlineFlowParser} with the specified name and using the * provided job repository ref. - * + * * @param flowName the name of the flow * @param jobFactoryRef the reference to the {@link JobParserJobFactoryBean} * from the enclosing tag @@ -42,19 +44,23 @@ public InlineFlowParser(String flowName, String jobFactoryRef) { } + @Override + protected boolean shouldGenerateId() { + return true; + } + /** * @param element the top level element containing a flow definition * @param parserContext the {@link ParserContext} */ @Override protected void doParse(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { - builder.getRawBeanDefinition().setAttribute("flowName", flowName); builder.addPropertyValue("name", flowName); + builder.addPropertyValue("stateTransitionComparator", new RuntimeBeanReference(DefaultStateTransitionComparator.STATE_TRANSITION_COMPARATOR)); super.doParse(element, parserContext, builder); builder.setRole(BeanDefinition.ROLE_INFRASTRUCTURE); parserContext.popAndRegisterContainingComponent(); } - } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/InlineStepParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/InlineStepParser.java index 75b4532d00..e8e8fa9985 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/InlineStepParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/InlineStepParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobExecutionListenerParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobExecutionListenerParser.java index 7065b75f67..ddd61ff0a8 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobExecutionListenerParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobExecutionListenerParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,7 +32,7 @@ public class JobExecutionListenerParser extends AbstractListenerParser { @Override - protected Class getBeanClass() { + protected Class> getBeanClass() { return JobListenerFactoryBean.class; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobParser.java index a89cdf3c0e..7edab3740a 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,7 +32,7 @@ import org.w3c.dom.Element; /** - * Parser for the lt;job/gt; element in the Batch namespace. Sets up and returns + * Parser for the <job/> element in the Batch namespace. Sets up and returns * a bean definition for a {@link org.springframework.batch.core.Job}. * * @author Dave Syer @@ -63,14 +63,14 @@ protected Class getBeanClass(Element element) { * @see AbstractSingleBeanDefinitionParser#doParse(Element, ParserContext, * BeanDefinitionBuilder) */ - @SuppressWarnings("unchecked") @Override protected void doParse(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { if (!CoreNamespaceUtils.namespaceMatchesVersion(element)) { parserContext.getReaderContext().error( - "You cannot use spring-batch-2.0.xsd with Spring Batch 2.1. Please upgrade your schema declarations " - + "(or use the spring-batch.xsd alias if you are feeling lucky).", element); + "You are using a version of the spring-batch XSD that is not compatible with Spring Batch 3.0." + + " Please upgrade your schema declarations (or use the spring-batch.xsd alias if you are " + + "feeling lucky).", element); return; } @@ -133,7 +133,7 @@ protected void doParse(Element element, ParserContext parserContext, BeanDefinit CompositeComponentDefinition compositeDef = new CompositeComponentDefinition(listenersElement.getTagName(), parserContext.extractSource(element)); parserContext.pushContainingComponent(compositeDef); - ManagedList listeners = new ManagedList(); + ManagedList listeners = new ManagedList<>(); listeners.setMergeEnabled(listenersElement.hasAttribute(MERGE_ATTR) && Boolean.valueOf(listenersElement.getAttribute(MERGE_ATTR))); List listenerElements = DomUtils.getChildElementsByTagName(listenersElement, "listener"); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobParserJobFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobParserJobFactoryBean.java index 202281f631..d099f029be 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobParserJobFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobParserJobFactoryBean.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,7 +35,7 @@ * @author Dave Syer * @since 2.0.1 */ -class JobParserJobFactoryBean implements SmartFactoryBean { +public class JobParserJobFactoryBean implements SmartFactoryBean { private String name; @@ -56,7 +56,7 @@ public JobParserJobFactoryBean(String name) { } @Override - public final Object getObject() throws Exception { + public final FlowJob getObject() throws Exception { Assert.isTrue(StringUtils.hasText(name), "The job must have an id."); FlowJob flowJob = new FlowJob(name); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobRepositoryParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobRepositoryParser.java index 6d81f82f25..b2a36074e6 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobRepositoryParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/JobRepositoryParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,7 +27,7 @@ import org.w3c.dom.Element; /** - * Parser for the lt;job-repository/gt; element in the Batch namespace. Sets up + * Parser for the <job-repository/> element in the Batch namespace. Sets up * and returns a JobRepositoryFactoryBean. * * @author Thomas Risberg @@ -65,6 +65,8 @@ protected void doParse(Element element, ParserContext parserContext, BeanDefinit CoreNamespaceUtils.autoregisterBeansForNamespace(parserContext, element); String dataSource = element.getAttribute("data-source"); + + String jdbcOperations = element.getAttribute("jdbc-operations"); String transactionManager = element.getAttribute("transaction-manager"); @@ -82,6 +84,9 @@ protected void doParse(Element element, ParserContext parserContext, BeanDefinit builder.addPropertyValue("dataSource", ds); RuntimeBeanReference tx = new RuntimeBeanReference(transactionManager); builder.addPropertyValue("transactionManager", tx); + if (StringUtils.hasText(jdbcOperations)) { + builder.addPropertyReference("jdbcOperations", jdbcOperations); + } if (StringUtils.hasText(isolationLevelForCreate)) { builder.addPropertyValue("isolationLevelForCreate", DefaultTransactionDefinition.PREFIX_ISOLATION + isolationLevelForCreate); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/SimpleFlowFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/SimpleFlowFactoryBean.java index b963d1d052..daf7914d72 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/SimpleFlowFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/SimpleFlowFactoryBean.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,6 +18,7 @@ import java.util.ArrayList; import java.util.Collection; import java.util.Collections; +import java.util.Comparator; import java.util.List; import org.springframework.batch.core.job.flow.Flow; @@ -40,10 +41,9 @@ * that form, in which case it is not modified). * * @author Dave Syer - * + * @author Michael Minella */ -@SuppressWarnings("rawtypes") -public class SimpleFlowFactoryBean implements FactoryBean, InitializingBean { +public class SimpleFlowFactoryBean implements FactoryBean, InitializingBean { private String name; @@ -51,6 +51,25 @@ public class SimpleFlowFactoryBean implements FactoryBean, InitializingBean { private String prefix; + private Comparator stateTransitionComparator; + + private Class flowType; + + /** + * @param stateTransitionComparator {@link Comparator} implementation that addresses + * the ordering of state evaluation + */ + public void setStateTransitionComparator(Comparator stateTransitionComparator) { + this.stateTransitionComparator = stateTransitionComparator; + } + + /** + * @param flowType Used to inject the type of flow (regular Spring Batch or JSR-352) + */ + public void setFlowType(Class flowType) { + this.flowType = flowType; + } + /** * The name of the flow that is created by this factory. * @@ -75,19 +94,27 @@ public void setStateTransitions(List stateTransitions) { /** * Check mandatory properties (name). * - * @throws Exception + * @throws Exception thrown if error occurs. */ @Override public void afterPropertiesSet() throws Exception { Assert.hasText(name, "The flow must have a name"); + + if(flowType == null) { + flowType = SimpleFlow.class; + } } + /* (non-Javadoc) + * @see org.springframework.beans.factory.FactoryBean#getObject() + */ @Override - public Object getObject() throws Exception { + public SimpleFlow getObject() throws Exception { + SimpleFlow flow = flowType.getConstructor(String.class).newInstance(name); - SimpleFlow flow = new SimpleFlow(name); + flow.setStateTransitionComparator(stateTransitionComparator); - List updatedTransitions = new ArrayList(); + List updatedTransitions = new ArrayList<>(); for (StateTransition stateTransition : stateTransitions) { State state = getProxyState(stateTransition.getState()); updatedTransitions.add(StateTransition.switchOriginAndDestination(stateTransition, state, @@ -122,11 +149,25 @@ private State getProxyState(State state) { } String stateName = prefix + oldName; if (state instanceof StepState) { - return new StepState(stateName, ((StepState) state).getStep()); + return createNewStepState(state, oldName, stateName); } return new DelegateState(stateName, state); } + /** + * Provides an extension point to provide alternative {@link StepState} + * implementations within a {@link SimpleFlow} + * + * @param state The state that will be used to create the StepState + * @param oldName The name to be replaced + * @param stateName The name for the new State + * @return a state for the requested data + */ + protected State createNewStepState(State state, String oldName, + String stateName) { + return new StepState(stateName, ((StepState) state).getStep(oldName)); + } + @Override public Class getObjectType() { return SimpleFlow.class; @@ -144,7 +185,7 @@ public boolean isSingleton() { * @author Dave Syer * */ - private static class DelegateState extends AbstractState implements FlowHolder { + public static class DelegateState extends AbstractState implements FlowHolder { private final State state; private DelegateState(String name, State state) { @@ -152,6 +193,10 @@ private DelegateState(String name, State state) { this.state = state; } + public State getState() { + return this.state; + } + @Override public boolean isEndState() { return state.isEndState(); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/SplitParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/SplitParser.java index 5209dc22bf..e364fcc402 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/SplitParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/SplitParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -44,9 +44,6 @@ */ public class SplitParser { - /** - * - */ private static final String PARENT_ATTR = "parent"; private final String jobFactoryRef; @@ -84,15 +81,13 @@ public Collection parse(Element element, ParserContext parserCon stateBuilder.addPropertyValue("taskExecutor", taskExecutorRef); } - @SuppressWarnings("unchecked") List flowElements = DomUtils.getChildElementsByTagName(element, "flow"); if (flowElements.size() < 2) { parserContext.getReaderContext().error("A must contain at least two 'flow' elements.", element); } - @SuppressWarnings("unchecked") - Collection flows = new ManagedList(); + Collection flows = new ManagedList<>(); int i = 0; String prefix = idAttribute; for (Element nextElement : flowElements) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/StandaloneStepParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/StandaloneStepParser.java index 5de0846c07..364eaf5cbc 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/StandaloneStepParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/StandaloneStepParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,6 +35,7 @@ public class StandaloneStepParser extends AbstractStepParser { * * @param element the <step/gt; element to parse * @param parserContext the parser context for the bean factory + * @return {@link AbstractBeanDefinition} instance. */ public AbstractBeanDefinition parse(Element element, ParserContext parserContext) { return parseStep(element, parserContext, null); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/StepListenerParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/StepListenerParser.java index bc33146f65..11792aee2d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/StepListenerParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/StepListenerParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -54,7 +54,7 @@ public StepListenerParser(ListenerMetaData[] listenerMetaData) { } @Override - protected Class getBeanClass() { + protected Class> getBeanClass() { return StepListenerFactoryBean.class; } @@ -63,7 +63,7 @@ protected ListenerMetaData[] getMetaDataValues() { return listenerMetaData; } - @SuppressWarnings({"unchecked", "rawtypes"}) + @SuppressWarnings("unchecked") public void handleListenersElement(Element stepElement, BeanDefinition beanDefinition, ParserContext parserContext) { MutablePropertyValues propertyValues = beanDefinition.getPropertyValues(); @@ -73,9 +73,9 @@ public void handleListenersElement(Element stepElement, BeanDefinition beanDefin CompositeComponentDefinition compositeDef = new CompositeComponentDefinition(listenersElement.getTagName(), parserContext.extractSource(stepElement)); parserContext.pushContainingComponent(compositeDef); - ManagedList listenerBeans = new ManagedList(); + ManagedList listenerBeans = new ManagedList<>(); if (propertyValues.contains("listeners")) { - listenerBeans = (ManagedList) propertyValues.getPropertyValue("listeners").getValue(); + listenerBeans = (ManagedList) propertyValues.getPropertyValue("listeners").getValue(); } listenerBeans.setMergeEnabled(listenersElement.hasAttribute(MERGE_ATTR) && Boolean.valueOf(listenersElement.getAttribute(MERGE_ATTR))); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/StepParserStepFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/StepParserStepFactoryBean.java index 9cecd4c431..ba9a68260c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/StepParserStepFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/StepParserStepFactoryBean.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,12 +16,22 @@ package org.springframework.batch.core.configuration.xml; +import java.io.Serializable; import java.util.Collection; import java.util.HashMap; import java.util.HashSet; import java.util.LinkedHashSet; import java.util.Map; +import java.util.Queue; import java.util.Set; +import java.util.concurrent.locks.ReentrantLock; +import javax.batch.api.chunk.listener.RetryProcessListener; +import javax.batch.api.chunk.listener.RetryReadListener; +import javax.batch.api.chunk.listener.RetryWriteListener; +import javax.batch.api.chunk.listener.SkipProcessListener; +import javax.batch.api.chunk.listener.SkipReadListener; +import javax.batch.api.chunk.listener.SkipWriteListener; +import javax.batch.api.partition.PartitionCollector; import org.springframework.batch.core.ChunkListener; import org.springframework.batch.core.ItemProcessListener; @@ -33,6 +43,16 @@ import org.springframework.batch.core.StepExecutionListener; import org.springframework.batch.core.StepListener; import org.springframework.batch.core.job.flow.Flow; +import org.springframework.batch.core.jsr.ChunkListenerAdapter; +import org.springframework.batch.core.jsr.ItemProcessListenerAdapter; +import org.springframework.batch.core.jsr.ItemReadListenerAdapter; +import org.springframework.batch.core.jsr.ItemWriteListenerAdapter; +import org.springframework.batch.core.jsr.RetryProcessListenerAdapter; +import org.springframework.batch.core.jsr.RetryReadListenerAdapter; +import org.springframework.batch.core.jsr.RetryWriteListenerAdapter; +import org.springframework.batch.core.jsr.SkipListenerAdapter; +import org.springframework.batch.core.jsr.StepListenerAdapter; +import org.springframework.batch.core.jsr.partition.PartitionCollectorAdapter; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.partition.PartitionHandler; import org.springframework.batch.core.partition.support.Partitioner; @@ -83,13 +103,14 @@ * * @author Dan Garrette * @author Josh Long + * @author Michael Minella + * @author Chris Schaefer * @see SimpleStepFactoryBean * @see FaultTolerantStepFactoryBean * @see TaskletStep * @since 2.0 */ -@SuppressWarnings("rawtypes") -class StepParserStepFactoryBean implements FactoryBean, BeanNameAware { +public class StepParserStepFactoryBean implements FactoryBean, BeanNameAware { // // Step Attributes @@ -109,7 +130,7 @@ class StepParserStepFactoryBean implements FactoryBean, BeanNameAware { private PlatformTransactionManager transactionManager; - private Set stepExecutionListeners = new LinkedHashSet(); + private Set stepExecutionListeners = new LinkedHashSet<>(); // // Flow Elements @@ -138,6 +159,10 @@ class StepParserStepFactoryBean implements FactoryBean, BeanNameAware { private int gridSize = DEFAULT_GRID_SIZE; + private Queue partitionQueue; + + private ReentrantLock partitionLock; + // // Tasklet Elements // @@ -149,7 +174,7 @@ class StepParserStepFactoryBean implements FactoryBean, BeanNameAware { private Isolation isolation; - private Set chunkListeners = new LinkedHashSet(); + private Set chunkListeners = new LinkedHashSet<>(); // // Chunk Attributes @@ -193,19 +218,21 @@ class StepParserStepFactoryBean implements FactoryBean, BeanNameAware { // private RetryListener[] retryListeners; - private Map, Boolean> skippableExceptionClasses = new HashMap, Boolean>(); + private Map, Boolean> skippableExceptionClasses = new HashMap<>(); - private Map, Boolean> retryableExceptionClasses = new HashMap, Boolean>(); + private Map, Boolean> retryableExceptionClasses = new HashMap<>(); private ItemStream[] streams; - private Set> readListeners = new LinkedHashSet>(); + private Set> readListeners = new LinkedHashSet<>(); + + private Set> writeListeners = new LinkedHashSet<>(); - private Set> writeListeners = new LinkedHashSet>(); + private Set> processListeners = new LinkedHashSet<>(); - private Set> processListeners = new LinkedHashSet>(); + private Set> skipListeners = new LinkedHashSet<>(); - private Set> skipListeners = new LinkedHashSet>(); + private Set jsrRetryListeners = new LinkedHashSet<>(); // // Additional @@ -214,7 +241,21 @@ class StepParserStepFactoryBean implements FactoryBean, BeanNameAware { private StepExecutionAggregator stepExecutionAggregator; - private StepListener[] listeners; + /** + * @param queue The {@link Queue} that is used for communication between {@link javax.batch.api.partition.PartitionCollector} and {@link javax.batch.api.partition.PartitionAnalyzer} + */ + public void setPartitionQueue(Queue queue) { + this.partitionQueue = queue; + } + + /** + * Used to coordinate access to the partition queue between the {@link javax.batch.api.partition.PartitionCollector} and {@link javax.batch.api.partition.AbstractPartitionAnalyzer} + * + * @param lock a lock that will be locked around accessing the partition queue + */ + public void setPartitionLock(ReentrantLock lock) { + this.partitionLock = lock; + } /** * Create a {@link Step} from the configuration provided. @@ -222,12 +263,13 @@ class StepParserStepFactoryBean implements FactoryBean, BeanNameAware { * @see FactoryBean#getObject() */ @Override - public final Object getObject() throws Exception { + public Step getObject() throws Exception { if (hasChunkElement) { Assert.isNull(tasklet, "Step [" + name + "] has both a element and a 'ref' attribute referencing a Tasklet."); validateFaultTolerantSettings(); + if (isFaultTolerant()) { return createFaultTolerantStep(); } @@ -255,7 +297,10 @@ public boolean requiresTransactionManager() { return hasChunkElement || tasklet != null; } - private void enhanceCommonStep(StepBuilderHelper builder) { + /** + * @param builder {@link StepBuilderHelper} representing the step to be enhanced + */ + protected void enhanceCommonStep(StepBuilderHelper builder) { if (allowStartIfComplete != null) { builder.allowStartIfComplete(allowStartIfComplete); } @@ -264,12 +309,16 @@ private void enhanceCommonStep(StepBuilderHelper builder) { } builder.repository(jobRepository); builder.transactionManager(transactionManager); - for (StepExecutionListener listener : stepExecutionListeners) { - builder.listener(listener); + for (Object listener : stepExecutionListeners) { + if(listener instanceof StepExecutionListener) { + builder.listener((StepExecutionListener) listener); + } else if(listener instanceof javax.batch.api.listener.StepListener) { + builder.listener(new StepListenerAdapter((javax.batch.api.listener.StepListener) listener)); + } } } - private Step createPartitionStep() { + protected Step createPartitionStep() { PartitionStepBuilder builder; if (partitioner != null) { @@ -294,13 +343,14 @@ private Step createPartitionStep() { } - private Step createFaultTolerantStep() { + protected Step createFaultTolerantStep() { - FaultTolerantStepBuilder builder = new FaultTolerantStepBuilder(new StepBuilder(name)); + FaultTolerantStepBuilder builder = getFaultTolerantStepBuilder(this.name); if (commitInterval != null) { builder.chunk(commitInterval); } + builder.chunk(chunkCompletionPolicy); enhanceTaskletStepBuilder(builder); builder.reader(itemReader); @@ -311,10 +361,18 @@ private Step createFaultTolerantStep() { builder.processorNonTransactional(); } + if (readerTransactionalQueue!=null && readerTransactionalQueue==true) { + builder.readerIsTransactionalQueue(); + } + for (SkipListener listener : skipListeners) { builder.listener(listener); } + for (org.springframework.batch.core.jsr.RetryListener listener : jsrRetryListeners) { + builder.listener(listener); + } + registerItemListeners(builder); if (skipPolicy != null) { @@ -369,7 +427,11 @@ else if (skipLimit!=null) { } - private void registerItemListeners(SimpleStepBuilder builder) { + protected FaultTolerantStepBuilder getFaultTolerantStepBuilder(String stepName) { + return new FaultTolerantStepBuilder<>(new StepBuilder(stepName)); + } + + protected void registerItemListeners(SimpleStepBuilder builder) { for (ItemReadListener listener : readListeners) { builder.listener(listener); } @@ -381,32 +443,52 @@ private void registerItemListeners(SimpleStepBuilder builder) { } } - @SuppressWarnings("unchecked") - private Step createSimpleStep() { - SimpleStepBuilder builder = new SimpleStepBuilder(new StepBuilder(name)); - if (commitInterval != null) { - builder.chunk(commitInterval); - } + protected Step createSimpleStep() { + SimpleStepBuilder builder = getSimpleStepBuilder(name); + + setChunk(builder); + enhanceTaskletStepBuilder(builder); registerItemListeners(builder); builder.reader(itemReader); builder.writer(itemWriter); builder.processor(itemProcessor); - builder.chunk(chunkCompletionPolicy); return builder.build(); } - private TaskletStep createTaskletStep() { + protected void setChunk(SimpleStepBuilder builder) { + if (commitInterval != null) { + builder.chunk(commitInterval); + } + builder.chunk(chunkCompletionPolicy); + } + + protected CompletionPolicy getCompletionPolicy() { + return this.chunkCompletionPolicy; + } + + protected SimpleStepBuilder getSimpleStepBuilder(String stepName) { + return new SimpleStepBuilder<>(new StepBuilder(stepName)); + } + + /** + * @return a new {@link TaskletStep} + */ + protected TaskletStep createTaskletStep() { TaskletStepBuilder builder = new StepBuilder(name).tasklet(tasklet); enhanceTaskletStepBuilder(builder); return builder.build(); } @SuppressWarnings("serial") - private void enhanceTaskletStepBuilder(AbstractTaskletStepBuilder builder) { + protected void enhanceTaskletStepBuilder(AbstractTaskletStepBuilder builder) { enhanceCommonStep(builder); for (ChunkListener listener : chunkListeners) { + if(listener instanceof PartitionCollectorAdapter) { + ((PartitionCollectorAdapter) listener).setPartitionLock(partitionLock); + } + builder.listener(listener); } @@ -427,7 +509,7 @@ private void enhanceTaskletStepBuilder(AbstractTaskletStepBuilder builder) { if (transactionTimeout != null) { attribute.setTimeout(transactionTimeout); } - Collection> exceptions = noRollbackExceptionClasses == null ? new HashSet>() + Collection> exceptions = noRollbackExceptionClasses == null ? new HashSet<>() : noRollbackExceptionClasses; final BinaryExceptionClassifier classifier = new BinaryExceptionClassifier(exceptions, false); builder.transactionAttribute(new DefaultTransactionAttribute(attribute) { @@ -445,7 +527,7 @@ public boolean rollbackOn(Throwable ex) { } - private Step createFlowStep() { + protected Step createFlowStep() { FlowStepBuilder builder = new StepBuilder(name).flow(flow); enhanceCommonStep(builder); return builder.build(); @@ -461,7 +543,10 @@ private Step createJobStep() throws Exception { } - private void validateFaultTolerantSettings() { + /** + * Validates that all components required to build a fault tolerant step are set + */ + protected void validateFaultTolerantSettings() { validateDependency("skippable-exception-classes", skippableExceptionClasses, "skip-limit", skipLimit, true); validateDependency("retryable-exception-classes", retryableExceptionClasses, "retry-limit", retryLimit, true); validateDependency("retry-listeners", retryListeners, "retry-limit", retryLimit, false); @@ -514,7 +599,10 @@ private boolean isPresent(Object o) { return o != null; } - private boolean isFaultTolerant() { + /** + * @return true if the step is configured with any components that require fault tolerance + */ + protected boolean isFaultTolerant() { return backOffPolicy != null || skipPolicy != null || retryPolicy != null || isPositive(skipLimit) || isPositive(retryLimit) || isPositive(cacheCapacity) || isTrue(readerTransactionalQueue); } @@ -560,6 +648,10 @@ public void setName(String name) { this.name = name; } + public String getName() { + return this.name; + } + // ========================================================= // Flow Attributes // ========================================================= @@ -605,6 +697,13 @@ public void setStepExecutionAggregator(StepExecutionAggregator stepExecutionAggr this.stepExecutionAggregator = stepExecutionAggregator; } + /** + * @return stepExecutionAggregator the current step's {@link StepExecutionAggregator} + */ + protected StepExecutionAggregator getStepExecutionAggergator() { + return this.stepExecutionAggregator; + } + /** * @param partitionHandler the partitionHandler to set */ @@ -612,6 +711,13 @@ public void setPartitionHandler(PartitionHandler partitionHandler) { this.partitionHandler = partitionHandler; } + /** + * @return partitionHandler the current step's {@link PartitionHandler} + */ + protected PartitionHandler getPartitionHandler() { + return this.partitionHandler; + } + /** * @param gridSize the gridSize to set */ @@ -651,7 +757,7 @@ public JobRepository getJobRepository() { /** * Public setter for {@link JobRepository}. * - * @param jobRepository + * @param jobRepository {@link JobRepository} instance to be used by the step. */ public void setJobRepository(JobRepository jobRepository) { this.jobRepository = jobRepository; @@ -660,7 +766,7 @@ public void setJobRepository(JobRepository jobRepository) { /** * The number of times that the step should be allowed to start * - * @param startLimit + * @param startLimit int containing the number of times a step should be allowed to start. */ public void setStartLimit(int startLimit) { this.startLimit = startLimit; @@ -669,14 +775,19 @@ public void setStartLimit(int startLimit) { /** * A preconfigured {@link Tasklet} to use. * - * @param tasklet + * @param tasklet {@link Tasklet} instance to be used by step. */ public void setTasklet(Tasklet tasklet) { this.tasklet = tasklet; } + protected Tasklet getTasklet() { + return this.tasklet; + } + /** - * @return transactionManager + * @return transactionManager instance of {@link PlatformTransactionManager} + * used by the step. */ public PlatformTransactionManager getTransactionManager() { return transactionManager; @@ -699,37 +810,78 @@ public void setTransactionManager(PlatformTransactionManager transactionManager) * * @param listeners an array of listeners */ - public void setListeners(StepListener[] listeners) { - this.listeners = listeners; // useful for testing - for (StepListener listener : listeners) { + @SuppressWarnings("unchecked") + public void setListeners(Object[] listeners) { + for (Object listener : listeners) { if (listener instanceof SkipListener) { - @SuppressWarnings("unchecked") SkipListener skipListener = (SkipListener) listener; skipListeners.add(skipListener); } + if(listener instanceof SkipReadListener) { + SkipListener skipListener = new SkipListenerAdapter<>((SkipReadListener) listener, null, null); + skipListeners.add(skipListener); + } + if(listener instanceof SkipProcessListener) { + SkipListener skipListener = new SkipListenerAdapter<>(null, (SkipProcessListener) listener, null); + skipListeners.add(skipListener); + } + if(listener instanceof SkipWriteListener) { + SkipListener skipListener = new SkipListenerAdapter<>(null, null, (SkipWriteListener) listener); + skipListeners.add(skipListener); + } if (listener instanceof StepExecutionListener) { StepExecutionListener stepExecutionListener = (StepExecutionListener) listener; stepExecutionListeners.add(stepExecutionListener); } + if(listener instanceof javax.batch.api.listener.StepListener) { + StepExecutionListener stepExecutionListener = new StepListenerAdapter((javax.batch.api.listener.StepListener) listener); + stepExecutionListeners.add(stepExecutionListener); + } if (listener instanceof ChunkListener) { ChunkListener chunkListener = (ChunkListener) listener; chunkListeners.add(chunkListener); } + if(listener instanceof javax.batch.api.chunk.listener.ChunkListener) { + ChunkListener chunkListener = new ChunkListenerAdapter((javax.batch.api.chunk.listener.ChunkListener) listener); + chunkListeners.add(chunkListener); + } if (listener instanceof ItemReadListener) { - @SuppressWarnings("unchecked") ItemReadListener readListener = (ItemReadListener) listener; readListeners.add(readListener); } + if(listener instanceof javax.batch.api.chunk.listener.ItemReadListener) { + ItemReadListener itemListener = new ItemReadListenerAdapter<>((javax.batch.api.chunk.listener.ItemReadListener) listener); + readListeners.add(itemListener); + } if (listener instanceof ItemWriteListener) { - @SuppressWarnings("unchecked") ItemWriteListener writeListener = (ItemWriteListener) listener; writeListeners.add(writeListener); } + if(listener instanceof javax.batch.api.chunk.listener.ItemWriteListener) { + ItemWriteListener itemListener = new ItemWriteListenerAdapter<>((javax.batch.api.chunk.listener.ItemWriteListener) listener); + writeListeners.add(itemListener); + } if (listener instanceof ItemProcessListener) { - @SuppressWarnings("unchecked") ItemProcessListener processListener = (ItemProcessListener) listener; processListeners.add(processListener); } + if(listener instanceof javax.batch.api.chunk.listener.ItemProcessListener) { + ItemProcessListener itemListener = new ItemProcessListenerAdapter<>((javax.batch.api.chunk.listener.ItemProcessListener) listener); + processListeners.add(itemListener); + } + if(listener instanceof RetryReadListener) { + jsrRetryListeners.add(new RetryReadListenerAdapter((RetryReadListener) listener)); + } + if(listener instanceof RetryProcessListener) { + jsrRetryListeners.add(new RetryProcessListenerAdapter((RetryProcessListener) listener)); + } + if(listener instanceof RetryWriteListener) { + jsrRetryListeners.add(new RetryWriteListenerAdapter((RetryWriteListener) listener)); + } + if(listener instanceof PartitionCollector) { + PartitionCollectorAdapter adapter = new PartitionCollectorAdapter(partitionQueue, (PartitionCollector) listener); + chunkListeners.add(adapter); + } } } @@ -810,11 +962,11 @@ public void setKeyGenerator(KeyGenerator keyGenerator) { /** * Public setter for the capacity of the cache in the retry policy. If more items than this fail without being * skipped or recovered an exception will be thrown. This is to guard against inadvertent infinite loops generated - * by item identity problems.
      - *

      + * by item identity problems.
      + *
      * The default value should be high enough and more for most purposes. To breach the limit in a single-threaded step * typically you have to have this many failures in a single transaction. Defaults to the value in the - * {@link MapRetryContextCache}.
      + * {@link MapRetryContextCache}.
      * * @param cacheCapacity the cache capacity to set (greater than 0 else ignored) */ @@ -842,6 +994,10 @@ public void setCommitInterval(int commitInterval) { this.commitInterval = commitInterval; } + protected Integer getCommitInterval() { + return this.commitInterval; + } + /** * Flag to signal that the reader is transactional (usually a JMS consumer) so that items are re-presented after a * rollback. The default is false and readers are assumed to be forward-only. @@ -950,7 +1106,8 @@ public void setRetryListeners(RetryListener... retryListeners) { * Public setter for exception classes that when raised won't crash the job but will result in transaction rollback * and the item which handling caused the exception will be skipped. * - * @param exceptionClasses + * @param exceptionClasses {@link Map} containing the {@link Throwable}s as + * the keys and the values are {@link Boolean}s, that if true the item is skipped. */ public void setSkippableExceptionClasses(Map, Boolean> exceptionClasses) { this.skippableExceptionClasses = exceptionClasses; @@ -980,10 +1137,30 @@ public void setStreams(ItemStream[] streams) { // ========================================================= /** - * @param hasChunkElement + * @param hasChunkElement true if step has <chunk/> element. */ public void setHasChunkElement(boolean hasChunkElement) { this.hasChunkElement = hasChunkElement; } + /** + * @return true if the defined step has a <chunk/> element + */ + protected boolean hasChunkElement() { + return this.hasChunkElement; + } + + /** + * @return true if the defined step has a <tasklet/> element + */ + protected boolean hasTasklet() { + return this.tasklet != null; + } + + /** + * @return true if the defined step has a <partition/> element + */ + protected boolean hasPartitionElement() { + return this.partitionHandler != null; + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TaskletParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TaskletParser.java index 25b8e39715..6db4f597f7 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TaskletParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TaskletParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -42,9 +42,6 @@ */ public class TaskletParser { - /** - * - */ private static final String TRANSACTION_MANAGER_ATTR = "transaction-manager"; private static final String TASKLET_REF_ATTR = "ref"; @@ -76,11 +73,8 @@ public void parseTasklet(Element stepElement, Element taskletElement, AbstractBe String taskletRef = taskletElement.getAttribute(TASKLET_REF_ATTR); String taskletMethod = taskletElement.getAttribute(TASKLET_METHOD_ATTR); - @SuppressWarnings("unchecked") List chunkElements = DomUtils.getChildElementsByTagName(taskletElement, CHUNK_ELE); - @SuppressWarnings("unchecked") List beanElements = DomUtils.getChildElementsByTagName(taskletElement, BEAN_ELE); - @SuppressWarnings("unchecked") List refElements = DomUtils.getChildElementsByTagName(taskletElement, REF_ELE); validateTaskletAttributesAndSubelements(taskletElement, parserContext, stepUnderspecified, taskletRef, @@ -193,7 +187,6 @@ private void handleTaskletElement(Element taskletElement, AbstractBeanDefinition } private void handleTransactionAttributesElement(Element stepElement, MutablePropertyValues propertyValues) { - @SuppressWarnings("unchecked") List txAttrElements = DomUtils.getChildElementsByTagName(stepElement, TX_ATTRIBUTES_ELE); if (txAttrElements.size() == 1) { Element txAttrElement = txAttrElements.get(0); @@ -212,13 +205,12 @@ private void handleTransactionAttributesElement(Element stepElement, MutableProp } } - @SuppressWarnings("unchecked") private void handleExceptionElement(Element element, ParserContext parserContext, MutablePropertyValues propertyValues, String exceptionListName, String propertyName) { List children = DomUtils.getChildElementsByTagName(element, exceptionListName); if (children.size() == 1) { Element exceptionClassesElement = children.get(0); - ManagedList list = new ManagedList(); + ManagedList list = new ManagedList<>(); list.setMergeEnabled(exceptionClassesElement.hasAttribute(MERGE_ATTR) && Boolean.valueOf(exceptionClassesElement.getAttribute(MERGE_ATTR))); addExceptionClasses("include", exceptionClassesElement, list, parserContext); @@ -231,10 +223,9 @@ else if (children.size() > 1) { } } - @SuppressWarnings("unchecked") - private void addExceptionClasses(String elementName, Element exceptionClassesElement, ManagedList list, + private void addExceptionClasses(String elementName, Element exceptionClassesElement, ManagedList list, ParserContext parserContext) { - for (Element child : (List) DomUtils.getChildElementsByTagName(exceptionClassesElement, elementName)) { + for (Element child : DomUtils.getChildElementsByTagName(exceptionClassesElement, elementName)) { String className = child.getAttribute("class"); list.add(new TypedStringValue(className, Class.class)); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelFlowParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelFlowParser.java index a6970d1e97..69f3fb9b17 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelFlowParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelFlowParser.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2013 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,6 +15,8 @@ */ package org.springframework.batch.core.configuration.xml; +import org.springframework.batch.core.job.flow.support.DefaultStateTransitionComparator; +import org.springframework.beans.factory.config.RuntimeBeanReference; import org.springframework.beans.factory.support.BeanDefinitionBuilder; import org.springframework.beans.factory.xml.ParserContext; import org.springframework.util.StringUtils; @@ -22,12 +24,11 @@ /** * @author Dave Syer - * + * @author Michael Minella + * */ public class TopLevelFlowParser extends AbstractFlowParser { - private static final String ID_ATTR = "id"; - private static final String ABSTRACT_ATTR = "abstract"; /** @@ -40,6 +41,7 @@ protected void doParse(Element element, ParserContext parserContext, BeanDefinit String flowName = element.getAttribute(ID_ATTR); builder.getRawBeanDefinition().setAttribute("flowName", flowName); builder.addPropertyValue("name", flowName); + builder.addPropertyValue("stateTransitionComparator", new RuntimeBeanReference(DefaultStateTransitionComparator.STATE_TRANSITION_COMPARATOR)); String abstractAttr = element.getAttribute(ABSTRACT_ATTR); if (StringUtils.hasText(abstractAttr)) { builder.setAbstract(abstractAttr.equals("true")); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelJobListenerParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelJobListenerParser.java index c74beaa9f4..a5f38ffcad 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelJobListenerParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelJobListenerParser.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.core.listener.AbstractListenerFactoryBean; @@ -23,7 +38,7 @@ protected void doParse(Element element, ParserContext parserContext, BeanDefinit } @Override - protected Class getBeanClass(Element element) { + protected Class> getBeanClass(Element element) { return jobListenerParser.getBeanClass(); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelStepListenerParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelStepListenerParser.java index 070dddddcc..f5bdb36e09 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelStepListenerParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelStepListenerParser.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.core.listener.AbstractListenerFactoryBean; @@ -23,7 +38,7 @@ protected void doParse(Element element, ParserContext parserContext, BeanDefinit } @Override - protected Class getBeanClass(Element element) { + protected Class> getBeanClass(Element element) { return stepListenerParser.getBeanClass(); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelStepParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelStepParser.java index d43be0157d..c2d85c0b5b 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelStepParser.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/TopLevelStepParser.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,7 +21,7 @@ import org.w3c.dom.Element; /** - * Parser for the lt;step/gt; top level element in the Batch namespace. Sets up + * Parser for the <step/> top level element in the Batch namespace. Sets up * and returns a bean definition for a * {@link org.springframework.batch.core.Step}. * diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/package-info.java new file mode 100644 index 0000000000..06d7e96347 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/xml/package-info.java @@ -0,0 +1,10 @@ +/** + * Parsers for XML based configuration + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.configuration.xml; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/converter/DefaultJobParametersConverter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/converter/DefaultJobParametersConverter.java index 995c64ef3a..b7ea06eb5a 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/converter/DefaultJobParametersConverter.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/converter/DefaultJobParametersConverter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,6 +15,14 @@ */ package org.springframework.batch.core.converter; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameter; +import org.springframework.batch.core.JobParameter.ParameterType; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.lang.Nullable; +import org.springframework.util.StringUtils; + import java.text.DateFormat; import java.text.DecimalFormat; import java.text.NumberFormat; @@ -27,13 +35,6 @@ import java.util.Map.Entry; import java.util.Properties; -import org.springframework.batch.core.JobInstance; -import org.springframework.batch.core.JobParameter; -import org.springframework.batch.core.JobParameter.ParameterType; -import org.springframework.batch.core.JobParameters; -import org.springframework.batch.core.JobParametersBuilder; -import org.springframework.util.StringUtils; - /** * Converter for {@link JobParameters} instances using a simple naming * convention for property keys. Key names that are prefixed with a - are @@ -50,14 +51,15 @@ * The literal values are converted to the correct type using the default Spring * strategies, augmented if necessary by the custom editors provided. * - *
      - * + *
      + * * If you need to be able to parse and format local-specific dates and numbers, * you can inject formatters ({@link #setDateFormat(DateFormat)} and * {@link #setNumberFormat(NumberFormat)}). * * @author Dave Syer * @author Michael Minella + * @author Mahmoud Ben Hassine * */ public class DefaultJobParametersConverter implements JobParametersConverter { @@ -92,7 +94,7 @@ public class DefaultJobParametersConverter implements JobParametersConverter { * @see org.springframework.batch.core.converter.JobParametersConverter#getJobParameters(java.util.Properties) */ @Override - public JobParameters getJobParameters(Properties props) { + public JobParameters getJobParameters(@Nullable Properties props) { if (props == null || props.isEmpty()) { return new JobParameters(); @@ -114,13 +116,15 @@ public JobParameters getJobParameters(Properties props) { if (key.endsWith(DATE_TYPE)) { Date date; - try { - date = dateFormat.parse(value); - } - catch (ParseException ex) { - String suffix = (dateFormat instanceof SimpleDateFormat) ? ", use " - + ((SimpleDateFormat) dateFormat).toPattern() : ""; - throw new IllegalArgumentException("Date format is invalid: [" + value + "]" + suffix); + synchronized (dateFormat) { + try { + date = dateFormat.parse(value); + } + catch (ParseException ex) { + String suffix = (dateFormat instanceof SimpleDateFormat) ? ", use " + + ((SimpleDateFormat) dateFormat).toPattern() : ""; + throw new IllegalArgumentException("Date format is invalid: [" + value + "]" + suffix); + } } propertiesBuilder.addDate(StringUtils.replace(key, DATE_TYPE, ""), date, identifying); } @@ -164,24 +168,29 @@ private boolean isIdentifyingKey(String key) { * Delegate to {@link NumberFormat} to parse the value */ private Number parseNumber(String value) { - try { - return numberFormat.parse(value); - } - catch (ParseException ex) { - String suffix = (numberFormat instanceof DecimalFormat) ? ", use " - + ((DecimalFormat) numberFormat).toPattern() : ""; - throw new IllegalArgumentException("Number format is invalid: [" + value + "], use " + suffix); + synchronized (numberFormat) { + try { + return numberFormat.parse(value); + } + catch (ParseException ex) { + String suffix = (numberFormat instanceof DecimalFormat) ? ", use " + + ((DecimalFormat) numberFormat).toPattern() : ""; + throw new IllegalArgumentException("Number format is invalid: [" + value + "], use " + suffix); + } } } /** * Use the same suffixes to create properties (omitting the string suffix - * because it is the default). + * because it is the default). Non-identifying parameters will be prefixed + * with the {@link #NON_IDENTIFYING_FLAG}. However, since parameters are + * identifying by default, they will not be prefixed with the + * {@link #IDENTIFYING_FLAG}. * * @see org.springframework.batch.core.converter.JobParametersConverter#getProperties(org.springframework.batch.core.JobParameters) */ @Override - public Properties getProperties(JobParameters params) { + public Properties getProperties(@Nullable JobParameters params) { if (params == null || params.isEmpty()) { return new Properties(); @@ -195,11 +204,16 @@ public Properties getProperties(JobParameters params) { JobParameter jobParameter = entry.getValue(); Object value = jobParameter.getValue(); if (value != null) { + key = (!jobParameter.isIdentifying()? NON_IDENTIFYING_FLAG : "") + key; if (jobParameter.getType() == ParameterType.DATE) { - result.setProperty(key + DATE_TYPE, dateFormat.format(value)); + synchronized (dateFormat) { + result.setProperty(key + DATE_TYPE, dateFormat.format(value)); + } } else if (jobParameter.getType() == ParameterType.LONG) { - result.setProperty(key + LONG_TYPE, longNumberFormat.format(value)); + synchronized (longNumberFormat) { + result.setProperty(key + LONG_TYPE, longNumberFormat.format(value)); + } } else if (jobParameter.getType() == ParameterType.DOUBLE) { result.setProperty(key + DOUBLE_TYPE, decimalFormat((Double)value)); @@ -218,7 +232,9 @@ else if (jobParameter.getType() == ParameterType.DOUBLE) { */ private String decimalFormat(double value) { if (numberFormat != DEFAULT_NUMBER_FORMAT) { - return numberFormat.format(value); + synchronized (numberFormat) { + return numberFormat.format(value); + } } return Double.toString(value); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/converter/JobParametersConverter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/converter/JobParametersConverter.java index 48a415edab..5bc3b86a90 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/converter/JobParametersConverter.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/converter/JobParametersConverter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.lang.Nullable; /** * A factory for {@link JobParameters} instances. A job can be executed with @@ -27,6 +28,7 @@ * This converter allows job parameters to be converted to and from Properties. * * @author Dave Syer + * @author Mahmoud Ben Hassine * * @see JobParametersBuilder * @@ -41,14 +43,14 @@ public interface JobParametersConverter { * @return a {@link JobParameters} properties converted to the correct * types. */ - public JobParameters getJobParameters(Properties properties); + JobParameters getJobParameters(@Nullable Properties properties); /** * The inverse operation: get a {@link Properties} instance. If given null * or empty JobParameters, an empty Properties should be returned. * - * @param params + * @param params the {@link JobParameters} instance to be converted. * @return a representation of the parameters as properties */ - public Properties getProperties(JobParameters params); + Properties getProperties(@Nullable JobParameters params); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/converter/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/converter/package-info.java new file mode 100644 index 0000000000..3330b0f300 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/converter/package-info.java @@ -0,0 +1,11 @@ +/** + * Support classes for implementations of the batch APIs. Things like converters and resource location and management + * concerns. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.converter; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/converter/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/converter/package.html deleted file mode 100644 index 72f5b5cf62..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/converter/package.html +++ /dev/null @@ -1,6 +0,0 @@ - - -

      Support classes for implementations of the batch APIs. Things -like converters and resource location and management concerns.

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/JobExplorer.java b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/JobExplorer.java index f1d81479d5..586ad67be1 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/JobExplorer.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/JobExplorer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,15 +21,19 @@ import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.launch.NoSuchJobException; import org.springframework.batch.item.ExecutionContext; +import org.springframework.lang.Nullable; /** * Entry point for browsing executions of running or historical jobs and steps. * Since the data may be re-hydrated from persistent storage, it may not contain * volatile fields that would have been present when the execution was active. - * + * * @author Dave Syer - * + * @author Michael Minella + * @author Will Schipp + * @author Mahmoud Ben Hassine * @since 2.0 */ public interface JobExplorer { @@ -37,7 +41,7 @@ public interface JobExplorer { /** * Fetch {@link JobInstance} values in descending order of creation (and * therefore usually of first execution). - * + * * @param jobName the name of the job to query * @param start the start index of the instances to return * @param count the maximum number of instances to return @@ -45,66 +49,119 @@ public interface JobExplorer { */ List getJobInstances(String jobName, int start, int count); + /** + * Find the last job instance by Id for the given job. + * @param jobName name of the job + * @return the last job instance by Id if any or null otherwise + * + * @since 4.2 + */ + @Nullable + default JobInstance getLastJobInstance(String jobName) { + throw new UnsupportedOperationException(); + } + /** * Retrieve a {@link JobExecution} by its id. The complete object graph for * this execution should be returned (unless otherwise indicated) including * the parent {@link JobInstance} and associated {@link ExecutionContext} * and {@link StepExecution} instances (also including their execution * contexts). - * + * * @param executionId the job execution id * @return the {@link JobExecution} with this id, or null if not found */ - JobExecution getJobExecution(Long executionId); + @Nullable + JobExecution getJobExecution(@Nullable Long executionId); /** * Retrieve a {@link StepExecution} by its id and parent * {@link JobExecution} id. The execution context for the step should be * available in the result, and the parent job execution should have its * primitive properties, but may not contain the job instance information. - * + * * @param jobExecutionId the parent job execution id * @param stepExecutionId the step execution id * @return the {@link StepExecution} with this id, or null if not found - * + * * @see #getJobExecution(Long) */ - StepExecution getStepExecution(Long jobExecutionId, Long stepExecutionId); + @Nullable + StepExecution getStepExecution(@Nullable Long jobExecutionId, @Nullable Long stepExecutionId); /** - * @param instanceId + * @param instanceId {@link Long} id for the jobInstance to obtain. * @return the {@link JobInstance} with this id, or null */ - JobInstance getJobInstance(Long instanceId); + @Nullable + JobInstance getJobInstance(@Nullable Long instanceId); /** * Retrieve job executions by their job instance. The corresponding step * executions may not be fully hydrated (e.g. their execution context may be * missing), depending on the implementation. Use * {@link #getStepExecution(Long, Long)} to hydrate them in that case. - * + * * @param jobInstance the {@link JobInstance} to query * @return the set of all executions for the specified {@link JobInstance} */ List getJobExecutions(JobInstance jobInstance); + /** + * Find the last {@link JobExecution} that has been created for a given + * {@link JobInstance}. + * @param jobInstance the {@link JobInstance} + * @return the last {@link JobExecution} that has been created for this instance or + * {@code null} if no job execution is found for the given job instance. + * + * @since 4.2 + */ + @Nullable + default JobExecution getLastJobExecution(JobInstance jobInstance) { + throw new UnsupportedOperationException(); + } + /** * Retrieve running job executions. The corresponding step executions may * not be fully hydrated (e.g. their execution context may be missing), * depending on the implementation. Use * {@link #getStepExecution(Long, Long)} to hydrate them in that case. - * + * * @param jobName the name of the job * @return the set of running executions for jobs with the specified name */ - Set findRunningJobExecutions(String jobName); + Set findRunningJobExecutions(@Nullable String jobName); /** * Query the repository for all unique {@link JobInstance} names (sorted * alphabetically). - * + * * @return the set of job names that have been executed */ List getJobNames(); + + /** + * Fetch {@link JobInstance} values in descending order of creation (and + * there for usually of first execution) with a 'like'/wildcard criteria. + * + * @param jobName the name of the job to query for. + * @param start the start index of the instances to return. + * @param count the maximum number of instances to return. + * @return a list of {@link JobInstance} for the job name requested. + */ + List findJobInstancesByJobName(String jobName, int start, int count); + + /** + * Query the repository for the number of unique {@link JobInstance}s + * associated with the supplied job name. + * + * @param jobName the name of the job to query for + * @return the number of {@link JobInstance}s that exist within the + * associated job repository + * + * @throws NoSuchJobException thrown when there is no {@link JobInstance} + * for the jobName specified. + */ + int getJobInstanceCount(@Nullable String jobName) throws NoSuchJobException; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/package-info.java new file mode 100644 index 0000000000..b5671f50be --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/package-info.java @@ -0,0 +1,10 @@ +/** + * Interfaces and related classes to support meta data browsing. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.explore; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/package.html deleted file mode 100644 index d8d027732d..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Interfaces and related classes to support meta data browsing. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/AbstractJobExplorerFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/AbstractJobExplorerFactoryBean.java index a6219d8d1a..e7d3a8ed00 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/AbstractJobExplorerFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/AbstractJobExplorerFactoryBean.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,21 +34,34 @@ * @author Dave Syer * @since 2.0 */ -@SuppressWarnings("rawtypes") -public abstract class AbstractJobExplorerFactoryBean implements FactoryBean { +public abstract class AbstractJobExplorerFactoryBean implements FactoryBean { /** * @return fully configured {@link JobInstanceDao} implementation. + * + * @throws Exception thrown if error occurs during JobInstanceDao creation. */ protected abstract JobInstanceDao createJobInstanceDao() throws Exception; /** * @return fully configured {@link JobExecutionDao} implementation. + * + * @throws Exception thrown if error occurs during JobExecutionDao creation. */ protected abstract JobExecutionDao createJobExecutionDao() throws Exception; + /** + * @return fully configured {@link StepExecutionDao} implementation. + * + * @throws Exception thrown if error occurs during StepExecutionDao creation. + */ protected abstract StepExecutionDao createStepExecutionDao() throws Exception; + /** + * @return fully configured {@link ExecutionContextDao} implementation. + * + * @throws Exception thrown if error occurs during ExecutionContextDao creation. + */ protected abstract ExecutionContextDao createExecutionContextDao() throws Exception; /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/JobExplorerFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/JobExplorerFactoryBean.java index f60b57920c..759ae17518 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/JobExplorerFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/JobExplorerFactoryBean.java @@ -1,11 +1,11 @@ /* - * Copyright 2002-2013 the original author or authors. + * Copyright 2002-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,9 +18,11 @@ import javax.sql.DataSource; +import org.springframework.batch.core.explore.JobExplorer; import org.springframework.batch.core.repository.ExecutionContextSerializer; import org.springframework.batch.core.repository.dao.AbstractJdbcBatchMetadataDao; import org.springframework.batch.core.repository.dao.ExecutionContextDao; +import org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer; import org.springframework.batch.core.repository.dao.JdbcExecutionContextDao; import org.springframework.batch.core.repository.dao.JdbcJobExecutionDao; import org.springframework.batch.core.repository.dao.JdbcJobInstanceDao; @@ -28,7 +30,6 @@ import org.springframework.batch.core.repository.dao.JobExecutionDao; import org.springframework.batch.core.repository.dao.JobInstanceDao; import org.springframework.batch.core.repository.dao.StepExecutionDao; -import org.springframework.batch.core.repository.dao.XStreamExecutionContextStringSerializer; import org.springframework.batch.item.ExecutionContext; import org.springframework.beans.factory.FactoryBean; import org.springframework.beans.factory.InitializingBean; @@ -45,6 +46,7 @@ * to describe what kind of database they are using. * * @author Dave Syer + * @author Mahmoud Ben Hassine * @since 2.0 */ public class JobExplorerFactoryBean extends AbstractJobExplorerFactoryBean @@ -52,7 +54,7 @@ public class JobExplorerFactoryBean extends AbstractJobExplorerFactoryBean private DataSource dataSource; - private JdbcOperations jdbcTemplate; + private JdbcOperations jdbcOperations; private String tablePrefix = AbstractJdbcBatchMetadataDao.DEFAULT_TABLE_PREFIX; @@ -69,9 +71,9 @@ protected long getNextKey() { /** * A custom implementation of the {@link ExecutionContextSerializer}. - * The default, if not injected, is the {@link XStreamExecutionContextStringSerializer}. + * The default, if not injected, is the {@link Jackson2ExecutionContextStringSerializer}. * - * @param serializer + * @param serializer used to serialize/deserialize an {@link org.springframework.batch.item.ExecutionContext} * @see ExecutionContextSerializer */ public void setSerializer(ExecutionContextSerializer serializer) { @@ -87,11 +89,20 @@ public void setSerializer(ExecutionContextSerializer serializer) { public void setDataSource(DataSource dataSource) { this.dataSource = dataSource; } + + /** + * Public setter for the {@link JdbcOperations}. If this property is not set explicitly, + * a new {@link JdbcTemplate} will be created for the configured DataSource by default. + * @param jdbcOperations a {@link JdbcOperations} + */ + public void setJdbcOperations(JdbcOperations jdbcOperations) { + this.jdbcOperations = jdbcOperations; + } /** * Sets the table prefix for all the batch meta-data tables. * - * @param tablePrefix + * @param tablePrefix prefix for the batch meta-data tables */ public void setTablePrefix(String tablePrefix) { this.tablePrefix = tablePrefix; @@ -101,7 +112,7 @@ public void setTablePrefix(String tablePrefix) { * The lob handler to use when saving {@link ExecutionContext} instances. * Defaults to null which works for most databases. * - * @param lobHandler + * @param lobHandler Large object handler for saving {@link org.springframework.batch.item.ExecutionContext} */ public void setLobHandler(LobHandler lobHandler) { this.lobHandler = lobHandler; @@ -112,17 +123,16 @@ public void afterPropertiesSet() throws Exception { Assert.notNull(dataSource, "DataSource must not be null."); - jdbcTemplate = new JdbcTemplate(dataSource); + if (jdbcOperations == null) { + jdbcOperations = new JdbcTemplate(dataSource); + } if(serializer == null) { - XStreamExecutionContextStringSerializer defaultSerializer = new XStreamExecutionContextStringSerializer(); - defaultSerializer.afterPropertiesSet(); - - serializer = defaultSerializer; + serializer = new Jackson2ExecutionContextStringSerializer(); } } - private Object getTarget() throws Exception { + private JobExplorer getTarget() throws Exception { return new SimpleJobExplorer(createJobInstanceDao(), createJobExecutionDao(), createStepExecutionDao(), createExecutionContextDao()); @@ -131,7 +141,7 @@ private Object getTarget() throws Exception { @Override protected ExecutionContextDao createExecutionContextDao() throws Exception { JdbcExecutionContextDao dao = new JdbcExecutionContextDao(); - dao.setJdbcTemplate(jdbcTemplate); + dao.setJdbcTemplate(jdbcOperations); dao.setLobHandler(lobHandler); dao.setTablePrefix(tablePrefix); dao.setSerializer(serializer); @@ -142,7 +152,7 @@ protected ExecutionContextDao createExecutionContextDao() throws Exception { @Override protected JobInstanceDao createJobInstanceDao() throws Exception { JdbcJobInstanceDao dao = new JdbcJobInstanceDao(); - dao.setJdbcTemplate(jdbcTemplate); + dao.setJdbcTemplate(jdbcOperations); dao.setJobIncrementer(incrementer); dao.setTablePrefix(tablePrefix); dao.afterPropertiesSet(); @@ -152,7 +162,7 @@ protected JobInstanceDao createJobInstanceDao() throws Exception { @Override protected JobExecutionDao createJobExecutionDao() throws Exception { JdbcJobExecutionDao dao = new JdbcJobExecutionDao(); - dao.setJdbcTemplate(jdbcTemplate); + dao.setJdbcTemplate(jdbcOperations); dao.setJobExecutionIncrementer(incrementer); dao.setTablePrefix(tablePrefix); dao.afterPropertiesSet(); @@ -162,7 +172,7 @@ protected JobExecutionDao createJobExecutionDao() throws Exception { @Override protected StepExecutionDao createStepExecutionDao() throws Exception { JdbcStepExecutionDao dao = new JdbcStepExecutionDao(); - dao.setJdbcTemplate(jdbcTemplate); + dao.setJdbcTemplate(jdbcOperations); dao.setStepExecutionIncrementer(incrementer); dao.setTablePrefix(tablePrefix); dao.afterPropertiesSet(); @@ -170,7 +180,7 @@ protected StepExecutionDao createStepExecutionDao() throws Exception { } @Override - public Object getObject() throws Exception { + public JobExplorer getObject() throws Exception { return getTarget(); } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/MapJobExplorerFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/MapJobExplorerFactoryBean.java index b8df7e9774..080cedd88e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/MapJobExplorerFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/MapJobExplorerFactoryBean.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,6 +16,7 @@ package org.springframework.batch.core.explore.support; +import org.springframework.batch.core.explore.JobExplorer; import org.springframework.batch.core.repository.dao.ExecutionContextDao; import org.springframework.batch.core.repository.dao.JobExecutionDao; import org.springframework.batch.core.repository.dao.JobInstanceDao; @@ -39,7 +40,7 @@ public class MapJobExplorerFactoryBean extends AbstractJobExplorerFactoryBean im /** * Create an instance with the provided {@link MapJobRepositoryFactoryBean} * as a source of Dao instances. - * @param repositoryFactory + * @param repositoryFactory provides the used {@link org.springframework.batch.core.repository.JobRepository} */ public MapJobExplorerFactoryBean(MapJobRepositoryFactoryBean repositoryFactory) { this.repositoryFactory = repositoryFactory; @@ -62,7 +63,8 @@ public void setRepositoryFactory(MapJobRepositoryFactoryBean repositoryFactory) } /** - * @throws Exception + * @throws Exception thrown if error occurs. + * * @see org.springframework.beans.factory.InitializingBean#afterPropertiesSet() */ @Override @@ -92,7 +94,7 @@ protected ExecutionContextDao createExecutionContextDao() throws Exception { } @Override - public Object getObject() throws Exception { + public JobExplorer getObject() throws Exception { return new SimpleJobExplorer(createJobInstanceDao(), createJobExecutionDao(), createStepExecutionDao(), createExecutionContextDao()); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/SimpleJobExplorer.java b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/SimpleJobExplorer.java index 6839d6953f..81a1d9c333 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/SimpleJobExplorer.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/SimpleJobExplorer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,23 +16,28 @@ package org.springframework.batch.core.explore.support; -import java.util.List; -import java.util.Set; - import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.launch.NoSuchJobException; import org.springframework.batch.core.repository.dao.ExecutionContextDao; import org.springframework.batch.core.repository.dao.JobExecutionDao; import org.springframework.batch.core.repository.dao.JobInstanceDao; import org.springframework.batch.core.repository.dao.StepExecutionDao; +import org.springframework.lang.Nullable; + +import java.util.List; +import java.util.Set; /** * Implementation of {@link JobExplorer} using the injected DAOs. * * @author Dave Syer * @author Lucas Ward + * @author Michael Minella + * @author Will Schipp + * @author Mahmoud Ben Hassine * * @see JobExplorer * @see JobInstanceDao @@ -85,6 +90,18 @@ public List getJobExecutions(JobInstance jobInstance) { return executions; } + /* + * (non-Javadoc) + * + * @see + * org.springframework.batch.core.explore.JobExplorer#getLastJobExecution( + * org.springframework.batch.core.JobInstance) + */ + @Nullable + public JobExecution getLastJobExecution(JobInstance jobInstance) { + return jobExecutionDao.getLastJobExecution(jobInstance); + } + /* * (non-Javadoc) * @@ -93,7 +110,7 @@ public List getJobExecutions(JobInstance jobInstance) { * (java.lang.String) */ @Override - public Set findRunningJobExecutions(String jobName) { + public Set findRunningJobExecutions(@Nullable String jobName) { Set executions = jobExecutionDao.findRunningJobExecutions(jobName); for (JobExecution jobExecution : executions) { getJobExecutionDependencies(jobExecution); @@ -111,8 +128,9 @@ public Set findRunningJobExecutions(String jobName) { * org.springframework.batch.core.explore.JobExplorer#getJobExecution(java * .lang.Long) */ + @Nullable @Override - public JobExecution getJobExecution(Long executionId) { + public JobExecution getJobExecution(@Nullable Long executionId) { if (executionId == null) { return null; } @@ -134,12 +152,14 @@ public JobExecution getJobExecution(Long executionId) { * org.springframework.batch.core.explore.JobExplorer#getStepExecution(java * .lang.Long) */ + @Nullable @Override - public StepExecution getStepExecution(Long jobExecutionId, Long executionId) { + public StepExecution getStepExecution(@Nullable Long jobExecutionId, @Nullable Long executionId) { JobExecution jobExecution = jobExecutionDao.getJobExecution(jobExecutionId); if (jobExecution == null) { return null; } + getJobExecutionDependencies(jobExecution); StepExecution stepExecution = stepExecutionDao.getStepExecution(jobExecution, executionId); getStepExecutionDependencies(stepExecution); return stepExecution; @@ -152,11 +172,25 @@ public StepExecution getStepExecution(Long jobExecutionId, Long executionId) { * org.springframework.batch.core.explore.JobExplorer#getJobInstance(java * .lang.Long) */ + @Nullable @Override - public JobInstance getJobInstance(Long instanceId) { + public JobInstance getJobInstance(@Nullable Long instanceId) { return jobInstanceDao.getJobInstance(instanceId); } + /* + * (non-Javadoc) + * + * @see + * org.springframework.batch.core.explore.JobExplorer#getLastJobInstance(java + * .lang.String) + */ + @Nullable + @Override + public JobInstance getLastJobInstance(String jobName) { + return jobInstanceDao.getLastJobInstance(jobName); + } + /* * (non-Javadoc) * @@ -179,12 +213,19 @@ public List getJobNames() { return jobInstanceDao.getJobNames(); } + /* (non-Javadoc) + * @see org.springframework.batch.core.explore.JobExplorer#getJobInstanceCount(java.lang.String) + */ + @Override + public int getJobInstanceCount(@Nullable String jobName) throws NoSuchJobException { + return jobInstanceDao.getJobInstanceCount(jobName); + } + /* * Find all dependencies for a JobExecution, including JobInstance (which * requires JobParameters) plus StepExecutions */ private void getJobExecutionDependencies(JobExecution jobExecution) { - JobInstance jobInstance = jobInstanceDao.getJobInstance(jobExecution); stepExecutionDao.addStepExecutions(jobExecution); jobExecution.setJobInstance(jobInstance); @@ -198,4 +239,8 @@ private void getStepExecutionDependencies(StepExecution stepExecution) { } } + @Override + public List findJobInstancesByJobName(String jobName, int start, int count) { + return jobInstanceDao.findJobInstancesByName(jobName, start, count); + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/package-info.java new file mode 100644 index 0000000000..6150d736cb --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/package-info.java @@ -0,0 +1,10 @@ +/** + * Specific implementations of explorer concerns. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.explore.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/package.html deleted file mode 100644 index 68494a733f..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/explore/support/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Specific implementations of explorer concerns. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/AbstractJob.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/AbstractJob.java index 0373bece0b..808f1c4509 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/AbstractJob.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/AbstractJob.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,6 +19,9 @@ import java.util.Collection; import java.util.Date; +import io.micrometer.core.instrument.LongTaskTimer; +import io.micrometer.core.instrument.Tag; +import io.micrometer.core.instrument.Timer; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.core.BatchStatus; @@ -36,12 +39,15 @@ import org.springframework.batch.core.launch.NoSuchJobException; import org.springframework.batch.core.launch.support.ExitCodeMapper; import org.springframework.batch.core.listener.CompositeJobExecutionListener; +import org.springframework.batch.core.metrics.BatchMetrics; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.batch.core.scope.context.JobSynchronizationManager; import org.springframework.batch.core.step.StepLocator; import org.springframework.batch.repeat.RepeatException; import org.springframework.beans.factory.BeanNameAware; import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.ClassUtils; @@ -53,6 +59,7 @@ * * @author Lucas Ward * @author Dave Syer + * @author Mahmoud Ben Hassine */ public abstract class AbstractJob implements Job, StepLocator, BeanNameAware, InitializingBean { @@ -84,7 +91,7 @@ public AbstractJob() { * Convenience constructor to immediately add name (which is mandatory but * not final). * - * @param name + * @param name name of the job */ public AbstractJob(String name) { super(); @@ -133,6 +140,8 @@ public void setBeanName(String name) { * Set the name property. Always overrides the default value if this object * is a Spring bean. * + * @param name the name to be associated with the job. + * * @see #setBeanName(java.lang.String) */ public void setName(String name) { @@ -153,7 +162,7 @@ public String getName() { * Retrieve the step with the given name. If there is no Step with the given * name, then return null. * - * @param stepName + * @param stepName name of the step * @return the Step */ @Override @@ -208,6 +217,7 @@ public void setJobParametersIncrementer( * @see org.springframework.batch.core.Job#getJobParametersIncrementer() */ @Override + @Nullable public JobParametersIncrementer getJobParametersIncrementer() { return this.jobParametersIncrementer; } @@ -241,7 +251,7 @@ public void registerJobExecutionListener(JobExecutionListener listener) { * state of the batch meta domain (jobs, steps, executions) during the life * of a job. * - * @param jobRepository + * @param jobRepository repository to use during the job execution */ public void setJobRepository(JobRepository jobRepository) { this.jobRepository = jobRepository; @@ -284,8 +294,16 @@ abstract protected void doExecute(JobExecution execution) @Override public final void execute(JobExecution execution) { - logger.debug("Job execution starting: " + execution); + Assert.notNull(execution, "jobExecution must not be null"); + + if (logger.isDebugEnabled()) { + logger.debug("Job execution starting: " + execution); + } + JobSynchronizationManager.register(execution); + LongTaskTimer longTaskTimer = BatchMetrics.createLongTaskTimer("job.active", "Active jobs"); + LongTaskTimer.Sample longTaskTimerSample = longTaskTimer.start(); + Timer.Sample timerSample = BatchMetrics.createTimerSample(); try { jobParametersValidator.validate(execution.getJobParameters()); @@ -299,7 +317,9 @@ public final void execute(JobExecution execution) { try { doExecute(execution); - logger.debug("Job execution complete: " + execution); + if (logger.isDebugEnabled()) { + logger.debug("Job execution complete: " + execution); + } } catch (RepeatException e) { throw e.getCause(); } @@ -309,7 +329,9 @@ public final void execute(JobExecution execution) { // with it in the same way as any other interruption. execution.setStatus(BatchStatus.STOPPED); execution.setExitStatus(ExitStatus.COMPLETED); - logger.debug("Job execution was stopped: " + execution); + if (logger.isDebugEnabled()) { + logger.debug("Job execution was stopped: " + execution); + } } @@ -319,34 +341,42 @@ public final void execute(JobExecution execution) { if (logger.isDebugEnabled()) { logger.debug("Full exception", e); } - execution.setExitStatus(getDefaultExitStatusForFailure(e)); + execution.setExitStatus(getDefaultExitStatusForFailure(e, execution)); execution.setStatus(BatchStatus.max(BatchStatus.STOPPED, e.getStatus())); execution.addFailureException(e); } catch (Throwable t) { logger.error("Encountered fatal error executing job", t); - execution.setExitStatus(getDefaultExitStatusForFailure(t)); + execution.setExitStatus(getDefaultExitStatusForFailure(t, execution)); execution.setStatus(BatchStatus.FAILED); execution.addFailureException(t); } finally { + try { + if (execution.getStatus().isLessThanOrEqualTo(BatchStatus.STOPPED) + && execution.getStepExecutions().isEmpty()) { + ExitStatus exitStatus = execution.getExitStatus(); + ExitStatus newExitStatus = + ExitStatus.NOOP.addExitDescription("All steps already completed or no steps configured for this job."); + execution.setExitStatus(exitStatus.and(newExitStatus)); + } - if (execution.getStatus().isLessThanOrEqualTo(BatchStatus.STOPPED) - && execution.getStepExecutions().isEmpty()) { - ExitStatus exitStatus = execution.getExitStatus(); - execution - .setExitStatus(exitStatus.and(ExitStatus.NOOP - .addExitDescription("All steps already completed or no steps configured for this job."))); - } + timerSample.stop(BatchMetrics.createTimer("job", "Job duration", + Tag.of("name", execution.getJobInstance().getJobName()), + Tag.of("status", execution.getExitStatus().getExitCode()) + )); + longTaskTimerSample.stop(); + execution.setEndTime(new Date()); - execution.setEndTime(new Date()); + try { + listener.afterJob(execution); + } catch (Exception e) { + logger.error("Exception encountered in afterJob callback", e); + } - try { - listener.afterJob(execution); - } catch (Exception e) { - logger.error("Exception encountered in afterStep callback", e); + jobRepository.update(execution); + } finally { + JobSynchronizationManager.release(); } - jobRepository.update(execution); - } } @@ -384,11 +414,11 @@ protected final StepExecution handleStep(Step step, JobExecution execution) /** * Default mapping from throwable to {@link ExitStatus}. * - * @param ex - * the cause of the failure + * @param ex the cause of the failure + * @param execution the {@link JobExecution} instance. * @return an {@link ExitStatus} */ - private ExitStatus getDefaultExitStatusForFailure(Throwable ex) { + protected ExitStatus getDefaultExitStatusForFailure(Throwable ex, JobExecution execution) { ExitStatus exitStatus; if (ex instanceof JobInterruptedException || ex.getCause() instanceof JobInterruptedException) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/CompositeJobParametersValidator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/CompositeJobParametersValidator.java index c8e449aec0..9c327cc39f 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/CompositeJobParametersValidator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/CompositeJobParametersValidator.java @@ -1,11 +1,11 @@ /* - * Copyright 2011-2013 the original author or authors. + * Copyright 2011-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,7 @@ import org.springframework.batch.core.JobParametersInvalidException; import org.springframework.batch.core.JobParametersValidator; import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -28,6 +29,7 @@ * injected JobParametersValidators * * @author Morten Andersen-Gott + * @author Mahmoud Ben Hassine * */ public class CompositeJobParametersValidator implements JobParametersValidator, InitializingBean { @@ -42,7 +44,7 @@ public class CompositeJobParametersValidator implements JobParametersValidator, * @throws JobParametersInvalidException if the parameters are invalid */ @Override - public void validate(JobParameters parameters) throws JobParametersInvalidException { + public void validate(@Nullable JobParameters parameters) throws JobParametersInvalidException { for (JobParametersValidator validator : validators) { validator.validate(parameters); } @@ -50,7 +52,7 @@ public void validate(JobParameters parameters) throws JobParametersInvalidExcept /** * Public setter for the validators - * @param validators + * @param validators list of validators to be used by the CompositeJobParametersValidator. */ public void setValidators(List validators) { this.validators = validators; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/DefaultJobParametersValidator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/DefaultJobParametersValidator.java index 2e81f4731c..23c2284296 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/DefaultJobParametersValidator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/DefaultJobParametersValidator.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,12 +24,14 @@ import org.springframework.batch.core.JobParametersInvalidException; import org.springframework.batch.core.JobParametersValidator; import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** * Default implementation of {@link JobParametersValidator}. * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public class DefaultJobParametersValidator implements JobParametersValidator, InitializingBean { @@ -68,7 +70,7 @@ public DefaultJobParametersValidator(String[] requiredKeys, String[] optionalKey @Override public void afterPropertiesSet() throws IllegalStateException { for (String key : requiredKeys) { - Assert.state(!optionalKeys.contains(key), "Optional keys canot be required: " + key); + Assert.state(!optionalKeys.contains(key), "Optional keys cannot be required: " + key); } } @@ -83,7 +85,7 @@ public void afterPropertiesSet() throws IllegalStateException { * @throws JobParametersInvalidException if the parameters are not valid */ @Override - public void validate(JobParameters parameters) throws JobParametersInvalidException { + public void validate(@Nullable JobParameters parameters) throws JobParametersInvalidException { if (parameters == null) { throw new JobParametersInvalidException("The JobParameters can not be null"); @@ -95,7 +97,7 @@ public void validate(JobParameters parameters) throws JobParametersInvalidExcept // group, or in the required group. if (!optionalKeys.isEmpty()) { - Collection missingKeys = new HashSet(); + Collection missingKeys = new HashSet<>(); for (String key : keys) { if (!optionalKeys.contains(key) && !requiredKeys.contains(key)) { missingKeys.add(key); @@ -108,7 +110,7 @@ public void validate(JobParameters parameters) throws JobParametersInvalidExcept } - Collection missingKeys = new HashSet(); + Collection missingKeys = new HashSet<>(); for (String key : requiredKeys) { if (!keys.contains(key)) { missingKeys.add(key); @@ -130,7 +132,7 @@ public void validate(JobParameters parameters) throws JobParametersInvalidExcept * @see #setOptionalKeys(String[]) */ public final void setRequiredKeys(String[] requiredKeys) { - this.requiredKeys = new HashSet(Arrays.asList(requiredKeys)); + this.requiredKeys = new HashSet<>(Arrays.asList(requiredKeys)); } /** @@ -144,7 +146,7 @@ public final void setRequiredKeys(String[] requiredKeys) { * @see #setRequiredKeys(String[]) */ public final void setOptionalKeys(String[] optionalKeys) { - this.optionalKeys = new HashSet(Arrays.asList(optionalKeys)); + this.optionalKeys = new HashSet<>(Arrays.asList(optionalKeys)); } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/SimpleJob.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/SimpleJob.java index 3ea13e82a1..1ac8cef781 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/SimpleJob.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/SimpleJob.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -28,6 +28,7 @@ import org.springframework.batch.core.Step; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.batch.core.step.StepLocator; /** * Simple implementation of {@link Job} interface providing the ability to run a @@ -37,10 +38,12 @@ * * @author Lucas Ward * @author Dave Syer + * @author Michael Minella + * @author Mahmoud Ben Hassine */ public class SimpleJob extends AbstractJob { - private List steps = new ArrayList(); + private List steps = new ArrayList<>(); /** * Default constructor for job with null name @@ -50,7 +53,7 @@ public SimpleJob() { } /** - * @param name + * @param name the job name. */ public SimpleJob(String name) { super(name); @@ -74,9 +77,13 @@ public void setSteps(List steps) { */ @Override public Collection getStepNames() { - List names = new ArrayList(); + List names = new ArrayList<>(); for (Step step : steps) { names.add(step.getName()); + + if(step instanceof StepLocator) { + names.addAll(((StepLocator)step).getStepNames()); + } } return names; } @@ -101,6 +108,11 @@ public Step getStep(String stepName) { for (Step step : this.steps) { if (step.getName().equals(stepName)) { return step; + } else if(step instanceof StepLocator) { + Step result = ((StepLocator)step).getStep(stepName); + if(result != null) { + return result; + } } } return null; @@ -134,7 +146,9 @@ protected void doExecute(JobExecution execution) throws JobInterruptedException, // Update the job status to be the same as the last step // if (stepExecution != null) { - logger.debug("Upgrading JobExecution status: " + stepExecution); + if (logger.isDebugEnabled()) { + logger.debug("Upgrading JobExecution status: " + stepExecution); + } execution.upgradeStatus(stepExecution.getStatus()); execution.setExitStatus(stepExecution.getExitStatus()); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/SimpleStepHandler.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/SimpleStepHandler.java index 21a494085e..bab83122a0 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/SimpleStepHandler.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/SimpleStepHandler.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -54,15 +54,15 @@ public SimpleStepHandler() { } /** - * @param jobRepository + * @param jobRepository a {@link org.springframework.batch.core.repository.JobRepository} */ public SimpleStepHandler(JobRepository jobRepository) { this(jobRepository, new ExecutionContext()); } /** - * @param jobRepository - * @param executionContext + * @param jobRepository a {@link org.springframework.batch.core.repository.JobRepository} + * @param executionContext the {@link org.springframework.batch.item.ExecutionContext} for the current Step */ public SimpleStepHandler(JobRepository jobRepository, ExecutionContext executionContext) { this.jobRepository = jobRepository; @@ -79,6 +79,13 @@ public void afterPropertiesSet() throws Exception { Assert.state(jobRepository != null, "A JobRepository must be provided"); } + /** + * @return the used jobRepository + */ + protected JobRepository getJobRepository() { + return this.jobRepository; + } + /** * @param jobRepository the jobRepository to set */ @@ -116,7 +123,7 @@ public StepExecution handleStep(Step step, JobExecution execution) throws JobInt } StepExecution currentStepExecution = lastStepExecution; - if (shouldStart(lastStepExecution, jobInstance, step)) { + if (shouldStart(lastStepExecution, execution, step)) { currentStepExecution = execution.createStepExecution(step.getName()); @@ -125,6 +132,10 @@ public StepExecution handleStep(Step step, JobExecution execution) throws JobInt if (isRestart) { currentStepExecution.setExecutionContext(lastStepExecution.getExecutionContext()); + + if(lastStepExecution.getExecutionContext().containsKey("batch.executed")) { + currentStepExecution.getExecutionContext().remove("batch.executed"); + } } else { currentStepExecution.setExecutionContext(new ExecutionContext(executionContext)); @@ -135,6 +146,7 @@ public StepExecution handleStep(Step step, JobExecution execution) throws JobInt logger.info("Executing step: [" + step.getName() + "]"); try { step.execute(currentStepExecution); + currentStepExecution.getExecutionContext().put("batch.executed", true); } catch (JobInterruptedException e) { // Ensure that the job gets the message that it is stopping @@ -154,9 +166,6 @@ public StepExecution handleStep(Step step, JobExecution execution) throws JobInt } } - else { - // currentStepExecution.setExitStatus(ExitStatus.NOOP); - } return currentStepExecution; } @@ -165,7 +174,7 @@ public StepExecution handleStep(Step step, JobExecution execution) throws JobInt * Detect whether a step execution belongs to this job execution. * @param jobExecution the current job execution * @param stepExecution an existing step execution - * @return + * @return true if the {@link org.springframework.batch.core.StepExecution} is part of the {@link org.springframework.batch.core.JobExecution} */ private boolean stepExecutionPartOfExistingJobExecution(JobExecution jobExecution, StepExecution stepExecution) { return stepExecution != null && stepExecution.getJobExecutionId() != null @@ -176,15 +185,16 @@ private boolean stepExecutionPartOfExistingJobExecution(JobExecution jobExecutio * Given a step and configuration, return true if the step should start, * false if it should not, and throw an exception if the job should finish. * @param lastStepExecution the last step execution - * @param jobInstance - * @param step + * @param jobExecution the {@link JobExecution} instance to be evaluated. + * @param step the {@link Step} instance to be evaluated. + * @return true if step should start, false if it should not. * * @throws StartLimitExceededException if the start limit has been exceeded * for this step * @throws JobRestartException if the job is in an inconsistent state from * an earlier failure */ - private boolean shouldStart(StepExecution lastStepExecution, JobInstance jobInstance, Step step) + protected boolean shouldStart(StepExecution lastStepExecution, JobExecution jobExecution, Step step) throws JobRestartException, StartLimitExceededException { BatchStatus stepStatus; @@ -201,7 +211,7 @@ private boolean shouldStart(StepExecution lastStepExecution, JobInstance jobInst + "so it may be dangerous to proceed. Manual intervention is probably necessary."); } - if ((stepStatus == BatchStatus.COMPLETED && step.isAllowStartIfComplete() == false) + if ((stepStatus == BatchStatus.COMPLETED && !step.isAllowStartIfComplete()) || stepStatus == BatchStatus.ABANDONED) { // step is complete, false should be returned, indicating that the // step should not be started @@ -209,7 +219,7 @@ private boolean shouldStart(StepExecution lastStepExecution, JobInstance jobInst return false; } - if (jobRepository.getStepExecutionCount(jobInstance, step.getName()) < step.getStartLimit()) { + if (jobRepository.getStepExecutionCount(jobExecution.getJobInstance(), step.getName()) < step.getStartLimit()) { // step start count is less than start max, return true return true; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/StepHandler.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/StepHandler.java index 10df108924..a34641c413 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/StepHandler.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/StepHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,7 +36,7 @@ public interface StepHandler { * Handle a step and return the execution for it. Does not save the * {@link JobExecution}, but should manage the persistence of the * {@link StepExecution} if required (e.g. at least it needs to be added to - * a repository before the step can eb executed). + * a repository before the step can be executed). * * @param step a {@link Step} * @param jobExecution a {@link JobExecution} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/FlowBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/FlowBuilder.java index f4e5e157f3..8a3a16ae3d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/FlowBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/FlowBuilder.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -56,9 +56,9 @@ public class FlowBuilder { private String prefix; - private List transitions = new ArrayList(); + private List transitions = new ArrayList<>(); - private Map tos = new HashMap(); + private Map tos = new HashMap<>(); private State currentState; @@ -74,7 +74,7 @@ public class FlowBuilder { private int endCounter = 0; - private Map states = new HashMap(); + private Map states = new HashMap<>(); private SimpleFlow flow; @@ -144,7 +144,7 @@ public FlowBuilder from(Step step) { */ public UnterminatedFlowBuilder next(JobExecutionDecider decider) { doNext(decider); - return new UnterminatedFlowBuilder(this); + return new UnterminatedFlowBuilder<>(this); } /** @@ -155,7 +155,7 @@ public UnterminatedFlowBuilder next(JobExecutionDecider decider) { */ public UnterminatedFlowBuilder start(JobExecutionDecider decider) { doStart(decider); - return new UnterminatedFlowBuilder(this); + return new UnterminatedFlowBuilder<>(this); } /** @@ -166,7 +166,7 @@ public UnterminatedFlowBuilder start(JobExecutionDecider decider) { */ public UnterminatedFlowBuilder from(JobExecutionDecider decider) { doFrom(decider); - return new UnterminatedFlowBuilder(this); + return new UnterminatedFlowBuilder<>(this); } /** @@ -207,7 +207,7 @@ public FlowBuilder start(Flow flow) { * @return a builder to enable fluent chaining */ public SplitBuilder split(TaskExecutor executor) { - return new SplitBuilder(this, executor); + return new SplitBuilder<>(this, executor); } /** @@ -219,7 +219,7 @@ public SplitBuilder split(TaskExecutor executor) { * @return a builder to enable fluent chaining */ public TransitionBuilder on(String pattern) { - return new TransitionBuilder(this, pattern); + return new TransitionBuilder<>(this, pattern); } /** @@ -316,14 +316,14 @@ private SplitState createState(Collection flows, TaskExecutor executor) { } private void addDanglingEndStates() { - Set froms = new HashSet(); + Set froms = new HashSet<>(); for (StateTransition transition : transitions) { froms.add(transition.getState().getName()); } if (tos.isEmpty() && currentState != null) { tos.put(currentState.getName(), currentState); } - Map copy = new HashMap(tos); + Map copy = new HashMap<>(tos); // Find all the states that are really end states but not explicitly declared as such for (String to : copy.keySet()) { if (!froms.contains(to)) { @@ -334,7 +334,7 @@ private void addDanglingEndStates() { } } } - copy = new HashMap(tos); + copy = new HashMap<>(tos); // Then find the states that do not have a default transition for (String from : copy.keySet()) { currentState = copy.get(from); @@ -380,11 +380,11 @@ private void addTransition(String pattern, State next) { dirty = true; } - private void stop(String pattern) { + protected void stop(String pattern) { addTransition(pattern, stoppedState); } - private void stop(String pattern, State restart) { + protected void stop(String pattern, State restart) { EndState next = new EndState(FlowExecutionStatus.STOPPED, "STOPPED", prefix + "stop" + (endCounter++), true); addTransition(pattern, next); currentState = next; @@ -427,7 +427,7 @@ public UnterminatedFlowBuilder(FlowBuilder parent) { * @return a TransitionBuilder */ public TransitionBuilder on(String pattern) { - return new TransitionBuilder(parent, pattern); + return new TransitionBuilder<>(parent, pattern); } } @@ -548,6 +548,7 @@ public FlowBuilder end() { /** * Signal the end of the flow with the status provided. * + * @param status {@link String} containing the status. * @return a FlowBuilder */ public FlowBuilder end(String status) { @@ -582,7 +583,34 @@ public FlowBuilder fail() { * * In this example, a flow consisting of step1 will be executed in parallel with flow. * + * Note: Adding a split to a chain of states is not supported. For example, the following configuration + * is not supported. Instead, the configuration would need to create a flow3 that was the split flow and assemble + * them separately. + * + *
      +	 * // instead of this
      +	 * Flow complexFlow = new FlowBuilder<SimpleFlow>("ComplexParallelFlow")
      +	 *                       .start(flow1)
      +	 *                       .next(flow2)
      +	 *                       .split(new SimpleAsyncTaskExecutor())
      +	 *                       .add(flow3, flow4)
      +	 *                       .build();
      +	 *
      +	 * // do this
      +	 * Flow splitFlow = new FlowBuilder<SimpleFlow>("parallelFlow")
      +	 *                       .start(flow3)
      +	 *                       .split(new SimpleAsyncTaskExecutor())
      +	 *                       .add(flow4).build();
      +	 *
      +	 * Flow complexFlow = new FlowBuilder<SimpleFlow>("ComplexParallelFlow")
      +	 *                       .start(flow1)
      +	 *                       .next(flow2)
      +	 *                       .next(splitFlow)
      +	 *                       .build();
      +	 * 
      + * * @author Dave Syer + * @author Michael Minella * * @param the result of the parent builder's build() */ @@ -608,16 +636,19 @@ public SplitBuilder(FlowBuilder parent, TaskExecutor executor) { * @return the parent builder */ public FlowBuilder add(Flow... flows) { - Collection list = new ArrayList(Arrays.asList(flows)); + Collection list = new ArrayList<>(Arrays.asList(flows)); String name = "split" + (parent.splitCounter++); int counter = 0; State one = parent.currentState; Flow flow = null; - if (!(one instanceof FlowState)) { - FlowBuilder stateBuilder = new FlowBuilder(name + "_" + (counter++)); + if (!(one == null || one instanceof FlowState)) { + FlowBuilder stateBuilder = new FlowBuilder<>(name + "_" + (counter++)); stateBuilder.currentState = one; flow = stateBuilder.build(); + } else if (one instanceof FlowState && parent.states.size() == 1) { + list.add(((FlowState) one).getFlows().iterator().next()); } + if (flow != null) { list.add(flow); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/FlowBuilderException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/FlowBuilderException.java index 7a065b5ffc..11c00a7515 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/FlowBuilderException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/FlowBuilderException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,7 @@ * @since 2.2 * */ +@SuppressWarnings("serial") public class FlowBuilderException extends RuntimeException { public FlowBuilderException(String msg, Exception e) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/FlowJobBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/FlowJobBuilder.java index 3ada9d7489..ec26ec7606 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/FlowJobBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/FlowJobBuilder.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobBuilder.java index dfdf37256e..eb412148fd 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobBuilder.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobBuilderException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobBuilderException.java index c59ff9757c..1e27421932 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobBuilderException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobBuilderException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,7 @@ * @since 2.2 * */ +@SuppressWarnings("serial") public class JobBuilderException extends RuntimeException { public JobBuilderException(Exception e) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobBuilderHelper.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobBuilderHelper.java index 97d5e24eaa..4c98cdbf17 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobBuilderHelper.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobBuilderHelper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -164,7 +164,7 @@ protected void enhance(Job target) { public static class CommonJobProperties { - private Set jobExecutionListeners = new LinkedHashSet(); + private Set jobExecutionListeners = new LinkedHashSet<>(); private boolean restartable = true; @@ -181,7 +181,7 @@ public CommonJobProperties(CommonJobProperties properties) { this.name = properties.name; this.restartable = properties.restartable; this.jobRepository = properties.jobRepository; - this.jobExecutionListeners = new LinkedHashSet(properties.jobExecutionListeners); + this.jobExecutionListeners = new LinkedHashSet<>(properties.jobExecutionListeners); this.jobParametersIncrementer = properties.jobParametersIncrementer; this.jobParametersValidator = properties.jobParametersValidator; } @@ -219,7 +219,7 @@ public void setName(String name) { } public List getJobExecutionListeners() { - return new ArrayList(jobExecutionListeners); + return new ArrayList<>(jobExecutionListeners); } public void addStepExecutionListeners(List jobExecutionListeners) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobFlowBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobFlowBuilder.java index 091ebece1d..e95db4fcc3 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobFlowBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/JobFlowBuilder.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,6 +18,7 @@ import org.springframework.batch.core.Step; import org.springframework.batch.core.job.flow.Flow; import org.springframework.batch.core.job.flow.JobExecutionDecider; +import org.springframework.beans.factory.InitializingBean; /** * @author Dave Syer @@ -59,6 +60,16 @@ public JobFlowBuilder(FlowJobBuilder parent, Flow flow) { @Override public FlowJobBuilder build() { Flow flow = flow(); + + if(flow instanceof InitializingBean) { + try { + ((InitializingBean) flow).afterPropertiesSet(); + } + catch (Exception e) { + throw new FlowBuilderException(e); + } + } + parent.flow(flow); return parent; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/SimpleJobBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/SimpleJobBuilder.java index 92b499f607..9a827f8be2 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/SimpleJobBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/SimpleJobBuilder.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -33,7 +33,7 @@ */ public class SimpleJobBuilder extends JobBuilderHelper { - private List steps = new ArrayList(); + private List steps = new ArrayList<>(); private JobFlowBuilder builder; @@ -157,7 +157,7 @@ public SimpleJobBuilder next(Step step) { } /** - * @param executor + * @param executor instance of {@link TaskExecutor} to be used. * @return builder for fluent chaining */ public JobFlowBuilder.SplitBuilder split(TaskExecutor executor) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/package-info.java new file mode 100644 index 0000000000..8ebd881b89 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/builder/package-info.java @@ -0,0 +1,10 @@ +/** + * Job and flow level builders for java based configuration of batch jobs + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.job.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/Flow.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/Flow.java index 18a9eccd39..9688ae0120 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/Flow.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/Flow.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,21 +32,25 @@ public interface Flow { * Retrieve the State with the given name. If there is no State with the * given name, then return null. * - * @param stateName + * @param stateName the name of the state to retrieve * @return the State */ State getState(String stateName); /** - * @throws FlowExecutionException + * @param executor the {@link FlowExecutor} instance to use for the flow execution. + * @return a {@link FlowExecution} containing the exit status of the flow. + * + * @throws FlowExecutionException thrown if error occurs during flow execution. */ FlowExecution start(FlowExecutor executor) throws FlowExecutionException; /** - * @param stateName the name of the state to resume on - * @param executor the context to be passed into each state executed - * @return a {@link FlowExecution} containing the exit status of the flow - * @throws FlowExecutionException + * @param stateName the name of the state to resume on. + * @param executor the context to be passed into each state executed. + * @return a {@link FlowExecution} containing the exit status of the flow. + * + * @throws FlowExecutionException thrown if error occurs during flow execution. */ FlowExecution resume(String stateName, FlowExecutor executor) throws FlowExecutionException; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecution.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecution.java index c20c356125..af6ae4d1b4 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecution.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecution.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,8 +26,8 @@ public class FlowExecution implements Comparable { private final FlowExecutionStatus status; /** - * @param name - * @param status + * @param name the flow name to be associated with the FlowExecution. + * @param status the {@link FlowExecutionStatus} to be associated with the FlowExecution. */ public FlowExecution(String name, FlowExecutionStatus status) { this.name = name; @@ -54,7 +54,7 @@ public FlowExecutionStatus getStatus() { * * @see Comparable#compareTo(Object) * - * @param other + * @param other the {@link FlowExecution} instance to compare with this instance. * @return negative, zero or positive as per the contract */ @Override diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecutionException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecutionException.java index 896fcb01ed..6edbf43e7f 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecutionException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecutionException.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2013 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,18 +19,19 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class FlowExecutionException extends Exception { /** - * @param message + * @param message the error message. */ public FlowExecutionException(String message) { super(message); } /** - * @param message - * @param cause + * @param message the error message. + * @param cause instance of {@link Throwable} that caused this exception. */ public FlowExecutionException(String message, Throwable cause) { super(message, cause); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecutionStatus.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecutionStatus.java index 24416fe93e..b16d4e43e2 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecutionStatus.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecutionStatus.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -61,10 +61,10 @@ static Status match(String value) { return COMPLETED; } - }; + } /** - * @param status + * @param status String status value. */ public FlowExecutionStatus(String status) { this.name = status; @@ -104,7 +104,7 @@ private boolean isComplete() { * * @see Comparable#compareTo(Object) * - * @param other + * @param other instance of {@link FlowExecutionStatus} to compare this instance with. * @return negative, zero or positive as per the contract */ @Override diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecutor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecutor.java index a1d2716b75..db11767247 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecutor.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowExecutor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,12 +21,14 @@ import org.springframework.batch.core.Step; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.lang.Nullable; /** * Context and execution strategy for {@link FlowJob} to allow it to delegate * its execution step by step. * * @author Dave Syer + * @author Mahmoud Ben Hassine * @since 2.0 */ public interface FlowExecutor { @@ -34,9 +36,9 @@ public interface FlowExecutor { /** * @param step a {@link Step} to execute * @return the exit status that drives the surrounding {@link Flow} - * @throws StartLimitExceededException - * @throws JobRestartException - * @throws JobInterruptedException + * @throws StartLimitExceededException thrown if start limit is exceeded. + * @throws JobRestartException thrown if job restart is not allowed. + * @throws JobInterruptedException thrown if job was interrupted. */ String executeStep(Step step) throws JobInterruptedException, JobRestartException, StartLimitExceededException; @@ -48,6 +50,7 @@ public interface FlowExecutor { /** * @return the latest {@link StepExecution} or null if there is none */ + @Nullable StepExecution getStepExecution(); /** @@ -66,6 +69,8 @@ public interface FlowExecutor { /** * Handle any status changes that might be needed in the * {@link JobExecution}. + * + * @param status status to update the {@link JobExecution} to. */ void updateJobExecutionStatus(FlowExecutionStatus status); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowHolder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowHolder.java index 3c35558203..8675dc0dee 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowHolder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowHolder.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowJob.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowJob.java index 596512b4af..6a762b3038 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowJob.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowJob.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -33,15 +33,16 @@ * steps, rather than requiring sequential execution. In general, this job * implementation was designed to be used behind a parser, allowing for a * namespace to abstract away details. - * + * * @author Dave Syer + * @author Mahmoud Ben Hassine * @since 2.0 */ public class FlowJob extends AbstractJob { - private Flow flow; + protected Flow flow; - private Map stepMap = new ConcurrentHashMap(); + private Map stepMap = new ConcurrentHashMap<>(); private volatile boolean initialized = false; @@ -54,6 +55,8 @@ public FlowJob() { /** * Create a {@link FlowJob} with provided name and no flow (invalid state). + * + * @param name the name to be associated with the FlowJob. */ public FlowJob(String name) { super(name); @@ -61,7 +64,7 @@ public FlowJob(String name) { /** * Public setter for the flow. - * + * * @param flow the flow to set */ public void setFlow(Flow flow) { @@ -93,7 +96,12 @@ private void init() { private void findSteps(Flow flow, Map map) { for (State state : flow.getStates()) { - if (state instanceof StepHolder) { + if (state instanceof StepLocator) { + StepLocator locator = (StepLocator) state; + for (String name : locator.getStepNames()) { + map.put(name, locator.getStep(name)); + } + } else if (state instanceof StepHolder) { Step step = ((StepHolder) state).getStep(); String name = step.getName(); stepMap.put(name, step); @@ -103,12 +111,6 @@ else if (state instanceof FlowHolder) { findSteps(subflow, map); } } - else if (state instanceof StepLocator) { - StepLocator locator = (StepLocator) state; - for (String name : locator.getStepNames()) { - map.put(name, locator.getStep(name)); - } - } } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowStep.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowStep.java index 735ff41493..93953d0a97 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowStep.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/FlowStep.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.job.flow; import org.springframework.batch.core.JobExecutionException; @@ -34,6 +49,8 @@ public FlowStep() { /** * Constructor for a {@link FlowStep} that sets the flow and of the step * explicitly. + * + * @param flow the {@link Flow} instance to be associated with this step. */ public FlowStep(Flow flow) { super(flow.getName()); @@ -69,6 +86,7 @@ public void afterPropertiesSet() throws Exception { @Override protected void doExecute(StepExecution stepExecution) throws Exception { try { + stepExecution.getExecutionContext().put(STEP_TYPE_KEY, this.getClass().getName()); StepHandler stepHandler = new SimpleStepHandler(getJobRepository(), stepExecution.getExecutionContext()); FlowExecutor executor = new JobFlowExecutor(getJobRepository(), stepHandler, stepExecution.getJobExecution()); executor.updateJobExecutionStatus(flow.start(executor).getStatus()); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/JobExecutionDecider.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/JobExecutionDecider.java index e61e97d55e..64cc0dea1c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/JobExecutionDecider.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/JobExecutionDecider.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,6 +17,7 @@ import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.StepExecution; +import org.springframework.lang.Nullable; /** * Interface allowing for programmatic access to the decision on what the status @@ -25,6 +26,7 @@ * implementation could check that value to determine the status of the flow. * * @author Dave Syer + * @author Mahmoud Ben Hassine * @since 2.0 */ public interface JobExecutionDecider { @@ -35,9 +37,9 @@ public interface JobExecutionDecider { * determine the next step in the job. * * @param jobExecution a job execution - * @param stepExecution the latest step execution (may be null) + * @param stepExecution the latest step execution (may be {@code null}) * @return the exit status code */ - FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution); + FlowExecutionStatus decide(JobExecution jobExecution, @Nullable StepExecution stepExecution); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/JobFlowExecutor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/JobFlowExecutor.java index 4f2938ddde..6a3003cbd0 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/JobFlowExecutor.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/JobFlowExecutor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,6 +26,7 @@ import org.springframework.batch.core.job.StepHandler; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.lang.Nullable; /** * Implementation of {@link FlowExecutor} for use in components that need to @@ -33,22 +34,25 @@ * * @author Dave Syer * @author Michael Minella + * @author Mahmoud Ben Hassine * */ public class JobFlowExecutor implements FlowExecutor { - private final ThreadLocal stepExecutionHolder = new ThreadLocal(); + private final ThreadLocal stepExecutionHolder = new ThreadLocal<>(); private final JobExecution execution; - private ExitStatus exitStatus = ExitStatus.EXECUTING; + protected ExitStatus exitStatus = ExitStatus.EXECUTING; private final StepHandler stepHandler; private final JobRepository jobRepository; /** - * @param execution + * @param jobRepository instance of {@link JobRepository}. + * @param stepHandler instance of {@link StepHandler}. + * @param execution instance of {@link JobExecution}. */ public JobFlowExecutor(JobRepository jobRepository, StepHandler stepHandler, JobExecution execution) { this.jobRepository = jobRepository; @@ -106,6 +110,7 @@ public JobExecution getJobExecution() { } @Override + @Nullable public StepExecution getStepExecution() { return stepExecutionHolder.get(); } @@ -134,10 +139,10 @@ public void addExitStatus(String code) { } /** - * @param status - * @return + * @param status {@link FlowExecutionStatus} to convert. + * @return A {@link BatchStatus} appropriate for the {@link FlowExecutionStatus} provided */ - private BatchStatus findBatchStatus(FlowExecutionStatus status) { + protected BatchStatus findBatchStatus(FlowExecutionStatus status) { for (BatchStatus batchStatus : BatchStatus.values()) { if (status.getName().startsWith(batchStatus.toString())) { return batchStatus; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/State.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/State.java index c9ba62ca67..659e47f21f 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/State.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/State.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,7 +35,7 @@ public interface State { * {@link FlowExecution}. The context can be used by implementations to do * whatever they need to do. The same context will be passed to all * {@link State} instances, so implementations should be careful that the - * context is thread safe, or used in a thread safe manner. + * context is thread-safe, or used in a thread-safe manner. * * @param executor the context passed in by the caller * @return a status for the execution diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/package-info.java new file mode 100644 index 0000000000..f64efbb060 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/package-info.java @@ -0,0 +1,10 @@ +/** + * Flow related constructs including Flow interface, executors, and related exceptions + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.job.flow; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/DefaultStateTransitionComparator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/DefaultStateTransitionComparator.java new file mode 100644 index 0000000000..2b36e74c15 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/DefaultStateTransitionComparator.java @@ -0,0 +1,62 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.job.flow.support; + +import org.springframework.util.StringUtils; + +import java.util.Comparator; + +/** + * Sorts by decreasing specificity of pattern, based on just counting + * wildcards (with * taking precedence over ?). If wildcard counts are equal + * then falls back to alphabetic comparison. Hence * > foo* > ??? > + * fo? > foo. + * + * @see Comparator + * @author Michael Minella + * @since 3.0 + */ +public class DefaultStateTransitionComparator implements Comparator { + public static final String STATE_TRANSITION_COMPARATOR = "batch_state_transition_comparator"; + + /* (non-Javadoc) + * @see java.util.Comparator#compare(java.lang.Object, java.lang.Object) + */ + @Override + public int compare(StateTransition arg0, StateTransition arg1) { + String value = arg1.getPattern(); + if (arg0.getPattern().equals(value)) { + return 0; + } + int patternCount = StringUtils.countOccurrencesOf(arg0.getPattern(), "*"); + int valueCount = StringUtils.countOccurrencesOf(value, "*"); + if (patternCount > valueCount) { + return 1; + } + if (patternCount < valueCount) { + return -1; + } + patternCount = StringUtils.countOccurrencesOf(arg0.getPattern(), "?"); + valueCount = StringUtils.countOccurrencesOf(value, "?"); + if (patternCount > valueCount) { + return 1; + } + if (patternCount < valueCount) { + return -1; + } + return arg0.getPattern().compareTo(value); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/SimpleFlow.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/SimpleFlow.java index e59a8cea1b..d14bbb61c3 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/SimpleFlow.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/SimpleFlow.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,17 +17,18 @@ import java.util.ArrayList; import java.util.Collection; +import java.util.Comparator; import java.util.HashMap; import java.util.HashSet; +import java.util.LinkedHashSet; import java.util.List; import java.util.Map; import java.util.Set; -import java.util.SortedSet; import java.util.TreeSet; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; -import org.springframework.batch.core.JobExecutionException; + import org.springframework.batch.core.Step; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.job.flow.Flow; @@ -55,14 +56,20 @@ public class SimpleFlow implements Flow, InitializingBean { private State startState; - private Map> transitionMap = new HashMap>(); + private Map> transitionMap = new HashMap<>(); - private Map stateMap = new HashMap(); + private Map stateMap = new HashMap<>(); - private List stateTransitions = new ArrayList(); + private List stateTransitions = new ArrayList<>(); private final String name; + private Comparator stateTransitionComparator; + + public void setStateTransitionComparator(Comparator stateTransitionComparator) { + this.stateTransitionComparator = stateTransitionComparator; + } + /** * Create a flow with the given name. * @@ -72,6 +79,10 @@ public SimpleFlow(String name) { this.name = name; } + public State getStartState() { + return this.startState; + } + /** * Get the name for this flow. * @@ -105,7 +116,7 @@ public State getState(String stateName) { */ @Override public Collection getStates() { - return new HashSet(stateMap.values()); + return new HashSet<>(stateMap.values()); } /** @@ -115,7 +126,9 @@ public Collection getStates() { */ @Override public void afterPropertiesSet() throws Exception { - initializeTransitions(); + if (startState == null) { + initializeTransitions(); + } } /** @@ -140,7 +153,9 @@ public FlowExecution resume(String stateName, FlowExecutor executor) throws Flow FlowExecutionStatus status = FlowExecutionStatus.UNKNOWN; State state = stateMap.get(stateName); - logger.debug("Resuming state="+stateName+" with status="+status); + if (logger.isDebugEnabled()) { + logger.debug("Resuming state="+stateName+" with status="+status); + } StepExecution stepExecution = null; // Terminate if there are no more states @@ -148,7 +163,9 @@ public FlowExecution resume(String stateName, FlowExecutor executor) throws Flow stateName = state.getName(); try { - logger.debug("Handling state="+stateName); + if (logger.isDebugEnabled()) { + logger.debug("Handling state="+stateName); + } status = state.handle(executor); stepExecution = executor.getStepExecution(); } @@ -159,12 +176,14 @@ public FlowExecution resume(String stateName, FlowExecutor executor) throws Flow catch (Exception e) { executor.close(new FlowExecution(stateName, status)); throw new FlowExecutionException(String.format("Ended flow=%s at state=%s with exception", name, - stateName), e); + stateName), e); } - logger.debug("Completed state="+stateName+" with status="+status); + if (logger.isDebugEnabled()) { + logger.debug("Completed state="+stateName+" with status="+status); + } - state = nextState(stateName, status); + state = nextState(stateName, status, stepExecution); } FlowExecution result = new FlowExecution(stateName, status); @@ -173,37 +192,32 @@ public FlowExecution resume(String stateName, FlowExecutor executor) throws Flow } - private boolean isFlowContinued(State state, FlowExecutionStatus status, StepExecution stepExecution) { - boolean continued = true; - - continued = state != null && status!=FlowExecutionStatus.STOPPED; - - if(stepExecution != null) { - Boolean reRun = (Boolean) stepExecution.getExecutionContext().get("batch.restart"); - - if(reRun != null && reRun && status == FlowExecutionStatus.STOPPED && !state.getName().endsWith(stepExecution.getStepName())) { - continued = true; - } - } + protected Map> getTransitionMap() { + return transitionMap; + } - return continued; + protected Map getStateMap() { + return stateMap; } /** + * @param stateName the name of the next state. + * @param status {@link FlowExecutionStatus} instance. + * @param stepExecution {@link StepExecution} instance. * @return the next {@link Step} (or null if this is the end) - * @throws JobExecutionException + * @throws FlowExecutionException thrown if error occurs during nextState processing. */ - private State nextState(String stateName, FlowExecutionStatus status) throws FlowExecutionException { - + protected State nextState(String stateName, FlowExecutionStatus status, StepExecution stepExecution) throws FlowExecutionException { Set set = transitionMap.get(stateName); if (set == null) { throw new FlowExecutionException(String.format("No transitions found in flow=%s for state=%s", getName(), - stateName)); + stateName)); } String next = null; String exitCode = status.getName(); + for (StateTransition stateTransition : set) { if (stateTransition.matches(exitCode) || (exitCode.equals("PENDING") && stateTransition.matches("STOPPED"))) { if (stateTransition.isEnd()) { @@ -216,19 +230,37 @@ private State nextState(String stateName, FlowExecutionStatus status) throws Flo } if (next == null) { - throw new FlowExecutionException(String.format( - "Next state not found in flow=%s for state=%s with exit status=%s", getName(), stateName, status.getName())); + throw new FlowExecutionException(String.format("Next state not found in flow=%s for state=%s with exit status=%s", getName(), stateName, status.getName())); } if (!stateMap.containsKey(next)) { throw new FlowExecutionException(String.format("Next state not specified in flow=%s for next=%s", - getName(), next)); + getName(), next)); } - State state = stateMap.get(next); + return stateMap.get(next); - return state; + } + protected boolean isFlowContinued(State state, FlowExecutionStatus status, StepExecution stepExecution) { + boolean continued = true; + + continued = state != null && status!=FlowExecutionStatus.STOPPED; + + if(stepExecution != null) { + Boolean reRun = (Boolean) stepExecution.getExecutionContext().get("batch.restart"); + Boolean executed = (Boolean) stepExecution.getExecutionContext().get("batch.executed"); + + if((executed == null || !executed) && reRun != null && reRun && status == FlowExecutionStatus.STOPPED && !state.getName().endsWith(stepExecution.getStepName()) ) { + continued = true; + } + } + + return continued; + } + + private boolean stateNameEndsWithStepName(State state, StepExecution stepExecution) { + return !(stepExecution == null || state == null) && !state.getName().endsWith(stepExecution.getStepName()); } /** @@ -270,9 +302,15 @@ private void initializeTransitions() { String name = state.getName(); - SortedSet set = transitionMap.get(name); + Set set = transitionMap.get(name); if (set == null) { - set = new TreeSet(); + // If no comparator is provided, we will maintain the order of insertion + if(stateTransitionComparator == null) { + set = new LinkedHashSet<>(); + } else { + set = new TreeSet<>(stateTransitionComparator); + } + transitionMap.put(name, set); } set.add(stateTransition); @@ -281,7 +319,7 @@ private void initializeTransitions() { if (!hasEndStep) { throw new IllegalArgumentException( - "No end state was found. You must specify at least one transition with no next state."); + "No end state was found. You must specify at least one transition with no next state."); } startState = stateTransitions.get(0).getState(); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/StateTransition.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/StateTransition.java index 4afafcead6..7f9709e4ee 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/StateTransition.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/StateTransition.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,6 +18,7 @@ import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.job.flow.State; import org.springframework.batch.support.PatternMatcher; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.StringUtils; @@ -28,9 +29,11 @@ * execution of the originating State. * * @author Dave Syer + * @author Michael Minella + * @author Mahmoud Ben Hassine * @since 2.0 */ -public final class StateTransition implements Comparable { +public final class StateTransition { private final State state; @@ -38,6 +41,13 @@ public final class StateTransition implements Comparable { private final String next; + /** + * @return the pattern the {@link ExitStatus#getExitCode()} will be compared against. + */ + public String getPattern() { + return this.pattern; + } + /** * Create a new end state {@link StateTransition} specification. This * transition explicitly goes unconditionally to an end state (i.e. no more @@ -45,6 +55,7 @@ public final class StateTransition implements Comparable { * * @param state the {@link State} used to generate the outcome for this * transition + * @return {@link StateTransition} that was created. */ public static StateTransition createEndStateTransition(State state) { return createStateTransition(state, null, null); @@ -59,6 +70,7 @@ public static StateTransition createEndStateTransition(State state) { * transition * @param pattern the pattern to match in the exit status of the * {@link State} + * @return {@link StateTransition} that was created. */ public static StateTransition createEndStateTransition(State state, String pattern) { return createStateTransition(state, pattern, null); @@ -72,7 +84,7 @@ public static StateTransition createEndStateTransition(State state, String patte * @param state the new state for the origin * @param next the new name for the destination * - * @return a {@link StateTransition} + * @return {@link StateTransition} that was created. */ public static StateTransition switchOriginAndDestination(StateTransition stateTransition, State state, String next) { return createStateTransition(state, stateTransition.pattern, next); @@ -85,6 +97,7 @@ public static StateTransition switchOriginAndDestination(StateTransition stateTr * @param state the {@link State} used to generate the outcome for this * transition * @param next the name of the next {@link State} to execute + * @return {@link StateTransition} that was created. */ public static StateTransition createStateTransition(State state, String next) { return createStateTransition(state, null, next); @@ -97,14 +110,15 @@ public static StateTransition createStateTransition(State state, String next) { * @param state the {@link State} used to generate the outcome for this * transition * @param pattern the pattern to match in the exit status of the - * {@link State} - * @param next the name of the next {@link State} to execute + * {@link State} (can be {@code null}) + * @param next the name of the next {@link State} to execute (can be {@code null}) + * @return {@link StateTransition} that was created. */ - public static StateTransition createStateTransition(State state, String pattern, String next) { + public static StateTransition createStateTransition(State state, @Nullable String pattern, @Nullable String next) { return new StateTransition(state, pattern, next); } - private StateTransition(State state, String pattern, String next) { + private StateTransition(State state, @Nullable String pattern, @Nullable String next) { super(); if (!StringUtils.hasText(pattern)) { this.pattern = "*"; @@ -158,38 +172,6 @@ public boolean isEnd() { return next == null; } - /** - * Sorts by decreasing specificity of pattern, based on just counting - * wildcards (with * taking precedence over ?). If wildcard counts are equal - * then falls back to alphabetic comparison. Hence * > foo* > ??? > - * fo? > foo. - * @see Comparable#compareTo(Object) - */ - @Override - public int compareTo(StateTransition other) { - String value = other.pattern; - if (pattern.equals(value)) { - return 0; - } - int patternCount = StringUtils.countOccurrencesOf(pattern, "*"); - int valueCount = StringUtils.countOccurrencesOf(value, "*"); - if (patternCount > valueCount) { - return 1; - } - if (patternCount < valueCount) { - return -1; - } - patternCount = StringUtils.countOccurrencesOf(pattern, "?"); - valueCount = StringUtils.countOccurrencesOf(value, "?"); - if (patternCount > valueCount) { - return 1; - } - if (patternCount < valueCount) { - return -1; - } - return pattern.compareTo(value); - } - /* * (non-Javadoc) * diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/package-info.java new file mode 100644 index 0000000000..3764d3be60 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/package-info.java @@ -0,0 +1,10 @@ +/** + * Basic implementations of flow constructs + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.job.flow.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/AbstractState.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/AbstractState.java index ee531dfa73..b7301ef4cb 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/AbstractState.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/AbstractState.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,7 +29,7 @@ public abstract class AbstractState implements State { private final String name; /** - * + * @param name of the state. */ public AbstractState(String name) { this.name = name; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/DecisionState.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/DecisionState.java index 704ebba5be..63f9b594c2 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/DecisionState.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/DecisionState.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,7 +31,8 @@ public class DecisionState extends AbstractState { private final JobExecutionDecider decider; /** - * @param name + * @param decider the {@link JobExecutionDecider} instance to make the status decision. + * @param name the name of the decision state. */ public DecisionState(JobExecutionDecider decider, String name) { super(name); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/EndState.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/EndState.java index c8782ff311..04c450eaa4 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/EndState.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/EndState.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -48,6 +48,7 @@ public EndState(FlowExecutionStatus status, String name) { /** * @param status The {@link FlowExecutionStatus} to end with * @param name The name of the state + * @param code The exit status to save */ public EndState(FlowExecutionStatus status, String code, String name) { this(status, code, name, false); @@ -56,6 +57,7 @@ public EndState(FlowExecutionStatus status, String code, String name) { /** * @param status The {@link FlowExecutionStatus} to end with * @param name The name of the state + * @param code The exit status to save * @param abandon flag to indicate that previous step execution can be * marked as abandoned (if there is one) * @@ -67,6 +69,18 @@ public EndState(FlowExecutionStatus status, String code, String name, boolean ab this.abandon = abandon; } + protected FlowExecutionStatus getStatus() { + return this.status; + } + + protected boolean isAbandon() { + return this.abandon; + } + + protected String getCode() { + return this.code; + } + /** * Return the {@link FlowExecutionStatus} stored. * @@ -108,12 +122,23 @@ public FlowExecutionStatus handle(FlowExecutor executor) throws Exception { } } - executor.addExitStatus(code); + setExitStatus(executor, code); + return status; } } + /** + * Performs any logic to update the exit status for the current flow. + * + * @param executor {@link FlowExecutor} for the current flow + * @param code The exit status to save + */ + protected void setExitStatus(FlowExecutor executor, String code) { + executor.addExitStatus(code); + } + /* * (non-Javadoc) * diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/FlowExecutionAggregator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/FlowExecutionAggregator.java index aae45a25ba..eb28126d7e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/FlowExecutionAggregator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/FlowExecutionAggregator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/FlowState.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/FlowState.java index 4c75971e95..617a975656 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/FlowState.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/FlowState.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,7 +35,8 @@ public class FlowState extends AbstractState implements FlowHolder { private final Flow flow; /** - * @param name + * @param flow the {@link Flow} to delegate to. + * @param name the name of the state. */ public FlowState(Flow flow, String name) { super(name); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/MaxValueFlowExecutionAggregator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/MaxValueFlowExecutionAggregator.java index b66b3e275f..2382af0dda 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/MaxValueFlowExecutionAggregator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/MaxValueFlowExecutionAggregator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/SplitState.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/SplitState.java index a5e7ed3ae6..0e945bce67 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/SplitState.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/SplitState.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -49,7 +49,8 @@ public class SplitState extends AbstractState implements FlowHolder { private FlowExecutionAggregator aggregator = new MaxValueFlowExecutionAggregator(); /** - * @param name + * @param flows collection of {@link Flow} instances. + * @param name the name of the state. */ public SplitState(Collection flows, String name) { super(name); @@ -83,16 +84,16 @@ public FlowExecutionStatus handle(final FlowExecutor executor) throws Exception // TODO: collect the last StepExecution from the flows as well, so they // can be abandoned if necessary - Collection> tasks = new ArrayList>(); + Collection> tasks = new ArrayList<>(); for (final Flow flow : flows) { - final FutureTask task = new FutureTask(new Callable() { - @Override - public FlowExecution call() throws Exception { - return flow.start(executor); - } - }); + final FutureTask task = new FutureTask<>(new Callable() { + @Override + public FlowExecution call() throws Exception { + return flow.start(executor); + } + }); tasks.add(task); @@ -105,7 +106,7 @@ public FlowExecution call() throws Exception { } - Collection results = new ArrayList(); + Collection results = new ArrayList<>(); // Could use a CompletionService here? for (Future task : tasks) { @@ -123,8 +124,11 @@ public FlowExecution call() throws Exception { } } - return aggregator.aggregate(results); + return doAggregation(results, executor); + } + protected FlowExecutionStatus doAggregation(Collection results, FlowExecutor executor) { + return aggregator.aggregate(results); } /* diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/StepState.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/StepState.java index b319908b4a..bde475e47c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/StepState.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/StepState.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,20 +16,28 @@ package org.springframework.batch.core.job.flow.support.state; +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; + import org.springframework.batch.core.Step; import org.springframework.batch.core.job.flow.FlowExecutionStatus; import org.springframework.batch.core.job.flow.FlowExecutor; import org.springframework.batch.core.job.flow.State; +import org.springframework.batch.core.step.NoSuchStepException; import org.springframework.batch.core.step.StepHolder; +import org.springframework.batch.core.step.StepLocator; /** * {@link State} implementation that delegates to a {@link FlowExecutor} to * execute the specified {@link Step}. * * @author Dave Syer + * @author Michael Minella + * @author Mahmoud Ben Hassine * @since 2.0 */ -public class StepState extends AbstractState implements StepHolder { +public class StepState extends AbstractState implements StepLocator, StepHolder { private final Step step; @@ -60,9 +68,6 @@ public FlowExecutionStatus handle(FlowExecutor executor) throws Exception { return new FlowExecutionStatus(executor.executeStep(step)); } - /** - * @return the step - */ @Override public Step getStep() { return step; @@ -76,4 +81,35 @@ public boolean isEndState() { return false; } + /* (non-Javadoc) + * @see org.springframework.batch.core.step.StepLocator#getStepNames() + */ + @Override + public Collection getStepNames() { + List names = new ArrayList<>(); + + names.add(step.getName()); + + if(step instanceof StepLocator) { + names.addAll(((StepLocator)step).getStepNames()); + } + + return names; + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.step.StepLocator#getStep(java.lang.String) + */ + @Override + public Step getStep(String stepName) throws NoSuchStepException { + Step result = null; + + if(step.getName().equals(stepName)) { + result = step; + } else if(step instanceof StepLocator) { + result = ((StepLocator) step).getStep(stepName); + } + + return result; + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/package-info.java new file mode 100644 index 0000000000..4277ac4e97 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/flow/support/state/package-info.java @@ -0,0 +1,10 @@ +/** + * States used in defining the underlying Spring Batch state machine + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.job.flow.support.state; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/job/package-info.java new file mode 100644 index 0000000000..a96f7e8954 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/job/package-info.java @@ -0,0 +1,10 @@ +/** + * Specific implementations of job concerns. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.job; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/job/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/job/package.html deleted file mode 100644 index 3e790dbd76..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/job/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Specific implementations of job concerns. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/ChunkListenerAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/ChunkListenerAdapter.java new file mode 100644 index 0000000000..982da32371 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/ChunkListenerAdapter.java @@ -0,0 +1,74 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import javax.batch.operations.BatchRuntimeException; + +import org.springframework.batch.core.ChunkListener; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.UncheckedTransactionException; +import org.springframework.util.Assert; + +/** + * Wrapper class to adapt the {@link javax.batch.api.chunk.listener.ChunkListener} to + * a {@link ChunkListener}. + * + * @author Michael Minella + * @since 3.0 + */ +public class ChunkListenerAdapter implements ChunkListener { + + private final javax.batch.api.chunk.listener.ChunkListener delegate; + + /** + * @param delegate to be called within the step chunk lifecycle + */ + public ChunkListenerAdapter(javax.batch.api.chunk.listener.ChunkListener delegate) { + Assert.notNull(delegate, "A ChunkListener is required"); + this.delegate = delegate; + } + + @Override + public void beforeChunk(ChunkContext context) { + try { + delegate.beforeChunk(); + } catch (Exception e) { + throw new UncheckedTransactionException(e); + } + } + + @Override + public void afterChunk(ChunkContext context) { + try { + delegate.afterChunk(); + } catch (Exception e) { + throw new UncheckedTransactionException(e); + } + } + + @Override + public void afterChunkError(ChunkContext context) { + if(context != null) { + try { + delegate.onError((Exception) context.getAttribute(ChunkListener.ROLLBACK_EXCEPTION_KEY)); + } catch (Exception e) { + throw new UncheckedTransactionException(e); + } + } else { + throw new BatchRuntimeException("Unable to retrieve causing exception due to null ChunkContext"); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/ItemProcessListenerAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/ItemProcessListenerAdapter.java new file mode 100644 index 0000000000..ae246144d0 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/ItemProcessListenerAdapter.java @@ -0,0 +1,72 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import javax.batch.operations.BatchRuntimeException; + +import org.springframework.batch.core.ItemProcessListener; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * Wrapper class for {@link javax.batch.api.chunk.listener.ItemProcessListener} + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * + * @param input type + * @param output type + * @since 3.0 + */ +public class ItemProcessListenerAdapter implements ItemProcessListener { + + private javax.batch.api.chunk.listener.ItemProcessListener delegate; + + /** + * @param delegate to be called within the batch lifecycle + */ + public ItemProcessListenerAdapter(javax.batch.api.chunk.listener.ItemProcessListener delegate) { + Assert.notNull(delegate, "An ItemProcessListener is required"); + this.delegate = delegate; + } + + @Override + public void beforeProcess(T item) { + try { + delegate.beforeProcess(item); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + @Override + public void afterProcess(T item, @Nullable S result) { + try { + delegate.afterProcess(item, result); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + @Override + public void onProcessError(T item, Exception e) { + try { + delegate.onProcessError(item, e); + } catch (Exception e1) { + throw new BatchRuntimeException(e1); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/ItemReadListenerAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/ItemReadListenerAdapter.java new file mode 100644 index 0000000000..e1a4df3d1d --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/ItemReadListenerAdapter.java @@ -0,0 +1,69 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import javax.batch.operations.BatchRuntimeException; + +import org.springframework.batch.core.ItemReadListener; +import org.springframework.batch.item.ItemReader; +import org.springframework.util.Assert; + +/** + * Wrapper class to adapt the {@link javax.batch.api.chunk.listener.ItemReadListener} to + * a {@link ItemReadListener}. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * + * @param type to be returned via a read on the associated {@link ItemReader} + * @since 3.0 + */ +public class ItemReadListenerAdapter implements ItemReadListener { + + private javax.batch.api.chunk.listener.ItemReadListener delegate; + + public ItemReadListenerAdapter(javax.batch.api.chunk.listener.ItemReadListener delegate) { + Assert.notNull(delegate, "An ItemReadListener is required"); + this.delegate = delegate; + } + + @Override + public void beforeRead() { + try { + delegate.beforeRead(); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + @Override + public void afterRead(T item) { + try { + delegate.afterRead(item); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + @Override + public void onReadError(Exception ex) { + try { + delegate.onReadError(ex); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/ItemWriteListenerAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/ItemWriteListenerAdapter.java new file mode 100644 index 0000000000..c4a4b9662b --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/ItemWriteListenerAdapter.java @@ -0,0 +1,73 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import java.util.List; + +import javax.batch.operations.BatchRuntimeException; + +import org.springframework.batch.core.ItemWriteListener; +import org.springframework.batch.item.ItemWriter; +import org.springframework.util.Assert; + +/** + * Wrapper class to adapt the {@link javax.batch.api.chunk.listener.ItemWriteListener} to + * a {@link ItemWriteListener}. + * + * @author Michael Minella + * + * @param type to be written by the associated {@link ItemWriter} + * @since 3.0 + */ +public class ItemWriteListenerAdapter implements ItemWriteListener { + + private javax.batch.api.chunk.listener.ItemWriteListener delegate; + + public ItemWriteListenerAdapter(javax.batch.api.chunk.listener.ItemWriteListener delegate) { + Assert.notNull(delegate, "An ItemWriteListener is required"); + this.delegate = delegate; + } + + @SuppressWarnings("unchecked") + @Override + public void beforeWrite(List items) { + try { + delegate.beforeWrite((List) items); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + @SuppressWarnings("unchecked") + @Override + public void afterWrite(List items) { + try { + delegate.afterWrite((List) items); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + @SuppressWarnings("unchecked") + @Override + public void onWriteError(Exception exception, List items) { + try { + delegate.onWriteError((List) items, exception); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JobListenerAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JobListenerAdapter.java new file mode 100644 index 0000000000..e8d16ffa3d --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JobListenerAdapter.java @@ -0,0 +1,61 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import javax.batch.api.listener.JobListener; +import javax.batch.operations.BatchRuntimeException; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobExecutionListener; +import org.springframework.util.Assert; + +/** + * Wrapper class to adapt the {@link JobListener} to + * a {@link JobExecutionListener}. + * + * @author Michael Minella + * @since 3.0 + */ +public class JobListenerAdapter implements JobExecutionListener { + + private JobListener delegate; + + /** + * @param delegate to be delegated to + */ + public JobListenerAdapter(JobListener delegate) { + Assert.notNull(delegate, "Delegate is required"); + this.delegate = delegate; + } + + @Override + public void beforeJob(JobExecution jobExecution) { + try { + delegate.beforeJob(); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + @Override + public void afterJob(JobExecution jobExecution) { + try { + delegate.afterJob(); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobContext.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobContext.java new file mode 100644 index 0000000000..86db6c7b7d --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobContext.java @@ -0,0 +1,126 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import java.util.Properties; +import java.util.concurrent.atomic.AtomicBoolean; + +import javax.batch.runtime.BatchStatus; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * Wrapper class to provide the {@link javax.batch.runtime.context.JobContext} functionality + * as specified in JSR-352. Wrapper delegates to the underlying {@link JobExecution} to + * obtain the related contextual information. + * + * @author Michael Minella + * @author Chris Schaefer + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class JsrJobContext implements javax.batch.runtime.context.JobContext { + private Object transientUserData; + private Properties properties; + private JobExecution jobExecution; + private AtomicBoolean exitStatusSet = new AtomicBoolean(); + + public void setJobExecution(JobExecution jobExecution) { + Assert.notNull(jobExecution, "A JobExecution is required"); + this.jobExecution = jobExecution; + } + + public void setProperties(@Nullable Properties properties) { + this.properties = properties != null ? properties : new Properties(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.JobContext#getJobName() + */ + @Override + public String getJobName() { + return jobExecution.getJobInstance().getJobName(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.JobContext#getTransientUserData() + */ + @Override + public Object getTransientUserData() { + return transientUserData; + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.JobContext#setTransientUserData(java.lang.Object) + */ + @Override + public void setTransientUserData(Object data) { + transientUserData = data; + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.JobContext#getInstanceId() + */ + @Override + public long getInstanceId() { + return jobExecution.getJobInstance().getId(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.JobContext#getExecutionId() + */ + @Override + public long getExecutionId() { + return jobExecution.getId(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.JobContext#getProperties() + */ + @Override + public Properties getProperties() { + return properties; + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.JobContext#getBatchStatus() + */ + @Override + public BatchStatus getBatchStatus() { + return jobExecution.getStatus().getBatchStatus(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.JobContext#getExitStatus() + */ + @Override + @Nullable + public String getExitStatus() { + return exitStatusSet.get() ? jobExecution.getExitStatus().getExitCode() : null; + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.JobContext#setExitStatus(java.lang.String) + */ + @Override + public void setExitStatus(String status) { + jobExecution.setExitStatus(new ExitStatus(status)); + exitStatusSet.set(true); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobContextFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobContextFactoryBean.java new file mode 100644 index 0000000000..50186355e6 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobContextFactoryBean.java @@ -0,0 +1,134 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import java.util.Properties; + +import javax.batch.runtime.StepExecution; +import javax.batch.runtime.context.JobContext; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.beans.factory.FactoryBean; +import org.springframework.beans.factory.FactoryBeanNotInitializedException; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.util.Assert; + +/** + * Provides a single {@link JobContext} for each thread in a running job. + * Subsequent calls to {@link FactoryBean#getObject()} on the same thread will + * return the same instance. The {@link JobContext} wraps a {@link JobExecution} + * which is obtained in one of two ways: + *
        + *
      • The current step scope (getting it from the current {@link StepExecution}
      • + *
      • The provided {@link JobExecution} via the {@link #setJobExecution(JobExecution)} + *
      + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrJobContextFactoryBean implements FactoryBean { + + private JobExecution jobExecution; + @Autowired + private BatchPropertyContext propertyContext; + + private static final ThreadLocal contextHolder = new ThreadLocal<>(); + + /* (non-Javadoc) + * @see org.springframework.beans.factory.FactoryBean#getObject() + */ + @Override + public JobContext getObject() throws Exception { + return getCurrent(); + } + + /* (non-Javadoc) + * @see org.springframework.beans.factory.FactoryBean#getObjectType() + */ + @Override + public Class getObjectType() { + return JobContext.class; + } + + /* (non-Javadoc) + * @see org.springframework.beans.factory.FactoryBean#isSingleton() + */ + @Override + public boolean isSingleton() { + return false; + } + + /** + * Used to provide {@link JobContext} instances to batch artifacts that + * are not within the scope of a given step. + * + * @param jobExecution set the current {@link JobExecution} + */ + public void setJobExecution(JobExecution jobExecution) { + Assert.notNull(jobExecution, "A JobExecution is required"); + this.jobExecution = jobExecution; + } + + /** + * @param propertyContext the {@link BatchPropertyContext} to obtain job properties from + */ + public void setBatchPropertyContext(BatchPropertyContext propertyContext) { + this.propertyContext = propertyContext; + } + + /** + * Used to remove the {@link JobContext} for the current thread. Not used via + * normal processing but useful for testing. + */ + public void close() { + if(contextHolder.get() != null) { + contextHolder.remove(); + } + } + + private JobContext getCurrent() { + if(contextHolder.get() == null) { + JobExecution curJobExecution = null; + + if(StepSynchronizationManager.getContext() != null) { + curJobExecution = StepSynchronizationManager.getContext().getStepExecution().getJobExecution(); + } + + if(curJobExecution != null) { + jobExecution = curJobExecution; + } + + if(jobExecution == null) { + throw new FactoryBeanNotInitializedException("A JobExecution is required"); + } + + JsrJobContext jobContext = new JsrJobContext(); + jobContext.setJobExecution(jobExecution); + + if(propertyContext != null) { + jobContext.setProperties(propertyContext.getJobProperties()); + } else { + jobContext.setProperties(new Properties()); + } + + contextHolder.set(jobContext); + } + + return contextHolder.get(); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobExecution.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobExecution.java new file mode 100644 index 0000000000..3d56085543 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobExecution.java @@ -0,0 +1,122 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import java.util.Date; +import java.util.Properties; + +import javax.batch.runtime.BatchStatus; + +import org.springframework.batch.core.converter.JobParametersConverter; +import org.springframework.util.Assert; + +/** + * Wrapper class to adapt the {@link javax.batch.runtime.JobExecution} to + * a {@link org.springframework.batch.core.JobExecution}. + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrJobExecution implements javax.batch.runtime.JobExecution { + + private org.springframework.batch.core.JobExecution execution; + private JobParametersConverter parametersConverter; + + /** + * @param execution for all information to be delegated from. + * @param parametersConverter instance of {@link JobParametersConverter}. + */ + public JsrJobExecution(org.springframework.batch.core.JobExecution execution, JobParametersConverter parametersConverter) { + Assert.notNull(execution, "A JobExecution is required"); + this.execution = execution; + + this.parametersConverter = parametersConverter; + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JobExecution#getExecutionId() + */ + @Override + public long getExecutionId() { + return this.execution.getId(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JobExecution#getJobName() + */ + @Override + public String getJobName() { + return this.execution.getJobInstance().getJobName(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JobExecution#getBatchStatus() + */ + @Override + public BatchStatus getBatchStatus() { + return this.execution.getStatus().getBatchStatus(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JobExecution#getStartTime() + */ + @Override + public Date getStartTime() { + return this.execution.getStartTime(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JobExecution#getEndTime() + */ + @Override + public Date getEndTime() { + return this.execution.getEndTime(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JobExecution#getExitStatus() + */ + @Override + public String getExitStatus() { + return this.execution.getExitStatus().getExitCode(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JobExecution#getCreateTime() + */ + @Override + public Date getCreateTime() { + return this.execution.getCreateTime(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JobExecution#getLastUpdatedTime() + */ + @Override + public Date getLastUpdatedTime() { + return this.execution.getLastUpdated(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JobExecution#getJobParameters() + */ + @Override + public Properties getJobParameters() { + Properties properties = parametersConverter.getProperties(this.execution.getJobParameters()); + properties.remove(JsrJobParametersConverter.JOB_RUN_ID); + return properties; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobListenerMetaData.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobListenerMetaData.java new file mode 100644 index 0000000000..d1fa03ae60 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobListenerMetaData.java @@ -0,0 +1,87 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import java.lang.annotation.Annotation; +import java.util.HashMap; +import java.util.Map; + +import javax.batch.api.listener.JobListener; + +import org.springframework.batch.core.listener.ListenerMetaData; + +/** + * Enumeration for {@link JobListener} meta data, which ties together the names + * of methods, their interfaces, annotation, and expected arguments. + * + * @author Michael Minella + * @since 3.0 + */ +public enum JsrJobListenerMetaData implements ListenerMetaData { + BEFORE_JOB("beforeJob", "jsr-before-job"), + AFTER_JOB("afterJob", "jsr-after-job"); + + private final String methodName; + private final String propertyName; + private static final Map propertyMap; + + JsrJobListenerMetaData(String methodName, String propertyName) { + this.methodName = methodName; + this.propertyName = propertyName; + } + + static{ + propertyMap = new HashMap<>(); + for(JsrJobListenerMetaData metaData : values()){ + propertyMap.put(metaData.getPropertyName(), metaData); + } + } + + @Override + public String getMethodName() { + return methodName; + } + + @Override + public Class getAnnotation() { + return null; + } + + @Override + public Class getListenerInterface() { + return JobListener.class; + } + + @Override + public String getPropertyName() { + return propertyName; + } + + @Override + public Class[] getParamTypes() { + return new Class[0]; + } + + /** + * Return the relevant meta data for the provided property name. + * + * @param propertyName the name of the property to return. + * @return meta data with supplied property name, null if none exists. + */ + public static JsrJobListenerMetaData fromPropertyName(String propertyName){ + return propertyMap.get(propertyName); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobParametersConverter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobParametersConverter.java new file mode 100644 index 0000000000..642b42fc4c --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrJobParametersConverter.java @@ -0,0 +1,135 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import java.util.Map; +import java.util.Properties; + +import javax.sql.DataSource; + +import org.springframework.batch.core.JobParameter; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.converter.JobParametersConverter; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.repository.dao.AbstractJdbcBatchMetadataDao; +import org.springframework.batch.item.database.support.DataFieldMaxValueIncrementerFactory; +import org.springframework.batch.item.database.support.DefaultDataFieldMaxValueIncrementerFactory; +import org.springframework.batch.support.DatabaseType; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.jdbc.support.incrementer.DataFieldMaxValueIncrementer; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * Provides default conversion methodology for JSR-352's implementation. + * + * Since Spring Batch uses job parameters as a way of identifying a job + * instance, this converter will add an additional identifying parameter if + * it does not exist already in the list. The id for the identifying parameter + * will come from the JOB_SEQ sequence as used to generate the unique ids + * for BATCH_JOB_INSTANCE records. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class JsrJobParametersConverter implements JobParametersConverter, InitializingBean { + + public static final String JOB_RUN_ID = "jsr_batch_run_id"; + public DataFieldMaxValueIncrementer incrementer; + public String tablePrefix = AbstractJdbcBatchMetadataDao.DEFAULT_TABLE_PREFIX; + public DataSource dataSource; + + /** + * Main constructor. + * + * @param dataSource used to gain access to the database to get unique ids. + */ + public JsrJobParametersConverter(DataSource dataSource) { + Assert.notNull(dataSource, "A DataSource is required"); + this.dataSource = dataSource; + } + + /** + * The table prefix used in the current {@link JobRepository} + * + * @param tablePrefix the table prefix used for the job repository tables + */ + public void setTablePrefix(String tablePrefix) { + this.tablePrefix = tablePrefix; + } + + @Override + public void afterPropertiesSet() throws Exception { + DataFieldMaxValueIncrementerFactory factory = new DefaultDataFieldMaxValueIncrementerFactory(dataSource); + + this.incrementer = factory.getIncrementer(DatabaseType.fromMetaData(dataSource).name(), tablePrefix + "JOB_SEQ"); + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.converter.JobParametersConverter#getJobParameters(java.util.Properties) + */ + @Override + public JobParameters getJobParameters(@Nullable Properties properties) { + JobParametersBuilder builder = new JobParametersBuilder(); + boolean runIdFound = false; + + if(properties != null) { + for (Map.Entry curParameter : properties.entrySet()) { + if(curParameter.getValue() != null) { + if(curParameter.getKey().equals(JOB_RUN_ID)) { + runIdFound = true; + builder.addLong(curParameter.getKey().toString(), Long.valueOf((String) curParameter.getValue()), true); + } else { + builder.addString(curParameter.getKey().toString(), curParameter.getValue().toString(), false); + } + } + } + } + + if(!runIdFound) { + builder.addLong(JOB_RUN_ID, incrementer.nextLongValue()); + } + + return builder.toJobParameters(); + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.converter.JobParametersConverter#getProperties(org.springframework.batch.core.JobParameters) + */ + @Override + public Properties getProperties(@Nullable JobParameters params) { + Properties properties = new Properties(); + boolean runIdFound = false; + + if(params != null) { + for(Map.Entry curParameter: params.getParameters().entrySet()) { + if(curParameter.getKey().equals(JOB_RUN_ID)) { + runIdFound = true; + } + + properties.setProperty(curParameter.getKey(), curParameter.getValue().getValue().toString()); + } + } + + if(!runIdFound) { + properties.setProperty(JOB_RUN_ID, String.valueOf(incrementer.nextLongValue())); + } + + return properties; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrStepContext.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrStepContext.java new file mode 100644 index 0000000000..213ac512c9 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrStepContext.java @@ -0,0 +1,181 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import java.io.Serializable; +import java.util.List; +import java.util.Properties; +import java.util.concurrent.atomic.AtomicBoolean; + +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.Metric; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.item.util.ExecutionContextUserSupport; +import org.springframework.util.Assert; +import org.springframework.util.ClassUtils; + +/** + * Wrapper class to provide the {@link javax.batch.runtime.context.StepContext} functionality + * as specified in JSR-352. Wrapper delegates to the underlying {@link StepExecution} to + * obtain the related contextual information. + * + * @author Michael Minella + * @author Chris Schaefer + * @since 3.0 + */ +public class JsrStepContext implements javax.batch.runtime.context.StepContext { + private final static String PERSISTENT_USER_DATA_KEY = "batch_jsr_persistentUserData"; + private StepExecution stepExecution; + private Object transientUserData; + private Properties properties = new Properties(); + private AtomicBoolean exitStatusSet = new AtomicBoolean(); + private final ExecutionContextUserSupport executionContextUserSupport = new ExecutionContextUserSupport(ClassUtils.getShortName(JsrStepContext.class)); + + public JsrStepContext(StepExecution stepExecution, Properties properties) { + Assert.notNull(stepExecution, "A StepExecution is required"); + + this.stepExecution = stepExecution; + this.properties = properties; + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.StepContext#getStepName() + */ + @Override + public String getStepName() { + return stepExecution.getStepName(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.StepContext#getTransientUserData() + */ + @Override + public Object getTransientUserData() { + return transientUserData; + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.StepContext#setTransientUserData(java.lang.Object) + */ + @Override + public void setTransientUserData(Object data) { + this.transientUserData = data; + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.StepContext#getStepExecutionId() + */ + @Override + public long getStepExecutionId() { + return stepExecution.getId(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.StepContext#getProperties() + */ + @Override + public Properties getProperties() { + return properties != null ? properties : new Properties(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.StepContext#getPersistentUserData() + */ + @Override + public Serializable getPersistentUserData() { + return (Serializable) stepExecution.getExecutionContext().get(executionContextUserSupport.getKey(PERSISTENT_USER_DATA_KEY)); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.StepContext#setPersistentUserData(java.io.Serializable) + */ + @Override + public void setPersistentUserData(Serializable data) { + stepExecution.getExecutionContext().put(executionContextUserSupport.getKey(PERSISTENT_USER_DATA_KEY), data); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.StepContext#getBatchStatus() + */ + @Override + public BatchStatus getBatchStatus() { + return stepExecution.getStatus().getBatchStatus(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.StepContext#getExitStatus() + */ + @Override + public String getExitStatus() { + return exitStatusSet.get() ? stepExecution.getExitStatus().getExitCode() : null; + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.StepContext#setExitStatus(java.lang.String) + */ + @Override + public void setExitStatus(String status) { + stepExecution.setExitStatus(new ExitStatus(status)); + exitStatusSet.set(true); + } + + /** + * To support both JSR-352's requirement to return the most recent exception + * and Spring Batch's support for {@link Throwable}, this implementation will + * return the most recent exception in the underlying {@link StepExecution}'s + * failure exceptions list. If the exception there extends {@link Throwable} + * instead of {@link Exception}, it will be wrapped in an {@link Exception} and + * then returned. + * + * @see javax.batch.runtime.context.StepContext#getException() + */ + @Override + public Exception getException() { + List failureExceptions = stepExecution.getFailureExceptions(); + if(failureExceptions == null || failureExceptions.isEmpty()) { + return null; + } else { + Throwable t = failureExceptions.get(failureExceptions.size() - 1); + + if(t instanceof Exception) { + return (Exception) t; + } else { + return new Exception(t); + } + } + } + + /* (non-Javadoc) + * @see javax.batch.runtime.context.StepContext#getMetrics() + */ + @Override + public Metric[] getMetrics() { + Metric[] metrics = new Metric[8]; + + metrics[0] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.COMMIT_COUNT, stepExecution.getCommitCount()); + metrics[1] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.FILTER_COUNT, stepExecution.getFilterCount()); + metrics[2] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.PROCESS_SKIP_COUNT, stepExecution.getProcessSkipCount()); + metrics[3] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.READ_COUNT, stepExecution.getReadCount()); + metrics[4] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.READ_SKIP_COUNT, stepExecution.getReadSkipCount()); + metrics[5] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.ROLLBACK_COUNT, stepExecution.getRollbackCount()); + metrics[6] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.WRITE_COUNT, stepExecution.getWriteCount()); + metrics[7] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.WRITE_SKIP_COUNT, stepExecution.getWriteSkipCount()); + + return metrics; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrStepContextFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrStepContextFactoryBean.java new file mode 100644 index 0000000000..9759246708 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrStepContextFactoryBean.java @@ -0,0 +1,113 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import java.util.Properties; + +import javax.batch.runtime.context.StepContext; + +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.beans.factory.FactoryBean; +import org.springframework.beans.factory.FactoryBeanNotInitializedException; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.util.Assert; + +/** + * {@link FactoryBean} implementation used to create {@link javax.batch.runtime.context.StepContext} + * instances within the step scope. + * + * @author Michael Minella + * @author Chris Schaefer + * @since 3.0 + */ +public class JsrStepContextFactoryBean implements FactoryBean, InitializingBean { + @Autowired + private BatchPropertyContext batchPropertyContext; + + private static final ThreadLocal contextHolder = new ThreadLocal<>(); + + protected void setBatchPropertyContext(BatchPropertyContext batchPropertyContext) { + this.batchPropertyContext = batchPropertyContext; + } + + /* (non-Javadoc) + * @see org.springframework.beans.factory.FactoryBean#getObject() + */ + @Override + public StepContext getObject() throws Exception { + return getCurrent(); + } + + private javax.batch.runtime.context.StepContext getCurrent() { + org.springframework.batch.core.StepExecution curStepExecution = null; + + if(StepSynchronizationManager.getContext() != null) { + curStepExecution = StepSynchronizationManager.getContext().getStepExecution(); + } + + if(curStepExecution == null) { + throw new FactoryBeanNotInitializedException("A StepExecution is required"); + } + + StepContext context = contextHolder.get(); + + // If the current context applies to the current step, use it + if(context != null && context.getStepExecutionId() == curStepExecution.getId()) { + return context; + } + + Properties stepProperties = batchPropertyContext.getStepProperties(curStepExecution.getStepName()); + + if(stepProperties != null) { + context = new JsrStepContext(curStepExecution, stepProperties); + } else { + context = new JsrStepContext(curStepExecution, new Properties()); + } + + contextHolder.set(context); + + return context; + } + + /* (non-Javadoc) + * @see org.springframework.beans.factory.FactoryBean#getObjectType() + */ + @Override + public Class getObjectType() { + return StepContext.class; + } + + /* (non-Javadoc) + * @see org.springframework.beans.factory.FactoryBean#isSingleton() + */ + @Override + public boolean isSingleton() { + return false; + } + + @Override + public void afterPropertiesSet() throws Exception { + Assert.notNull(batchPropertyContext, "BatchPropertyContext is required"); + } + + public void remove() { + if(contextHolder.get() != null) { + contextHolder.remove(); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrStepExecution.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrStepExecution.java new file mode 100644 index 0000000000..8863a947ae --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrStepExecution.java @@ -0,0 +1,134 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import java.io.Serializable; +import java.util.Date; + +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.Metric; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.item.util.ExecutionContextUserSupport; +import org.springframework.util.Assert; +import org.springframework.util.ClassUtils; + +/** + * Implementation of the JsrStepExecution as defined in JSR-352. This implementation + * wraps a {@link org.springframework.batch.core.StepExecution} as it's source of + * data. + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrStepExecution implements javax.batch.runtime.StepExecution{ + + private final static String PERSISTENT_USER_DATA_KEY = "batch_jsr_persistentUserData"; + private final org.springframework.batch.core.StepExecution stepExecution; + // The API for the persistent user data is handled by the JsrStepContext which is why the name here is based on the JsrStepContext. + private final ExecutionContextUserSupport executionContextUserSupport = new ExecutionContextUserSupport(ClassUtils.getShortName(JsrStepContext.class)); + + /** + * @param stepExecution The {@link org.springframework.batch.core.StepExecution} used + * as the basis for the data. + */ + public JsrStepExecution(org.springframework.batch.core.StepExecution stepExecution) { + Assert.notNull(stepExecution, "A StepExecution is required"); + + this.stepExecution = stepExecution; + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JsrStepExecution#getStepExecutionId() + */ + @Override + public long getStepExecutionId() { + return stepExecution.getId(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JsrStepExecution#getStepName() + */ + @Override + public String getStepName() { + return stepExecution.getStepName(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JsrStepExecution#getBatchStatus() + */ + @Override + public BatchStatus getBatchStatus() { + return stepExecution.getStatus().getBatchStatus(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JsrStepExecution#getStartTime() + */ + @Override + public Date getStartTime() { + return stepExecution.getStartTime(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JsrStepExecution#getEndTime() + */ + @Override + public Date getEndTime() { + return stepExecution.getEndTime(); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JsrStepExecution#getExitStatus() + */ + @Override + public String getExitStatus() { + ExitStatus status = stepExecution.getExitStatus(); + + if(status == null) { + return null; + } else { + return status.getExitCode(); + } + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JsrStepExecution#getPersistentUserData() + */ + @Override + public Serializable getPersistentUserData() { + return (Serializable) stepExecution.getExecutionContext().get(executionContextUserSupport.getKey(PERSISTENT_USER_DATA_KEY)); + } + + /* (non-Javadoc) + * @see javax.batch.runtime.JsrStepExecution#getMetrics() + */ + @Override + public Metric[] getMetrics() { + Metric[] metrics = new Metric[8]; + + metrics[0] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.COMMIT_COUNT, stepExecution.getCommitCount()); + metrics[1] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.FILTER_COUNT, stepExecution.getFilterCount()); + metrics[2] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.PROCESS_SKIP_COUNT, stepExecution.getProcessSkipCount()); + metrics[3] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.READ_COUNT, stepExecution.getReadCount()); + metrics[4] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.READ_SKIP_COUNT, stepExecution.getReadSkipCount()); + metrics[5] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.ROLLBACK_COUNT, stepExecution.getRollbackCount()); + metrics[6] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.WRITE_COUNT, stepExecution.getWriteCount()); + metrics[7] = new SimpleMetric(javax.batch.runtime.Metric.MetricType.WRITE_SKIP_COUNT, stepExecution.getWriteSkipCount()); + + return metrics; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrStepListenerMetaData.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrStepListenerMetaData.java new file mode 100644 index 0000000000..21a6efafc3 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/JsrStepListenerMetaData.java @@ -0,0 +1,123 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import java.lang.annotation.Annotation; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +import javax.batch.api.chunk.listener.ChunkListener; +import javax.batch.api.chunk.listener.ItemProcessListener; +import javax.batch.api.chunk.listener.ItemReadListener; +import javax.batch.api.chunk.listener.ItemWriteListener; +import javax.batch.api.chunk.listener.RetryProcessListener; +import javax.batch.api.chunk.listener.RetryReadListener; +import javax.batch.api.chunk.listener.RetryWriteListener; +import javax.batch.api.chunk.listener.SkipProcessListener; +import javax.batch.api.chunk.listener.SkipReadListener; +import javax.batch.api.chunk.listener.SkipWriteListener; +import javax.batch.api.listener.StepListener; + +import org.springframework.batch.core.listener.ListenerMetaData; +import org.springframework.batch.core.listener.StepListenerFactoryBean; + +/** + * Enumeration for the JSR specific {@link StepListener} meta data, which + * ties together the names of methods, their interfaces, and expected arguments. + * + * @author Michael Minella + * @author Chris Schaefer + * @since 3.0 + * @see StepListenerFactoryBean + */ +public enum JsrStepListenerMetaData implements ListenerMetaData { + BEFORE_STEP("beforeStep", "jsr-before-step", StepListener.class), + AFTER_STEP("afterStep", "jsr-after-step", StepListener.class), + BEFORE_CHUNK("beforeChunk", "jsr-before-chunk", ChunkListener.class), + AFTER_CHUNK("afterChunk", "jsr-after-chunk", ChunkListener.class), + AFTER_CHUNK_ERROR("onError", "jsr-after-chunk-error", ChunkListener.class, Exception.class), + BEFORE_READ("beforeRead", "jsr-before-read", ItemReadListener.class), + AFTER_READ("afterRead", "jsr-after-read", ItemReadListener.class, Object.class), + AFTER_READ_ERROR("onReadError", "jsr-after-read-error", ItemReadListener.class, Exception.class), + BEFORE_PROCESS("beforeProcess", "jsr-before-process", ItemProcessListener.class, Object.class), + AFTER_PROCESS("afterProcess", "jsr-after-process", ItemProcessListener.class, Object.class, Object.class), + AFTER_PROCESS_ERROR("onProcessError", "jsr-after-process-error", ItemProcessListener.class, Object.class, Exception.class), + BEFORE_WRITE("beforeWrite", "jsr-before-write", ItemWriteListener.class, List.class), + AFTER_WRITE("afterWrite", "jsr-after-write", ItemWriteListener.class, List.class), + AFTER_WRITE_ERROR("onWriteError", "jsr-after-write-error", ItemWriteListener.class, List.class, Exception.class), + SKIP_READ("onSkipReadItem", "jsr-skip-read", SkipReadListener.class, Exception.class), + SKIP_PROCESS("onSkipProcessItem", "jsr-skip-process", SkipProcessListener.class, Object.class, Exception.class), + SKIP_WRITE("onSkipWriteItem", "jsr-skip-write", SkipWriteListener.class, List.class, Exception.class), + RETRY_READ("onRetryReadException", "jsr-retry-read", RetryReadListener.class, Exception.class), + RETRY_PROCESS("onRetryProcessException", "jsr-retry-process", RetryProcessListener.class, Object.class, Exception.class), + RETRY_WRITE("onRetryWriteException", "jsr-retry-write", RetryWriteListener.class, List.class, Exception.class); + + private final String methodName; + private final String propertyName; + private final Class listenerInterface; + private static final Map propertyMap; + private final Class[] paramTypes; + + JsrStepListenerMetaData(String methodName, String propertyName, Class listenerInterface, Class... paramTypes) { + this.propertyName = propertyName; + this.methodName = methodName; + this.listenerInterface = listenerInterface; + this.paramTypes = paramTypes; + } + + static{ + propertyMap = new HashMap<>(); + for(JsrStepListenerMetaData metaData : values()){ + propertyMap.put(metaData.getPropertyName(), metaData); + } + } + + @Override + public String getMethodName() { + return methodName; + } + + @Override + public Class getAnnotation() { + return null; + } + + @Override + public Class getListenerInterface() { + return listenerInterface; + } + + @Override + public Class[] getParamTypes() { + return paramTypes; + } + + @Override + public String getPropertyName() { + return propertyName; + } + + /** + * Return the relevant meta data for the provided property name. + * + * @param propertyName the name of the property to return. + * @return meta data with supplied property name, null if none exists. + */ + public static JsrStepListenerMetaData fromPropertyName(String propertyName){ + return propertyMap.get(propertyName); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/RetryListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/RetryListener.java new file mode 100644 index 0000000000..6527cd524b --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/RetryListener.java @@ -0,0 +1,30 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import org.springframework.batch.core.StepListener; + +/** + *

      + * Interface used internally by RetryListener adapters to provide consistent naming. + * Extends {@link StepListener} to allow registration with existing listener methods. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public interface RetryListener extends StepListener { +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/RetryProcessListenerAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/RetryProcessListenerAdapter.java new file mode 100644 index 0000000000..c2c076d650 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/RetryProcessListenerAdapter.java @@ -0,0 +1,44 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import javax.batch.api.chunk.listener.RetryProcessListener; +import javax.batch.operations.BatchRuntimeException; + +/** + *

      + * Wrapper class to adapt a {@link RetryProcessListener} to a {@link RetryListener}. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class RetryProcessListenerAdapter implements RetryListener, RetryProcessListener { + private RetryProcessListener retryProcessListener; + + public RetryProcessListenerAdapter(RetryProcessListener retryProcessListener) { + this.retryProcessListener = retryProcessListener; + } + + @Override + public void onRetryProcessException(Object item, Exception ex) throws Exception { + try { + retryProcessListener.onRetryProcessException(item, ex); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/RetryReadListenerAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/RetryReadListenerAdapter.java new file mode 100644 index 0000000000..c3392287b7 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/RetryReadListenerAdapter.java @@ -0,0 +1,44 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import javax.batch.api.chunk.listener.RetryReadListener; +import javax.batch.operations.BatchRuntimeException; + +/** + *

      + * Wrapper class to adapt a {@link RetryReadListener} to a {@link RetryListener}. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class RetryReadListenerAdapter implements RetryListener, RetryReadListener { + private RetryReadListener retryReadListener; + + public RetryReadListenerAdapter(RetryReadListener retryReadListener) { + this.retryReadListener = retryReadListener; + } + + @Override + public void onRetryReadException(Exception ex) throws Exception { + try { + retryReadListener.onRetryReadException(ex); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/RetryWriteListenerAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/RetryWriteListenerAdapter.java new file mode 100644 index 0000000000..cfd7c1a074 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/RetryWriteListenerAdapter.java @@ -0,0 +1,45 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import java.util.List; +import javax.batch.api.chunk.listener.RetryWriteListener; +import javax.batch.operations.BatchRuntimeException; + +/** + *

      + * Wrapper class to adapt a {@link RetryWriteListener} to a {@link RetryListener}. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class RetryWriteListenerAdapter implements RetryListener, RetryWriteListener { + private RetryWriteListener retryWriteListener; + + public RetryWriteListenerAdapter(RetryWriteListener retryWriteListener) { + this.retryWriteListener = retryWriteListener; + } + + @Override + public void onRetryWriteException(List items, Exception ex) throws Exception { + try { + retryWriteListener.onRetryWriteException(items, ex); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/SimpleMetric.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/SimpleMetric.java new file mode 100644 index 0000000000..955245f50c --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/SimpleMetric.java @@ -0,0 +1,63 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import javax.batch.runtime.Metric; + +import org.springframework.util.Assert; + +/** + * Simple implementation of the {@link Metric} interface as required by JSR-352. + * + * @author Michael Minella + * @since 3.0 + */ +public class SimpleMetric implements Metric { + + private final MetricType type; + private final long value; + + /** + * Basic constructor. The attributes are immutable so this class is + * thread-safe. + * + * @param type as defined by JSR-352 + * @param value the count of the times the related type has occurred. + */ + public SimpleMetric(MetricType type, long value) { + Assert.notNull(type, "A MetricType is required"); + + this.type = type; + this.value = value; + } + + /* (non-Javadoc) + * @see javax.batch.runtime.Metric#getType() + */ + @Override + public MetricType getType() { + return type; + } + + /* (non-Javadoc) + * @see javax.batch.runtime.Metric#getValue() + */ + @Override + public long getValue() { + return value; + } + +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/SkipListenerAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/SkipListenerAdapter.java new file mode 100644 index 0000000000..c86b664adc --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/SkipListenerAdapter.java @@ -0,0 +1,75 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import javax.batch.api.chunk.listener.SkipProcessListener; +import javax.batch.api.chunk.listener.SkipReadListener; +import javax.batch.api.chunk.listener.SkipWriteListener; +import javax.batch.operations.BatchRuntimeException; + +import org.springframework.batch.core.SkipListener; + +import java.util.List; + +public class SkipListenerAdapter implements SkipListener { + private final SkipReadListener skipReadDelegate; + private final SkipProcessListener skipProcessDelegate; + private final SkipWriteListener skipWriteDelegate; + + public SkipListenerAdapter(SkipReadListener skipReadDelegate, SkipProcessListener skipProcessDelegate, SkipWriteListener skipWriteDelegate) { + this.skipReadDelegate = skipReadDelegate; + this.skipProcessDelegate = skipProcessDelegate; + this.skipWriteDelegate = skipWriteDelegate; + } + + @Override + public void onSkipInRead(Throwable t) { + if(skipReadDelegate != null && t instanceof Exception) { + try { + skipReadDelegate.onSkipReadItem((Exception) t); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + } + + @SuppressWarnings("unchecked") + @Override + public void onSkipInWrite(S item, Throwable t) { + if(skipWriteDelegate != null && t instanceof Exception) { + try { + /* + * assuming this SkipListenerAdapter will only be called from JsrFaultTolerantChunkProcessor, + * which calls onSkipInWrite() with the whole chunk (List) of items instead of single item + */ + skipWriteDelegate.onSkipWriteItem((List) item, (Exception) t); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + } + + @Override + public void onSkipInProcess(T item, Throwable t) { + if(skipProcessDelegate != null && t instanceof Exception) { + try { + skipProcessDelegate.onSkipProcessItem(item, (Exception) t); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/StepListenerAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/StepListenerAdapter.java new file mode 100644 index 0000000000..f4294df41b --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/StepListenerAdapter.java @@ -0,0 +1,66 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import javax.batch.api.listener.StepListener; +import javax.batch.operations.BatchRuntimeException; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.StepExecutionListener; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * Wrapper class to adapt the {@link StepListener} to + * a {@link StepExecutionListener}. + * + * @author Michael Minella + * @since 3.0 + */ +public class StepListenerAdapter implements StepExecutionListener { + + private final StepListener delegate; + + /** + * @param delegate instance of {@link StepListener}. + */ + public StepListenerAdapter(StepListener delegate) { + Assert.notNull(delegate, "A listener is required"); + this.delegate = delegate; + } + + @Override + public void beforeStep(StepExecution stepExecution) { + try { + delegate.beforeStep(); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + @Nullable + @Override + public ExitStatus afterStep(StepExecution stepExecution) { + try { + delegate.afterStep(); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + + return stepExecution.getExitStatus(); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/BaseContextListFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/BaseContextListFactoryBean.java new file mode 100644 index 0000000000..58fd746525 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/BaseContextListFactoryBean.java @@ -0,0 +1,56 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.support; + +import java.util.ArrayList; +import java.util.List; + +import org.springframework.beans.factory.FactoryBean; + +/** + * A simple factory bean that consolidates the list of locations to look for the base context for the JSR-352 + * functionality + * + * @author Michael Minella + * @since 3.0.3 + */ +public class BaseContextListFactoryBean implements FactoryBean>{ + + @Override + public List getObject() throws Exception { + String overrideContextLocation = System.getProperty("JSR-352-BASE-CONTEXT"); + + List contextLocations = new ArrayList<>(2); + + contextLocations.add("baseContext.xml"); + + if(overrideContextLocation != null) { + contextLocations.add(overrideContextLocation); + } + + return contextLocations; + } + + @Override + public Class getObjectType() { + return List.class; + } + + @Override + public boolean isSingleton() { + return true; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/BatchArtifactType.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/BatchArtifactType.java new file mode 100644 index 0000000000..a60d16481a --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/BatchArtifactType.java @@ -0,0 +1,31 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.support; + +/** + *

      + * Enum to identify batch artifact types. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public enum BatchArtifactType { + STEP, + STEP_ARTIFACT, + ARTIFACT, + JOB +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/BatchPropertyContext.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/BatchPropertyContext.java new file mode 100644 index 0000000000..027e8932e4 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/BatchPropertyContext.java @@ -0,0 +1,244 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.support; + +import java.util.Enumeration; +import java.util.HashMap; +import java.util.Map; +import java.util.Properties; + +import org.springframework.util.Assert; + +/** + *

      + * Context object to hold parsed JSR-352 batch properties, mapping properties to beans / + * "batch artifacts". Used internally when parsing property tags from a batch configuration + * file and to obtain corresponding values when injecting into batch artifacts. + *

      + * + * @author Chris Schaefer + * @author Michael Minella + * @since 3.0 + */ +public class BatchPropertyContext { + private static final String PARTITION_INDICATOR = ":partition"; + + private Properties jobProperties = new Properties(); + private Map stepProperties = new HashMap<>(); + private Map artifactProperties = new HashMap<>(); + private Map> stepArtifactProperties = new HashMap<>(); + + /** + *

      + * Obtains the Job level properties. + *

      + * + * @return the Job level properties + */ + public Properties getJobProperties() { + return jobProperties; + } + + /** + *

      + * Adds Job level properties to the context. + *

      + * + * @param properties the job {@link Properties} to add + */ + public void setJobProperties(Properties properties) { + Assert.notNull(properties, "Job properties cannot be null"); + this.jobProperties.putAll(properties); + } + + /** + *

      + * Obtains the Step level properties for the provided Step name. + *

      + * + * @param stepName the Step name to obtain properties for + * @return the {@link Properties} for the Step + */ + public Properties getStepProperties(String stepName) { + Assert.hasText(stepName, "Step name must be provided"); + Properties properties = new Properties(); + + if(stepProperties.containsKey(stepName)) { + properties.putAll(stepProperties.get(stepName)); + } + + if(stepName.contains(PARTITION_INDICATOR)) { + String parentStepName = stepName.substring(0, stepName.indexOf(PARTITION_INDICATOR)); + properties.putAll(getStepProperties(parentStepName)); + } + + return properties; + } + + /** + *

      + * Adds Step level properties to the context. + *

      + * + * @param properties the step {@link Properties} to add + */ + public void setStepProperties(Map properties) { + Assert.notNull(properties, "Step properties cannot be null"); + + for(Map.Entry propertiesEntry : properties.entrySet()) { + String stepName = propertiesEntry.getKey(); + Properties stepProperties = propertiesEntry.getValue(); + + if (!stepProperties.isEmpty()) { + if (this.stepProperties.containsKey(stepName)) { + Properties existingStepProperties = this.stepProperties.get(stepName); + + Enumeration stepPropertyNames = stepProperties.propertyNames(); + + while(stepPropertyNames.hasMoreElements()) { + String propertyEntryName = (String) stepPropertyNames.nextElement(); + existingStepProperties.put(propertyEntryName, stepProperties.getProperty(propertyEntryName)); + } + + this.stepProperties.put(stepName, existingStepProperties); + } else { + this.stepProperties.put(stepName, propertiesEntry.getValue()); + } + } + } + } + + /** + *

      + * Convenience method to set step level properties. Simply wraps the provided parameters + * and delegates to {@link #setStepProperties(java.util.Map)}. + *

      + * + * @param stepName the step name to set {@link Properties} for + * @param properties the {@link Properties} to set + */ + public void setStepProperties(String stepName, Properties properties) { + Assert.hasText(stepName, "Step name must be provided"); + Assert.notNull(properties, "Step properties must not be null"); + + Map stepProperties = new HashMap<>(); + stepProperties.put(stepName, properties); + + setStepProperties(stepProperties); + } + + /** + *

      + * Obtains the batch {@link Properties} for the provided artifact name. + *

      + * + * @param artifactName the batch artifact to obtain properties for + * @return the {@link Properties} for the provided batch artifact + */ + public Properties getArtifactProperties(String artifactName) { + Properties properties = new Properties(); + + if (artifactProperties.containsKey(artifactName)) { + properties.putAll(artifactProperties.get(artifactName)); + } + + return properties; + } + + /** + *

      + * Adds non-step artifact properties to the context. + *

      + * + * @param properties the artifact {@link Properties} to add + */ + public void setArtifactProperties(Map properties) { + Assert.notNull(properties, "Step properties cannot be null"); + + for(Map.Entry propertiesEntry : properties.entrySet()) { + String artifactName = propertiesEntry.getKey(); + Properties artifactProperties = propertiesEntry.getValue(); + + if(!artifactProperties.isEmpty()) { + this.artifactProperties.put(artifactName, artifactProperties); + } + } + } + + /** + *

      + * Obtains the batch {@link Properties} for the provided Step and artifact name. + *

      + * + * @param stepName the Step name the artifact is associated with + * @param artifactName the artifact name to obtain {@link Properties} for + * @return the {@link Properties} for the provided Step artifact + */ + public Properties getStepArtifactProperties(String stepName, String artifactName) { + Properties properties = new Properties(); + properties.putAll(getStepProperties(stepName)); + + Map artifactProperties = stepArtifactProperties.get(stepName); + + if (artifactProperties != null && artifactProperties.containsKey(artifactName)) { + properties.putAll(artifactProperties.get(artifactName)); + } + + if(stepName.contains(PARTITION_INDICATOR)) { + String parentStepName = stepName.substring(0, stepName.indexOf(PARTITION_INDICATOR)); + properties.putAll(getStepProperties(parentStepName)); + + Map parentArtifactProperties = stepArtifactProperties.get(parentStepName); + + if (parentArtifactProperties != null && parentArtifactProperties.containsKey(artifactName)) { + properties.putAll(parentArtifactProperties.get(artifactName)); + } + } + + return properties; + } + + /** + *

      + * Adds Step artifact properties to the context. + *

      + * + * @param properties the step artifact {@link Properties} to add + */ + @SuppressWarnings("serial") + public void setStepArtifactProperties(Map> properties) { + Assert.notNull(properties, "Step artifact properties cannot be null"); + + for(Map.Entry> propertyEntries : properties.entrySet()) { + String stepName = propertyEntries.getKey(); + + for(Map.Entry artifactEntries : propertyEntries.getValue().entrySet()) { + final String artifactName = artifactEntries.getKey(); + final Properties props = artifactEntries.getValue(); + + Map artifactProperties = stepArtifactProperties.get(stepName); + + if (artifactProperties == null) { + stepArtifactProperties.put(stepName, new HashMap() {{ + put(artifactName, props); + }}); + } else { + artifactProperties.put(artifactName, props); + } + } + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/JsrAutowiredAnnotationBeanPostProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/JsrAutowiredAnnotationBeanPostProcessor.java new file mode 100644 index 0000000000..2a888b6c24 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/JsrAutowiredAnnotationBeanPostProcessor.java @@ -0,0 +1,44 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.support; + +import java.lang.annotation.Annotation; +import java.lang.reflect.AccessibleObject; + +import javax.batch.api.BatchProperty; + +import org.springframework.beans.factory.annotation.InjectionMetadata; + +/** + *

      This class overrides methods in the copied {@link SpringAutowiredAnnotationBeanPostProcessor} class + * to check for the {@link BatchProperty} annotation before processing injection annotations. If the annotation + * is found, further injection processing for the field is skipped.

      + */ +public class JsrAutowiredAnnotationBeanPostProcessor extends SpringAutowiredAnnotationBeanPostProcessor { + @Override + protected InjectionMetadata findAutowiringMetadata(Class clazz) { + return super.buildAutowiringMetadata(clazz); + } + + @Override + protected Annotation findAutowiredAnnotation(AccessibleObject ao) { + if (ao.getAnnotation(BatchProperty.class) != null) { + return null; + } + + return super.findAutowiredAnnotation(ao); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/JsrExpressionParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/JsrExpressionParser.java new file mode 100644 index 0000000000..2097da0058 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/JsrExpressionParser.java @@ -0,0 +1,130 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.support; + +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import org.springframework.beans.factory.config.BeanExpressionContext; +import org.springframework.beans.factory.config.BeanExpressionResolver; +import org.springframework.util.StringUtils; + +/** + *

      + * Support class for parsing JSR-352 expressions. The JSR-352 expression syntax, for + * example conditional/elvis statements need to be transformed a bit to be valid SPeL expressions. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class JsrExpressionParser { + private static final String QUOTE = "'"; + private static final String NULL = "null"; + private static final String ELVIS_RHS = ":"; + private static final String ELVIS_LHS = "\\?"; + private static final String ELVIS_OPERATOR = "?:"; + private static final String EXPRESSION_SUFFIX = "}"; + private static final String EXPRESSION_PREFIX = "#{"; + private static final String DEFAULT_VALUE_SEPARATOR = ";"; + private static final Pattern CONDITIONAL_EXPRESSION = Pattern.compile("(((\\bnull\\b)|(#\\{\\w))[^;]+)"); + + private BeanExpressionContext beanExpressionContext; + private BeanExpressionResolver beanExpressionResolver; + + /** + *

      + * Creates a new instance of this expression parser without and expression resolver. Creating + * an instance via this constructor will still parse expressions but no resolution of operators + * will occur as its expected the caller will. + *

      + */ + public JsrExpressionParser() { } + + /** + *

      + * Creates a new instances of this expression parser with the provided expression resolver and context to evaluate + * against. + *

      + * + * @param beanExpressionResolver the expression resolver to use when resolving expressions + * @param beanExpressionContext the expression context to resolve expressions against + */ + public JsrExpressionParser(BeanExpressionResolver beanExpressionResolver, BeanExpressionContext beanExpressionContext) { + this.beanExpressionContext = beanExpressionContext; + this.beanExpressionResolver = beanExpressionResolver; + } + + /** + *

      + * Parses the provided expression, applying any transformations needed to evaluate as a SPeL expression. + *

      + * + * @param expression the expression to parse and transform + * @return a JSR-352 transformed expression that can be evaluated by a SPeL parser + */ + public String parseExpression(String expression) { + String expressionToParse = expression; + + if (StringUtils.countOccurrencesOf(expressionToParse, ELVIS_OPERATOR) > 0) { + expressionToParse = parseConditionalExpressions(expressionToParse); + } + + return evaluateExpression(expressionToParse); + } + + private String parseConditionalExpressions(String expression) { + String expressionToParse = expression; + + Matcher conditionalExpressionMatcher = CONDITIONAL_EXPRESSION.matcher(expressionToParse); + + while (conditionalExpressionMatcher.find()) { + String conditionalExpression = conditionalExpressionMatcher.group(1); + + String value = conditionalExpression.split(ELVIS_LHS)[0]; + String defaultValue = conditionalExpression.split(ELVIS_RHS)[1]; + + StringBuilder parsedExpression = new StringBuilder(); + + if(beanExpressionResolver != null) { + parsedExpression.append(EXPRESSION_PREFIX) + .append(evaluateExpression(value)) + .append(ELVIS_OPERATOR) + .append(QUOTE) + .append(evaluateExpression(defaultValue)) + .append(QUOTE) + .append(EXPRESSION_SUFFIX); + } else { + if(NULL.equals(value)) { + parsedExpression.append(defaultValue); + } else { + parsedExpression.append(value); + } + } + + expressionToParse = expressionToParse.replace(conditionalExpression, parsedExpression); + } + + return expressionToParse.replace(DEFAULT_VALUE_SEPARATOR, ""); + } + + private String evaluateExpression(String expression) { + if(beanExpressionResolver != null) { + return (String) beanExpressionResolver.evaluate(expression, beanExpressionContext); + } + + return expression; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/SpringAutowiredAnnotationBeanPostProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/SpringAutowiredAnnotationBeanPostProcessor.java new file mode 100644 index 0000000000..56a7f6eaf2 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/SpringAutowiredAnnotationBeanPostProcessor.java @@ -0,0 +1,603 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.support; + +import java.beans.PropertyDescriptor; +import java.lang.annotation.Annotation; +import java.lang.reflect.AccessibleObject; +import java.lang.reflect.Constructor; +import java.lang.reflect.Field; +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; +import java.lang.reflect.Modifier; +import java.util.ArrayList; +import java.util.Iterator; +import java.util.LinkedHashSet; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.beans.BeanUtils; +import org.springframework.beans.BeansException; +import org.springframework.beans.PropertyValues; +import org.springframework.beans.TypeConverter; +import org.springframework.beans.factory.BeanCreationException; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.beans.factory.BeanFactoryAware; +import org.springframework.beans.factory.BeanFactoryUtils; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.InjectionMetadata; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; +import org.springframework.beans.factory.config.DependencyDescriptor; +import org.springframework.beans.factory.config.InstantiationAwareBeanPostProcessorAdapter; +import org.springframework.beans.factory.config.RuntimeBeanReference; +import org.springframework.beans.factory.support.MergedBeanDefinitionPostProcessor; +import org.springframework.beans.factory.support.RootBeanDefinition; +import org.springframework.core.BridgeMethodResolver; +import org.springframework.core.GenericTypeResolver; +import org.springframework.core.MethodParameter; +import org.springframework.core.Ordered; +import org.springframework.core.PriorityOrdered; +import org.springframework.core.annotation.AnnotationUtils; +import org.springframework.util.Assert; +import org.springframework.util.ClassUtils; +import org.springframework.util.ReflectionUtils; + +/** + *

      This is a copy of AutowiredAnnotationBeanPostProcessor with modifications allow a subclass to + * do additional checks on other field annotations before processing injection annotations.

      + * + *

      This class is considered a quick work around and needs to be refactored / removed.

      + * + *

      The in addition to making this class package private, the following methods were modified to be protected:

      + *
        + *
      • findAutowiringMetadata(Class<?> clazz)
      • + *
      • buildAutowiringMetadata(Class<?> clazz)
      • + *
      • findAutowiredAnnotation(AccessibleObject ao)
      • + *
      + */ +class SpringAutowiredAnnotationBeanPostProcessor extends InstantiationAwareBeanPostProcessorAdapter + implements MergedBeanDefinitionPostProcessor, PriorityOrdered, BeanFactoryAware { + + protected final Log logger = LogFactory.getLog(getClass()); + + private final Set> autowiredAnnotationTypes = + new LinkedHashSet<>(); + + private String requiredParameterName = "required"; + + private boolean requiredParameterValue = true; + + private int order = Ordered.LOWEST_PRECEDENCE - 2; + + private ConfigurableListableBeanFactory beanFactory; + + private final Map, Constructor[]> candidateConstructorsCache = + new ConcurrentHashMap<>(64); + + private final Map, InjectionMetadata> injectionMetadataCache = + new ConcurrentHashMap<>(64); + + + /** + * Create a new AutowiredAnnotationBeanPostProcessor + * for Spring's standard {@link org.springframework.beans.factory.annotation.Autowired} annotation. + *

      Also supports JSR-330's {@link javax.inject.Inject} annotation, if available. + */ + @SuppressWarnings("unchecked") + public SpringAutowiredAnnotationBeanPostProcessor() { + this.autowiredAnnotationTypes.add(Autowired.class); + this.autowiredAnnotationTypes.add(Value.class); + ClassLoader cl = SpringAutowiredAnnotationBeanPostProcessor.class.getClassLoader(); + try { + this.autowiredAnnotationTypes.add((Class) cl.loadClass("javax.inject.Inject")); + logger.info("JSR-330 'javax.inject.Inject' annotation found and supported for autowiring"); + } + catch (ClassNotFoundException ex) { + // JSR-330 API not available - simply skip. + } + } + + + /** + * Set the 'autowired' annotation type, to be used on constructors, fields, + * setter methods and arbitrary config methods. + *

      The default autowired annotation type is the Spring-provided + * {@link Autowired} annotation, as well as {@link Value}. + *

      This setter property exists so that developers can provide their own + * (non-Spring-specific) annotation type to indicate that a member is + * supposed to be autowired. + * + * @param autowiredAnnotationType type to be used by constructors, fields and methods. + */ + public void setAutowiredAnnotationType(Class autowiredAnnotationType) { + Assert.notNull(autowiredAnnotationType, "'autowiredAnnotationType' must not be null"); + this.autowiredAnnotationTypes.clear(); + this.autowiredAnnotationTypes.add(autowiredAnnotationType); + } + + /** + * Set the 'autowired' annotation types, to be used on constructors, fields, + * setter methods and arbitrary config methods. + *

      The default autowired annotation type is the Spring-provided + * {@link Autowired} annotation, as well as {@link Value}. + *

      This setter property exists so that developers can provide their own + * (non-Spring-specific) annotation types to indicate that a member is + * supposed to be autowired. + + * @param autowiredAnnotationTypes set of types to be used by constructors, fields and methods. + */ + public void setAutowiredAnnotationTypes(Set> autowiredAnnotationTypes) { + Assert.notEmpty(autowiredAnnotationTypes, "'autowiredAnnotationTypes' must not be empty"); + this.autowiredAnnotationTypes.clear(); + this.autowiredAnnotationTypes.addAll(autowiredAnnotationTypes); + } + + /** + * Set the name of a parameter of the annotation that specifies + * whether it is required. + * + * @param requiredParameterName the name of the parameter. + * + * @see #setRequiredParameterValue(boolean) + */ + public void setRequiredParameterName(String requiredParameterName) { + this.requiredParameterName = requiredParameterName; + } + + /** + * Set the boolean value that marks a dependency as required + *

      For example if using 'required=true' (the default), + * this value should be true; but if using + * 'optional=false', this value should be false. + * + * @param requiredParameterValue true if dependency is required. + * + * @see #setRequiredParameterName(String) + */ + public void setRequiredParameterValue(boolean requiredParameterValue) { + this.requiredParameterValue = requiredParameterValue; + } + + public void setOrder(int order) { + this.order = order; + } + + @Override + public int getOrder() { + return this.order; + } + + @Override + public void setBeanFactory(BeanFactory beanFactory) throws BeansException { + if (!(beanFactory instanceof ConfigurableListableBeanFactory)) { + throw new IllegalArgumentException( + "AutowiredAnnotationBeanPostProcessor requires a ConfigurableListableBeanFactory"); + } + this.beanFactory = (ConfigurableListableBeanFactory) beanFactory; + } + + + @Override + public void postProcessMergedBeanDefinition(RootBeanDefinition beanDefinition, Class beanType, String beanName) { + if (beanType != null) { + InjectionMetadata metadata = findAutowiringMetadata(beanType); + metadata.checkConfigMembers(beanDefinition); + } + } + + @Override + public Constructor[] determineCandidateConstructors(Class beanClass, String beanName) throws BeansException { + // Quick check on the concurrent map first, with minimal locking. + Constructor[] candidateConstructors = this.candidateConstructorsCache.get(beanClass); + if (candidateConstructors == null) { + synchronized (this.candidateConstructorsCache) { + candidateConstructors = this.candidateConstructorsCache.get(beanClass); + if (candidateConstructors == null) { + Constructor[] rawCandidates = beanClass.getDeclaredConstructors(); + List> candidates = new ArrayList<>(rawCandidates.length); + Constructor requiredConstructor = null; + Constructor defaultConstructor = null; + for (Constructor candidate : rawCandidates) { + Annotation annotation = findAutowiredAnnotation(candidate); + if (annotation != null) { + if (requiredConstructor != null) { + throw new BeanCreationException("Invalid autowire-marked constructor: " + candidate + + ". Found another constructor with 'required' Autowired annotation: " + + requiredConstructor); + } + if (candidate.getParameterTypes().length == 0) { + throw new IllegalStateException( + "Autowired annotation requires at least one argument: " + candidate); + } + boolean required = determineRequiredStatus(annotation); + if (required) { + if (!candidates.isEmpty()) { + throw new BeanCreationException( + "Invalid autowire-marked constructors: " + candidates + + ". Found another constructor with 'required' Autowired annotation: " + + requiredConstructor); + } + requiredConstructor = candidate; + } + candidates.add(candidate); + } + else if (candidate.getParameterTypes().length == 0) { + defaultConstructor = candidate; + } + } + if (!candidates.isEmpty()) { + // Add default constructor to list of optional constructors, as fallback. + if (requiredConstructor == null && defaultConstructor != null) { + candidates.add(defaultConstructor); + } + candidateConstructors = candidates.toArray(new Constructor[candidates.size()]); + } + else { + candidateConstructors = new Constructor[0]; + } + this.candidateConstructorsCache.put(beanClass, candidateConstructors); + } + } + } + return (candidateConstructors.length > 0 ? candidateConstructors : null); + } + + @Override + public PropertyValues postProcessPropertyValues( + PropertyValues pvs, PropertyDescriptor[] pds, Object bean, String beanName) throws BeansException { + + InjectionMetadata metadata = findAutowiringMetadata(bean.getClass()); + try { + metadata.inject(bean, beanName, pvs); + } + catch (Throwable ex) { + throw new BeanCreationException(beanName, "Injection of autowired dependencies failed", ex); + } + return pvs; + } + + /** + * 'Native' processing method for direct calls with an arbitrary target instance, + * resolving all of its fields and methods which are annotated with @Autowired. + * @param bean the target instance to process + * @throws BeansException if autowiring failed + */ + public void processInjection(Object bean) throws BeansException { + Class clazz = bean.getClass(); + InjectionMetadata metadata = findAutowiringMetadata(clazz); + try { + metadata.inject(bean, null, null); + } + catch (Throwable ex) { + throw new BeanCreationException("Injection of autowired dependencies failed for class [" + clazz + "]", ex); + } + } + + + protected InjectionMetadata findAutowiringMetadata(Class clazz) { + // Quick check on the concurrent map first, with minimal locking. + InjectionMetadata metadata = this.injectionMetadataCache.get(clazz); + if (metadata == null) { + synchronized (this.injectionMetadataCache) { + metadata = this.injectionMetadataCache.get(clazz); + if (metadata == null) { + metadata = buildAutowiringMetadata(clazz); + this.injectionMetadataCache.put(clazz, metadata); + } + } + } + return metadata; + } + + protected InjectionMetadata buildAutowiringMetadata(Class clazz) { + LinkedList elements = new LinkedList<>(); + Class targetClass = clazz; + + do { + LinkedList currElements = new LinkedList<>(); + for (Field field : targetClass.getDeclaredFields()) { + Annotation annotation = findAutowiredAnnotation(field); + if (annotation != null) { + if (Modifier.isStatic(field.getModifiers())) { + if (logger.isWarnEnabled()) { + logger.warn("Autowired annotation is not supported on static fields: " + field); + } + continue; + } + boolean required = determineRequiredStatus(annotation); + currElements.add(new AutowiredFieldElement(field, required)); + } + } + for (Method method : targetClass.getDeclaredMethods()) { + Method bridgedMethod = BridgeMethodResolver.findBridgedMethod(method); + Annotation annotation = BridgeMethodResolver.isVisibilityBridgeMethodPair(method, bridgedMethod) ? + findAutowiredAnnotation(bridgedMethod) : findAutowiredAnnotation(method); + if (annotation != null && method.equals(ClassUtils.getMostSpecificMethod(method, clazz))) { + if (Modifier.isStatic(method.getModifiers())) { + if (logger.isWarnEnabled()) { + logger.warn("Autowired annotation is not supported on static methods: " + method); + } + continue; + } + if (method.getParameterTypes().length == 0) { + if (logger.isWarnEnabled()) { + logger.warn("Autowired annotation should be used on methods with actual parameters: " + method); + } + } + boolean required = determineRequiredStatus(annotation); + PropertyDescriptor pd = BeanUtils.findPropertyForMethod(method); + currElements.add(new AutowiredMethodElement(method, required, pd)); + } + } + elements.addAll(0, currElements); + targetClass = targetClass.getSuperclass(); + } + while (targetClass != null && targetClass != Object.class); + + return new InjectionMetadata(clazz, elements); + } + + protected Annotation findAutowiredAnnotation(AccessibleObject ao) { + for (Class type : this.autowiredAnnotationTypes) { + Annotation annotation = AnnotationUtils.getAnnotation(ao, type); + if (annotation != null) { + return annotation; + } + } + return null; + } + + /** + * Obtain all beans of the given type as autowire candidates. + * + * @param type the type of the bean. + * @param the type of the bean. + * @return the target beans, or an empty Collection if no bean of this type is found + * + * @throws BeansException if bean retrieval failed + */ + protected Map findAutowireCandidates(Class type) throws BeansException { + if (this.beanFactory == null) { + throw new IllegalStateException("No BeanFactory configured - " + + "override the getBeanOfType method or specify the 'beanFactory' property"); + } + return BeanFactoryUtils.beansOfTypeIncludingAncestors(this.beanFactory, type); + } + + /** + * Determine if the annotated field or method requires its dependency. + *

      A 'required' dependency means that autowiring should fail when no beans + * are found. Otherwise, the autowiring process will simply bypass the field + * or method when no beans are found. + * @param annotation the Autowired annotation + * @return whether the annotation indicates that a dependency is required + */ + protected boolean determineRequiredStatus(Annotation annotation) { + try { + Method method = ReflectionUtils.findMethod(annotation.annotationType(), this.requiredParameterName); + if (method == null) { + // annotations like @Inject and @Value don't have a method (attribute) named "required" + // -> default to required status + return true; + } + return (this.requiredParameterValue == (Boolean) ReflectionUtils.invokeMethod(method, annotation)); + } + catch (Exception ex) { + // an exception was thrown during reflective invocation of the required attribute + // -> default to required status + return true; + } + } + + /** + * Register the specified bean as dependent on the autowired beans. + */ + private void registerDependentBeans(String beanName, Set autowiredBeanNames) { + if (beanName != null) { + for (String autowiredBeanName : autowiredBeanNames) { + if (this.beanFactory.containsBean(autowiredBeanName)) { + this.beanFactory.registerDependentBean(autowiredBeanName, beanName); + } + if (logger.isDebugEnabled()) { + logger.debug("Autowiring by type from bean name '" + beanName + + "' to bean named '" + autowiredBeanName + "'"); + } + } + } + } + + /** + * Resolve the specified cached method argument or field value. + */ + private Object resolvedCachedArgument(String beanName, Object cachedArgument) { + if (cachedArgument instanceof DependencyDescriptor) { + DependencyDescriptor descriptor = (DependencyDescriptor) cachedArgument; + TypeConverter typeConverter = this.beanFactory.getTypeConverter(); + return this.beanFactory.resolveDependency(descriptor, beanName, null, typeConverter); + } + else if (cachedArgument instanceof RuntimeBeanReference) { + return this.beanFactory.getBean(((RuntimeBeanReference) cachedArgument).getBeanName()); + } + else { + return cachedArgument; + } + } + + + /** + * Class representing injection information about an annotated field. + */ + private class AutowiredFieldElement extends InjectionMetadata.InjectedElement { + + private final boolean required; + + private volatile boolean cached = false; + + private volatile Object cachedFieldValue; + + public AutowiredFieldElement(Field field, boolean required) { + super(field, null); + this.required = required; + } + + @Override + protected void inject(Object bean, String beanName, PropertyValues pvs) throws Throwable { + Field field = (Field) this.member; + try { + Object value; + if (this.cached) { + value = resolvedCachedArgument(beanName, this.cachedFieldValue); + } + else { + DependencyDescriptor descriptor = new DependencyDescriptor(field, this.required); + Set autowiredBeanNames = new LinkedHashSet<>(1); + TypeConverter typeConverter = beanFactory.getTypeConverter(); + value = beanFactory.resolveDependency(descriptor, beanName, autowiredBeanNames, typeConverter); + synchronized (this) { + if (!this.cached) { + if (value != null || this.required) { + this.cachedFieldValue = descriptor; + registerDependentBeans(beanName, autowiredBeanNames); + if (autowiredBeanNames.size() == 1) { + String autowiredBeanName = autowiredBeanNames.iterator().next(); + if (beanFactory.containsBean(autowiredBeanName)) { + if (beanFactory.isTypeMatch(autowiredBeanName, field.getType())) { + this.cachedFieldValue = new RuntimeBeanReference(autowiredBeanName); + } + } + } + } + else { + this.cachedFieldValue = null; + } + this.cached = true; + } + } + } + if (value != null) { + ReflectionUtils.makeAccessible(field); + field.set(bean, value); + } + } + catch (Throwable ex) { + throw new BeanCreationException("Could not autowire field: " + field, ex); + } + } + } + + + /** + * Class representing injection information about an annotated method. + */ + private class AutowiredMethodElement extends InjectionMetadata.InjectedElement { + + private final boolean required; + + private volatile boolean cached = false; + + private volatile Object[] cachedMethodArguments; + + public AutowiredMethodElement(Method method, boolean required, PropertyDescriptor pd) { + super(method, pd); + this.required = required; + } + + @Override + protected void inject(Object bean, String beanName, PropertyValues pvs) throws Throwable { + if (checkPropertySkipping(pvs)) { + return; + } + Method method = (Method) this.member; + try { + Object[] arguments; + if (this.cached) { + // Shortcut for avoiding synchronization... + arguments = resolveCachedArguments(beanName); + } + else { + Class[] paramTypes = method.getParameterTypes(); + arguments = new Object[paramTypes.length]; + DependencyDescriptor[] descriptors = new DependencyDescriptor[paramTypes.length]; + Set autowiredBeanNames = new LinkedHashSet<>(paramTypes.length); + TypeConverter typeConverter = beanFactory.getTypeConverter(); + for (int i = 0; i < arguments.length; i++) { + MethodParameter methodParam = new MethodParameter(method, i); + GenericTypeResolver.resolveParameterType(methodParam, bean.getClass()); + descriptors[i] = new DependencyDescriptor(methodParam, this.required); + arguments[i] = beanFactory.resolveDependency( + descriptors[i], beanName, autowiredBeanNames, typeConverter); + if (arguments[i] == null && !this.required) { + arguments = null; + break; + } + } + synchronized (this) { + if (!this.cached) { + if (arguments != null) { + this.cachedMethodArguments = new Object[arguments.length]; + for (int i = 0; i < arguments.length; i++) { + this.cachedMethodArguments[i] = descriptors[i]; + } + registerDependentBeans(beanName, autowiredBeanNames); + if (autowiredBeanNames.size() == paramTypes.length) { + Iterator it = autowiredBeanNames.iterator(); + for (int i = 0; i < paramTypes.length; i++) { + String autowiredBeanName = it.next(); + if (beanFactory.containsBean(autowiredBeanName)) { + if (beanFactory.isTypeMatch(autowiredBeanName, paramTypes[i])) { + this.cachedMethodArguments[i] = new RuntimeBeanReference(autowiredBeanName); + } + } + } + } + } + else { + this.cachedMethodArguments = null; + } + this.cached = true; + } + } + } + if (arguments != null) { + ReflectionUtils.makeAccessible(method); + method.invoke(bean, arguments); + } + } + catch (InvocationTargetException ex) { + throw ex.getTargetException(); + } + catch (Throwable ex) { + throw new BeanCreationException("Could not autowire method: " + method, ex); + } + } + + private Object[] resolveCachedArguments(String beanName) { + if (this.cachedMethodArguments == null) { + return null; + } + Object[] arguments = new Object[this.cachedMethodArguments.length]; + for (int i = 0; i < arguments.length; i++) { + arguments[i] = resolvedCachedArgument(beanName, this.cachedMethodArguments[i]); + } + return arguments; + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/ThreadLocalClassloaderBeanPostProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/ThreadLocalClassloaderBeanPostProcessor.java new file mode 100644 index 0000000000..22464fad6a --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/ThreadLocalClassloaderBeanPostProcessor.java @@ -0,0 +1,74 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.support; + +import org.springframework.beans.BeansException; +import org.springframework.beans.PropertyValue; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.config.BeanFactoryPostProcessor; +import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; +import org.springframework.beans.factory.config.RuntimeBeanReference; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.support.DefaultListableBeanFactory; +import org.springframework.core.PriorityOrdered; + +/** + * After the {@link BeanFactory} is created, this post processor will evaluate to see + * if any of the beans referenced from a job definition (as defined by JSR-352) point + * to class names instead of bean names. If this is the case, a new {@link BeanDefinition} + * is added with the name of the class as the bean name. + * + * @author Michael Minella + * @since 3.0 + */ +public class ThreadLocalClassloaderBeanPostProcessor implements BeanFactoryPostProcessor, PriorityOrdered { + /* (non-Javadoc) + * @see org.springframework.beans.factory.config.BeanFactoryPostProcessor#postProcessBeanFactory(org.springframework.beans.factory.config.ConfigurableListableBeanFactory) + */ + @Override + public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException { + String[] beanNames = beanFactory.getBeanDefinitionNames(); + + for (String curName : beanNames) { + BeanDefinition beanDefinition = beanFactory.getBeanDefinition(curName); + PropertyValue[] values = beanDefinition.getPropertyValues().getPropertyValues(); + + for (PropertyValue propertyValue : values) { + Object value = propertyValue.getValue(); + + if(value instanceof RuntimeBeanReference) { + RuntimeBeanReference ref = (RuntimeBeanReference) value; + if(!beanFactory.containsBean(ref.getBeanName())) { + AbstractBeanDefinition newBeanDefinition = BeanDefinitionBuilder.genericBeanDefinition(ref.getBeanName()).getBeanDefinition(); + newBeanDefinition.setScope("step"); + ((DefaultListableBeanFactory) beanFactory).registerBeanDefinition(ref.getBeanName(), newBeanDefinition); + } + } + } + } + } + + /** + * Sets this {@link BeanFactoryPostProcessor} to the lowest precedence so that + * it is executed as late as possible in the chain of {@link BeanFactoryPostProcessor}s + */ + @Override + public int getOrder() { + return PriorityOrdered.LOWEST_PRECEDENCE; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/package-info.java new file mode 100644 index 0000000000..bd359dd441 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/support/package-info.java @@ -0,0 +1,10 @@ +/** + * Extensions of Spring components to support JSR-352 functionality. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.configuration.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/BatchParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/BatchParser.java new file mode 100644 index 0000000000..e09e79b042 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/BatchParser.java @@ -0,0 +1,80 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.support.BeanDefinitionRegistry; +import org.springframework.beans.factory.xml.AbstractBeanDefinitionParser; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.xml.DomUtils; +import org.w3c.dom.Element; + +/** + * Parser used to parse the batch.xml file as defined in JSR-352. It is not + * recommended to use the batch.xml approach with Spring to manage bean instantiation. + * It is recommended that standard Spring bean configurations (via XML or Java Config) + * be used. + * + * @author Michael Minella + * @since 3.0 + */ +public class BatchParser extends AbstractBeanDefinitionParser { + + private static final Log logger = LogFactory.getLog(BatchParser.class); + + @Override + protected boolean shouldGenerateIdAsFallback() { + return true; + } + + @Override + protected AbstractBeanDefinition parseInternal(Element element, + ParserContext parserContext) { + BeanDefinitionRegistry registry = parserContext.getRegistry(); + + parseRefElements(element, registry); + + return null; + } + + private void parseRefElements(Element element, + BeanDefinitionRegistry registry) { + List beanElements = DomUtils.getChildElementsByTagName(element, "ref"); + + if(beanElements.size() > 0) { + for (Element curElement : beanElements) { + AbstractBeanDefinition beanDefinition = BeanDefinitionBuilder.genericBeanDefinition(curElement.getAttribute("class")) + .getBeanDefinition(); + + beanDefinition.setScope("step"); + + String beanName = curElement.getAttribute("id"); + + if(!registry.containsBeanDefinition(beanName)) { + registry.registerBeanDefinition(beanName, beanDefinition); + } else { + logger.info("Ignoring batch.xml bean definition for " + beanName + " because another bean of the same name has been registered"); + } + } + } + + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/BatchletParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/BatchletParser.java new file mode 100644 index 0000000000..953449ff93 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/BatchletParser.java @@ -0,0 +1,55 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.springframework.batch.core.jsr.configuration.support.BatchArtifactType; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.config.RuntimeBeanReference; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.xml.AbstractSingleBeanDefinitionParser; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.StringUtils; +import org.w3c.dom.Element; + +/** + * Parser for the <batchlet /> tag defined in JSR-352. The current state + * of this parser parses a batchlet element into a {@link Tasklet} (the ref + * attribute is expected to point to an implementation of Tasklet). + * + * @author Michael Minella + * @author Chris Schaefer + * @since 3.0 + */ +public class BatchletParser extends AbstractSingleBeanDefinitionParser { + private static final String REF = "ref"; + + public void parseBatchlet(Element batchletElement, AbstractBeanDefinition bd, ParserContext parserContext, String stepName) { + bd.setBeanClass(StepFactoryBean.class); + bd.setAttribute("isNamespaceStep", false); + + String taskletRef = batchletElement.getAttribute(REF); + + if (StringUtils.hasText(taskletRef)) { + bd.getPropertyValues().addPropertyValue("stepTasklet", new RuntimeBeanReference(taskletRef)); + } + + bd.setRole(BeanDefinition.ROLE_SUPPORT); + bd.setSource(parserContext.extractSource(batchletElement)); + + new PropertyParser(taskletRef, parserContext, BatchArtifactType.STEP_ARTIFACT, stepName).parseProperties(batchletElement); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/ChunkParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/ChunkParser.java new file mode 100644 index 0000000000..9f12c65d51 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/ChunkParser.java @@ -0,0 +1,186 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.util.List; + +import org.springframework.batch.core.configuration.xml.ExceptionElementParser; +import org.springframework.batch.core.jsr.configuration.support.BatchArtifactType; +import org.springframework.batch.core.step.item.ChunkOrientedTasklet; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemWriter; +import org.springframework.beans.MutablePropertyValues; +import org.springframework.beans.factory.config.RuntimeBeanReference; +import org.springframework.beans.factory.config.TypedStringValue; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.ManagedList; +import org.springframework.beans.factory.support.ManagedMap; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.StringUtils; +import org.springframework.util.xml.DomUtils; +import org.w3c.dom.Element; +import org.w3c.dom.Node; +import org.w3c.dom.NodeList; + +/** + * Parser for the <chunk /> element as specified in JSR-352. The current state + * parses a chunk element into it's related batch artifacts ({@link ChunkOrientedTasklet}, {@link ItemReader}, + * {@link ItemProcessor}, and {@link ItemWriter}). + * + * @author Michael Minella + * @author Chris Schaefer + * @since 3.0 + * + */ +public class ChunkParser { + private static final String TIME_LIMIT_ATTRIBUTE = "time-limit"; + private static final String ITEM_COUNT_ATTRIBUTE = "item-count"; + private static final String CHECKPOINT_ALGORITHM_ELEMENT = "checkpoint-algorithm"; + private static final String CLASS_ATTRIBUTE = "class"; + private static final String INCLUDE_ELEMENT = "include"; + private static final String NO_ROLLBACK_EXCEPTION_CLASSES_ELEMENT = "no-rollback-exception-classes"; + private static final String RETRYABLE_EXCEPTION_CLASSES_ELEMENT = "retryable-exception-classes"; + private static final String SKIPPABLE_EXCEPTION_CLASSES_ELEMENT = "skippable-exception-classes"; + private static final String WRITER_ELEMENT = "writer"; + private static final String PROCESSOR_ELEMENT = "processor"; + private static final String READER_ELEMENT = "reader"; + private static final String REF_ATTRIBUTE = "ref"; + private static final String RETRY_LIMIT_ATTRIBUTE = "retry-limit"; + private static final String SKIP_LIMIT_ATTRIBUTE = "skip-limit"; + private static final String CUSTOM_CHECKPOINT_POLICY = "custom"; + private static final String ITEM_CHECKPOINT_POLICY = "item"; + private static final String CHECKPOINT_POLICY_ATTRIBUTE = "checkpoint-policy"; + + public void parse(Element element, AbstractBeanDefinition bd, ParserContext parserContext, String stepName) { + MutablePropertyValues propertyValues = bd.getPropertyValues(); + bd.setBeanClass(StepFactoryBean.class); + bd.setAttribute("isNamespaceStep", false); + + propertyValues.addPropertyValue("hasChunkElement", Boolean.TRUE); + + String checkpointPolicy = element.getAttribute(CHECKPOINT_POLICY_ATTRIBUTE); + if(StringUtils.hasText(checkpointPolicy)) { + if(checkpointPolicy.equals(ITEM_CHECKPOINT_POLICY)) { + String itemCount = element.getAttribute(ITEM_COUNT_ATTRIBUTE); + if (StringUtils.hasText(itemCount)) { + propertyValues.addPropertyValue("commitInterval", itemCount); + } else { + propertyValues.addPropertyValue("commitInterval", "10"); + } + + parseSimpleAttribute(element, propertyValues, TIME_LIMIT_ATTRIBUTE, "timeout"); + } else if(checkpointPolicy.equals(CUSTOM_CHECKPOINT_POLICY)) { + parseCustomCheckpointAlgorithm(element, parserContext, propertyValues, stepName); + } + } else { + String itemCount = element.getAttribute(ITEM_COUNT_ATTRIBUTE); + if (StringUtils.hasText(itemCount)) { + propertyValues.addPropertyValue("commitInterval", itemCount); + } else { + propertyValues.addPropertyValue("commitInterval", "10"); + } + + parseSimpleAttribute(element, propertyValues, TIME_LIMIT_ATTRIBUTE, "timeout"); + } + + parseSimpleAttribute(element, propertyValues, SKIP_LIMIT_ATTRIBUTE, "skipLimit"); + parseSimpleAttribute(element, propertyValues, RETRY_LIMIT_ATTRIBUTE, "retryLimit"); + + NodeList children = element.getChildNodes(); + for (int i = 0; i < children.getLength(); i++) { + Node nd = children.item(i); + + parseChildElement(element, parserContext, propertyValues, nd, stepName); + } + } + + private void parseSimpleAttribute(Element element, + MutablePropertyValues propertyValues, String attributeName, String propertyName) { + String propertyValue = element.getAttribute(attributeName); + if (StringUtils.hasText(propertyValue)) { + propertyValues.addPropertyValue(propertyName, propertyValue); + } + } + + private void parseChildElement(Element element, ParserContext parserContext, + MutablePropertyValues propertyValues, Node nd, String stepName) { + if (nd instanceof Element) { + Element nestedElement = (Element) nd; + String name = nestedElement.getLocalName(); + String artifactName = nestedElement.getAttribute(REF_ATTRIBUTE); + + if(name.equals(READER_ELEMENT)) { + if (StringUtils.hasText(artifactName)) { + propertyValues.addPropertyValue("stepItemReader", new RuntimeBeanReference(artifactName)); + } + + new PropertyParser(artifactName, parserContext, BatchArtifactType.STEP_ARTIFACT, stepName).parseProperties(nestedElement); + } else if(name.equals(PROCESSOR_ELEMENT)) { + if (StringUtils.hasText(artifactName)) { + propertyValues.addPropertyValue("stepItemProcessor", new RuntimeBeanReference(artifactName)); + } + + new PropertyParser(artifactName, parserContext, BatchArtifactType.STEP_ARTIFACT, stepName).parseProperties(nestedElement); + } else if(name.equals(WRITER_ELEMENT)) { + if (StringUtils.hasText(artifactName)) { + propertyValues.addPropertyValue("stepItemWriter", new RuntimeBeanReference(artifactName)); + } + + new PropertyParser(artifactName, parserContext, BatchArtifactType.STEP_ARTIFACT, stepName).parseProperties(nestedElement); + } else if(name.equals(SKIPPABLE_EXCEPTION_CLASSES_ELEMENT)) { + ManagedMap exceptionClasses = new ExceptionElementParser().parse(element, parserContext, SKIPPABLE_EXCEPTION_CLASSES_ELEMENT); + if(exceptionClasses != null) { + propertyValues.addPropertyValue("skippableExceptionClasses", exceptionClasses); + } + } else if(name.equals(RETRYABLE_EXCEPTION_CLASSES_ELEMENT)) { + ManagedMap exceptionClasses = new ExceptionElementParser().parse(element, parserContext, RETRYABLE_EXCEPTION_CLASSES_ELEMENT); + if(exceptionClasses != null) { + propertyValues.addPropertyValue("retryableExceptionClasses", exceptionClasses); + } + } else if(name.equals(NO_ROLLBACK_EXCEPTION_CLASSES_ELEMENT)) { + //TODO: Update to support excludes + ManagedList list = new ManagedList<>(); + + for (Element child : DomUtils.getChildElementsByTagName(nestedElement, INCLUDE_ELEMENT)) { + String className = child.getAttribute(CLASS_ATTRIBUTE); + list.add(new TypedStringValue(className, Class.class)); + } + + propertyValues.addPropertyValue("noRollbackExceptionClasses", list); + } + } + } + + private void parseCustomCheckpointAlgorithm(Element element, ParserContext parserContext, MutablePropertyValues propertyValues, String stepName) { + List elements = DomUtils.getChildElementsByTagName(element, CHECKPOINT_ALGORITHM_ELEMENT); + + if(elements.size() == 1) { + Element checkpointAlgorithmElement = elements.get(0); + + String name = checkpointAlgorithmElement.getAttribute(REF_ATTRIBUTE); + if(StringUtils.hasText(name)) { + propertyValues.addPropertyValue("stepChunkCompletionPolicy", new RuntimeBeanReference(name)); + } + + new PropertyParser(name, parserContext, BatchArtifactType.STEP_ARTIFACT, stepName).parseProperties(checkpointAlgorithmElement); + } else if(elements.size() > 1){ + parserContext.getReaderContext().error( + "The element may not appear more than once in a single <" + + element.getNodeName() + "/>.", element); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/DecisionStepFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/DecisionStepFactoryBean.java new file mode 100644 index 0000000000..83150128f7 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/DecisionStepFactoryBean.java @@ -0,0 +1,98 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import javax.batch.api.Decider; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.jsr.step.DecisionStep; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.beans.factory.FactoryBean; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.util.Assert; + +/** + * {@link FactoryBean} for creating a {@link DecisionStep}. + * + * @author Michael Minella + * @since 3.0 + */ +public class DecisionStepFactoryBean implements FactoryBean, InitializingBean { + + private Decider jsrDecider; + private String name; + private JobRepository jobRepository; + + /** + * @param jobRepository All steps need to be able to reference a {@link JobRepository} + */ + public void setJobRepository(JobRepository jobRepository) { + this.jobRepository = jobRepository; + } + + /** + * @param decider a {@link Decider} + * @throws IllegalArgumentException if the type passed in is not a valid type + */ + public void setDecider(Decider decider) { + this.jsrDecider = decider; + } + + /** + * The name of the state + * + * @param name the name to be used by the DecisionStep. + */ + public void setName(String name) { + this.name = name; + } + + /* (non-Javadoc) + * @see org.springframework.beans.factory.FactoryBean#getObject() + */ + @Override + public Step getObject() throws Exception { + + DecisionStep decisionStep = new DecisionStep(jsrDecider); + decisionStep.setName(name); + decisionStep.setJobRepository(jobRepository); + decisionStep.setAllowStartIfComplete(true); + + return decisionStep; + } + + /* (non-Javadoc) + * @see org.springframework.beans.factory.FactoryBean#getObjectType() + */ + @Override + public Class getObjectType() { + return DecisionStep.class; + } + + /* (non-Javadoc) + * @see org.springframework.beans.factory.FactoryBean#isSingleton() + */ + @Override + public boolean isSingleton() { + return true; + } + + @Override + public void afterPropertiesSet() throws Exception { + Assert.isTrue(jsrDecider != null, "A decider implementation is required"); + Assert.notNull(name, "A name is required for a decision state"); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/FlowParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/FlowParser.java new file mode 100644 index 0000000000..1b4fe3341d --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/FlowParser.java @@ -0,0 +1,275 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import org.springframework.batch.core.configuration.xml.AbstractFlowParser; +import org.springframework.batch.core.job.flow.FlowExecutionStatus; +import org.springframework.batch.core.jsr.job.flow.support.JsrFlow; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.support.ManagedList; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.StringUtils; +import org.springframework.util.xml.DomUtils; +import org.w3c.dom.Element; +import org.w3c.dom.Node; +import org.w3c.dom.NodeList; + +/** + * Parses flows as defined in JSR-352. The current state parses a flow + * as it is within a regular Spring Batch job/flow. + * + * @author Michael Minella + * @author Chris Schaefer + * @since 3.0 + */ +public class FlowParser extends AbstractFlowParser { + private static final String NEXT_ATTRIBUTE = "next"; + private static final String EXIT_STATUS_ATTRIBUTE = "exit-status"; + private static final List TRANSITION_TYPES = new ArrayList<>(); + + static { + TRANSITION_TYPES.add(NEXT_ELE); + TRANSITION_TYPES.add(STOP_ELE); + TRANSITION_TYPES.add(END_ELE); + TRANSITION_TYPES.add(FAIL_ELE); + } + + private String flowName; + private String jobFactoryRef; + private StepParser stepParser = new StepParser(); + + /** + * @param flowName The name of the flow + * @param jobFactoryRef The bean name for the job factory + */ + public FlowParser(String flowName, String jobFactoryRef) { + super.setJobFactoryRef(jobFactoryRef); + this.jobFactoryRef = jobFactoryRef; + this.flowName = flowName; + } + + @Override + protected Class getBeanClass(Element element) { + return JsrFlowFactoryBean.class; + } + + @Override + protected void doParse(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { + builder.getRawBeanDefinition().setAttribute("flowName", flowName); + builder.addPropertyValue("name", flowName); + builder.addPropertyValue("flowType", JsrFlow.class); + + List stateTransitions = new ArrayList<>(); + + Map> reachableElementMap = new HashMap<>(); + String startElement = null; + NodeList children = element.getChildNodes(); + for (int i = 0; i < children.getLength(); i++) { + Node node = children.item(i); + if (node instanceof Element) { + String nodeName = node.getLocalName(); + Element child = (Element) node; + if (nodeName.equals(STEP_ELE)) { + stateTransitions.addAll(stepParser.parse(child, parserContext, builder)); + } else if(nodeName.equals(SPLIT_ELE)) { + stateTransitions.addAll(new JsrSplitParser(flowName).parse(child, parserContext)); + } else if(nodeName.equals(DECISION_ELE)) { + stateTransitions.addAll(new JsrDecisionParser().parse(child, parserContext, flowName)); + } else if(nodeName.equals(FLOW_ELE)) { + stateTransitions.addAll(parseFlow(child, parserContext, builder)); + } + } + } + + Set allReachableElements = new HashSet<>(); + findAllReachableElements(startElement, reachableElementMap, allReachableElements); + for (String elementId : reachableElementMap.keySet()) { + if (!allReachableElements.contains(elementId)) { + parserContext.getReaderContext().error("The element [" + elementId + "] is unreachable", element); + } + } + + ManagedList managedList = new ManagedList<>(); + managedList.addAll(stateTransitions); + builder.addPropertyValue("stateTransitions", managedList); + } + + private Collection parseFlow(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { + String idAttribute = element.getAttribute(ID_ATTRIBUTE); + + BeanDefinitionBuilder stateBuilder = BeanDefinitionBuilder + .genericBeanDefinition("org.springframework.batch.core.job.flow.support.state.FlowState"); + + FlowParser flowParser = new FlowParser(idAttribute, jobFactoryRef); + + stateBuilder.addConstructorArgValue(flowParser.parse(element, parserContext)); + stateBuilder.addConstructorArgValue(idAttribute); + + builder.getRawBeanDefinition().setAttribute("flowName", idAttribute); + builder.addPropertyValue("name", idAttribute); + + doParse(element, parserContext, builder); + builder.setRole(BeanDefinition.ROLE_INFRASTRUCTURE); + + return FlowParser.getNextElements(parserContext, null, stateBuilder.getBeanDefinition(), element); + } + + public static Collection getNextElements(ParserContext parserContext, BeanDefinition stateDef, + Element element) { + return getNextElements(parserContext, null, stateDef, element); + } + + public static Collection getNextElements(ParserContext parserContext, String stepId, + BeanDefinition stateDef, Element element) { + + Collection list = new ArrayList<>(); + + boolean transitionElementExists = false; + boolean failedTransitionElementExists = false; + + List childElements = DomUtils.getChildElements(element); + for(Element childElement : childElements) { + if(isChildElementTransitionElement(childElement)) { + list.addAll(parseTransitionElement(childElement, stepId, stateDef, parserContext)); + failedTransitionElementExists = failedTransitionElementExists || hasFailedTransitionElement(childElement); + transitionElementExists = true; + } + } + + String shortNextAttribute = element.getAttribute(NEXT_ATTRIBUTE); + boolean hasNextAttribute = StringUtils.hasText(shortNextAttribute); + + if (!transitionElementExists) { + list.addAll(createTransition(FlowExecutionStatus.FAILED, FlowExecutionStatus.FAILED.getName(), null, null, + stateDef, parserContext, false)); + list.addAll(createTransition(FlowExecutionStatus.UNKNOWN, FlowExecutionStatus.UNKNOWN.getName(), null, null, + stateDef, parserContext, false)); + } + + if (hasNextAttribute) { + if (transitionElementExists && !failedTransitionElementExists) { + list.addAll(createTransition(FlowExecutionStatus.FAILED, FlowExecutionStatus.FAILED.getName(), null, null, + stateDef, parserContext, false)); + } + + list.add(getStateTransitionReference(parserContext, stateDef, null, shortNextAttribute)); + } else { + list.addAll(createTransition(FlowExecutionStatus.COMPLETED, FlowExecutionStatus.COMPLETED.getName(), null, null, stateDef, parserContext, + false)); + } + + return list; + } + + private static boolean isChildElementTransitionElement(Element childElement) { + return TRANSITION_TYPES.contains(childElement.getLocalName()); + } + + private static boolean hasFailedTransitionElement(Element childName) { + return FAIL_ELE.equals(childName.getLocalName()); + } + + protected static Collection parseTransitionElement(Element transitionElement, String stateId, + BeanDefinition stateDef, ParserContext parserContext) { + FlowExecutionStatus status = getBatchStatusFromEndTransitionName(transitionElement.getNodeName()); + String onAttribute = transitionElement.getAttribute(ON_ATTR); + String restartAttribute = transitionElement.getAttribute(RESTART_ATTR); + String nextAttribute = transitionElement.getAttribute(TO_ATTR); + + if (!StringUtils.hasText(nextAttribute)) { + nextAttribute = restartAttribute; + } + String exitCodeAttribute = transitionElement.getAttribute(EXIT_STATUS_ATTRIBUTE); + + return createTransition(status, onAttribute, nextAttribute, restartAttribute, exitCodeAttribute, stateDef, parserContext, false); + } + + /** + * @param status The batch status that this transition will set. Use + * BatchStatus.UNKNOWN if not applicable. + * @param on The pattern that this transition should match. Use null for + * "no restriction" (same as "*"). + * @param next The state to which this transition should go. Use null if not + * applicable. + * @param restart The restart attribute this transition will set. + * @param exitCode The exit code that this transition will set. Use null to + * default to batchStatus. + * @param stateDef The bean definition for the current state + * @param parserContext the parser context for the bean factory + * @param abandon the abandon state this transition will set. + * @return a collection of + * {@link org.springframework.batch.core.job.flow.support.StateTransition} + * references + */ + protected static Collection createTransition(FlowExecutionStatus status, String on, String next, + String restart, String exitCode, BeanDefinition stateDef, ParserContext parserContext, boolean abandon) { + + BeanDefinition endState = null; + + if (status.isEnd()) { + + BeanDefinitionBuilder endBuilder = BeanDefinitionBuilder + .genericBeanDefinition("org.springframework.batch.core.jsr.job.flow.support.state.JsrEndState"); + + boolean exitCodeExists = StringUtils.hasText(exitCode); + + endBuilder.addConstructorArgValue(status); + + endBuilder.addConstructorArgValue(exitCodeExists ? exitCode : status.getName()); + + String endName = (status == FlowExecutionStatus.STOPPED ? STOP_ELE + : status == FlowExecutionStatus.FAILED ? FAIL_ELE : END_ELE) + + (endCounter++); + endBuilder.addConstructorArgValue(endName); + + endBuilder.addConstructorArgValue(restart); + + endBuilder.addConstructorArgValue(abandon); + + endBuilder.addConstructorArgReference("jobRepository"); + + String nextOnEnd = exitCodeExists ? null : next; + endState = getStateTransitionReference(parserContext, endBuilder.getBeanDefinition(), null, nextOnEnd); + next = endName; + + } + + Collection list = new ArrayList<>(); + list.add(getStateTransitionReference(parserContext, stateDef, on, next)); + + if(StringUtils.hasText(restart)) { + list.add(getStateTransitionReference(parserContext, stateDef, on + ".RESTART", restart)); + } + + if (endState != null) { + // + // Must be added after the state to ensure that the state is the + // first in the list + // + list.add(endState); + } + return list; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JobFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JobFactoryBean.java new file mode 100644 index 0000000000..c6eb8287a4 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JobFactoryBean.java @@ -0,0 +1,168 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import javax.batch.api.listener.JobListener; + +import org.springframework.batch.core.JobExecutionListener; +import org.springframework.batch.core.JobParametersIncrementer; +import org.springframework.batch.core.JobParametersValidator; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.job.flow.Flow; +import org.springframework.batch.core.job.flow.FlowJob; +import org.springframework.batch.core.jsr.JobListenerAdapter; +import org.springframework.batch.core.jsr.job.flow.JsrFlowJob; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.beans.factory.FactoryBean; +import org.springframework.beans.factory.SmartFactoryBean; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * This {@link FactoryBean} is used by the JSR-352 namespace parser to create + * {@link FlowJob} objects. It stores all of the properties that are + * configurable on the <job/>. + * + * @author Michael Minella + * @since 3.0 + */ +public class JobFactoryBean implements SmartFactoryBean { + + private String name; + + private Boolean restartable; + + private JobRepository jobRepository; + + private JobParametersValidator jobParametersValidator; + + private JobExecutionListener[] jobExecutionListeners; + + private JobParametersIncrementer jobParametersIncrementer; + + private Flow flow; + + private JobExplorer jobExplorer; + + public JobFactoryBean(String name) { + this.name = name; + } + + @Override + public final FlowJob getObject() throws Exception { + Assert.isTrue(StringUtils.hasText(name), "The job must have an id."); + JsrFlowJob flowJob = new JsrFlowJob(name); + flowJob.setJobExplorer(jobExplorer); + + if (restartable != null) { + flowJob.setRestartable(restartable); + } + + if (jobRepository != null) { + flowJob.setJobRepository(jobRepository); + } + + if (jobParametersValidator != null) { + flowJob.setJobParametersValidator(jobParametersValidator); + } + + if (jobExecutionListeners != null) { + flowJob.setJobExecutionListeners(jobExecutionListeners); + } + + if (jobParametersIncrementer != null) { + flowJob.setJobParametersIncrementer(jobParametersIncrementer); + } + + if (flow != null) { + flowJob.setFlow(flow); + } + + flowJob.afterPropertiesSet(); + return flowJob; + } + + public void setJobExplorer(JobExplorer jobExplorer) { + this.jobExplorer = jobExplorer; + } + + public void setRestartable(Boolean restartable) { + this.restartable = restartable; + } + + public void setJobRepository(JobRepository jobRepository) { + this.jobRepository = jobRepository; + } + + public void setJobParametersValidator(JobParametersValidator jobParametersValidator) { + this.jobParametersValidator = jobParametersValidator; + } + + public JobRepository getJobRepository() { + return this.jobRepository; + } + + public void setJobParametersIncrementer(JobParametersIncrementer jobParametersIncrementer) { + this.jobParametersIncrementer = jobParametersIncrementer; + } + + public void setFlow(Flow flow) { + this.flow = flow; + } + + @Override + public Class getObjectType() { + return FlowJob.class; + } + + @Override + public boolean isSingleton() { + return true; + } + + @Override + public boolean isEagerInit() { + return true; + } + + @Override + public boolean isPrototype() { + return false; + } + + /** + * Addresses wrapping {@link JobListener} as needed to be used with + * the framework. + * + * @param jobListeners a list of all job listeners + */ + public void setJobExecutionListeners(Object[] jobListeners) { + if(jobListeners != null) { + JobExecutionListener[] listeners = new JobExecutionListener[jobListeners.length]; + + for(int i = 0; i < jobListeners.length; i++) { + Object curListener = jobListeners[i]; + if(curListener instanceof JobExecutionListener) { + listeners[i] = (JobExecutionListener) curListener; + } else if(curListener instanceof JobListener){ + listeners[i] = new JobListenerAdapter((JobListener) curListener); + } + } + + this.jobExecutionListeners = listeners; + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrBeanDefinitionDocumentReader.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrBeanDefinitionDocumentReader.java new file mode 100644 index 0000000000..e11ea43b95 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrBeanDefinitionDocumentReader.java @@ -0,0 +1,310 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.util.Enumeration; +import java.util.HashMap; +import java.util.Map; +import java.util.Properties; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.core.jsr.configuration.support.JsrExpressionParser; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.support.BeanDefinitionRegistry; +import org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader; +import org.springframework.util.ClassUtils; +import org.w3c.dom.Element; +import org.w3c.dom.NamedNodeMap; +import org.w3c.dom.Node; +import org.w3c.dom.NodeList; +import org.w3c.dom.ls.DOMImplementationLS; +import org.w3c.dom.traversal.DocumentTraversal; +import org.w3c.dom.traversal.NodeFilter; +import org.w3c.dom.traversal.NodeIterator; + +/** + *

      + * {@link DefaultBeanDefinitionDocumentReader} extension to hook into the pre processing of the provided + * XML document, ensuring any references to property operators such as jobParameters and jobProperties are + * resolved prior to loading the context. Since we know these initial values upfront, doing this transformation + * allows us to ensure values are retrieved in their resolved form prior to loading the context and property + * operators can be used on any element. This document reader will also look for references to artifacts by + * the same name and create new bean definitions to provide the ability to create new instances. + *

      + * + * @author Chris Schaefer + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class JsrBeanDefinitionDocumentReader extends DefaultBeanDefinitionDocumentReader { + private static final String NULL = "null"; + private static final String ROOT_JOB_ELEMENT_NAME = "job"; + private static final String JOB_PROPERTY_ELEMENT_NAME = "property"; + private static final String JOB_PROPERTIES_ELEMENT_NAME = "properties"; + private static final String JOB_PROPERTY_ELEMENT_NAME_ATTRIBUTE = "name"; + private static final String JOB_PROPERTY_ELEMENT_VALUE_ATTRIBUTE = "value"; + private static final String JOB_PROPERTIES_KEY_NAME = "jobProperties"; + private static final String JOB_PARAMETERS_KEY_NAME = "jobParameters"; + private static final String JOB_PARAMETERS_BEAN_DEFINITION_NAME = "jsr_jobParameters"; + private static final Log LOG = LogFactory.getLog(JsrBeanDefinitionDocumentReader.class); + private static final Pattern PROPERTY_KEY_SEPARATOR = Pattern.compile("'([^']*?)'"); + private static final Pattern OPERATOR_PATTERN = Pattern.compile("(#\\{(job(Properties|Parameters))[^}]+\\})"); + + private BeanDefinitionRegistry beanDefinitionRegistry; + private JsrExpressionParser expressionParser = new JsrExpressionParser(); + private Map propertyMap = new HashMap<>(); + + /** + *

      + * Creates a new {@link JsrBeanDefinitionDocumentReader} instance. + *

      + */ + public JsrBeanDefinitionDocumentReader() { } + + /** + *

      + * Create a new {@link JsrBeanDefinitionDocumentReader} instance with the provided + * {@link BeanDefinitionRegistry}. + *

      + * + * @param beanDefinitionRegistry the {@link BeanDefinitionRegistry} to use + */ + public JsrBeanDefinitionDocumentReader(BeanDefinitionRegistry beanDefinitionRegistry) { + this.beanDefinitionRegistry = beanDefinitionRegistry; + } + + @Override + protected void preProcessXml(Element root) { + if (ROOT_JOB_ELEMENT_NAME.equals(root.getLocalName())) { + initProperties(root); + transformDocument(root); + + if (LOG.isDebugEnabled()) { + LOG.debug("Transformed XML from preProcessXml: " + elementToString(root)); + } + } + } + + protected void initProperties(Element root) { + propertyMap.put(JOB_PARAMETERS_KEY_NAME, initJobParameters()); + propertyMap.put(JOB_PROPERTIES_KEY_NAME, initJobProperties(root)); + + resolvePropertyValues(propertyMap.get(JOB_PARAMETERS_KEY_NAME)); + resolvePropertyValues(propertyMap.get(JOB_PROPERTIES_KEY_NAME)); + } + + private Properties initJobParameters() { + Properties jobParameters = new Properties(); + + if (getBeanDefinitionRegistry().containsBeanDefinition(JOB_PARAMETERS_BEAN_DEFINITION_NAME)) { + BeanDefinition beanDefinition = getBeanDefinitionRegistry().getBeanDefinition(JOB_PARAMETERS_BEAN_DEFINITION_NAME); + + Properties properties = (Properties) beanDefinition.getConstructorArgumentValues() + .getGenericArgumentValue(Properties.class) + .getValue(); + + if (properties == null) { + return new Properties(); + } + + Enumeration propertyNames = properties.propertyNames(); + + while(propertyNames.hasMoreElements()) { + String curName = (String) propertyNames.nextElement(); + jobParameters.put(curName, properties.getProperty(curName)); + } + } + + return jobParameters; + } + + private Properties initJobProperties(Element root) { + Properties properties = new Properties(); + Node propertiesNode = root.getElementsByTagName(JOB_PROPERTIES_ELEMENT_NAME).item(0); + + if(propertiesNode != null) { + NodeList children = propertiesNode.getChildNodes(); + + for(int i=0; i < children.getLength(); i++) { + Node child = children.item(i); + + if(JOB_PROPERTY_ELEMENT_NAME.equals(child.getLocalName())) { + NamedNodeMap attributes = child.getAttributes(); + Node name = attributes.getNamedItem(JOB_PROPERTY_ELEMENT_NAME_ATTRIBUTE); + Node value = attributes.getNamedItem(JOB_PROPERTY_ELEMENT_VALUE_ATTRIBUTE); + + properties.setProperty(name.getNodeValue(), value.getNodeValue()); + } + } + } + + return properties; + } + + private void resolvePropertyValues(Properties properties) { + for (String propertyKey : properties.stringPropertyNames()) { + String resolvedPropertyValue = resolvePropertyValue(properties.getProperty(propertyKey)); + + if(!properties.getProperty(propertyKey).equals(resolvedPropertyValue)) { + properties.setProperty(propertyKey, resolvedPropertyValue); + } + } + } + + private String resolvePropertyValue(String propertyValue) { + String resolvedValue = resolveValue(propertyValue); + + Matcher jobParameterMatcher = OPERATOR_PATTERN.matcher(resolvedValue); + + while (jobParameterMatcher.find()) { + resolvedValue = resolvePropertyValue(resolvedValue); + } + + return resolvedValue; + } + + private String resolveValue(String value) { + StringBuffer valueBuffer = new StringBuffer(); + Matcher jobParameterMatcher = OPERATOR_PATTERN.matcher(value); + + while (jobParameterMatcher.find()) { + Matcher jobParameterKeyMatcher = PROPERTY_KEY_SEPARATOR.matcher(jobParameterMatcher.group(1)); + + if (jobParameterKeyMatcher.find()) { + String propertyType = jobParameterMatcher.group(2); + String extractedProperty = jobParameterKeyMatcher.group(1); + + Properties properties = propertyMap.get(propertyType); + + if(properties == null) { + throw new IllegalArgumentException("Unknown property type: " + propertyType); + } + + String resolvedProperty = properties.getProperty(extractedProperty, NULL); + + if (NULL.equals(resolvedProperty)) { + LOG.info(propertyType + " with key of: " + extractedProperty + " could not be resolved. Possible configuration error?"); + } + + jobParameterMatcher.appendReplacement(valueBuffer, resolvedProperty); + } + } + + jobParameterMatcher.appendTail(valueBuffer); + String resolvedValue = valueBuffer.toString(); + + if (NULL.equals(resolvedValue)) { + return ""; + } + + return expressionParser.parseExpression(resolvedValue); + } + + private BeanDefinitionRegistry getBeanDefinitionRegistry() { + return beanDefinitionRegistry != null ? beanDefinitionRegistry : getReaderContext().getRegistry(); + } + + private void transformDocument(Element root) { + DocumentTraversal traversal = (DocumentTraversal) root.getOwnerDocument(); + NodeIterator iterator = traversal.createNodeIterator(root, NodeFilter.SHOW_ELEMENT, null, true); + + BeanDefinitionRegistry registry = getBeanDefinitionRegistry(); + Map referenceCountMap = new HashMap<>(); + + for (Node n = iterator.nextNode(); n != null; n = iterator.nextNode()) { + NamedNodeMap map = n.getAttributes(); + + if (map.getLength() > 0) { + for (int i = 0; i < map.getLength(); i++) { + Node node = map.item(i); + + String nodeName = node.getNodeName(); + String nodeValue = node.getNodeValue(); + String resolvedValue = resolveValue(nodeValue); + String newNodeValue = resolvedValue; + + if("ref".equals(nodeName)) { + if(!referenceCountMap.containsKey(resolvedValue)) { + referenceCountMap.put(resolvedValue, 0); + } + + boolean isClass = isClass(resolvedValue); + Integer referenceCount = referenceCountMap.get(resolvedValue); + + // possibly fully qualified class name in ref tag in the JSL or pointer to bean/artifact ref. + if(isClass && !registry.containsBeanDefinition(resolvedValue)) { + AbstractBeanDefinition beanDefinition = BeanDefinitionBuilder.genericBeanDefinition(resolvedValue) + .getBeanDefinition(); + beanDefinition.setScope("step"); + registry.registerBeanDefinition(resolvedValue, beanDefinition); + + newNodeValue = resolvedValue; + } else { + if(registry.containsBeanDefinition(resolvedValue)) { + referenceCount++; + referenceCountMap.put(resolvedValue, referenceCount); + + newNodeValue = resolvedValue + referenceCount; + + BeanDefinition beanDefinition = registry.getBeanDefinition(resolvedValue); + registry.registerBeanDefinition(newNodeValue, beanDefinition); + } + } + } + + if(!nodeValue.equals(newNodeValue)) { + node.setNodeValue(newNodeValue); + } + } + } else { + String nodeValue = n.getTextContent(); + String resolvedValue = resolveValue(nodeValue); + + if(!nodeValue.equals(resolvedValue)) { + n.setTextContent(resolvedValue); + } + } + } + } + + private boolean isClass(String className) { + try { + Class.forName(className, false, ClassUtils.getDefaultClassLoader()); + } catch (ClassNotFoundException e) { + return false; + } + + return true; + } + + protected Properties getJobParameters() { + return propertyMap.get(JOB_PARAMETERS_KEY_NAME); + } + + protected Properties getJobProperties() { + return propertyMap.get(JOB_PROPERTIES_KEY_NAME); + } + + private String elementToString(Element root) { + DOMImplementationLS domImplLS = (DOMImplementationLS) root.getOwnerDocument().getImplementation(); + return domImplLS.createLSSerializer().writeToString(root); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrDecisionParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrDecisionParser.java new file mode 100644 index 0000000000..78ddaaa34a --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrDecisionParser.java @@ -0,0 +1,69 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.util.Collection; + +import org.springframework.batch.core.job.flow.JobExecutionDecider; +import org.springframework.batch.core.jsr.configuration.support.BatchArtifactType; +import org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.config.RuntimeBeanReference; +import org.springframework.beans.factory.parsing.BeanComponentDefinition; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.StringUtils; +import org.w3c.dom.Element; + +/** + * Parser for the <decision /> element as specified in JSR-352. The current state + * parses a decision element and assumes that it refers to a {@link JobExecutionDecider} + * + * @author Michael Minella + * @author Chris Schaefer + * @since 3.0 + */ +public class JsrDecisionParser { + + private static final String ID_ATTRIBUTE = "id"; + private static final String REF_ATTRIBUTE = "ref"; + + public Collection parse(Element element, ParserContext parserContext, String jobFactoryRef) { + BeanDefinitionBuilder factoryBuilder = BeanDefinitionBuilder.genericBeanDefinition(); + AbstractBeanDefinition factoryDefinition = factoryBuilder.getRawBeanDefinition(); + factoryDefinition.setBeanClass(DecisionStepFactoryBean.class); + + BeanDefinitionBuilder stateBuilder = BeanDefinitionBuilder.genericBeanDefinition(JsrStepState.class); + + String idAttribute = element.getAttribute(ID_ATTRIBUTE); + + parserContext.registerBeanComponent(new BeanComponentDefinition(factoryDefinition, idAttribute)); + stateBuilder.addConstructorArgReference(idAttribute); + + String refAttribute = element.getAttribute(REF_ATTRIBUTE); + factoryDefinition.getPropertyValues().add("decider", new RuntimeBeanReference(refAttribute)); + factoryDefinition.getPropertyValues().add("name", idAttribute); + + if(StringUtils.hasText(jobFactoryRef)) { + factoryDefinition.setAttribute("jobParserJobFactoryBeanRef", jobFactoryRef); + } + + new PropertyParser(refAttribute, parserContext, BatchArtifactType.STEP_ARTIFACT, idAttribute).parseProperties(element); + + return FlowParser.getNextElements(parserContext, stateBuilder.getBeanDefinition(), element); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrFlowFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrFlowFactoryBean.java new file mode 100644 index 0000000000..3238a2b06b --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrFlowFactoryBean.java @@ -0,0 +1,39 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.springframework.batch.core.configuration.xml.SimpleFlowFactoryBean; +import org.springframework.batch.core.job.flow.State; +import org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState; + +/** + * Extension to the {@link SimpleFlowFactoryBean} that provides {@link org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState} + * implementations for JSR-352 based jobs. + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrFlowFactoryBean extends SimpleFlowFactoryBean { + + /* (non-Javadoc) + * @see org.springframework.batch.core.configuration.xml.SimpleFlowFactoryBean#createNewStepState(org.springframework.batch.core.job.flow.State, java.lang.String, java.lang.String) + */ + @Override + protected State createNewStepState(State state, String oldName, + String stateName) { + return new JsrStepState(stateName, ((JsrStepState) state).getStep(oldName)); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrJobListenerFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrJobListenerFactoryBean.java new file mode 100644 index 0000000000..5a0b455515 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrJobListenerFactoryBean.java @@ -0,0 +1,63 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +import javax.batch.api.listener.JobListener; + +import org.springframework.batch.core.JobExecutionListener; +import org.springframework.batch.core.jsr.JsrJobListenerMetaData; +import org.springframework.batch.core.listener.JobListenerMetaData; +import org.springframework.batch.core.listener.ListenerMetaData; +import org.springframework.beans.factory.FactoryBean; + +/** + * This {@link FactoryBean} is used by the JSR-352 namespace parser to create + * {@link JobExecutionListener} objects. + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrJobListenerFactoryBean extends org.springframework.batch.core.listener.JobListenerFactoryBean { + + @Override + public Class getObjectType() { + return JobListener.class; + } + + @Override + protected ListenerMetaData[] getMetaDataValues() { + List values = new ArrayList<>(); + Collections.addAll(values, JobListenerMetaData.values()); + Collections.addAll(values, JsrJobListenerMetaData.values()); + + return values.toArray(new ListenerMetaData[0]); + } + + @Override + protected ListenerMetaData getMetaDataFromPropertyName(String propertyName) { + ListenerMetaData result = JobListenerMetaData.fromPropertyName(propertyName); + + if(result == null) { + result = JsrJobListenerMetaData.fromPropertyName(propertyName); + } + + return result; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrJobParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrJobParser.java new file mode 100644 index 0000000000..43334726ba --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrJobParser.java @@ -0,0 +1,78 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.springframework.batch.core.configuration.xml.CoreNamespaceUtils; +import org.springframework.batch.core.jsr.JsrStepContextFactoryBean; +import org.springframework.batch.core.jsr.configuration.support.BatchArtifactType; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.xml.AbstractSingleBeanDefinitionParser; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.StringUtils; +import org.w3c.dom.Element; + +/** + * Parses a <job /> tag as defined in JSR-352. Current state parses into + * the standard Spring Batch artifacts. + * + * @author Michael Minella + * @author Chris Schaefer + * @since 3.0 + */ +public class JsrJobParser extends AbstractSingleBeanDefinitionParser { + private static final String ID_ATTRIBUTE = "id"; + private static final String RESTARTABLE_ATTRIBUTE = "restartable"; + + @Override + protected Class getBeanClass(Element element) { + return JobFactoryBean.class; + } + + @Override + protected void doParse(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { + CoreNamespaceUtils.autoregisterBeansForNamespace(parserContext, parserContext.extractSource(element)); + JsrNamespaceUtils.autoregisterJsrBeansForNamespace(parserContext); + + String jobName = element.getAttribute(ID_ATTRIBUTE); + + builder.setLazyInit(true); + + builder.addConstructorArgValue(jobName); + + builder.addPropertyReference("jobExplorer", "jobExplorer"); + + String restartableAttribute = element.getAttribute(RESTARTABLE_ATTRIBUTE); + if (StringUtils.hasText(restartableAttribute)) { + builder.addPropertyValue("restartable", restartableAttribute); + } + + new PropertyParser(jobName, parserContext, BatchArtifactType.JOB).parseProperties(element); + + BeanDefinition flowDef = new FlowParser(jobName, jobName).parse(element, parserContext); + builder.addPropertyValue("flow", flowDef); + + AbstractBeanDefinition stepContextBeanDefinition = BeanDefinitionBuilder.genericBeanDefinition(JsrStepContextFactoryBean.class) + .getBeanDefinition(); + + stepContextBeanDefinition.setScope("step"); + + parserContext.getRegistry().registerBeanDefinition("stepContextFactory", stepContextBeanDefinition); + + new ListenerParser(JsrJobListenerFactoryBean.class, "jobExecutionListeners").parseListeners(element, parserContext, builder); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrNamespaceHandler.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrNamespaceHandler.java new file mode 100644 index 0000000000..61988a0627 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrNamespaceHandler.java @@ -0,0 +1,32 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.springframework.beans.factory.xml.NamespaceHandlerSupport; + +/** + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrNamespaceHandler extends NamespaceHandlerSupport { + + @Override + public void init() { + this.registerBeanDefinitionParser("job", new JsrJobParser()); + this.registerBeanDefinitionParser("batch-artifacts", new BatchParser()); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrNamespacePostProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrNamespacePostProcessor.java new file mode 100644 index 0000000000..8a4038895d --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrNamespacePostProcessor.java @@ -0,0 +1,58 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.config.BeanPostProcessor; +import org.springframework.context.ApplicationContext; +import org.springframework.context.ApplicationContextAware; + +/** + * @author Michael Minella + */ +public class JsrNamespacePostProcessor implements BeanPostProcessor, ApplicationContextAware { + + private static final String DEFAULT_JOB_REPOSITORY_NAME = "jobRepository"; + + private ApplicationContext applicationContext; + + @Override + public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { + this.applicationContext = applicationContext; + } + + @Override + public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException { + if(bean instanceof JobFactoryBean) { + JobFactoryBean fb = (JobFactoryBean) bean; + JobRepository jobRepository = fb.getJobRepository(); + if (jobRepository == null) { + fb.setJobRepository((JobRepository) applicationContext.getBean(DEFAULT_JOB_REPOSITORY_NAME)); + } + } + + return bean; + } + + @Override + public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException { + return bean; + } +} + + + diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrNamespaceUtils.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrNamespaceUtils.java new file mode 100644 index 0000000000..3e29244ac5 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrNamespaceUtils.java @@ -0,0 +1,108 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.springframework.batch.core.jsr.launch.support.BatchPropertyBeanPostProcessor; +import org.springframework.batch.core.jsr.configuration.support.JsrAutowiredAnnotationBeanPostProcessor; +import org.springframework.batch.core.jsr.partition.support.JsrBeanScopeBeanFactoryPostProcessor; +import org.springframework.batch.core.jsr.configuration.support.ThreadLocalClassloaderBeanPostProcessor; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.context.annotation.AnnotationConfigUtils; + +import java.util.HashMap; + +/** + * Utility methods used in parsing of the JSR-352 batch namespace and related helpers. + * + * @author Michael Minella + * @author Chris Schaefer + * @since 3.0 + */ +class JsrNamespaceUtils { + private static final String JOB_PROPERTIES_BEAN_NAME = "jobProperties"; + private static final String BATCH_PROPERTY_POST_PROCESSOR_BEAN_NAME = "batchPropertyPostProcessor"; + private static final String THREAD_LOCAL_CLASS_LOADER_BEAN_POST_PROCESSOR_BEAN_NAME = "threadLocalClassloaderBeanPostProcessor"; + private static final String BEAN_SCOPE_POST_PROCESSOR_BEAN_NAME = "beanScopeBeanPostProcessor"; + private static final String BATCH_PROPERTY_CONTEXT_BEAN_CLASS_NAME = "org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext"; + private static final String BATCH_PROPERTY_CONTEXT_BEAN_NAME = "batchPropertyContext"; + private static final String JSR_NAMESPACE_POST_PROCESSOR = "jsrNamespacePostProcessor"; + + static void autoregisterJsrBeansForNamespace(ParserContext parserContext) { + autoRegisterJobProperties(parserContext); + autoRegisterBatchPostProcessor(parserContext); + autoRegisterJsrAutowiredAnnotationBeanPostProcessor(parserContext); + autoRegisterThreadLocalClassloaderBeanPostProcessor(parserContext); + autoRegisterBeanScopeBeanFactoryPostProcessor(parserContext); + autoRegisterBatchPropertyContext(parserContext); + autoRegisterNamespacePostProcessor(parserContext); + } + + private static void autoRegisterNamespacePostProcessor(ParserContext parserContext) { + registerPostProcessor(parserContext, JsrNamespacePostProcessor.class, BeanDefinition.ROLE_INFRASTRUCTURE, JSR_NAMESPACE_POST_PROCESSOR); + } + + private static void autoRegisterBeanScopeBeanFactoryPostProcessor( + ParserContext parserContext) { + registerPostProcessor(parserContext, JsrBeanScopeBeanFactoryPostProcessor.class, BeanDefinition.ROLE_INFRASTRUCTURE, BEAN_SCOPE_POST_PROCESSOR_BEAN_NAME); + } + + private static void autoRegisterBatchPostProcessor(ParserContext parserContext) { + registerPostProcessor(parserContext, BatchPropertyBeanPostProcessor.class, BeanDefinition.ROLE_INFRASTRUCTURE, BATCH_PROPERTY_POST_PROCESSOR_BEAN_NAME); + } + + private static void autoRegisterJsrAutowiredAnnotationBeanPostProcessor(ParserContext parserContext) { + registerPostProcessor(parserContext, JsrAutowiredAnnotationBeanPostProcessor.class, BeanDefinition.ROLE_INFRASTRUCTURE, AnnotationConfigUtils.AUTOWIRED_ANNOTATION_PROCESSOR_BEAN_NAME); + } + + private static void autoRegisterThreadLocalClassloaderBeanPostProcessor(ParserContext parserContext) { + registerPostProcessor(parserContext, ThreadLocalClassloaderBeanPostProcessor.class, BeanDefinition.ROLE_INFRASTRUCTURE, THREAD_LOCAL_CLASS_LOADER_BEAN_POST_PROCESSOR_BEAN_NAME); + } + + private static void registerPostProcessor(ParserContext parserContext, Class clazz, int role, String beanName) { + BeanDefinitionBuilder beanDefinitionBuilder = BeanDefinitionBuilder.genericBeanDefinition(clazz); + + AbstractBeanDefinition beanDefinition = beanDefinitionBuilder.getBeanDefinition(); + beanDefinition.setRole(role); + + parserContext.getRegistry().registerBeanDefinition(beanName, beanDefinition); + } + + // Registers a bean by the name of {@link #JOB_PROPERTIES_BEAN_NAME} so job level properties can be obtained through + // for example a SPeL expression referencing #{jobProperties['key']} similar to systemProperties resolution. + private static void autoRegisterJobProperties(ParserContext parserContext) { + if (!parserContext.getRegistry().containsBeanDefinition(JOB_PROPERTIES_BEAN_NAME)) { + AbstractBeanDefinition jobPropertiesBeanDefinition = BeanDefinitionBuilder.genericBeanDefinition(HashMap.class).getBeanDefinition(); + jobPropertiesBeanDefinition.setRole(BeanDefinition.ROLE_INFRASTRUCTURE); + + parserContext.getRegistry().registerBeanDefinition(JOB_PROPERTIES_BEAN_NAME, jobPropertiesBeanDefinition); + } + } + + private static void autoRegisterBatchPropertyContext(ParserContext parserContext) { + if (!parserContext.getRegistry().containsBeanDefinition(BATCH_PROPERTY_CONTEXT_BEAN_NAME)) { + AbstractBeanDefinition batchPropertyContextBeanDefinition = + BeanDefinitionBuilder.genericBeanDefinition(BATCH_PROPERTY_CONTEXT_BEAN_CLASS_NAME) + .getBeanDefinition(); + + batchPropertyContextBeanDefinition.setRole(BeanDefinition.ROLE_INFRASTRUCTURE); + + parserContext.getRegistry().registerBeanDefinition(BATCH_PROPERTY_CONTEXT_BEAN_NAME, batchPropertyContextBeanDefinition); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrSplitParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrSplitParser.java new file mode 100644 index 0000000000..62440141d6 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrSplitParser.java @@ -0,0 +1,94 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.util.Collection; +import java.util.List; + +import org.springframework.beans.PropertyValue; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.config.RuntimeBeanReference; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.support.BeanDefinitionRegistry; +import org.springframework.beans.factory.support.ManagedList; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.util.xml.DomUtils; +import org.w3c.dom.Element; + +/** + * Parses a <split /> element as defined in JSR-352. + * + * @author Michael Minella + * @author Chris Schaefer + * @since 3.0 + */ +public class JsrSplitParser { + private static final String TASK_EXECUTOR_PROPERTY_NAME = "taskExecutor"; + private static final String JSR_352_SPLIT_TASK_EXECUTOR_BEAN_NAME = "jsr352splitTaskExecutor"; + + private String jobFactoryRef; + + public JsrSplitParser(String jobFactoryRef) { + this.jobFactoryRef = jobFactoryRef; + } + + public Collection parse(Element element, ParserContext parserContext) { + + String idAttribute = element.getAttribute("id"); + + BeanDefinitionBuilder stateBuilder = BeanDefinitionBuilder + .genericBeanDefinition("org.springframework.batch.core.jsr.job.flow.support.state.JsrSplitState"); + + List flowElements = DomUtils.getChildElementsByTagName(element, "flow"); + + if (flowElements.size() < 2) { + parserContext.getReaderContext().error("A must contain at least two 'flow' elements.", element); + } + + Collection flows = new ManagedList<>(); + int i = 0; + for (Element nextElement : flowElements) { + FlowParser flowParser = new FlowParser(idAttribute + "." + i, jobFactoryRef); + flows.add(flowParser.parse(nextElement, parserContext)); + i++; + } + + stateBuilder.addConstructorArgValue(flows); + stateBuilder.addConstructorArgValue(idAttribute); + + PropertyValue propertyValue = getSplitTaskExecutorPropertyValue(parserContext.getRegistry()); + stateBuilder.addPropertyValue(propertyValue.getName(), propertyValue.getValue()); + + return FlowParser.getNextElements(parserContext, null, stateBuilder.getBeanDefinition(), element); + } + + protected PropertyValue getSplitTaskExecutorPropertyValue(BeanDefinitionRegistry beanDefinitionRegistry) { + PropertyValue propertyValue; + + if (hasBeanDefinition(beanDefinitionRegistry, JSR_352_SPLIT_TASK_EXECUTOR_BEAN_NAME)) { + propertyValue = new PropertyValue(TASK_EXECUTOR_PROPERTY_NAME, new RuntimeBeanReference(JSR_352_SPLIT_TASK_EXECUTOR_BEAN_NAME)); + } else { + propertyValue = new PropertyValue(TASK_EXECUTOR_PROPERTY_NAME, new SimpleAsyncTaskExecutor()); + } + + return propertyValue; + } + + private boolean hasBeanDefinition(BeanDefinitionRegistry beanDefinitionRegistry, String beanName) { + return beanDefinitionRegistry.containsBeanDefinition(beanName); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrStepListenerFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrStepListenerFactoryBean.java new file mode 100644 index 0000000000..70f2c2d7f6 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrStepListenerFactoryBean.java @@ -0,0 +1,51 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.springframework.batch.core.jsr.JsrStepListenerMetaData; +import org.springframework.batch.core.listener.ListenerMetaData; +import org.springframework.batch.core.listener.StepListenerFactoryBean; +import org.springframework.batch.core.listener.StepListenerMetaData; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +/** + * @author Michael Minella + */ +public class JsrStepListenerFactoryBean extends StepListenerFactoryBean { + + @Override + protected ListenerMetaData getMetaDataFromPropertyName(String propertyName) { + ListenerMetaData metaData = StepListenerMetaData.fromPropertyName(propertyName); + + if(metaData == null) { + metaData = JsrStepListenerMetaData.fromPropertyName(propertyName); + } + + return metaData; + } + + @Override + protected ListenerMetaData[] getMetaDataValues() { + List values = new ArrayList<>(); + Collections.addAll(values, StepListenerMetaData.values()); + Collections.addAll(values, JsrStepListenerMetaData.values()); + + return values.toArray(new ListenerMetaData[values.size()]); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrXmlApplicationContext.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrXmlApplicationContext.java new file mode 100644 index 0000000000..802d07252b --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/JsrXmlApplicationContext.java @@ -0,0 +1,90 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.util.Properties; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.xml.XmlBeanDefinitionReader; +import org.springframework.context.support.GenericApplicationContext; +import org.springframework.core.io.Resource; + +/** + *

      + * {@link GenericApplicationContext} implementation providing JSR-352 related context operations. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class JsrXmlApplicationContext extends GenericApplicationContext { + private static final String JOB_PARAMETERS_BEAN_DEFINITION_NAME = "jsr_jobParameters"; + + private XmlBeanDefinitionReader reader = new XmlBeanDefinitionReader(this); + + /** + *

      + * Create a new context instance with no job parameters. + *

      + */ + public JsrXmlApplicationContext() { + reader.setDocumentReaderClass(JsrBeanDefinitionDocumentReader.class); + reader.setEnvironment(this.getEnvironment()); + } + + /** + *

      + * Create a new context instance using the provided {@link Properties} representing job + * parameters when pre-processing the job definition document. + *

      + * + * @param jobParameters the {@link Properties} representing job parameters + */ + public JsrXmlApplicationContext(Properties jobParameters) { + reader.setDocumentReaderClass(JsrBeanDefinitionDocumentReader.class); + reader.setEnvironment(this.getEnvironment()); + + storeJobParameters(jobParameters); + } + + private void storeJobParameters(Properties properties) { + BeanDefinition jobParameters = BeanDefinitionBuilder.genericBeanDefinition(Properties.class).getBeanDefinition(); + jobParameters.getConstructorArgumentValues().addGenericArgumentValue(properties != null ? properties : new Properties()); + + reader.getRegistry().registerBeanDefinition(JOB_PARAMETERS_BEAN_DEFINITION_NAME, jobParameters); + } + + protected XmlBeanDefinitionReader getReader() { + return reader; + } + + /** + * Set whether to use XML validation. Default is true. + * + * @param validating true if XML should be validated. + */ + public void setValidating(boolean validating) { + this.reader.setValidating(validating); + } + + /** + * Load bean definitions from the given XML resources. + * @param resources one or more resources to load from + */ + public void load(Resource... resources) { + this.reader.loadBeanDefinitions(resources); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/ListenerParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/ListenerParser.java new file mode 100644 index 0000000000..ae9c0de5b5 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/ListenerParser.java @@ -0,0 +1,140 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.util.List; + +import org.springframework.batch.core.jsr.configuration.support.BatchArtifactType; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.config.RuntimeBeanReference; +import org.springframework.beans.factory.parsing.CompositeComponentDefinition; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.support.BeanDefinitionRegistry; +import org.springframework.beans.factory.support.ManagedList; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.xml.DomUtils; +import org.w3c.dom.Element; + +/** + * Parses the various listeners defined in JSR-352. Current state assumes + * the ref attributes point to implementations of Spring Batch interfaces + * and not JSR interfaces + * + * @author Michael Minella + * @author Chris Schaefer + * @since 3.0 + */ +public class ListenerParser { + private static final String REF_ATTRIBUTE = "ref"; + private static final String LISTENER_ELEMENT = "listener"; + private static final String LISTENERS_ELEMENT = "listeners"; + private static final String SCOPE_STEP = "step"; + private static final String SCOPE_JOB = "job"; + + private Class listenerType; + private String propertyKey; + + public ListenerParser(Class listenerType, String propertyKey) { + this.propertyKey = propertyKey; + this.listenerType = listenerType; + } + + public void parseListeners(Element element, ParserContext parserContext, AbstractBeanDefinition bd, String stepName) { + ManagedList listeners = parseListeners(element, parserContext, stepName); + + if(listeners.size() > 0) { + bd.getPropertyValues().add(propertyKey, listeners); + } + } + + public void parseListeners(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { + ManagedList listeners = parseListeners(element, parserContext, ""); + + if(listeners.size() > 0) { + builder.addPropertyValue(propertyKey, listeners); + } + } + + private ManagedList parseListeners(Element element, ParserContext parserContext, String stepName) { + List listenersElements = DomUtils.getChildElementsByTagName(element, LISTENERS_ELEMENT); + + ManagedList listeners = new ManagedList<>(); + + if (listenersElements.size() == 1) { + Element listenersElement = listenersElements.get(0); + CompositeComponentDefinition compositeDef = new CompositeComponentDefinition(listenersElement.getTagName(), + parserContext.extractSource(element)); + parserContext.pushContainingComponent(compositeDef); + listeners.setMergeEnabled(false); + List listenerElements = DomUtils.getChildElementsByTagName(listenersElement, LISTENER_ELEMENT); + for (Element listenerElement : listenerElements) { + String beanName = listenerElement.getAttribute(REF_ATTRIBUTE); + + BeanDefinitionBuilder bd = BeanDefinitionBuilder.genericBeanDefinition(listenerType); + bd.addPropertyValue("delegate", new RuntimeBeanReference(beanName)); + + applyListenerScope(beanName, parserContext.getRegistry()); + + listeners.add(bd.getBeanDefinition()); + + new PropertyParser(beanName, parserContext, getBatchArtifactType(stepName), stepName).parseProperties(listenerElement); + } + parserContext.popAndRegisterContainingComponent(); + } + else if (listenersElements.size() > 1) { + parserContext.getReaderContext().error( + "The '' element may not appear more than once in a single " + element.getLocalName(), element); + } + + return listeners; + } + + protected void applyListenerScope(String beanName, BeanDefinitionRegistry beanDefinitionRegistry) { + BeanDefinition beanDefinition = getListenerBeanDefinition(beanName, beanDefinitionRegistry); + beanDefinition.setScope(getListenerScope()); + beanDefinition.setLazyInit(isLazyInit()); + + if (!beanDefinitionRegistry.containsBeanDefinition(beanName)) { + beanDefinitionRegistry.registerBeanDefinition(beanName, beanDefinition); + } + } + + private BeanDefinition getListenerBeanDefinition(String beanName, BeanDefinitionRegistry beanDefinitionRegistry) { + if (beanDefinitionRegistry.containsBeanDefinition(beanName)) { + return beanDefinitionRegistry.getBeanDefinition(beanName); + } + + return BeanDefinitionBuilder.genericBeanDefinition(beanName).getBeanDefinition(); + } + + private boolean isLazyInit() { + return listenerType == JsrJobListenerFactoryBean.class; + } + + private String getListenerScope() { + if (listenerType == JsrJobListenerFactoryBean.class) { + return SCOPE_JOB; + } + + return SCOPE_STEP; + } + + private BatchArtifactType getBatchArtifactType(String stepName) { + return (stepName != null && !"".equals(stepName)) ? BatchArtifactType.STEP_ARTIFACT + : BatchArtifactType.ARTIFACT; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/PartitionParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/PartitionParser.java new file mode 100644 index 0000000000..7c4ccb66b9 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/PartitionParser.java @@ -0,0 +1,187 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.util.List; +import java.util.concurrent.ConcurrentLinkedQueue; +import java.util.concurrent.locks.ReentrantLock; + +import org.springframework.batch.core.jsr.configuration.support.BatchArtifactType; +import org.springframework.batch.core.jsr.partition.JsrPartitionHandler; +import org.springframework.beans.MutablePropertyValues; +import org.springframework.beans.factory.config.RuntimeBeanReference; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.support.BeanDefinitionRegistry; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.StringUtils; +import org.springframework.util.xml.DomUtils; +import org.w3c.dom.Element; + +/** + * Parser for the <partition> element as defined by JSR-352. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class PartitionParser { + + private static final String REF = "ref"; + private static final String MAPPER_ELEMENT = "mapper"; + private static final String PLAN_ELEMENT = "plan"; + private static final String PARTITIONS_ATTRIBUTE = "partitions"; + private static final String THREADS_ATTRIBUTE = "threads"; + private static final String PROPERTIES_ELEMENT = "properties"; + private static final String ANALYZER_ELEMENT = "analyzer"; + private static final String COLLECTOR_ELEMENT = "collector"; + private static final String REDUCER_ELEMENT = "reducer"; + private static final String PARTITION_CONTEXT_PROPERTY = "propertyContext"; + private static final String PARTITION_MAPPER_PROPERTY = "partitionMapper"; + private static final String PARTITION_ANALYZER_PROPERTY = "partitionAnalyzer"; + private static final String PARTITION_REDUCER_PROPERTY = "partitionReducer"; + private static final String PARTITION_QUEUE_PROPERTY = "partitionDataQueue"; + private static final String LISTENERS_PROPERTY = "listeners"; + private static final String THREADS_PROPERTY = "threads"; + private static final String PARTITIONS_PROPERTY = "partitions"; + private static final String PARTITION_LOCK_PROPERTY = "partitionLock"; + + private final String name; + private boolean allowStartIfComplete = false; + + /** + * @param stepName the name of the step that is being partitioned + * @param allowStartIfComplete boolean to establish the allowStartIfComplete property for partition properties. + */ + public PartitionParser(String stepName, boolean allowStartIfComplete) { + this.name = stepName; + this.allowStartIfComplete = allowStartIfComplete; + } + + public void parse(Element element, AbstractBeanDefinition bd, ParserContext parserContext, String stepName) { + BeanDefinitionRegistry registry = parserContext.getRegistry(); + MutablePropertyValues factoryBeanProperties = bd.getPropertyValues(); + + AbstractBeanDefinition partitionHandlerDefinition = BeanDefinitionBuilder.genericBeanDefinition(JsrPartitionHandler.class) + .getBeanDefinition(); + + MutablePropertyValues properties = partitionHandlerDefinition.getPropertyValues(); + properties.addPropertyValue(PARTITION_CONTEXT_PROPERTY, new RuntimeBeanReference("batchPropertyContext")); + properties.addPropertyValue("jobRepository", new RuntimeBeanReference("jobRepository")); + properties.addPropertyValue("allowStartIfComplete", allowStartIfComplete); + + parseMapperElement(element, parserContext, properties); + parsePartitionPlan(element, parserContext, stepName, properties); + parseAnalyzerElement(element, parserContext, properties); + parseReducerElement(element, parserContext, factoryBeanProperties); + parseCollectorElement(element, parserContext, factoryBeanProperties, + properties); + + String partitionHandlerBeanName = name + ".partitionHandler"; + registry.registerBeanDefinition(partitionHandlerBeanName, partitionHandlerDefinition); + factoryBeanProperties.add("partitionHandler", new RuntimeBeanReference(partitionHandlerBeanName)); + } + + private void parseCollectorElement(Element element, + ParserContext parserContext, + MutablePropertyValues factoryBeanProperties, + MutablePropertyValues properties) { + Element collectorElement = DomUtils.getChildElementByTagName(element, COLLECTOR_ELEMENT); + + if(collectorElement != null) { + // Only needed if a collector is used + registerCollectorAnalyzerQueue(parserContext); + properties.add(PARTITION_QUEUE_PROPERTY, new RuntimeBeanReference(name + "PartitionQueue")); + properties.add(PARTITION_LOCK_PROPERTY, new RuntimeBeanReference(name + "PartitionLock")); + factoryBeanProperties.add("partitionQueue", new RuntimeBeanReference(name + "PartitionQueue")); + factoryBeanProperties.add("partitionLock", new RuntimeBeanReference(name + "PartitionLock")); + String collectorName = collectorElement.getAttribute(REF); + factoryBeanProperties.add(LISTENERS_PROPERTY, new RuntimeBeanReference(collectorName)); + new PropertyParser(collectorName, parserContext, BatchArtifactType.STEP_ARTIFACT, name).parseProperties(collectorElement); + } + } + + private void parseReducerElement(Element element, + ParserContext parserContext, + MutablePropertyValues factoryBeanProperties) { + Element reducerElement = DomUtils.getChildElementByTagName(element, REDUCER_ELEMENT); + + if(reducerElement != null) { + String reducerName = reducerElement.getAttribute(REF); + factoryBeanProperties.add(PARTITION_REDUCER_PROPERTY, new RuntimeBeanReference(reducerName)); + new PropertyParser(reducerName, parserContext, BatchArtifactType.STEP_ARTIFACT, name).parseProperties(reducerElement); + } + } + + private void parseAnalyzerElement(Element element, + ParserContext parserContext, MutablePropertyValues properties) { + Element analyzerElement = DomUtils.getChildElementByTagName(element, ANALYZER_ELEMENT); + + if(analyzerElement != null) { + String analyzerName = analyzerElement.getAttribute(REF); + properties.add(PARTITION_ANALYZER_PROPERTY, new RuntimeBeanReference(analyzerName)); + new PropertyParser(analyzerName, parserContext, BatchArtifactType.STEP_ARTIFACT, name).parseProperties(analyzerElement); + } + } + + private void parseMapperElement(Element element, + ParserContext parserContext, MutablePropertyValues properties) { + Element mapperElement = DomUtils.getChildElementByTagName(element, MAPPER_ELEMENT); + + if(mapperElement != null) { + String mapperName = mapperElement.getAttribute(REF); + properties.add(PARTITION_MAPPER_PROPERTY, new RuntimeBeanReference(mapperName)); + new PropertyParser(mapperName, parserContext, BatchArtifactType.STEP_ARTIFACT, name).parseProperties(mapperElement); + } + } + + private void registerCollectorAnalyzerQueue(ParserContext parserContext) { + AbstractBeanDefinition partitionQueueDefinition = BeanDefinitionBuilder.genericBeanDefinition(ConcurrentLinkedQueue.class) + .getBeanDefinition(); + AbstractBeanDefinition partitionLockDefinition = BeanDefinitionBuilder.genericBeanDefinition(ReentrantLock.class) + .getBeanDefinition(); + + parserContext.getRegistry().registerBeanDefinition(name + "PartitionQueue", partitionQueueDefinition); + parserContext.getRegistry().registerBeanDefinition(name + "PartitionLock", partitionLockDefinition); + } + + protected void parsePartitionPlan(Element element, + ParserContext parserContext, String stepName, + MutablePropertyValues properties) { + Element planElement = DomUtils.getChildElementByTagName(element, PLAN_ELEMENT); + + if(planElement != null) { + String partitions = planElement.getAttribute(PARTITIONS_ATTRIBUTE); + String threads = planElement.getAttribute(THREADS_ATTRIBUTE); + + if(!StringUtils.hasText(threads)) { + threads = partitions; + } + + List partitionProperties = DomUtils.getChildElementsByTagName(planElement, PROPERTIES_ELEMENT); + + if(partitionProperties != null) { + for (Element partition : partitionProperties) { + String partitionStepName = stepName + ":partition" + partition.getAttribute("partition"); + new PropertyParser(partitionStepName, parserContext, BatchArtifactType.STEP, partitionStepName).parseProperty(partition); + } + } + + properties.add(THREADS_PROPERTY, threads); + properties.add(PARTITIONS_PROPERTY, partitions); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/PropertyParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/PropertyParser.java new file mode 100644 index 0000000000..475ade25d9 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/PropertyParser.java @@ -0,0 +1,192 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Properties; + +import org.springframework.batch.core.jsr.configuration.support.BatchArtifactType; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.support.ManagedMap; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.xml.DomUtils; +import org.w3c.dom.Element; + +/** + *

      + * Parser for the <properties /> element defined by JSR-352. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class PropertyParser { + private static final String PROPERTY_ELEMENT = "property"; + private static final String PROPERTIES_ELEMENT = "properties"; + private static final String PROPERTY_NAME_ATTRIBUTE = "name"; + private static final String PROPERTY_VALUE_ATTRIBUTE = "value"; + private static final String JOB_PROPERTIES_BEAN_NAME = "jobProperties"; + private static final String BATCH_PROPERTY_CONTEXT_BEAN_NAME = "batchPropertyContext"; + private static final String JOB_PROPERTIES_PROPERTY_NAME = "jobProperties"; + private static final String STEP_PROPERTIES_PROPERTY_NAME = "stepProperties"; + private static final String ARTIFACT_PROPERTIES_PROPERTY_NAME = "artifactProperties"; + private static final String STEP_ARTIFACT_PROPERTIES_PROPERTY_NAME = "stepArtifactProperties"; + + private String beanName; + private String stepName; + private ParserContext parserContext; + private BatchArtifactType batchArtifactType; + + public PropertyParser(String beanName, ParserContext parserContext, BatchArtifactType batchArtifactType) { + this.beanName = beanName; + this.parserContext = parserContext; + this.batchArtifactType = batchArtifactType; + } + + public PropertyParser(String beanName, ParserContext parserContext, BatchArtifactType batchArtifactType, String stepName) { + this(beanName, parserContext, batchArtifactType); + this.stepName = stepName; + } + + /** + *

      + * Parses <property> tag values from the provided {@link Element} if it contains a <properties /> element. + * Only one <properties /> element may be present. <property> elements have a name and value attribute + * which represent the property entries key and value. + *

      + * + * @param element the element to parse looking for <properties /> + */ + public void parseProperties(Element element) { + List propertiesElements = DomUtils.getChildElementsByTagName(element, PROPERTIES_ELEMENT); + + if (propertiesElements.size() == 1) { + parsePropertyElement(propertiesElements.get(0)); + } else if (propertiesElements.size() > 1) { + parserContext.getReaderContext().error("The element may not appear more than once.", element); + } + } + + /** + *

      + * Parses a <property> tag value from the provided {@link Element}. <property> elements have a name and + * value attribute which represent the property entries key and value. + *

      + * + * @param element the element to parse looking for <property/> + */ + public void parseProperty(Element element) { + parsePropertyElement(element); + } + + private void parsePropertyElement(Element propertyElement) { + Properties properties = new Properties(); + + for (Element element : DomUtils.getChildElementsByTagName(propertyElement, PROPERTY_ELEMENT)) { + properties.put(element.getAttribute(PROPERTY_NAME_ATTRIBUTE), element.getAttribute(PROPERTY_VALUE_ATTRIBUTE)); + } + + setProperties(properties); + setJobPropertiesBean(properties); + } + + private void setProperties(Properties properties) { + Object propertyValue; + BeanDefinition beanDefinition = parserContext.getRegistry().getBeanDefinition(BATCH_PROPERTY_CONTEXT_BEAN_NAME); + + if(batchArtifactType.equals(BatchArtifactType.JOB)) { + propertyValue = getJobProperties(properties); + } else if (batchArtifactType.equals(BatchArtifactType.STEP)) { + propertyValue = getProperties(stepName, properties); + } else if (batchArtifactType.equals(BatchArtifactType.ARTIFACT)) { + propertyValue = getProperties(beanName, properties); + } else if (batchArtifactType.equals(BatchArtifactType.STEP_ARTIFACT)) { + propertyValue = getStepArtifactProperties(beanDefinition, properties); + } else { + throw new IllegalStateException("Unhandled BatchArtifactType of: " + batchArtifactType); + } + + beanDefinition.getPropertyValues().addPropertyValue(getPropertyName(batchArtifactType), propertyValue); + } + + private Map getProperties(String keyName, Properties properties) { + ManagedMap stepProperties = new ManagedMap<>(); + stepProperties.setMergeEnabled(true); + stepProperties.put(keyName, properties); + + return stepProperties; + } + + private Properties getJobProperties(Properties properties) { + return properties; + } + + @SuppressWarnings("unchecked") + private Map> getStepArtifactProperties(BeanDefinition beanDefinition, Properties properties) { + ManagedMap> stepArtifacts = new ManagedMap<>(); + stepArtifacts.setMergeEnabled(true); + + Map> existingArtifacts + = (Map>) beanDefinition.getPropertyValues().get(getPropertyName(batchArtifactType)); + + ManagedMap artifactProperties = new ManagedMap<>(); + artifactProperties.setMergeEnabled(true); + + if(existingArtifacts != null && existingArtifacts.containsKey(stepName)) { + Map existingArtifactsMap = existingArtifacts.get(stepName); + + for(Map.Entry existingArtifactEntry : existingArtifactsMap.entrySet()) { + artifactProperties.put(existingArtifactEntry.getKey(), existingArtifactEntry.getValue()); + } + } + + artifactProperties.put(beanName, properties); + stepArtifacts.put(stepName, artifactProperties); + + return stepArtifacts; + } + + private void setJobPropertiesBean(Properties properties) { + if (batchArtifactType.equals(BatchArtifactType.JOB)) { + Map jobProperties = new HashMap<>(); + + if (properties != null && !properties.isEmpty()) { + for (String param : properties.stringPropertyNames()) { + jobProperties.put(param, properties.getProperty(param)); + } + } + + BeanDefinition jobPropertiesBeanDefinition = parserContext.getRegistry().getBeanDefinition(JOB_PROPERTIES_BEAN_NAME); + jobPropertiesBeanDefinition.getConstructorArgumentValues().addGenericArgumentValue(jobProperties); + } + } + + private String getPropertyName(BatchArtifactType batchArtifactType) { + if(batchArtifactType.equals(BatchArtifactType.JOB)) { + return JOB_PROPERTIES_PROPERTY_NAME; + } else if (batchArtifactType.equals(BatchArtifactType.STEP)) { + return STEP_PROPERTIES_PROPERTY_NAME; + } else if (batchArtifactType.equals(BatchArtifactType.ARTIFACT)) { + return ARTIFACT_PROPERTIES_PROPERTY_NAME; + } else if (batchArtifactType.equals(BatchArtifactType.STEP_ARTIFACT)) { + return STEP_ARTIFACT_PROPERTIES_PROPERTY_NAME; + } else { + throw new IllegalStateException("Unhandled BatchArtifactType of: " + batchArtifactType); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/StepFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/StepFactoryBean.java new file mode 100644 index 0000000000..ce5509238e --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/StepFactoryBean.java @@ -0,0 +1,298 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import javax.batch.api.Batchlet; +import javax.batch.api.chunk.CheckpointAlgorithm; +import javax.batch.api.chunk.ItemProcessor; +import javax.batch.api.chunk.ItemReader; +import javax.batch.api.chunk.ItemWriter; +import javax.batch.api.partition.PartitionReducer; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.xml.StepParserStepFactoryBean; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.jsr.partition.JsrPartitionHandler; +import org.springframework.batch.core.jsr.step.batchlet.BatchletAdapter; +import org.springframework.batch.core.jsr.step.builder.JsrBatchletStepBuilder; +import org.springframework.batch.core.jsr.step.builder.JsrFaultTolerantStepBuilder; +import org.springframework.batch.core.jsr.step.builder.JsrPartitionStepBuilder; +import org.springframework.batch.core.jsr.step.builder.JsrSimpleStepBuilder; +import org.springframework.batch.core.step.builder.FaultTolerantStepBuilder; +import org.springframework.batch.core.step.builder.SimpleStepBuilder; +import org.springframework.batch.core.step.builder.StepBuilder; +import org.springframework.batch.core.step.builder.TaskletStepBuilder; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.batch.jsr.item.ItemProcessorAdapter; +import org.springframework.batch.jsr.item.ItemReaderAdapter; +import org.springframework.batch.jsr.item.ItemWriterAdapter; +import org.springframework.batch.jsr.repeat.CheckpointAlgorithmAdapter; +import org.springframework.batch.repeat.CompletionPolicy; +import org.springframework.batch.repeat.policy.CompositeCompletionPolicy; +import org.springframework.batch.repeat.policy.SimpleCompletionPolicy; +import org.springframework.batch.repeat.policy.TimeoutTerminationPolicy; +import org.springframework.beans.factory.FactoryBean; +import org.springframework.util.Assert; + +/** + * This {@link FactoryBean} is used by the JSR-352 namespace parser to create + * {@link Step} objects. It stores all of the properties that are + * configurable on the <step/>. + * + * @author Michael Minella + * @author Chris Schaefer + * @since 3.0 + */ +public class StepFactoryBean extends StepParserStepFactoryBean { + + @SuppressWarnings("unused") + private int partitions; + private BatchPropertyContext batchPropertyContext; + + private PartitionReducer reducer; + + private Integer timeout; + + public void setPartitionReducer(PartitionReducer reducer) { + this.reducer = reducer; + } + + public void setBatchPropertyContext(BatchPropertyContext context) { + this.batchPropertyContext = context; + } + + public void setPartitions(int partitions) { + this.partitions = partitions; + } + + /** + * Create a {@link Step} from the configuration provided. + * + * @see FactoryBean#getObject() + */ + @Override + public Step getObject() throws Exception { + if(hasPartitionElement()) { + return createPartitionStep(); + } + else if (hasChunkElement()) { + Assert.isTrue(!hasTasklet(), "Step [" + getName() + + "] has both a element and a 'ref' attribute referencing a Tasklet."); + + validateFaultTolerantSettings(); + + if (isFaultTolerant()) { + return createFaultTolerantStep(); + } + else { + return createSimpleStep(); + } + } + else if (hasTasklet()) { + return createTaskletStep(); + } + else { + return createFlowStep(); + } + } + + /** + * @return a new {@link TaskletStep} + */ + @Override + protected TaskletStep createTaskletStep() { + JsrBatchletStepBuilder jsrBatchletStepBuilder = new JsrBatchletStepBuilder(new StepBuilder(getName())); + jsrBatchletStepBuilder.setBatchPropertyContext(batchPropertyContext); + TaskletStepBuilder builder = jsrBatchletStepBuilder.tasklet(getTasklet()); + enhanceTaskletStepBuilder(builder); + return builder.build(); + } + + @Override + protected void setChunk(SimpleStepBuilder builder) { + if(timeout != null && getCommitInterval() != null) { + CompositeCompletionPolicy completionPolicy = new CompositeCompletionPolicy(); + CompletionPolicy [] policies = new CompletionPolicy[2]; + policies[0] = new SimpleCompletionPolicy(getCommitInterval()); + policies[1] = new TimeoutTerminationPolicy(timeout * 1000); + completionPolicy.setPolicies(policies); + builder.chunk(completionPolicy); + } else if(timeout != null) { + builder.chunk(new TimeoutTerminationPolicy(timeout * 1000)); + } else if(getCommitInterval() != null) { + builder.chunk(getCommitInterval()); + } + + if(getCompletionPolicy() != null) { + builder.chunk(getCompletionPolicy()); + } + } + + + @Override + protected Step createPartitionStep() { + // Creating a partitioned step for the JSR needs to create two steps...the partitioned step and the step being executed. + Step executedStep = null; + + if (hasChunkElement()) { + Assert.isTrue(!hasTasklet(), "Step [" + getName() + + "] has both a element and a 'ref' attribute referencing a Tasklet."); + + validateFaultTolerantSettings(); + + if (isFaultTolerant()) { + executedStep = createFaultTolerantStep(); + } + else { + executedStep = createSimpleStep(); + } + } + else if (hasTasklet()) { + executedStep = createTaskletStep(); + } + + ((JsrPartitionHandler) super.getPartitionHandler()).setStep(executedStep); + + JsrPartitionStepBuilder builder = new JsrSimpleStepBuilder(new StepBuilder(executedStep.getName())).partitioner(executedStep); + + enhanceCommonStep(builder); + + if (getPartitionHandler() != null) { + builder.partitionHandler(getPartitionHandler()); + } + + if(reducer != null) { + builder.reducer(reducer); + } + + builder.aggregator(getStepExecutionAggergator()); + + return builder.build(); + } + + /** + * Wraps a {@link Batchlet} in a {@link BatchletAdapter} if required for consumption + * by the rest of the framework. + * + * @param tasklet {@link Tasklet} or {@link Batchlet} implementation + * @throws IllegalArgumentException if tasklet does not implement either Tasklet or Batchlet + */ + public void setStepTasklet(Object tasklet) { + if(tasklet instanceof Tasklet) { + super.setTasklet((Tasklet) tasklet); + } else if(tasklet instanceof Batchlet){ + super.setTasklet(new BatchletAdapter((Batchlet) tasklet)); + } else { + throw new IllegalArgumentException("The field tasklet must reference an implementation of " + + "either org.springframework.batch.core.step.tasklet.Tasklet or javax.batch.api.Batchlet"); + } + } + + /** + * Wraps a {@link ItemReader} in a {@link ItemReaderAdapter} if required for consumption + * by the rest of the framework. + * + * @param itemReader {@link ItemReader} or {@link org.springframework.batch.item.ItemReader} implementation + * @throws IllegalArgumentException if itemReader does not implement either version of ItemReader + */ + @SuppressWarnings("unchecked") + public void setStepItemReader(Object itemReader) { + if(itemReader instanceof org.springframework.batch.item.ItemReader) { + super.setItemReader((org.springframework.batch.item.ItemReader) itemReader); + } else if(itemReader instanceof ItemReader){ + super.setItemReader(new ItemReaderAdapter<>((ItemReader) itemReader)); + } else { + throw new IllegalArgumentException("The definition of an item reader must implement either " + + "org.springframework.batch.item.ItemReader or javax.batch.api.chunk.ItemReader"); + } + } + + /** + * Wraps a {@link ItemProcessor} in a {@link ItemProcessorAdapter} if required for consumption + * by the rest of the framework. + * + * @param itemProcessor {@link ItemProcessor} or {@link org.springframework.batch.item.ItemProcessor} implementation + * @throws IllegalArgumentException if itemProcessor does not implement either version of ItemProcessor + */ + @SuppressWarnings("unchecked") + public void setStepItemProcessor(Object itemProcessor) { + if(itemProcessor instanceof org.springframework.batch.item.ItemProcessor) { + super.setItemProcessor((org.springframework.batch.item.ItemProcessor) itemProcessor); + } else if(itemProcessor instanceof ItemProcessor){ + super.setItemProcessor(new ItemProcessorAdapter<>((ItemProcessor) itemProcessor)); + } else { + throw new IllegalArgumentException("The definition of an item processor must implement either " + + "org.springframework.batch.item.ItemProcessor or javax.batch.api.chunk.ItemProcessor"); + } + } + + /** + * Wraps a {@link ItemWriter} in a {@link ItemWriterAdapter} if required for consumption + * by the rest of the framework. + * + * @param itemWriter {@link ItemWriter} or {@link org.springframework.batch.item.ItemWriter} implementation + * @throws IllegalArgumentException if itemWriter does not implement either version of ItemWriter + */ + @SuppressWarnings("unchecked") + public void setStepItemWriter(Object itemWriter) { + if(itemWriter instanceof org.springframework.batch.item.ItemWriter) { + super.setItemWriter((org.springframework.batch.item.ItemWriter) itemWriter); + } else if(itemWriter instanceof ItemWriter){ + super.setItemWriter(new ItemWriterAdapter<>((ItemWriter) itemWriter)); + } else { + throw new IllegalArgumentException("The definition of an item writer must implement either " + + "org.springframework.batch.item.ItemWriter or javax.batch.api.chunk.ItemWriter"); + } + } + + /** + * Wraps a {@link CheckpointAlgorithm} in a {@link CheckpointAlgorithmAdapter} if required for consumption + * by the rest of the framework. + * + * @param chunkCompletionPolicy {@link CompletionPolicy} or {@link CheckpointAlgorithm} implementation + * @throws IllegalArgumentException if chunkCompletionPolicy does not implement either CompletionPolicy or CheckpointAlgorithm + */ + public void setStepChunkCompletionPolicy(Object chunkCompletionPolicy) { + if(chunkCompletionPolicy instanceof CompletionPolicy) { + super.setChunkCompletionPolicy((CompletionPolicy) chunkCompletionPolicy); + } else if(chunkCompletionPolicy instanceof CheckpointAlgorithm) { + super.setChunkCompletionPolicy(new CheckpointAlgorithmAdapter((CheckpointAlgorithm) chunkCompletionPolicy)); + } else { + throw new IllegalArgumentException("The definition of a chunk completion policy must implement either " + + "org.springframework.batch.repeat.CompletionPolicy or javax.batch.api.chunk.CheckpointAlgorithm"); + } + } + + @Override + protected FaultTolerantStepBuilder getFaultTolerantStepBuilder(String stepName) { + JsrFaultTolerantStepBuilder jsrFaultTolerantStepBuilder = new JsrFaultTolerantStepBuilder<>( + new StepBuilder(stepName)); + jsrFaultTolerantStepBuilder.setBatchPropertyContext(batchPropertyContext); + return jsrFaultTolerantStepBuilder; + } + + @Override + protected SimpleStepBuilder getSimpleStepBuilder(String stepName) { + JsrSimpleStepBuilder jsrSimpleStepBuilder = new JsrSimpleStepBuilder<>(new StepBuilder(stepName)); + jsrSimpleStepBuilder.setBatchPropertyContext(batchPropertyContext); + return jsrSimpleStepBuilder; + } + + public void setTimeout(Integer timeout) { + this.timeout = timeout; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/StepParser.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/StepParser.java new file mode 100644 index 0000000000..a66d57c486 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/StepParser.java @@ -0,0 +1,103 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.springframework.batch.core.jsr.configuration.support.BatchArtifactType; +import org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.config.RuntimeBeanReference; +import org.springframework.beans.factory.parsing.BeanComponentDefinition; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.xml.AbstractSingleBeanDefinitionParser; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.StringUtils; +import org.w3c.dom.Element; +import org.w3c.dom.Node; +import org.w3c.dom.NodeList; + +import java.util.Collection; + +/** + * Parser for the <step /> element defined by JSR-352. + * + * @author Michael Minella + * @author Glenn Renfro + * @author Chris Schaefer + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class StepParser extends AbstractSingleBeanDefinitionParser { + private static final String CHUNK_ELEMENT = "chunk"; + private static final String BATCHLET_ELEMENT = "batchlet"; + private static final String ALLOW_START_IF_COMPLETE_ATTRIBUTE = "allow-start-if-complete"; + private static final String START_LIMIT_ATTRIBUTE = "start-limit"; + private static final String SPLIT_ID_ATTRIBUTE = "id"; + private static final String PARTITION_ELEMENT = "partition"; + + protected Collection parse(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { + BeanDefinitionBuilder defBuilder = BeanDefinitionBuilder.genericBeanDefinition(); + AbstractBeanDefinition bd = defBuilder.getRawBeanDefinition(); + bd.setBeanClass(StepFactoryBean.class); + bd.getPropertyValues().addPropertyValue("batchPropertyContext", new RuntimeBeanReference("batchPropertyContext")); + + BeanDefinitionBuilder stateBuilder = BeanDefinitionBuilder.genericBeanDefinition(JsrStepState.class); + + String stepName = element.getAttribute(SPLIT_ID_ATTRIBUTE); + builder.addPropertyValue("name", stepName); + + parserContext.registerBeanComponent(new BeanComponentDefinition(bd, stepName)); + stateBuilder.addConstructorArgReference(stepName); + + String startLimit = element.getAttribute(START_LIMIT_ATTRIBUTE); + if(StringUtils.hasText(startLimit)) { + bd.getPropertyValues().addPropertyValue("startLimit", startLimit); + } + + String allowStartIfComplete = element.getAttribute(ALLOW_START_IF_COMPLETE_ATTRIBUTE); + boolean allowStartIfCompleteValue = false; + if(StringUtils.hasText(allowStartIfComplete)) { + bd.getPropertyValues().addPropertyValue("allowStartIfComplete", + allowStartIfComplete); + allowStartIfCompleteValue = Boolean.valueOf(allowStartIfComplete); + } + + new ListenerParser(JsrStepListenerFactoryBean.class, "listeners").parseListeners(element, parserContext, bd, stepName); + new PropertyParser(stepName, parserContext, BatchArtifactType.STEP, stepName).parseProperties(element); + + // look at all nested elements + NodeList children = element.getChildNodes(); + + for (int i = 0; i < children.getLength(); i++) { + Node nd = children.item(i); + + if (nd instanceof Element) { + Element nestedElement = (Element) nd; + String name = nestedElement.getLocalName(); + + if(name.equalsIgnoreCase(BATCHLET_ELEMENT)) { + new BatchletParser().parseBatchlet(nestedElement, bd, parserContext, stepName); + } else if(name.equals(CHUNK_ELEMENT)) { + new ChunkParser().parse(nestedElement, bd, parserContext, stepName); + } else if(name.equals(PARTITION_ELEMENT)) { + new PartitionParser(stepName, allowStartIfCompleteValue).parse(nestedElement, bd, parserContext, stepName); + } + } + } + + return FlowParser.getNextElements(parserContext, stepName, stateBuilder.getBeanDefinition(), element); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/package-info.java new file mode 100644 index 0000000000..582712ada6 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/configuration/xml/package-info.java @@ -0,0 +1,10 @@ +/** + * XML parsers for JSR-352 based Job Specification Language (JSL). + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.configuration.xml; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/JsrStepHandler.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/JsrStepHandler.java new file mode 100644 index 0000000000..4a2e242edc --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/JsrStepHandler.java @@ -0,0 +1,153 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.job; + +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.StartLimitExceededException; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.job.SimpleStepHandler; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.util.Assert; +import org.springframework.util.CollectionUtils; +import org.springframework.util.StringUtils; + +/** + * Extends {@link SimpleStepHandler} to apply JSR-352 specific logic for whether to + * start a step. + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrStepHandler extends SimpleStepHandler { + + private static final Log logger = LogFactory.getLog(JsrStepHandler.class); + + private JobExplorer jobExplorer; + + /** + * @param jobRepository instance of {@link JobRepository}. + * @param jobExplorer instance of {@link JobExplorer}. + */ + public JsrStepHandler(JobRepository jobRepository, JobExplorer jobExplorer) { + super(jobRepository, new ExecutionContext()); + this.jobExplorer = jobExplorer; + } + + @Override + public void afterPropertiesSet() throws Exception { + super.afterPropertiesSet(); + Assert.state(jobExplorer != null, "A JobExplorer must be provided"); + } + + + /** + * Given a step and configuration, return true if the step should start, + * false if it should not, and throw an exception if the job should finish. + * @param lastStepExecution the last step execution + * @param jobExecution instance of {@link JobExecution} + * @param step instance of {@link Step} + * + * @throws StartLimitExceededException if the start limit has been exceeded + * for this step + * @throws JobRestartException if the job is in an inconsistent state from + * an earlier failure + */ + @Override + protected boolean shouldStart(StepExecution lastStepExecution, JobExecution jobExecution, Step step) + throws JobRestartException, StartLimitExceededException { + BatchStatus stepStatus; + String restartStep = null; + if (lastStepExecution == null) { + jobExecution.getExecutionContext().put("batch.startedStep", step.getName()); + stepStatus = BatchStatus.STARTING; + } + else { + stepStatus = lastStepExecution.getStatus(); + + JobExecution lastJobExecution = getLastJobExecution(jobExecution); + + if(lastJobExecution.getExecutionContext().containsKey("batch.restartStep")) { + restartStep = lastJobExecution.getExecutionContext().getString("batch.restartStep"); + + if(CollectionUtils.isEmpty(jobExecution.getStepExecutions()) && lastJobExecution.getStatus() == BatchStatus.STOPPED && StringUtils.hasText(restartStep)) { + if(!restartStep.equals(step.getName()) && !jobExecution.getExecutionContext().containsKey("batch.startedStep")) { + logger.info("Job was stopped and should restart at step " + restartStep + ". The current step is " + step.getName()); + return false; + } else { + // Indicates the starting point for execution evaluation per JSR-352 + jobExecution.getExecutionContext().put("batch.startedStep", step.getName()); + } + } + } + } + + if (stepStatus == BatchStatus.UNKNOWN) { + throw new JobRestartException("Cannot restart step from UNKNOWN status. " + + "The last execution ended with a failure that could not be rolled back, " + + "so it may be dangerous to proceed. Manual intervention is probably necessary."); + } + + if ((stepStatus == BatchStatus.COMPLETED && step.isAllowStartIfComplete() == false) + || stepStatus == BatchStatus.ABANDONED) { + // step is complete, false should be returned, indicating that the + // step should not be started + logger.info("Step already complete or not restartable, so no action to execute: " + lastStepExecution); + return false; + } + + if (getJobRepository().getStepExecutionCount(jobExecution.getJobInstance(), step.getName()) < step.getStartLimit()) { + // step start count is less than start max, return true + return true; + } + else { + // start max has been exceeded, throw an exception. + throw new StartLimitExceededException("Maximum start limit exceeded for step: " + step.getName() + + "StartMax: " + step.getStartLimit()); + } + } + + /** + * Since all JSR-352 jobs are run asynchronously, {@link JobRepository#getLastJobExecution(String, org.springframework.batch.core.JobParameters)} + * could return the currently running {@link JobExecution}. To get around this, we use the {@link JobExplorer} + * to get a list of the executions and get the most recent one that is not the currently running + * {@link JobExecution}. + * + * @param jobExecution + * @return the last executed JobExecution. + */ + private JobExecution getLastJobExecution(JobExecution jobExecution) { + List jobExecutions = jobExplorer.getJobExecutions(jobExecution.getJobInstance()); + JobExecution lastJobExecution = null; + + for (JobExecution curJobExecution : jobExecutions) { + if(lastJobExecution == null && curJobExecution.getId().longValue() != jobExecution.getId().longValue()) { + lastJobExecution = curJobExecution; + } else if(curJobExecution.getId().longValue() != jobExecution.getId().longValue() && (lastJobExecution == null || curJobExecution.getId().longValue() > lastJobExecution.getId().longValue())) { + lastJobExecution = curJobExecution; + } + } + return lastJobExecution; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/JsrFlowExecutor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/JsrFlowExecutor.java new file mode 100644 index 0000000000..4b955a0bb2 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/JsrFlowExecutor.java @@ -0,0 +1,69 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.job.flow; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.job.StepHandler; +import org.springframework.batch.core.job.flow.FlowExecutionStatus; +import org.springframework.batch.core.job.flow.JobFlowExecutor; +import org.springframework.batch.core.repository.JobRepository; + +/** + * JSR-352 specific {@link JobFlowExecutor}. Unlike the regular {@link JobFlowExecutor}, + * this extension does not promote an {@link ExitStatus} from a step to the job level if + * a custom exit status has been set on the job. + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrFlowExecutor extends JobFlowExecutor { + + public JsrFlowExecutor(JobRepository jobRepository, + StepHandler stepHandler, JobExecution execution) { + super(jobRepository, stepHandler, execution); + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.job.flow.JobFlowExecutor#addExitStatus(java.lang.String) + */ + @Override + public void addExitStatus(String code) { + ExitStatus status = new ExitStatus(code); + if((exitStatus != null && ExitStatus.isNonDefaultExitStatus(exitStatus)) && !ExitStatus.isNonDefaultExitStatus(status)) { + exitStatus = exitStatus.and(status); + } + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.job.flow.JobFlowExecutor#updateJobExecutionStatus(org.springframework.batch.core.job.flow.FlowExecutionStatus) + */ + @Override + public void updateJobExecutionStatus(FlowExecutionStatus status) { + JobExecution execution = super.getJobExecution(); + + execution.setStatus(findBatchStatus(status)); + + ExitStatus curStatus = execution.getExitStatus(); + if(ExitStatus.isNonDefaultExitStatus(curStatus)) { + exitStatus = exitStatus.and(new ExitStatus(status.getName())); + execution.setExitStatus(exitStatus); + } else { + exitStatus = exitStatus.and(curStatus); + execution.setExitStatus(exitStatus); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/JsrFlowJob.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/JsrFlowJob.java new file mode 100644 index 0000000000..bf69e20ad9 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/JsrFlowJob.java @@ -0,0 +1,141 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.job.flow; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobExecutionException; +import org.springframework.batch.core.JobInterruptedException; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.xml.SimpleFlowFactoryBean.DelegateState; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.job.AbstractJob; +import org.springframework.batch.core.job.flow.Flow; +import org.springframework.batch.core.job.flow.FlowExecutionException; +import org.springframework.batch.core.job.flow.FlowJob; +import org.springframework.batch.core.job.flow.JobFlowExecutor; +import org.springframework.batch.core.job.flow.State; +import org.springframework.batch.core.job.flow.support.state.FlowState; +import org.springframework.batch.core.jsr.job.JsrStepHandler; +import org.springframework.batch.core.jsr.job.flow.support.JsrFlow; +import org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState; +import org.springframework.batch.core.jsr.step.DecisionStep; +import org.springframework.batch.core.launch.NoSuchJobException; +import org.springframework.batch.core.launch.support.ExitCodeMapper; + +/** + * JSR-352 specific extension of the {@link FlowJob}. + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrFlowJob extends FlowJob { + + private JobExplorer jobExplorer; + + /** + * No arg constructor (invalid state) + */ + public JsrFlowJob() { + super(); + } + + /** + * Main constructor + * + * @param name of the flow + */ + public JsrFlowJob(String name) { + super(name); + } + + public void setJobExplorer(JobExplorer jobExplorer) { + this.jobExplorer = jobExplorer; + } + + /** + * @see AbstractJob#doExecute(JobExecution) + */ + @Override + protected void doExecute(final JobExecution execution) throws JobExecutionException { + try { + JobFlowExecutor executor = new JsrFlowExecutor(getJobRepository(), + new JsrStepHandler(getJobRepository(), jobExplorer), execution); + + State startState = ((JsrFlow)flow).getStartState(); + + validateFirstStep(startState); + + executor.updateJobExecutionStatus(flow.start(executor).getStatus()); + } + catch (FlowExecutionException e) { + if (e.getCause() instanceof JobExecutionException) { + throw (JobExecutionException) e.getCause(); + } + throw new JobExecutionException("Flow execution ended unexpectedly", e); + } + } + + private void validateFirstStep(State startState) + throws JobExecutionException { + while(true) { + if(startState instanceof DelegateState) { + startState = ((DelegateState) startState).getState(); + } else if(startState instanceof JsrStepState) { + String stepName = startState.getName().substring(startState.getName().indexOf(".") + 1, startState.getName().length()); + Step step = ((JsrStepState) startState).getStep(stepName); + if(step instanceof DecisionStep) { + throw new JobExecutionException("Decision step is an invalid first step"); + } else { + break; + } + } else if(startState instanceof FlowState){ + Flow firstFlow = ((FlowState) startState).getFlows().iterator().next(); + startState = firstFlow.getStates().iterator().next(); + } else { + break; + } + } + } + + /** + * Default mapping from throwable to {@link ExitStatus}. + * + * @param ex the cause of the failure + * @return an {@link ExitStatus} + */ + @Override + protected ExitStatus getDefaultExitStatusForFailure(Throwable ex, JobExecution execution) { + if(!ExitStatus.isNonDefaultExitStatus(execution.getExitStatus())) { + return execution.getExitStatus(); + } else { + ExitStatus exitStatus; + if (ex instanceof JobInterruptedException + || ex.getCause() instanceof JobInterruptedException) { + exitStatus = ExitStatus.STOPPED + .addExitDescription(JobInterruptedException.class.getName()); + } else if (ex instanceof NoSuchJobException + || ex.getCause() instanceof NoSuchJobException) { + exitStatus = new ExitStatus(ExitCodeMapper.NO_SUCH_JOB, ex + .getClass().getName()); + } else { + exitStatus = ExitStatus.FAILED.addExitDescription(ex); + } + + return exitStatus; + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/package-info.java new file mode 100644 index 0000000000..e40bd8b74e --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/package-info.java @@ -0,0 +1,10 @@ +/** + * JSR-352 specific extensions of Flow constructs (executor and job). + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.job.flow; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/JsrFlow.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/JsrFlow.java new file mode 100644 index 0000000000..442d6d392d --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/JsrFlow.java @@ -0,0 +1,152 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.job.flow.support; + +import java.util.Set; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.configuration.xml.SimpleFlowFactoryBean.DelegateState; +import org.springframework.batch.core.job.flow.Flow; +import org.springframework.batch.core.job.flow.FlowExecutionException; +import org.springframework.batch.core.job.flow.FlowExecutionStatus; +import org.springframework.batch.core.job.flow.State; +import org.springframework.batch.core.job.flow.support.SimpleFlow; +import org.springframework.batch.core.job.flow.support.StateTransition; +import org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.lang.Nullable; +import org.springframework.util.StringUtils; + +/** + * Implements JSR-352 specific logic around the execution of a flow. Specifically, this + * {@link Flow} implementation will attempt to find the next state based on the provided + * exit status. If none is found (the exit status isn't mapped), it will attempt to + * resolve the next state basing it on the last step's batch status. Only if both + * attempts fail, the flow will fail due to the inability to find the next state. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class JsrFlow extends SimpleFlow { + + private JsrStepState currentStep; + + /** + * @param name name of the flow + */ + public JsrFlow(String name) { + super(name); + } + + @Nullable + public String getMostRecentStepName() { + if(currentStep != null) { + return currentStep.getStep().getName(); + } else { + return null; + } + } + + @Override + protected boolean isFlowContinued(State state, FlowExecutionStatus status, StepExecution stepExecution) { + if(state instanceof DelegateState) { + state = ((DelegateState) state).getState(); + } + + if(state instanceof JsrStepState) { + currentStep = (JsrStepState) state; + } + + return super.isFlowContinued(state, status, stepExecution); + } + + @Override + protected State nextState(String stateName, FlowExecutionStatus status, StepExecution stepExecution) throws FlowExecutionException { + State nextState = findState(stateName, status, stepExecution); + + if(stepExecution != null) { + ExecutionContext executionContext = stepExecution.getJobExecution().getExecutionContext(); + if(executionContext.containsKey("batch.stoppedStep")) { + String stepName = executionContext.getString("batch.stoppedStep"); + + if(stateName.endsWith(stepName)) { + if(nextState != null && executionContext.containsKey("batch.restartStep") && StringUtils.hasText(executionContext.getString("batch.restartStep"))) { + nextState = findState(stateName, new FlowExecutionStatus(status.getName() + ".RESTART"), stepExecution); + } + } + } + } + + return nextState; + } + + /** + * @return the next {@link Step} (or null if this is the end) + * @throws FlowExecutionException + */ + private State findState(String stateName, FlowExecutionStatus status, StepExecution stepExecution) throws FlowExecutionException { + Set set = getTransitionMap().get(stateName); + + if (set == null) { + throw new FlowExecutionException(String.format("No transitions found in flow=%s for state=%s", getName(), + stateName)); + } + + String next = null; + String exitCode = status.getName(); + for (StateTransition stateTransition : set) { + if (stateTransition.matches(exitCode) || (exitCode.equals("PENDING") && stateTransition.matches("STOPPED"))) { + if (stateTransition.isEnd()) { + // End of job + return null; + } + next = stateTransition.getNext(); + break; + } + } + + if (next == null) { + if(stepExecution != null) { + exitCode = stepExecution.getStatus().toString(); + + for (StateTransition stateTransition : set) { + if (stateTransition.matches(exitCode) || (exitCode.equals("PENDING") && stateTransition.matches("STOPPED"))) { + if (stateTransition.isEnd()) { + // End of job + return null; + } + next = stateTransition.getNext(); + break; + } + } + } + + if(next == null) { + throw new FlowExecutionException(String.format( + "Next state not found in flow=%s for state=%s with exit status=%s", getName(), stateName, status.getName())); + } + } + + if (!getStateMap().containsKey(next)) { + throw new FlowExecutionException(String.format("Next state not specified in flow=%s for next=%s", + getName(), next)); + } + + return getStateMap().get(next); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/package-info.java new file mode 100644 index 0000000000..b919779714 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/package-info.java @@ -0,0 +1,10 @@ +/** + * JSR-352 specific flow extensions. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.job.flow.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/state/JsrEndState.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/state/JsrEndState.java new file mode 100644 index 0000000000..e5b3a7a3ad --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/state/JsrEndState.java @@ -0,0 +1,135 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.job.flow.support.state; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.job.flow.FlowExecutionStatus; +import org.springframework.batch.core.job.flow.FlowExecutor; +import org.springframework.batch.core.job.flow.State; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.item.ExecutionContext; + +/** + * {@link State} implementation for ending a job per JSR-352 rules if it is + * in progress and continuing if just starting. + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrEndState extends org.springframework.batch.core.job.flow.support.state.EndState { + + private JobRepository jobRepository; + private String restart; + + /** + * @param status The {@link FlowExecutionStatus} to end with + * @param name The name of the state + */ + public JsrEndState(FlowExecutionStatus status, String name) { + super(status, status.getName(), name); + } + + /** + * @param status The {@link FlowExecutionStatus} to end with + * @param name The name of the state + * @param code the exit status. + */ + public JsrEndState(FlowExecutionStatus status, String code, String name) { + super(status, code, name, false); + } + + /** + * @param status The {@link FlowExecutionStatus} to end with + * @param name The name of the state + * @param abandon flag to indicate that previous step execution can be + * marked as abandoned (if there is one) + * @param code the exit status. + * + */ + public JsrEndState(FlowExecutionStatus status, String code, String name, boolean abandon) { + super(status, code, name, abandon); + } + + public JsrEndState(FlowExecutionStatus status, String code, String name, String restart, boolean abandon, JobRepository jobRepository) { + super(status, code, name, abandon); + this.jobRepository = jobRepository; + this.restart = restart; + } + + @Override + public FlowExecutionStatus handle(FlowExecutor executor) + throws Exception { + synchronized (executor) { + + // Special case. If the last step execution could not complete we + // are in an unknown state (possibly unrecoverable). + StepExecution stepExecution = executor.getStepExecution(); + if (stepExecution != null && executor.getStepExecution().getStatus() == BatchStatus.UNKNOWN) { + return FlowExecutionStatus.UNKNOWN; + } + + if (getStatus().isStop()) { + JobExecution jobExecution = stepExecution.getJobExecution(); + ExecutionContext executionContext = jobExecution.getExecutionContext(); + executionContext.put("batch.restartStep", restart); + executionContext.put("batch.stoppedStep", stepExecution.getStepName()); + jobRepository.updateExecutionContext(jobExecution); + + if (!executor.isRestart()) { + /* + * If there are step executions, then we are not at the + * beginning of a restart. + */ + if (isAbandon()) { + /* + * Only if instructed to do so, upgrade the status of + * last step execution so it is not replayed on a + * restart... + */ + executor.abandonStepExecution(); + } + } + else { + /* + * If we are a stop state and we got this far then it must + * be a restart, so return COMPLETED. + */ + return FlowExecutionStatus.COMPLETED; + } + } + + setExitStatus(executor, getCode()); + + return getStatus(); + } + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.job.flow.support.state.EndState#setExitStatus(org.springframework.batch.core.job.flow.FlowExecutor, java.lang.String) + */ + @Override + protected void setExitStatus(FlowExecutor executor, String code) { + StepExecution stepExecution = executor.getStepExecution(); + + ExitStatus status = new ExitStatus(code); + if(!ExitStatus.isNonDefaultExitStatus(status)) { + stepExecution.getJobExecution().setExitStatus(status); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/state/JsrSplitState.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/state/JsrSplitState.java new file mode 100644 index 0000000000..c2bfd154ca --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/state/JsrSplitState.java @@ -0,0 +1,73 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.job.flow.support.state; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.job.flow.Flow; +import org.springframework.batch.core.job.flow.FlowExecution; +import org.springframework.batch.core.job.flow.FlowExecutionStatus; +import org.springframework.batch.core.job.flow.FlowExecutor; +import org.springframework.batch.core.jsr.job.flow.support.JsrFlow; + +/** + * JSR-352 states that artifacts cannot set the ExitStatus from within a split for a job. Because + * of this, this state will reset the exit status once the flows have completed (prior to aggregation + * of the results). + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrSplitState extends org.springframework.batch.core.job.flow.support.state.SplitState { + + /** + * @param flows {@link Flow}s to be executed in parallel + * @param name the name to be associated with the split state. + */ + public JsrSplitState(Collection flows, String name) { + super(flows, name); + } + + /** + * Resets the {@link JobExecution}'s exit status before aggregating the results of the flows within + * the split. + * + * @param results the {@link FlowExecution}s from each of the flows executed within this split + * @param executor the {@link FlowExecutor} used to execute the flows + */ + @Override + protected FlowExecutionStatus doAggregation(Collection results, FlowExecutor executor) { + List stepNames = new ArrayList<>(); + + for (Flow curFlow : getFlows()) { + JsrFlow flow = (JsrFlow) curFlow; + if(flow.getMostRecentStepName() != null) { + stepNames.add(flow.getMostRecentStepName()); + } + } + + if(!stepNames.isEmpty()) { + executor.getJobExecution().getExecutionContext().put("batch.lastSteps", stepNames); + } + + executor.getJobExecution().setExitStatus(null); + + return super.doAggregation(results, executor); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/state/JsrStepState.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/state/JsrStepState.java new file mode 100644 index 0000000000..c5e3dc1781 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/state/JsrStepState.java @@ -0,0 +1,59 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.job.flow.support.state; + +import java.util.Collections; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.job.flow.FlowExecutionStatus; +import org.springframework.batch.core.job.flow.FlowExecutor; + +/** + * Extends {@link org.springframework.batch.core.job.flow.support.state.StepState} to persist what the + * last step that was executed was (used in Decisions and restarts). + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrStepState extends org.springframework.batch.core.job.flow.support.state.StepState { + + /** + * @param step the step that will be executed + */ + public JsrStepState(Step step) { + super(step); + } + + /** + * @param name for the step that will be executed + * @param step the step that will be executed + */ + public JsrStepState(String name, Step step) { + super(name, step); + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.job.flow.support.state.StepState#handle(org.springframework.batch.core.job.flow.FlowExecutor) + */ + @Override + public FlowExecutionStatus handle(FlowExecutor executor) throws Exception { + FlowExecutionStatus result = super.handle(executor); + + executor.getJobExecution().getExecutionContext().put("batch.lastSteps", Collections.singletonList(getStep().getName())); + + return result; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/state/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/state/package-info.java new file mode 100644 index 0000000000..4335b22ea2 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/flow/support/state/package-info.java @@ -0,0 +1,10 @@ +/** + * JSR-352 specific states used in flow execution. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.job.flow.support.state; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/package-info.java new file mode 100644 index 0000000000..7592416e94 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/job/package-info.java @@ -0,0 +1,10 @@ +/** + * JSR-352 specific handler implementations. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.job; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/launch/JsrJobOperator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/launch/JsrJobOperator.java new file mode 100644 index 0000000000..51c11a9741 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/launch/JsrJobOperator.java @@ -0,0 +1,850 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.launch; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.Enumeration; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Properties; +import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.Semaphore; +import javax.batch.operations.BatchRuntimeException; +import javax.batch.operations.JobExecutionAlreadyCompleteException; +import javax.batch.operations.JobExecutionIsRunningException; +import javax.batch.operations.JobExecutionNotMostRecentException; +import javax.batch.operations.JobExecutionNotRunningException; +import javax.batch.operations.JobOperator; +import javax.batch.operations.JobRestartException; +import javax.batch.operations.JobSecurityException; +import javax.batch.operations.JobStartException; +import javax.batch.operations.NoSuchJobException; +import javax.batch.operations.NoSuchJobExecutionException; +import javax.batch.operations.NoSuchJobInstanceException; +import javax.batch.runtime.BatchRuntime; +import javax.batch.runtime.JobExecution; +import javax.batch.runtime.JobInstance; +import javax.batch.runtime.StepExecution; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.DuplicateJobException; +import org.springframework.batch.core.converter.JobParametersConverter; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.jsr.JsrJobContextFactoryBean; +import org.springframework.batch.core.jsr.JsrJobExecution; +import org.springframework.batch.core.jsr.JsrJobParametersConverter; +import org.springframework.batch.core.jsr.JsrStepExecution; +import org.springframework.batch.core.jsr.configuration.xml.JsrXmlApplicationContext; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.batch.core.step.NoSuchStepException; +import org.springframework.batch.core.step.StepLocator; +import org.springframework.batch.core.step.tasklet.StoppableTasklet; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.BeanCreationException; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.beans.factory.config.AutowireCapableBeanFactory; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.context.ApplicationContext; +import org.springframework.context.ApplicationContextAware; +import org.springframework.context.support.GenericXmlApplicationContext; +import org.springframework.core.convert.converter.Converter; +import org.springframework.core.io.ClassPathResource; +import org.springframework.core.io.Resource; +import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.core.task.TaskExecutor; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.util.Assert; + +/** + * The entrance for executing batch jobs as defined by JSR-352. This class provides + * a single base {@link ApplicationContext} that is the equivalent to the following: + * + * <beans> + * <batch:job-repository id="jobRepository" ... /> + * + * <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> + * ... + * </bean> + * + * <bean id="batchJobOperator" class="org.springframework.batch.core.launch.support.SimpleJobOperator"> + * ... + * </bean> + * + * <bean id="jobExplorer" class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean"> + * ... + * </bean> + * + * <bean id="dataSource" + * class="org.apache.commons.dbcp2.BasicDataSource"> + * ... + * </bean> + * + * <bean id="transactionManager" + * class="org.springframework.jdbc.datasource.DataSourceTransactionManager"> + * ... + * </bean> + * + * <bean id="jobParametersConverter" class="org.springframework.batch.core.jsr.JsrJobParametersConverter"/> + * + * <bean id="jobRegistry" class="org.springframework.batch.core.configuration.support.MapJobRegistry"/> + * + * <bean id="placeholderProperties" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> + * ... + * </bean> + * </beans> + * + * A custom configuration of the above components can be specified by providing a system property JSR-352-BASE-CONTEXT. + * The location that is provided by this system property will override any beans as defined in baseContext.xml. + * + * Calls to {@link JobOperator#start(String, Properties)} will provide a child context to the above context + * using the job definition and batch.xml if provided. + * + * By default, calls to start/restart will result in asynchronous execution of the batch job (via an asynchronous {@link TaskExecutor}. + * For synchronous behavior or customization of thread behavior, a different {@link TaskExecutor} implementation is required to + * be provided. + * + * Note: This class is intended to only be used for JSR-352 configured jobs. Use of + * this {@link JobOperator} to start/stop/restart Spring Batch jobs may result in unexpected behaviors due to + * how job instances are identified differently. + * + * @author Michael Minella + * @author Chris Schaefer + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class JsrJobOperator implements JobOperator, ApplicationContextAware, InitializingBean { + private static final String JSR_JOB_CONTEXT_BEAN_NAME = "jsr_jobContext"; + private final Log logger = LogFactory.getLog(getClass()); + + private JobExplorer jobExplorer; + private JobRepository jobRepository; + private TaskExecutor taskExecutor; + private JobParametersConverter jobParametersConverter; + private ApplicationContext baseContext; + private PlatformTransactionManager transactionManager; + private static ExecutingJobRegistry jobRegistry = new ExecutingJobRegistry(); + + /** + * Public constructor used by {@link BatchRuntime#getJobOperator()}. This will bootstrap a + * singleton ApplicationContext if one has not already been created (and will utilize the existing + * one if it has) to populate itself. + */ + public JsrJobOperator() { + + this.baseContext = BaseContextHolder.getInstance().getContext(); + + baseContext.getAutowireCapableBeanFactory().autowireBeanProperties(this, + AutowireCapableBeanFactory.AUTOWIRE_BY_TYPE, false); + + if(taskExecutor == null) { + taskExecutor = new SimpleAsyncTaskExecutor(); + } + } + + /** + * The no-arg constructor is used by the {@link BatchRuntime#getJobOperator()} and so bootstraps + * an {@link ApplicationContext}. This constructor does not and is therefore dependency injection + * friendly. Also useful for unit testing. + * + * @param jobExplorer an instance of Spring Batch's {@link JobExplorer}. + * @param jobRepository an instance of Spring Batch's {@link JobOperator}. + * @param jobParametersConverter an instance of Spring Batch's {@link JobParametersConverter}. + * @param transactionManager a {@link PlatformTransactionManager}. + */ + public JsrJobOperator(JobExplorer jobExplorer, JobRepository jobRepository, JobParametersConverter jobParametersConverter, PlatformTransactionManager transactionManager) { + Assert.notNull(jobExplorer, "A JobExplorer is required"); + Assert.notNull(jobRepository, "A JobRepository is required"); + Assert.notNull(jobParametersConverter, "A ParametersConverter is required"); + Assert.notNull(transactionManager, "A PlatformTransactionManager is required"); + + this.jobExplorer = jobExplorer; + this.jobRepository = jobRepository; + this.jobParametersConverter = jobParametersConverter; + this.transactionManager = transactionManager; + } + + public void setJobExplorer(JobExplorer jobExplorer) { + Assert.notNull(jobExplorer, "A JobExplorer is required"); + + this.jobExplorer = jobExplorer; + } + + public void setJobRepository(JobRepository jobRepository) { + Assert.notNull(jobRepository, "A JobRepository is required"); + + this.jobRepository = jobRepository; + } + + public void setTransactionManager(PlatformTransactionManager transactionManager) { + Assert.notNull(transactionManager, "A PlatformTransactionManager is required"); + + this.transactionManager = transactionManager; + } + + public void setTaskExecutor(TaskExecutor taskExecutor) { + this.taskExecutor = taskExecutor; + } + + protected TaskExecutor getTaskExecutor() { + return taskExecutor; + } + + @Override + public void afterPropertiesSet() throws Exception { + if (this.taskExecutor == null) { + this.taskExecutor = new SimpleAsyncTaskExecutor(); + } + } + + /** + * Used to convert the {@link Properties} objects used by JSR-352 to the {@link JobParameters} + * objects used in Spring Batch. The default implementation used will configure all parameters + * to be non-identifying (per the JSR). + * + * @param converter A {@link Converter} implementation used to convert {@link Properties} to + * {@link JobParameters} + */ + public void setJobParametersConverter(JobParametersConverter converter) { + Assert.notNull(converter, "A Converter is required"); + + this.jobParametersConverter = converter; + } + + /* (non-Javadoc) + * @see javax.batch.operations.JobOperator#abandon(long) + */ + @Override + public void abandon(long jobExecutionId) throws NoSuchJobExecutionException, + JobExecutionIsRunningException, JobSecurityException { + org.springframework.batch.core.JobExecution jobExecution = jobExplorer.getJobExecution(jobExecutionId); + + if(jobExecution == null) { + throw new NoSuchJobExecutionException("Unable to retrieve JobExecution for id " + jobExecutionId); + } + + if(jobExecution.isRunning()) { + throw new JobExecutionIsRunningException("Unable to abandon a job that is currently running"); + } + + jobExecution.upgradeStatus(BatchStatus.ABANDONED); + jobRepository.update(jobExecution); + } + + /* (non-Javadoc) + * @see javax.batch.operations.JobOperator#getJobExecution(long) + */ + @Override + public JobExecution getJobExecution(long executionId) + throws NoSuchJobExecutionException, JobSecurityException { + org.springframework.batch.core.JobExecution jobExecution = jobExplorer.getJobExecution(executionId); + + if(jobExecution == null) { + throw new NoSuchJobExecutionException("No execution was found for executionId " + executionId); + } + + return new JsrJobExecution(jobExecution, jobParametersConverter); + } + + /* (non-Javadoc) + * @see javax.batch.operations.JobOperator#getJobExecutions(javax.batch.runtime.JobInstance) + */ + @Override + public List getJobExecutions(JobInstance jobInstance) + throws NoSuchJobInstanceException, JobSecurityException { + if(jobInstance == null) { + throw new NoSuchJobInstanceException("A null JobInstance was provided"); + } + + org.springframework.batch.core.JobInstance instance = (org.springframework.batch.core.JobInstance) jobInstance; + List batchExecutions = jobExplorer.getJobExecutions(instance); + + if(batchExecutions == null || batchExecutions.size() == 0) { + throw new NoSuchJobInstanceException("Unable to find JobInstance " + jobInstance.getInstanceId()); + } + + List results = new ArrayList<>(batchExecutions.size()); + for (org.springframework.batch.core.JobExecution jobExecution : batchExecutions) { + results.add(new JsrJobExecution(jobExecution, jobParametersConverter)); + } + + return results; + } + + /* (non-Javadoc) + * @see javax.batch.operations.JobOperator#getJobInstance(long) + */ + @Override + public JobInstance getJobInstance(long executionId) + throws NoSuchJobExecutionException, JobSecurityException { + org.springframework.batch.core.JobExecution execution = jobExplorer.getJobExecution(executionId); + + if(execution == null) { + throw new NoSuchJobExecutionException("The JobExecution was not found"); + } + + return jobExplorer.getJobInstance(execution.getJobInstance().getId()); + } + + /* (non-Javadoc) + * @see javax.batch.operations.JobOperator#getJobInstanceCount(java.lang.String) + */ + @Override + public int getJobInstanceCount(String jobName) throws NoSuchJobException, + JobSecurityException { + try { + int count = jobExplorer.getJobInstanceCount(jobName); + + if(count <= 0) { + throw new NoSuchJobException("No job instances were found for job name " + jobName); + } else { + return count; + } + } catch (org.springframework.batch.core.launch.NoSuchJobException e) { + throw new NoSuchJobException("No job instances were found for job name " + jobName); + } + } + + /* (non-Javadoc) + * @see javax.batch.operations.JobOperator#getJobInstances(java.lang.String, int, int) + */ + @Override + public List getJobInstances(String jobName, int start, int count) + throws NoSuchJobException, JobSecurityException { + List jobInstances = jobExplorer.getJobInstances(jobName, start, count); + + if(jobInstances == null || jobInstances.size() == 0) { + throw new NoSuchJobException("The job was not found"); + } + + return new ArrayList<>(jobInstances); + } + + /* (non-Javadoc) + * @see javax.batch.operations.JobOperator#getJobNames() + */ + @Override + public Set getJobNames() throws JobSecurityException { + return new HashSet<>(jobExplorer.getJobNames()); + } + + /* (non-Javadoc) + * @see javax.batch.operations.JobOperator#getParameters(long) + */ + @Override + public Properties getParameters(long executionId) + throws NoSuchJobExecutionException, JobSecurityException { + org.springframework.batch.core.JobExecution execution = jobExplorer.getJobExecution(executionId); + + if(execution == null) { + throw new NoSuchJobExecutionException("Unable to find the JobExecution for id " + executionId); + } + + Properties properties = jobParametersConverter.getProperties(execution.getJobParameters()); + properties.remove(JsrJobParametersConverter.JOB_RUN_ID); + + return properties; + } + + /* (non-Javadoc) + * @see javax.batch.operations.JobOperator#getRunningExecutions(java.lang.String) + */ + @Override + public List getRunningExecutions(String name) + throws NoSuchJobException, JobSecurityException { + Set findRunningJobExecutions = jobExplorer.findRunningJobExecutions(name); + + if(findRunningJobExecutions.isEmpty()) { + throw new NoSuchJobException("Job name: " + name + " not found."); + } + + List results = new ArrayList<>(findRunningJobExecutions.size()); + + for (org.springframework.batch.core.JobExecution jobExecution : findRunningJobExecutions) { + results.add(jobExecution.getId()); + } + + return results; + } + + /* (non-Javadoc) + * @see javax.batch.operations.JobOperator#getStepExecutions(long) + */ + @Override + public List getStepExecutions(long executionId) + throws NoSuchJobExecutionException, JobSecurityException { + org.springframework.batch.core.JobExecution execution = jobExplorer.getJobExecution(executionId); + + if(execution == null) { + throw new NoSuchJobException("JobExecution with the id " + executionId + " was not found"); + } + + Collection executions = execution.getStepExecutions(); + + List batchExecutions = new ArrayList<>(); + + if(executions != null) { + for (org.springframework.batch.core.StepExecution stepExecution : executions) { + if(!stepExecution.getStepName().contains(":partition")) { + batchExecutions.add(new JsrStepExecution(jobExplorer.getStepExecution(executionId, stepExecution.getId()))); + } + } + } + + return batchExecutions; + } + + /** + * Creates a child {@link ApplicationContext} for the job being requested based upon + * the /META-INF/batch.xml (if exists) and the /META-INF/batch-jobs/<jobName>.xml + * configuration and restart the job. + * + * @param executionId the database id of the job execution to be restarted. + * @param params any job parameters to be used during the execution of this job. + * @throws JobExecutionAlreadyCompleteException thrown if the requested job execution has + * a status of COMPLETE + * @throws NoSuchJobExecutionException throw if the requested job execution does not exist + * in the repository + * @throws JobExecutionNotMostRecentException thrown if the requested job execution is not + * the most recent attempt for the job instance it's related to. + * @throws JobRestartException thrown for any general errors during the job restart process + */ + @Override + public long restart(long executionId, Properties params) + throws JobExecutionAlreadyCompleteException, + NoSuchJobExecutionException, JobExecutionNotMostRecentException, + JobRestartException, JobSecurityException { + org.springframework.batch.core.JobExecution previousJobExecution = jobExplorer.getJobExecution(executionId); + + if (previousJobExecution == null) { + throw new NoSuchJobExecutionException("No JobExecution found for id: [" + executionId + "]"); + } else if(previousJobExecution.getStatus().equals(BatchStatus.COMPLETED)) { + throw new JobExecutionAlreadyCompleteException("The requested job has already completed"); + } + + List previousExecutions = jobExplorer.getJobExecutions(previousJobExecution.getJobInstance()); + + for (org.springframework.batch.core.JobExecution jobExecution : previousExecutions) { + if(jobExecution.getCreateTime().compareTo(previousJobExecution.getCreateTime()) > 0) { + throw new JobExecutionNotMostRecentException("The requested JobExecution to restart was not the most recently run"); + } + + if(jobExecution.getStatus().equals(BatchStatus.ABANDONED)) { + throw new JobRestartException("JobExecution ID: " + jobExecution.getId() + " is abandoned and attempted to be restarted."); + } + } + + final String jobName = previousJobExecution.getJobInstance().getJobName(); + + Properties jobRestartProperties = getJobRestartProperties(params, previousJobExecution); + + final JsrXmlApplicationContext batchContext = new JsrXmlApplicationContext(jobRestartProperties); + batchContext.setValidating(false); + + Resource batchXml = new ClassPathResource("/META-INF/batch.xml"); + Resource jobXml = new ClassPathResource(previousJobExecution.getJobConfigurationName()); + + if(batchXml.exists()) { + batchContext.load(batchXml); + } + + if(jobXml.exists()) { + batchContext.load(jobXml); + } + + AbstractBeanDefinition beanDefinition = BeanDefinitionBuilder.genericBeanDefinition("org.springframework.batch.core.jsr.JsrJobContextFactoryBean").getBeanDefinition(); + beanDefinition.setScope(BeanDefinition.SCOPE_SINGLETON); + batchContext.registerBeanDefinition(JSR_JOB_CONTEXT_BEAN_NAME, beanDefinition); + + batchContext.setParent(baseContext); + + try { + batchContext.refresh(); + } catch (BeanCreationException e) { + throw new JobRestartException(e); + } + + final org.springframework.batch.core.JobExecution jobExecution; + + try { + JobParameters jobParameters = jobParametersConverter.getJobParameters(jobRestartProperties); + jobExecution = jobRepository.createJobExecution(previousJobExecution.getJobInstance(), jobParameters, previousJobExecution.getJobConfigurationName()); + } catch (Exception e) { + throw new JobRestartException(e); + } + + try { + final Semaphore semaphore = new Semaphore(1); + final List exceptionHolder = Collections.synchronizedList(new ArrayList<>()); + semaphore.acquire(); + + taskExecutor.execute(new Runnable() { + + @Override + public void run() { + JsrJobContextFactoryBean factoryBean = null; + try { + factoryBean = (JsrJobContextFactoryBean) batchContext.getBean("&" + JSR_JOB_CONTEXT_BEAN_NAME); + factoryBean.setJobExecution(jobExecution); + final Job job = batchContext.getBean(Job.class); + + if(!job.isRestartable()) { + throw new JobRestartException("Job " + jobName + " is not restartable"); + } + + semaphore.release(); + // Initialization of the JobExecution for job level dependencies + jobRegistry.register(job, jobExecution); + job.execute(jobExecution); + jobRegistry.remove(jobExecution); + } + catch (Exception e) { + exceptionHolder.add(e); + } finally { + if(factoryBean != null) { + factoryBean.close(); + } + + batchContext.close(); + + if(semaphore.availablePermits() == 0) { + semaphore.release(); + } + } + } + }); + + semaphore.acquire(); + if(exceptionHolder.size() > 0) { + semaphore.release(); + throw new JobRestartException(exceptionHolder.get(0)); + } + } + catch (Exception e) { + jobExecution.upgradeStatus(BatchStatus.FAILED); + if (jobExecution.getExitStatus().equals(ExitStatus.UNKNOWN)) { + jobExecution.setExitStatus(ExitStatus.FAILED.addExitDescription(e)); + } + + jobRepository.update(jobExecution); + + if(batchContext.isActive()) { + batchContext.close(); + } + + throw new JobRestartException(e); + } + + return jobExecution.getId(); + } + + protected Properties getJobRestartProperties(Properties params, org.springframework.batch.core.JobExecution previousJobExecution) { + Properties jobRestartProperties = new Properties(); + + if (previousJobExecution != null) { + JobParameters previousJobParameters = previousJobExecution.getJobParameters(); + + if (previousJobParameters != null && !previousJobParameters.isEmpty()) { + jobRestartProperties.putAll(previousJobParameters.toProperties()); + } + } + + if (params != null) { + Enumeration propertyNames = params.propertyNames(); + + while(propertyNames.hasMoreElements()) { + String curName = (String) propertyNames.nextElement(); + jobRestartProperties.setProperty(curName, params.getProperty(curName)); + } + } + + return jobRestartProperties; + } + + /** + * Creates a child {@link ApplicationContext} for the job being requested based upon + * the /META-INF/batch.xml (if exists) and the /META-INF/batch-jobs/<jobName>.xml + * configuration and launches the job. Per JSR-352, calls to this method will always + * create a new {@link JobInstance} (and related {@link JobExecution}). + * + * @param jobName the name of the job XML file without the .xml that is located within the + * /META-INF/batch-jobs directory. + * @param params any job parameters to be used during the execution of this job. + */ + @Override + public long start(String jobName, Properties params) throws JobStartException, + JobSecurityException { + final JsrXmlApplicationContext batchContext = new JsrXmlApplicationContext(params); + batchContext.setValidating(false); + + Resource batchXml = new ClassPathResource("/META-INF/batch.xml"); + String jobConfigurationLocation = "/META-INF/batch-jobs/" + jobName + ".xml"; + Resource jobXml = new ClassPathResource(jobConfigurationLocation); + + if(batchXml.exists()) { + batchContext.load(batchXml); + } + + if(jobXml.exists()) { + batchContext.load(jobXml); + } + + AbstractBeanDefinition beanDefinition = BeanDefinitionBuilder.genericBeanDefinition("org.springframework.batch.core.jsr.JsrJobContextFactoryBean").getBeanDefinition(); + beanDefinition.setScope(BeanDefinition.SCOPE_SINGLETON); + batchContext.registerBeanDefinition(JSR_JOB_CONTEXT_BEAN_NAME, beanDefinition); + + if(baseContext != null) { + batchContext.setParent(baseContext); + } else { + batchContext.getBeanFactory().registerSingleton("jobExplorer", jobExplorer); + batchContext.getBeanFactory().registerSingleton("jobRepository", jobRepository); + batchContext.getBeanFactory().registerSingleton("jobParametersConverter", jobParametersConverter); + batchContext.getBeanFactory().registerSingleton("transactionManager", transactionManager); + } + + try { + batchContext.refresh(); + } catch (BeanCreationException e) { + throw new JobStartException(e); + } + + Assert.notNull(jobName, "The job name must not be null."); + + final org.springframework.batch.core.JobExecution jobExecution; + + try { + JobParameters jobParameters = jobParametersConverter.getJobParameters(params); + String [] jobNames = batchContext.getBeanNamesForType(Job.class); + + if(jobNames == null || jobNames.length <= 0) { + throw new BatchRuntimeException("No Job defined in current context"); + } + + org.springframework.batch.core.JobInstance jobInstance = jobRepository.createJobInstance(jobNames[0], jobParameters); + jobExecution = jobRepository.createJobExecution(jobInstance, jobParameters, jobConfigurationLocation); + } catch (Exception e) { + throw new JobStartException(e); + } + + try { + final Semaphore semaphore = new Semaphore(1); + final List exceptionHolder = Collections.synchronizedList(new ArrayList<>()); + semaphore.acquire(); + + taskExecutor.execute(new Runnable() { + + @Override + public void run() { + JsrJobContextFactoryBean factoryBean = null; + try { + factoryBean = (JsrJobContextFactoryBean) batchContext.getBean("&" + JSR_JOB_CONTEXT_BEAN_NAME); + factoryBean.setJobExecution(jobExecution); + final Job job = batchContext.getBean(Job.class); + semaphore.release(); + // Initialization of the JobExecution for job level dependencies + jobRegistry.register(job, jobExecution); + job.execute(jobExecution); + jobRegistry.remove(jobExecution); + } + catch (Exception e) { + exceptionHolder.add(e); + } finally { + if(factoryBean != null) { + factoryBean.close(); + } + + batchContext.close(); + + if(semaphore.availablePermits() == 0) { + semaphore.release(); + } + } + } + }); + + semaphore.acquire(); + if(exceptionHolder.size() > 0) { + semaphore.release(); + throw new JobStartException(exceptionHolder.get(0)); + } + } + catch (Exception e) { + if(jobRegistry.exists(jobExecution.getId())) { + jobRegistry.remove(jobExecution); + } + jobExecution.upgradeStatus(BatchStatus.FAILED); + if (jobExecution.getExitStatus().equals(ExitStatus.UNKNOWN)) { + jobExecution.setExitStatus(ExitStatus.FAILED.addExitDescription(e)); + } + jobRepository.update(jobExecution); + + if(batchContext.isActive()) { + batchContext.close(); + } + + throw new JobStartException(e); + } + return jobExecution.getId(); + } + + /** + * Stops the running job execution if it is currently running. + * + * @param executionId the database id for the {@link JobExecution} to be stopped. + * @throws NoSuchJobExecutionException thrown if {@link JobExecution} instance does not exist. + * @throws JobExecutionNotRunningException thrown if {@link JobExecution} is not running. + */ + @Override + public void stop(long executionId) throws NoSuchJobExecutionException, + JobExecutionNotRunningException, JobSecurityException { + org.springframework.batch.core.JobExecution jobExecution = jobExplorer.getJobExecution(executionId); + // Indicate the execution should be stopped by setting it's status to + // 'STOPPING'. It is assumed that + // the step implementation will check this status at chunk boundaries. + BatchStatus status = jobExecution.getStatus(); + if (!(status == BatchStatus.STARTED || status == BatchStatus.STARTING)) { + throw new JobExecutionNotRunningException("JobExecution must be running so that it can be stopped: "+jobExecution); + } + jobExecution.setStatus(BatchStatus.STOPPING); + jobRepository.update(jobExecution); + + try { + Job job = jobRegistry.getJob(jobExecution.getId()); + if (job instanceof StepLocator) {//can only process as StepLocator is the only way to get the step object + //get the current stepExecution + for (org.springframework.batch.core.StepExecution stepExecution : jobExecution.getStepExecutions()) { + if (stepExecution.getStatus().isRunning()) { + try { + //have the step execution that's running -> need to 'stop' it + Step step = ((StepLocator)job).getStep(stepExecution.getStepName()); + if (step instanceof TaskletStep) { + Tasklet tasklet = ((TaskletStep)step).getTasklet(); + if (tasklet instanceof StoppableTasklet) { + StepSynchronizationManager.register(stepExecution); + ((StoppableTasklet)tasklet).stop(); + StepSynchronizationManager.release(); + } + } + } + catch (NoSuchStepException e) { + logger.warn("Step not found",e); + } + } + } + } + } + catch (NoSuchJobException e) { + logger.warn("Cannot find Job object",e); + } + } + + @Override + public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { + baseContext = applicationContext; + } + + private static class ExecutingJobRegistry { + + private Map registry = new ConcurrentHashMap<>(); + + public void register(Job job, org.springframework.batch.core.JobExecution jobExecution) throws DuplicateJobException { + + if(registry.containsKey(jobExecution.getId())) { + throw new DuplicateJobException("This job execution has already been registered"); + } else { + registry.put(jobExecution.getId(), job); + } + } + + public void remove(org.springframework.batch.core.JobExecution jobExecution) { + if(!registry.containsKey(jobExecution.getId())) { + throw new NoSuchJobExecutionException("The job execution " + jobExecution.getId() + " was not found"); + } else { + registry.remove(jobExecution.getId()); + } + } + + public boolean exists(long jobExecutionId) { + return registry.containsKey(jobExecutionId); + } + + public Job getJob(long jobExecutionId) { + if(!registry.containsKey(jobExecutionId)) { + throw new NoSuchJobExecutionException("The job execution " + jobExecutionId + " was not found"); + } else { + return registry.get(jobExecutionId); + } + } + } + + /** + * A singleton holder used to lazily bootstrap the base context used in JSR-352. + */ + protected static class BaseContextHolder { + + private ApplicationContext context; + + private static BaseContextHolder instance; + + private BaseContextHolder() { + synchronized (BaseContextHolder.class) { + if(this.context == null) { + String overrideContextLocation = System.getProperty("JSR-352-BASE-CONTEXT"); + + List contextLocations = new ArrayList<>(); + + contextLocations.add("jsrBaseContext.xml"); + + if(overrideContextLocation != null) { + contextLocations.add(overrideContextLocation); + } + + this.context = new GenericXmlApplicationContext( + contextLocations.toArray(new String[contextLocations.size()])); + } + } + } + + public static BaseContextHolder getInstance() { + synchronized (BaseContextHolder.class) { + if(instance == null) { + instance = new BaseContextHolder(); + } + } + + return instance; + } + + public ApplicationContext getContext() { + return this.context; + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/launch/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/launch/package-info.java new file mode 100644 index 0000000000..00f60e37ad --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/launch/package-info.java @@ -0,0 +1,10 @@ +/** + * Implementation of the JSR-352 specific job launching facilities. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.launch; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/launch/support/BatchPropertyBeanPostProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/launch/support/BatchPropertyBeanPostProcessor.java new file mode 100644 index 0000000000..6cf77dfeb2 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/launch/support/BatchPropertyBeanPostProcessor.java @@ -0,0 +1,183 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.launch.support; + +import java.lang.annotation.Annotation; +import java.lang.reflect.Field; +import java.lang.reflect.Modifier; +import java.util.HashSet; +import java.util.Properties; +import java.util.Set; + +import javax.batch.api.BatchProperty; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.jsr.configuration.support.JsrExpressionParser; +import org.springframework.batch.core.scope.StepScope; +import org.springframework.batch.core.scope.context.StepContext; +import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.beans.factory.BeanFactoryAware; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.config.BeanExpressionContext; +import org.springframework.beans.factory.config.BeanPostProcessor; +import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; +import org.springframework.context.expression.StandardBeanExpressionResolver; +import org.springframework.util.ReflectionUtils; +import org.springframework.util.StringUtils; + +/** + *

      + * {@link BeanPostProcessor} implementation used to inject JSR-352 String properties into batch artifact fields + * that are marked with the {@link BatchProperty} annotation. + *

      + * + * @author Chris Schaefer + * @author Michael Minella + * @since 3.0 + */ +@SuppressWarnings("unchecked") +public class BatchPropertyBeanPostProcessor implements BeanPostProcessor, BeanFactoryAware { + private static final String SCOPED_TARGET_BEAN_PREFIX = "scopedTarget."; + private static final Log LOGGER = LogFactory.getLog(BatchPropertyBeanPostProcessor.class); + private static final Set> REQUIRED_ANNOTATIONS = new HashSet<>(); + + private JsrExpressionParser jsrExpressionParser; + private BatchPropertyContext batchPropertyContext; + + static { + ClassLoader cl = BatchPropertyBeanPostProcessor.class.getClassLoader(); + + try { + REQUIRED_ANNOTATIONS.add((Class) cl.loadClass("javax.inject.Inject")); + } catch (ClassNotFoundException ex) { + LOGGER.warn("javax.inject.Inject not found - @BatchProperty marked fields will not be processed."); + } + + REQUIRED_ANNOTATIONS.add(BatchProperty.class); + } + + @Override + public Object postProcessBeforeInitialization(final Object artifact, String artifactName) throws BeansException { + Properties artifactProperties = getArtifactProperties(artifactName); + + if (artifactProperties.isEmpty()) { + return artifact; + } + + injectBatchProperties(artifact, artifactProperties); + + return artifact; + } + + @Override + public Object postProcessAfterInitialization(Object artifact, String artifactName) throws BeansException { + return artifact; + } + + private Properties getArtifactProperties(String artifactName) { + String originalArtifactName = artifactName; + + if(originalArtifactName.startsWith(SCOPED_TARGET_BEAN_PREFIX)) { + originalArtifactName = artifactName.substring(SCOPED_TARGET_BEAN_PREFIX.length()); + } + + StepContext stepContext = StepSynchronizationManager.getContext(); + + if (stepContext != null) { + return batchPropertyContext.getStepArtifactProperties(stepContext.getStepName(), originalArtifactName); + } + + return batchPropertyContext.getArtifactProperties(originalArtifactName); + } + + private void injectBatchProperties(final Object artifact, final Properties artifactProperties) { + ReflectionUtils.doWithFields(artifact.getClass(), new ReflectionUtils.FieldCallback() { + @Override + public void doWith(Field field) throws IllegalArgumentException, IllegalAccessException { + if (isValidFieldModifier(field) && isAnnotated(field)) { + boolean isAccessible = field.isAccessible(); + field.setAccessible(true); + + String batchProperty = getBatchPropertyFieldValue(field, artifactProperties); + + if (StringUtils.hasText(batchProperty)) { + field.set(artifact, batchProperty); + } + + field.setAccessible(isAccessible); + } + } + }); + } + + private String getBatchPropertyFieldValue(Field field, Properties batchArtifactProperties) { + BatchProperty batchProperty = field.getAnnotation(BatchProperty.class); + + if (!"".equals(batchProperty.name())) { + return getBatchProperty(batchProperty.name(), batchArtifactProperties); + } + + return getBatchProperty(field.getName(), batchArtifactProperties); + } + + private String getBatchProperty(String propertyKey, Properties batchArtifactProperties) { + if (batchArtifactProperties.containsKey(propertyKey)) { + String propertyValue = (String) batchArtifactProperties.get(propertyKey); + + return jsrExpressionParser.parseExpression(propertyValue); + } + + return null; + } + + private boolean isAnnotated(Field field) { + for (Class annotation : REQUIRED_ANNOTATIONS) { + if (!field.isAnnotationPresent(annotation)) { + return false; + } + } + + return true; + } + + private boolean isValidFieldModifier(Field field) { + return !Modifier.isStatic(field.getModifiers()) && !Modifier.isFinal(field.getModifiers()); + } + + @Override + public void setBeanFactory(BeanFactory beanFactory) throws BeansException { + if (!(beanFactory instanceof ConfigurableListableBeanFactory)) { + throw new IllegalArgumentException( + "BatchPropertyBeanPostProcessor requires a ConfigurableListableBeanFactory"); + } + + ConfigurableListableBeanFactory configurableListableBeanFactory = (ConfigurableListableBeanFactory) beanFactory; + + BeanExpressionContext beanExpressionContext = new BeanExpressionContext(configurableListableBeanFactory, + configurableListableBeanFactory.getBean(StepScope.class)); + + this.jsrExpressionParser = new JsrExpressionParser(new StandardBeanExpressionResolver(), beanExpressionContext); + } + + @Autowired + public void setBatchPropertyContext(BatchPropertyContext batchPropertyContext) { + this.batchPropertyContext = batchPropertyContext; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/package-info.java new file mode 100644 index 0000000000..45b3edde37 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/package-info.java @@ -0,0 +1,10 @@ +/** + * Extensions of core batch components to apply JSR-352 specific logic. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/JsrPartitionHandler.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/JsrPartitionHandler.java new file mode 100644 index 0000000000..8841dfc0fb --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/JsrPartitionHandler.java @@ -0,0 +1,509 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.partition; + +import java.io.Serializable; +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashSet; +import java.util.Iterator; +import java.util.List; +import java.util.Properties; +import java.util.Queue; +import java.util.Set; +import java.util.concurrent.Callable; +import java.util.concurrent.Future; +import java.util.concurrent.FutureTask; +import java.util.concurrent.LinkedBlockingQueue; +import java.util.concurrent.locks.ReentrantLock; + +import javax.batch.api.partition.PartitionAnalyzer; +import javax.batch.api.partition.PartitionCollector; +import javax.batch.api.partition.PartitionMapper; +import javax.batch.api.partition.PartitionPlan; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecutionException; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.partition.PartitionHandler; +import org.springframework.batch.core.partition.StepExecutionSplitter; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.core.task.TaskRejectedException; +import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor; +import org.springframework.util.Assert; + +/** + * Executes a step instance per thread using a {@link ThreadPoolTaskExecutor} in + * accordance with JSR-352. The results from each step is aggregated into a + * cumulative result. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class JsrPartitionHandler implements PartitionHandler, InitializingBean { + + private static final int DEFAULT_POLLING_INTERVAL = 500; + + // TODO: Replace with proper Channel and Messages once minimum support level for Spring is 4 + private Queue partitionDataQueue; + private ReentrantLock lock; + private Step step; + private int partitions; + private PartitionAnalyzer analyzer; + private PartitionMapper mapper; + private int threads; + private BatchPropertyContext propertyContext; + private JobRepository jobRepository; + private boolean allowStartIfComplete = false; + private Set partitionStepNames = new HashSet<>(); + private int pollingInterval = DEFAULT_POLLING_INTERVAL; + + /** + * @return the step that will be executed by each partition + */ + public Step getStep() { + return step; + } + + /** + * @return the names of each partitioned step + */ + public Collection getPartitionStepNames() { + return partitionStepNames; + } + + /** + * @param allowStartIfComplete flag stating if the step should restart if it + * was complete in a previous run + */ + public void setAllowStartIfComplete(boolean allowStartIfComplete) { + this.allowStartIfComplete = allowStartIfComplete; + } + + /** + * @param queue {@link Queue} to receive the output of the {@link PartitionCollector} + */ + public void setPartitionDataQueue(Queue queue) { + this.partitionDataQueue = queue; + } + + public void setPartitionLock(ReentrantLock lock) { + this.lock = lock; + } + + /** + * @param context {@link BatchPropertyContext} to resolve partition level step properties + */ + public void setPropertyContext(BatchPropertyContext context) { + this.propertyContext = context; + } + + /** + * @param mapper {@link PartitionMapper} used to configure partitioning + */ + public void setPartitionMapper(PartitionMapper mapper) { + this.mapper = mapper; + } + + /** + * @param step the step to be executed as a partitioned step + */ + public void setStep(Step step) { + this.step = step; + } + + /** + * @param analyzer {@link PartitionAnalyzer} + */ + public void setPartitionAnalyzer(PartitionAnalyzer analyzer) { + this.analyzer = analyzer; + } + + /** + * @param threads the number of threads to execute the partitions to be run + * within. The default is the number of partitions. + */ + public void setThreads(int threads) { + this.threads = threads; + } + + /** + * @param partitions the number of partitions to be executed + */ + public void setPartitions(int partitions) { + this.partitions = partitions; + } + + /** + * @param jobRepository {@link JobRepository} + */ + public void setJobRepository(JobRepository jobRepository) { + this.jobRepository = jobRepository; + } + + /** + * @param pollingInterval the duration of partitions completion polling interval + * (in milliseconds). The default value is 500ms. + */ + public void setPollingInterval(int pollingInterval) { + this.pollingInterval = pollingInterval; + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.partition.PartitionHandler#handle(org.springframework.batch.core.partition.StepExecutionSplitter, org.springframework.batch.core.StepExecution) + */ + @Override + public Collection handle(StepExecutionSplitter stepSplitter, + StepExecution stepExecution) throws Exception { + final List> tasks = new ArrayList<>(); + final Set result = new HashSet<>(); + final ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor(); + + int stepExecutionCount = jobRepository.getStepExecutionCount(stepExecution.getJobExecution().getJobInstance(), stepExecution.getStepName()); + + boolean isRestart = stepExecutionCount > 1; + + Set partitionStepExecutions = splitStepExecution(stepExecution, isRestart); + + for (StepExecution curStepExecution : partitionStepExecutions) { + partitionStepNames.add(curStepExecution.getStepName()); + } + + taskExecutor.setCorePoolSize(threads); + taskExecutor.setMaxPoolSize(threads); + + taskExecutor.initialize(); + + try { + for (final StepExecution curStepExecution : partitionStepExecutions) { + final FutureTask task = createTask(step, curStepExecution); + + try { + taskExecutor.execute(task); + tasks.add(task); + } catch (TaskRejectedException e) { + // couldn't execute one of the tasks + ExitStatus exitStatus = ExitStatus.FAILED + .addExitDescription("TaskExecutor rejected the task for this step."); + /* + * Set the status in case the caller is tracking it through the + * JobExecution. + */ + curStepExecution.setStatus(BatchStatus.FAILED); + curStepExecution.setExitStatus(exitStatus); + result.add(stepExecution); + } + } + + processPartitionResults(tasks, result); + } + finally { + taskExecutor.shutdown(); + } + + return result; + } + + /** + * Blocks until all partitioned steps have completed. As each step completes + * the PartitionAnalyzer analyzes the collector data received from each + * partition (if there is any). + * + * @param tasks The {@link Future} that contains the reference to the executing step + * @param result Set of completed {@link StepExecution}s + * @throws Exception + */ + private void processPartitionResults( + final List> tasks, + final Set result) throws Exception { + while(true) { + Thread.sleep(pollingInterval); + try { + lock.lock(); + while(!partitionDataQueue.isEmpty()) { + analyzer.analyzeCollectorData(partitionDataQueue.remove()); + } + + processFinishedPartitions(tasks, result); + + if(tasks.size() == 0) { + break; + } + } finally { + if(lock.isHeldByCurrentThread()) { + lock.unlock(); + } + } + } + } + + /** + * Uses either the {@link PartitionMapper} or the hard coded configuration to split + * the supplied manager StepExecution into the worker StepExecutions. + * + * @param stepExecution manager {@link StepExecution} + * @param isRestart true if this step is being restarted + * @return a {@link Set} of {@link StepExecution}s to be executed + * @throws Exception + * @throws JobExecutionException + */ + private Set splitStepExecution(StepExecution stepExecution, + boolean isRestart) throws Exception, JobExecutionException { + Set partitionStepExecutions = new HashSet<>(); + if(isRestart) { + if(mapper != null) { + PartitionPlan plan = mapper.mapPartitions(); + + if(plan.getPartitionsOverride()) { + partitionStepExecutions = applyPartitionPlan(stepExecution, plan, false); + + for (StepExecution curStepExecution : partitionStepExecutions) { + curStepExecution.setExecutionContext(new ExecutionContext()); + } + } else { + Properties[] partitionProps = plan.getPartitionProperties(); + + plan = (PartitionPlanState) stepExecution.getExecutionContext().get("partitionPlanState"); + plan.setPartitionProperties(partitionProps); + + partitionStepExecutions = applyPartitionPlan(stepExecution, plan, true); + } + + } else { + StepExecutionSplitter stepSplitter = new JsrStepExecutionSplitter(jobRepository, allowStartIfComplete, stepExecution.getStepName(), true); + partitionStepExecutions = stepSplitter.split(stepExecution, partitions); + } + } else { + if(mapper != null) { + PartitionPlan plan = mapper.mapPartitions(); + partitionStepExecutions = applyPartitionPlan(stepExecution, plan, true); + } else { + StepExecutionSplitter stepSplitter = new JsrStepExecutionSplitter(jobRepository, allowStartIfComplete, stepExecution.getStepName(), true); + partitionStepExecutions = stepSplitter.split(stepExecution, partitions); + } + } + return partitionStepExecutions; + } + + private Set applyPartitionPlan(StepExecution stepExecution, + PartitionPlan plan, boolean restoreState) throws JobExecutionException { + StepExecutionSplitter stepSplitter; + Set partitionStepExecutions; + if(plan.getThreads() > 0) { + threads = plan.getThreads(); + } else if(plan.getPartitions() > 0) { + threads = plan.getPartitions(); + } else { + throw new IllegalArgumentException("Either a number of threads or partitions are required"); + } + + PartitionPlanState partitionPlanState = new PartitionPlanState(); + partitionPlanState.setPartitionPlan(plan); + + stepExecution.getExecutionContext().put("partitionPlanState", partitionPlanState); + + stepSplitter = new JsrStepExecutionSplitter(jobRepository, allowStartIfComplete, stepExecution.getStepName(), restoreState); + partitionStepExecutions = stepSplitter.split(stepExecution, plan.getPartitions()); + registerPartitionProperties(partitionStepExecutions, plan); + return partitionStepExecutions; + } + + private void processFinishedPartitions( + final List> tasks, + final Set result) throws Exception { + for(int i = 0; i < tasks.size(); i++) { + Future curTask = tasks.get(i); + + if(curTask.isDone()) { + StepExecution curStepExecution = curTask.get(); + + if(analyzer != null) { + analyzer.analyzeStatus(curStepExecution.getStatus().getBatchStatus(), curStepExecution.getExitStatus().getExitCode()); + } + + result.add(curStepExecution); + + tasks.remove(i); + i--; + } + } + } + + private void registerPartitionProperties( + Set partitionStepExecutions, PartitionPlan plan) { + Properties[] partitionProperties = plan.getPartitionProperties(); + if(partitionProperties != null) { + Iterator executions = partitionStepExecutions.iterator(); + + int i = 0; + while(executions.hasNext()) { + StepExecution curExecution = executions.next(); + + if(i < partitionProperties.length) { + Properties partitionPropertyValues = partitionProperties[i]; + if(partitionPropertyValues != null) { + propertyContext.setStepProperties(curExecution.getStepName(), partitionPropertyValues); + } + + i++; + } else { + break; + } + } + } + } + + /** + * Creates the task executing the given step in the context of the given execution. + * + * @param step the step to execute + * @param stepExecution the given execution + * @return the task executing the given step + */ + protected FutureTask createTask(final Step step, + final StepExecution stepExecution) { + return new FutureTask<>(new Callable() { + @Override + public StepExecution call() throws Exception { + step.execute(stepExecution); + return stepExecution; + } + }); + } + + /* (non-Javadoc) + * @see org.springframework.beans.factory.InitializingBean#afterPropertiesSet() + */ + @Override + public void afterPropertiesSet() throws Exception { + Assert.notNull(propertyContext, "A BatchPropertyContext is required"); + Assert.isTrue(mapper != null || (threads > 0 || partitions > 0), "Either a mapper implementation or the number of partitions/threads is required"); + Assert.notNull(jobRepository, "A JobRepository is required"); + Assert.isTrue(pollingInterval >= 0, "The polling interval must be positive"); + + if(partitionDataQueue == null) { + partitionDataQueue = new LinkedBlockingQueue<>(); + } + + if(lock == null) { + lock = new ReentrantLock(); + } + } + + /** + * Since a {@link PartitionPlan} could provide dynamic data (different results from run to run), + * the batch runtime needs to save off the results for restarts. This class serves as a container + * used to save off that state. + * + * @author Michael Minella + * @since 3.0 + */ + public static class PartitionPlanState implements PartitionPlan, Serializable { + + private static final long serialVersionUID = 1L; + private Properties[] partitionProperties; + private int partitions; + private int threads; + + /** + * @param plan the {@link PartitionPlan} that is the source of the state + */ + public PartitionPlanState(PartitionPlan plan) { + partitionProperties = plan.getPartitionProperties(); + partitions = plan.getPartitions(); + threads = plan.getThreads(); + } + + public PartitionPlanState() { + } + + public void setPartitionPlan(PartitionPlan plan) { + this.partitionProperties = plan.getPartitionProperties(); + this.partitions = plan.getPartitions(); + this.threads = plan.getThreads(); + } + + /* (non-Javadoc) + * @see javax.batch.api.partition.PartitionPlan#getPartitionProperties() + */ + @Override + public Properties[] getPartitionProperties() { + return partitionProperties; + } + + /* (non-Javadoc) + * @see javax.batch.api.partition.PartitionPlan#getPartitions() + */ + @Override + public int getPartitions() { + return partitions; + } + + /* (non-Javadoc) + * @see javax.batch.api.partition.PartitionPlan#getThreads() + */ + @Override + public int getThreads() { + return threads; + } + + /* (non-Javadoc) + * @see javax.batch.api.partition.PartitionPlan#setPartitions(int) + */ + @Override + public void setPartitions(int count) { + this.partitions = count; + } + + /* (non-Javadoc) + * @see javax.batch.api.partition.PartitionPlan#setPartitionsOverride(boolean) + */ + @Override + public void setPartitionsOverride(boolean override) { + // Intentional No-op + } + + /* (non-Javadoc) + * @see javax.batch.api.partition.PartitionPlan#getPartitionsOverride() + */ + @Override + public boolean getPartitionsOverride() { + return false; + } + + /* (non-Javadoc) + * @see javax.batch.api.partition.PartitionPlan#setThreads(int) + */ + @Override + public void setThreads(int count) { + this.threads = count; + } + + /* (non-Javadoc) + * @see javax.batch.api.partition.PartitionPlan#setPartitionProperties(java.util.Properties[]) + */ + @Override + public void setPartitionProperties(Properties[] props) { + this.partitionProperties = props; + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/JsrStepExecutionSplitter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/JsrStepExecutionSplitter.java new file mode 100644 index 0000000000..257489de85 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/JsrStepExecutionSplitter.java @@ -0,0 +1,96 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.partition; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobExecutionException; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.jsr.launch.JsrJobOperator; +import org.springframework.batch.core.partition.support.SimpleStepExecutionSplitter; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.item.ExecutionContext; + +import java.util.Comparator; +import java.util.Set; +import java.util.TreeSet; + +/** + * Provides JSR-352 specific behavior for the splitting of {@link StepExecution}s. + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrStepExecutionSplitter extends SimpleStepExecutionSplitter { + + private String stepName; + private JobRepository jobRepository; + private boolean restoreState; + + public JsrStepExecutionSplitter(JobRepository jobRepository, boolean allowStartIfComplete, String stepName, boolean restoreState) { + super(jobRepository, allowStartIfComplete, stepName, null); + this.stepName = stepName; + this.jobRepository = jobRepository; + this.restoreState = restoreState; + } + + @Override + public String getStepName() { + return this.stepName; + } + + /** + * Returns the same number of {@link StepExecution}s as the gridSize specifies. Each + * of the child StepExecutions will not be available via the {@link JsrJobOperator} per + * JSR-352. + * + * @see https://java.net/projects/jbatch/lists/public/archive/2013-10/message/10 + */ + @Override + public Set split(StepExecution stepExecution, int gridSize) + throws JobExecutionException { + Set executions = new TreeSet<>(new Comparator() { + + @Override + public int compare(StepExecution arg0, StepExecution arg1) { + String r1 = ""; + String r2 = ""; + if (arg0 != null) { + r1 = arg0.getStepName(); + } + if (arg1 != null) { + r2 = arg1.getStepName(); + } + + return r1.compareTo(r2); + } + }); + JobExecution jobExecution = stepExecution.getJobExecution(); + + for(int i = 0; i < gridSize; i++) { + String stepName = this.stepName + ":partition" + i; + JobExecution curJobExecution = new JobExecution(jobExecution); + StepExecution curStepExecution = new StepExecution(stepName, curJobExecution); + + if(!restoreState || isStartable(curStepExecution, new ExecutionContext())) { + executions.add(curStepExecution); + } + } + + jobRepository.addAll(executions); + + return executions; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/PartitionCollectorAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/PartitionCollectorAdapter.java new file mode 100644 index 0000000000..248f1d589c --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/PartitionCollectorAdapter.java @@ -0,0 +1,100 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.partition; + +import java.io.Serializable; +import java.util.Queue; +import java.util.concurrent.locks.ReentrantLock; + +import javax.batch.api.partition.PartitionCollector; +import javax.batch.operations.BatchRuntimeException; + +import org.springframework.batch.core.ChunkListener; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.util.Assert; + +/** + * Adapter class used to wrap a {@link PartitionCollector} so that it can be consumed + * as a {@link ChunkListener}. A thread-safe {@link Queue} is required along with the + * {@link PartitionCollector}. The {@link Queue} is where the result of the call to + * the PartitionCollector will be placed. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class PartitionCollectorAdapter implements ChunkListener { + + private PartitionCollector collector; + private Queue partitionQueue; + private ReentrantLock lock; + + public PartitionCollectorAdapter(Queue queue, PartitionCollector collector) { + Assert.notNull(queue, "A thread-safe Queue is required"); + Assert.notNull(collector, "A PartitionCollector is required"); + + this.partitionQueue = queue; + this.collector = collector; + } + + public void setPartitionLock(ReentrantLock lock) { + this.lock = lock; + } + + @Override + public void beforeChunk(ChunkContext context) { + } + + @Override + public void afterChunk(ChunkContext context) { + try { + if(context.isComplete()) { + lock.lock(); + Serializable collectPartitionData = collector.collectPartitionData(); + + if(collectPartitionData != null) { + partitionQueue.add(collectPartitionData); + } + } + } catch (Throwable e) { + throw new BatchRuntimeException("An error occurred while collecting data from the PartitionCollector", e); + } finally { + if(lock.isHeldByCurrentThread()) { + lock.unlock(); + } + } + } + + @Override + public void afterChunkError(ChunkContext context) { + try { + lock.lock(); + if(context.isComplete()) { + Serializable collectPartitionData = collector.collectPartitionData(); + + if(collectPartitionData != null) { + partitionQueue.add(collectPartitionData); + } + } + } catch (Throwable e) { + throw new BatchRuntimeException("An error occurred while collecting data from the PartitionCollector", e); + } finally { + if(lock.isHeldByCurrentThread()) { + lock.unlock(); + } + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/package-info.java new file mode 100644 index 0000000000..7ab8ea49e7 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/package-info.java @@ -0,0 +1,10 @@ +/** + * Implementation of JSR-352 specific partitioning extensions. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.partition; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/support/JsrBeanScopeBeanFactoryPostProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/support/JsrBeanScopeBeanFactoryPostProcessor.java new file mode 100644 index 0000000000..fcd28a7594 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/support/JsrBeanScopeBeanFactoryPostProcessor.java @@ -0,0 +1,102 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.partition.support; + +import org.springframework.batch.core.jsr.configuration.xml.StepFactoryBean; +import org.springframework.batch.core.jsr.partition.JsrPartitionHandler; +import org.springframework.beans.BeansException; +import org.springframework.beans.PropertyValue; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.config.BeanFactoryPostProcessor; +import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; +import org.springframework.beans.factory.config.RuntimeBeanReference; + +import javax.batch.api.partition.PartitionAnalyzer; +import javax.batch.api.partition.PartitionMapper; +import javax.batch.api.partition.PartitionReducer; + +/** + * In order for property resolution to occur correctly within the scope of a JSR-352 + * batch job, initialization of job level artifacts must occur on the same thread that + * the job is executing. To allow this to occur, {@link PartitionMapper}, + * {@link PartitionReducer}, and {@link PartitionAnalyzer} are all configured to + * lazy initialization (equivalent to lazy-init="true"). + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrBeanScopeBeanFactoryPostProcessor implements BeanFactoryPostProcessor { + + private JobLevelBeanLazyInitializer initializer; + + /* (non-Javadoc) + * @see org.springframework.beans.factory.config.BeanFactoryPostProcessor#postProcessBeanFactory(org.springframework.beans.factory.config.ConfigurableListableBeanFactory) + */ + @Override + public void postProcessBeanFactory( + ConfigurableListableBeanFactory beanFactory) throws BeansException { + if (initializer == null) { + this.initializer = new JobLevelBeanLazyInitializer(beanFactory); + } + + String[] beanNames = beanFactory.getBeanDefinitionNames(); + + for (String curName : beanNames) { + initializer.visitBeanDefinition(beanFactory.getBeanDefinition(curName)); + } + } + + /** + * Looks for beans that may have dependencies that need to be lazily initialized and + * configures the corresponding {@link BeanDefinition} accordingly. + * + * @author Michael Minella + * @since 3.0 + */ + public static class JobLevelBeanLazyInitializer { + + private ConfigurableListableBeanFactory beanFactory; + + public JobLevelBeanLazyInitializer(ConfigurableListableBeanFactory beanFactory) { + this.beanFactory = beanFactory; + } + + public void visitBeanDefinition(BeanDefinition beanDefinition) { + String beanClassName = beanDefinition.getBeanClassName(); + + if(StepFactoryBean.class.getName().equals(beanClassName)) { + PropertyValue [] values = beanDefinition.getPropertyValues().getPropertyValues(); + for (PropertyValue propertyValue : values) { + if(propertyValue.getName().equalsIgnoreCase("partitionReducer")) { + RuntimeBeanReference ref = (RuntimeBeanReference) propertyValue.getValue(); + beanFactory.getBeanDefinition(ref.getBeanName()).setLazyInit(true); + } + } + } + + if(JsrPartitionHandler.class.getName().equals(beanClassName)) { + PropertyValue [] values = beanDefinition.getPropertyValues().getPropertyValues(); + for (PropertyValue propertyValue : values) { + String propertyName = propertyValue.getName(); + if(propertyName.equalsIgnoreCase("partitionMapper") || propertyName.equalsIgnoreCase("partitionAnalyzer")) { + RuntimeBeanReference ref = (RuntimeBeanReference) propertyValue.getValue(); + beanFactory.getBeanDefinition(ref.getBeanName()).setLazyInit(true); + } + } + } + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/support/JsrStepExecutionAggregator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/support/JsrStepExecutionAggregator.java new file mode 100644 index 0000000000..ad95e4260c --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/support/JsrStepExecutionAggregator.java @@ -0,0 +1,56 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.partition.support; + +import java.util.Collection; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.partition.support.StepExecutionAggregator; +import org.springframework.util.Assert; + +/** + * Aggregates {@link StepExecution}s based on the rules outlined in JSR-352. Specifically + * it aggregates all counts and determines the correct BatchStatus. However, the ExitStatus + * for each child StepExecution is ignored. + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrStepExecutionAggregator implements StepExecutionAggregator { + + /* (non-Javadoc) + * @see org.springframework.batch.core.partition.support.StepExecutionAggregator#aggregate(org.springframework.batch.core.StepExecution, java.util.Collection) + */ + @Override + public void aggregate(StepExecution result, + Collection executions) { + Assert.notNull(result, "To aggregate into a result it must be non-null."); + if (executions == null) { + return; + } + for (StepExecution stepExecution : executions) { + BatchStatus status = stepExecution.getStatus(); + result.setStatus(BatchStatus.max(result.getStatus(), status)); + result.setCommitCount(result.getCommitCount() + stepExecution.getCommitCount()); + result.setRollbackCount(result.getRollbackCount() + stepExecution.getRollbackCount()); + result.setReadCount(result.getReadCount() + stepExecution.getReadCount()); + result.setReadSkipCount(result.getReadSkipCount() + stepExecution.getReadSkipCount()); + result.setWriteCount(result.getWriteCount() + stepExecution.getWriteCount()); + result.setWriteSkipCount(result.getWriteSkipCount() + stepExecution.getWriteSkipCount()); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/support/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/support/package-info.java new file mode 100644 index 0000000000..226692c754 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/partition/support/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Support classes for JSR-352 partitioning configuration. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.partition.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/BatchletStep.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/BatchletStep.java new file mode 100644 index 0000000000..034d6e940f --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/BatchletStep.java @@ -0,0 +1,50 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step; + +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.util.Assert; + +/** + * Special sub class of the {@link TaskletStep} for use with JSR-352 jobs. This + * implementation addresses the registration of a {@link BatchPropertyContext} for + * resolution of late binding parameters. + * + * @author Michael Minella + * @since 3.0 + */ +public class BatchletStep extends TaskletStep { + + private BatchPropertyContext propertyContext; + + /** + * @param name name of the step + * @param propertyContext {@link BatchPropertyContext} used to resolve batch properties. + */ + public BatchletStep(String name, BatchPropertyContext propertyContext) { + super(name); + Assert.notNull(propertyContext, "A propertyContext is required"); + this.propertyContext = propertyContext; + } + + @Override + protected void doExecutionRegistration(StepExecution stepExecution) { + StepSynchronizationManager.register(stepExecution, propertyContext); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/DecisionStep.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/DecisionStep.java new file mode 100644 index 0000000000..25f9e63d7c --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/DecisionStep.java @@ -0,0 +1,95 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; + +import javax.batch.api.Decider; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.jsr.JsrStepExecution; +import org.springframework.batch.core.step.AbstractStep; +import org.springframework.batch.item.ExecutionContext; + +/** + * Implements a {@link Step} to follow the rules for a decision state + * as defined by JSR-352. Currently does not support the JSR requirement + * to provide all of the last {@link javax.batch.runtime.StepExecution}s from + * a split. + * + * @author Michael Minella + * @since 3.0 + */ +public class DecisionStep extends AbstractStep { + + private final Decider decider; + + /** + * @param decider a {@link Decider} implementation + */ + public DecisionStep(Decider decider) { + this.decider = decider; + } + + @SuppressWarnings("unchecked") + @Override + protected void doExecute(StepExecution stepExecution) throws Exception { + ExecutionContext executionContext = stepExecution.getJobExecution().getExecutionContext(); + List stepExecutions = new ArrayList<>(); + + if(executionContext.containsKey("batch.lastSteps")) { + List stepNames = (List) executionContext.get("batch.lastSteps"); + + for (String stepName : stepNames) { + StepExecution curStepExecution = getJobRepository().getLastStepExecution(stepExecution.getJobExecution().getJobInstance(), stepName); + stepExecutions.add(new JsrStepExecution(curStepExecution)); + } + } else { + Collection currentRunStepExecutions = stepExecution.getJobExecution().getStepExecutions(); + + StepExecution lastExecution = null; + + if(stepExecutions != null) { + for (StepExecution curStepExecution : currentRunStepExecutions) { + if(lastExecution == null || (curStepExecution.getEndTime() != null && curStepExecution.getEndTime().after(lastExecution.getEndTime()))) { + lastExecution = curStepExecution; + } + } + + stepExecutions.add(new JsrStepExecution(lastExecution)); + } + } + + try { + ExitStatus exitStatus = new ExitStatus(decider.decide(stepExecutions.toArray(new javax.batch.runtime.StepExecution[0]))); + + stepExecution.getJobExecution().setExitStatus(exitStatus); + stepExecution.setExitStatus(exitStatus); + + if(executionContext.containsKey("batch.lastSteps")) { + executionContext.remove("batch.lastSteps"); + } + } catch (Exception e) { + stepExecution.setTerminateOnly(); + stepExecution.addFailureException(e); + throw e; + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/PartitionStep.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/PartitionStep.java new file mode 100644 index 0000000000..cb69187ea6 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/PartitionStep.java @@ -0,0 +1,117 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step; + +import java.util.Collection; + +import javax.batch.api.partition.PartitionReducer; +import javax.batch.api.partition.PartitionReducer.PartitionStatus; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.JobExecutionException; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.jsr.partition.JsrPartitionHandler; +import org.springframework.batch.core.jsr.partition.support.JsrStepExecutionAggregator; +import org.springframework.batch.core.partition.PartitionHandler; +import org.springframework.batch.core.partition.StepExecutionSplitter; +import org.springframework.batch.core.partition.support.StepExecutionAggregator; +import org.springframework.batch.core.step.NoSuchStepException; +import org.springframework.batch.core.step.StepLocator; +import org.springframework.batch.item.ExecutionContext; + +/** + * An extension of the {@link PartitionStep} that provides additional semantics + * required by JSR-352. Specifically, this implementation adds the required + * lifecycle calls to the {@link PartitionReducer} if it is used. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class PartitionStep extends org.springframework.batch.core.partition.support.PartitionStep implements StepLocator { + + private PartitionReducer reducer; + private boolean hasReducer = false; + private StepExecutionAggregator stepExecutionAggregator = new JsrStepExecutionAggregator(); + + public void setPartitionReducer(PartitionReducer reducer) { + this.reducer = reducer; + hasReducer = reducer != null; + } + + /** + * Delegate execution to the {@link PartitionHandler} provided. The + * {@link StepExecution} passed in here becomes the parent or manager + * execution for the partition, summarizing the status on exit of the + * logical grouping of work carried out by the {@link PartitionHandler}. The + * individual step executions and their input parameters (through + * {@link ExecutionContext}) for the partition elements are provided by the + * {@link StepExecutionSplitter}. + * + * @param stepExecution the manager step execution for the partition + * + * @see Step#execute(StepExecution) + */ + @Override + protected void doExecute(StepExecution stepExecution) throws Exception { + + if(hasReducer) { + reducer.beginPartitionedStep(); + } + + // Wait for task completion and then aggregate the results + Collection stepExecutions = getPartitionHandler().handle(null, stepExecution); + stepExecution.upgradeStatus(BatchStatus.COMPLETED); + stepExecutionAggregator.aggregate(stepExecution, stepExecutions); + + if (stepExecution.getStatus().isUnsuccessful()) { + if (hasReducer) { + reducer.rollbackPartitionedStep(); + reducer.afterPartitionedStepCompletion(PartitionStatus.ROLLBACK); + } + throw new JobExecutionException("Partition handler returned an unsuccessful step"); + } + + if (hasReducer) { + reducer.beforePartitionedStepCompletion(); + reducer.afterPartitionedStepCompletion(PartitionStatus.COMMIT); + } + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.step.StepLocator#getStepNames() + */ + @Override + public Collection getStepNames() { + return ((JsrPartitionHandler) getPartitionHandler()).getPartitionStepNames(); + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.step.StepLocator#getStep(java.lang.String) + */ + @Override + public Step getStep(String stepName) throws NoSuchStepException { + JsrPartitionHandler partitionHandler = (JsrPartitionHandler) getPartitionHandler(); + Collection names = partitionHandler.getPartitionStepNames(); + + if(names.contains(stepName)) { + return partitionHandler.getStep(); + } else { + throw new NoSuchStepException(stepName + " was not found"); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/batchlet/BatchletAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/batchlet/BatchletAdapter.java new file mode 100644 index 0000000000..1bf71cfbdf --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/batchlet/BatchletAdapter.java @@ -0,0 +1,71 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.batchlet; + +import javax.batch.api.Batchlet; +import javax.batch.operations.BatchRuntimeException; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.StoppableTasklet; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * + * @author Michael Minella + * @since 3.0 + */ +public class BatchletAdapter implements StoppableTasklet { + + private Batchlet batchlet; + + public BatchletAdapter(Batchlet batchlet) { + Assert.notNull(batchlet, "A Batchlet implementation is required"); + this.batchlet = batchlet; + } + + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, + ChunkContext chunkContext) throws Exception { + String exitStatus; + try { + exitStatus = batchlet.process(); + } finally { + chunkContext.setComplete(); + } + + if(StringUtils.hasText(exitStatus)) { + contribution.setExitStatus(new ExitStatus(exitStatus)); + } + + + return RepeatStatus.FINISHED; + } + + @Override + public void stop() { + try { + batchlet.stop(); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/batchlet/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/batchlet/package-info.java new file mode 100644 index 0000000000..b97f3ed73e --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/batchlet/package-info.java @@ -0,0 +1,10 @@ +/** + * Classes for supporting JSR-352's {@link javax.batch.api.Batchlet}. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.step.batchlet; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/JsrBatchletStepBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/JsrBatchletStepBuilder.java new file mode 100644 index 0000000000..923f497de1 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/JsrBatchletStepBuilder.java @@ -0,0 +1,103 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.builder; + +import org.springframework.batch.core.ChunkListener; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.jsr.step.BatchletStep; +import org.springframework.batch.core.step.builder.StepBuilderException; +import org.springframework.batch.core.step.builder.StepBuilderHelper; +import org.springframework.batch.core.step.builder.TaskletStepBuilder; +import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.batch.item.ItemStream; +import org.springframework.batch.repeat.support.RepeatTemplate; +import org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate; + +/** + * Extension of the {@link TaskletStepBuilder} that uses a {@link BatchletStep} instead + * of a {@link TaskletStep}. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class JsrBatchletStepBuilder extends TaskletStepBuilder { + + private BatchPropertyContext batchPropertyContext; + + /** + * @param context used to resolve lazy bound properties + */ + public void setBatchPropertyContext(BatchPropertyContext context) { + this.batchPropertyContext = context; + } + + public JsrBatchletStepBuilder(StepBuilderHelper> parent) { + super(parent); + } + + /** + * Build the step from the components collected by the fluent setters. Delegates first to {@link #enhance(Step)} and + * then to {@link #createTasklet()} in subclasses to create the actual tasklet. + * + * @return a tasklet step fully configured and read to execute + */ + @Override + public TaskletStep build() { + + registerStepListenerAsChunkListener(); + + BatchletStep step = new BatchletStep(getName(), batchPropertyContext); + + super.enhance(step); + + step.setChunkListeners(chunkListeners.toArray(new ChunkListener[0])); + + if (getTransactionAttribute() != null) { + step.setTransactionAttribute(getTransactionAttribute()); + } + + if (getStepOperations() == null) { + + stepOperations(new RepeatTemplate()); + + if (getTaskExecutor() != null) { + TaskExecutorRepeatTemplate repeatTemplate = new TaskExecutorRepeatTemplate(); + repeatTemplate.setTaskExecutor(getTaskExecutor()); + repeatTemplate.setThrottleLimit(getThrottleLimit()); + stepOperations(repeatTemplate); + } + + ((RepeatTemplate) getStepOperations()).setExceptionHandler(getExceptionHandler()); + + } + step.setStepOperations(getStepOperations()); + step.setTasklet(createTasklet()); + + step.setStreams(getStreams().toArray(new ItemStream[0])); + + try { + step.afterPropertiesSet(); + } + catch (Exception e) { + throw new StepBuilderException(e); + } + + return step; + + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/JsrFaultTolerantStepBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/JsrFaultTolerantStepBuilder.java new file mode 100644 index 0000000000..7766357848 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/JsrFaultTolerantStepBuilder.java @@ -0,0 +1,157 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.builder; + +import java.util.ArrayList; +import java.util.List; + +import org.springframework.batch.core.ChunkListener; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepListener; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.jsr.step.BatchletStep; +import org.springframework.batch.core.jsr.step.item.JsrChunkProvider; +import org.springframework.batch.core.jsr.step.item.JsrFaultTolerantChunkProcessor; +import org.springframework.batch.core.step.builder.FaultTolerantStepBuilder; +import org.springframework.batch.core.step.builder.StepBuilder; +import org.springframework.batch.core.step.builder.StepBuilderException; +import org.springframework.batch.core.step.item.ChunkOrientedTasklet; +import org.springframework.batch.core.step.item.ChunkProcessor; +import org.springframework.batch.core.step.item.ChunkProvider; +import org.springframework.batch.core.step.skip.SkipPolicy; +import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.batch.item.ItemStream; +import org.springframework.batch.repeat.support.RepeatTemplate; +import org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate; + +/** + * A step builder that extends the {@link FaultTolerantStepBuilder} to create JSR-352 + * specific {@link ChunkProvider} and {@link ChunkProcessor} supporting both the chunking + * pattern defined by the spec as well as skip/retry logic. + * + * @author Michael Minella + * @author Chris Schaefer + * + * @param The input type for the step + * @param The output type for the step + */ +public class JsrFaultTolerantStepBuilder extends FaultTolerantStepBuilder { + + private BatchPropertyContext batchPropertyContext; + + public void setBatchPropertyContext(BatchPropertyContext batchPropertyContext) { + this.batchPropertyContext = batchPropertyContext; + } + + public JsrFaultTolerantStepBuilder(StepBuilder parent) { + super(parent); + } + + @Override + public FaultTolerantStepBuilder faultTolerant() { + return this; + } + + + /** + * Build the step from the components collected by the fluent setters. Delegates first to {@link #enhance(Step)} and + * then to {@link #createTasklet()} in subclasses to create the actual tasklet. + * + * @return a tasklet step fully configured and read to execute + */ + @Override + public TaskletStep build() { + registerStepListenerAsSkipListener(); + registerAsStreamsAndListeners(getReader(), getProcessor(), getWriter()); + + registerStepListenerAsChunkListener(); + + BatchletStep step = new BatchletStep(getName(), batchPropertyContext); + + super.enhance(step); + + step.setChunkListeners(chunkListeners.toArray(new ChunkListener[0])); + + if (getTransactionAttribute() != null) { + step.setTransactionAttribute(getTransactionAttribute()); + } + + if (getStepOperations() == null) { + + stepOperations(new RepeatTemplate()); + + if (getTaskExecutor() != null) { + TaskExecutorRepeatTemplate repeatTemplate = new TaskExecutorRepeatTemplate(); + repeatTemplate.setTaskExecutor(getTaskExecutor()); + repeatTemplate.setThrottleLimit(getThrottleLimit()); + stepOperations(repeatTemplate); + } + + ((RepeatTemplate) getStepOperations()).setExceptionHandler(getExceptionHandler()); + + } + step.setStepOperations(getStepOperations()); + step.setTasklet(createTasklet()); + + step.setStreams(getStreams().toArray(new ItemStream[0])); + + try { + step.afterPropertiesSet(); + } + catch (Exception e) { + throw new StepBuilderException(e); + } + + return step; + + } + + @Override + protected ChunkProvider createChunkProvider() { + return new JsrChunkProvider<>(); + } + + /** + * Provides a JSR-352 specific implementation of a {@link ChunkProcessor} for use + * within the {@link ChunkOrientedTasklet} + * + * @return a JSR-352 implementation of the {@link ChunkProcessor} + * @see JsrFaultTolerantChunkProcessor + */ + @Override + protected ChunkProcessor createChunkProcessor() { + SkipPolicy skipPolicy = getFatalExceptionAwareProxy(createSkipPolicy()); + JsrFaultTolerantChunkProcessor chunkProcessor = + new JsrFaultTolerantChunkProcessor<>(getReader(), getProcessor(), + getWriter(), createChunkOperations(), createRetryOperations()); + chunkProcessor.setSkipPolicy(skipPolicy); + chunkProcessor.setRollbackClassifier(getRollbackClassifier()); + detectStreamInReader(); + chunkProcessor.setChunkMonitor(getChunkMonitor()); + chunkProcessor.setListeners(getChunkListeners()); + + return chunkProcessor; + } + + private List getChunkListeners() { + List listeners = new ArrayList<>(); + listeners.addAll(getItemListeners()); + listeners.addAll(getSkipListeners()); + listeners.addAll(getJsrRetryListeners()); + + return listeners; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/JsrPartitionStepBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/JsrPartitionStepBuilder.java new file mode 100644 index 0000000000..48a76e80ea --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/JsrPartitionStepBuilder.java @@ -0,0 +1,129 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.builder; + +import javax.batch.api.partition.PartitionReducer; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.jsr.step.PartitionStep; +import org.springframework.batch.core.partition.support.SimpleStepExecutionSplitter; +import org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler; +import org.springframework.batch.core.step.builder.PartitionStepBuilder; +import org.springframework.batch.core.step.builder.StepBuilderException; +import org.springframework.batch.core.step.builder.StepBuilderHelper; +import org.springframework.core.task.SyncTaskExecutor; + +/** + * An extension of the {@link PartitionStepBuilder} that uses {@link PartitionStep} + * so that JSR-352 specific semantics are honored. + * + * @author Michael Minella + * @since 3.0 + */ +public class JsrPartitionStepBuilder extends PartitionStepBuilder { + + private PartitionReducer reducer; + + /** + * @param parent parent step builder for basic step properties + */ + public JsrPartitionStepBuilder(StepBuilderHelper parent) { + super(parent); + } + + /** + * @param reducer used to provide a single callback at the beginning and end + * of a partitioned step. + * + * @return this + */ + public JsrPartitionStepBuilder reducer(PartitionReducer reducer) { + this.reducer = reducer; + return this; + } + + @Override + public JsrPartitionStepBuilder step(Step step) { + super.step(step); + return this; + } + + @Override + public Step build() { + PartitionStep step = new PartitionStep(); + step.setName(getName()); + super.enhance(step); + + if (getPartitionHandler() != null) { + step.setPartitionHandler(getPartitionHandler()); + } + else { + TaskExecutorPartitionHandler partitionHandler = new TaskExecutorPartitionHandler(); + partitionHandler.setStep(getStep()); + if (getTaskExecutor() == null) { + taskExecutor(new SyncTaskExecutor()); + } + partitionHandler.setGridSize(getGridSize()); + partitionHandler.setTaskExecutor(getTaskExecutor()); + step.setPartitionHandler(partitionHandler); + } + + if (getSplitter() != null) { + step.setStepExecutionSplitter(getSplitter()); + } + else { + + boolean allowStartIfComplete = isAllowStartIfComplete(); + String name = getStepName(); + if (getStep() != null) { + try { + allowStartIfComplete = getStep().isAllowStartIfComplete(); + name = getStep().getName(); + } + catch (Exception e) { + logger.info("Ignored exception from step asking for name and allowStartIfComplete flag. " + + "Using default from enclosing PartitionStep (" + name + "," + allowStartIfComplete + ")."); + } + } + SimpleStepExecutionSplitter splitter = new SimpleStepExecutionSplitter(); + splitter.setPartitioner(getPartitioner()); + splitter.setJobRepository(getJobRepository()); + splitter.setAllowStartIfComplete(allowStartIfComplete); + splitter.setStepName(name); + splitter(splitter); + step.setStepExecutionSplitter(splitter); + + } + + if (getAggregator() != null) { + step.setStepExecutionAggregator(getAggregator()); + } + + if(reducer != null) { + step.setPartitionReducer(reducer); + } + + try { + step.afterPropertiesSet(); + } + catch (Exception e) { + throw new StepBuilderException(e); + } + + return step; + + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/JsrSimpleStepBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/JsrSimpleStepBuilder.java new file mode 100644 index 0000000000..254f203743 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/JsrSimpleStepBuilder.java @@ -0,0 +1,140 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.builder; + +import java.util.ArrayList; + +import org.springframework.batch.core.ChunkListener; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.jsr.step.BatchletStep; +import org.springframework.batch.core.jsr.step.item.JsrChunkProcessor; +import org.springframework.batch.core.jsr.step.item.JsrChunkProvider; +import org.springframework.batch.core.step.builder.FaultTolerantStepBuilder; +import org.springframework.batch.core.step.builder.SimpleStepBuilder; +import org.springframework.batch.core.step.builder.StepBuilder; +import org.springframework.batch.core.step.builder.StepBuilderException; +import org.springframework.batch.core.step.item.ChunkOrientedTasklet; +import org.springframework.batch.core.step.item.ChunkProcessor; +import org.springframework.batch.core.step.item.ChunkProvider; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.batch.item.ItemStream; +import org.springframework.batch.repeat.RepeatOperations; +import org.springframework.batch.repeat.support.RepeatTemplate; +import org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate; +import org.springframework.util.Assert; + +/** + * A step builder that extends the {@link FaultTolerantStepBuilder} to create JSR-352 + * specific {@link ChunkProvider} and {@link ChunkProcessor} supporting the chunking + * pattern defined by the spec. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * + * @param The input type for the step + * @param The output type for the step + */ +public class JsrSimpleStepBuilder extends SimpleStepBuilder { + + private BatchPropertyContext batchPropertyContext; + + public JsrSimpleStepBuilder(StepBuilder parent) { + super(parent); + } + + public JsrPartitionStepBuilder partitioner(Step step) { + return new JsrPartitionStepBuilder(this).step(step); + } + + public void setBatchPropertyContext(BatchPropertyContext batchPropertyContext) { + this.batchPropertyContext = batchPropertyContext; + } + + /** + * Build the step from the components collected by the fluent setters. Delegates first to {@link #enhance(Step)} and + * then to {@link #createTasklet()} in subclasses to create the actual tasklet. + * + * @return a tasklet step fully configured and read to execute + */ + @Override + public TaskletStep build() { + registerStepListenerAsItemListener(); + registerAsStreamsAndListeners(getReader(), getProcessor(), getWriter()); + registerStepListenerAsChunkListener(); + + BatchletStep step = new BatchletStep(getName(), batchPropertyContext); + + super.enhance(step); + + step.setChunkListeners(chunkListeners.toArray(new ChunkListener[0])); + + if (getTransactionAttribute() != null) { + step.setTransactionAttribute(getTransactionAttribute()); + } + + if (getStepOperations() == null) { + + stepOperations(new RepeatTemplate()); + + if (getTaskExecutor() != null) { + TaskExecutorRepeatTemplate repeatTemplate = new TaskExecutorRepeatTemplate(); + repeatTemplate.setTaskExecutor(getTaskExecutor()); + repeatTemplate.setThrottleLimit(getThrottleLimit()); + stepOperations(repeatTemplate); + } + + ((RepeatTemplate) getStepOperations()).setExceptionHandler(getExceptionHandler()); + + } + step.setStepOperations(getStepOperations()); + step.setTasklet(createTasklet()); + + ItemStream[] streams = getStreams().toArray(new ItemStream[0]); + step.setStreams(streams); + + try { + step.afterPropertiesSet(); + } + catch (Exception e) { + throw new StepBuilderException(e); + } + + return step; + + } + + @Override + protected Tasklet createTasklet() { + Assert.state(getReader() != null, "ItemReader must be provided"); + Assert.state(getProcessor() != null || getWriter() != null, "ItemWriter or ItemProcessor must be provided"); + RepeatOperations repeatOperations = createRepeatOperations(); + ChunkProvider chunkProvider = new JsrChunkProvider<>(); + JsrChunkProcessor chunkProcessor = new JsrChunkProcessor<>(getReader(), getProcessor(), getWriter(), repeatOperations); + chunkProcessor.setListeners(new ArrayList<>(getItemListeners())); + ChunkOrientedTasklet tasklet = new ChunkOrientedTasklet<>(chunkProvider, chunkProcessor); + tasklet.setBuffering(!isReaderTransactionalQueue()); + return tasklet; + } + + private RepeatOperations createRepeatOperations() { + RepeatTemplate repeatOperations = new RepeatTemplate(); + repeatOperations.setCompletionPolicy(getChunkCompletionPolicy()); + repeatOperations.setExceptionHandler(getExceptionHandler()); + return repeatOperations; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/package-info.java new file mode 100644 index 0000000000..2dbcacea87 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/builder/package-info.java @@ -0,0 +1,10 @@ +/** + * Extensions to step related builders to implement JSR-352 specific functionality + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.step.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/item/JsrChunkProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/item/JsrChunkProcessor.java new file mode 100644 index 0000000000..e2c4909286 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/item/JsrChunkProcessor.java @@ -0,0 +1,253 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.item; + +import java.util.List; +import java.util.concurrent.atomic.AtomicInteger; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.StepListener; +import org.springframework.batch.core.listener.MulticasterBatchListener; +import org.springframework.batch.core.step.item.Chunk; +import org.springframework.batch.core.step.item.ChunkProcessor; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.repeat.RepeatCallback; +import org.springframework.batch.repeat.RepeatContext; +import org.springframework.batch.repeat.RepeatOperations; +import org.springframework.batch.repeat.RepeatStatus; + +/** + * {@link ChunkProcessor} implementation that implements JSR-352's chunking pattern + * (read and process in a loop until the chunk is complete then write). This + * implementation is responsible for all three phases of chunk based processing + * (reading, processing and writing). + * + * @author Michael Minella + * + * @param The input type for the step + * @param The output type for the step + */ +public class JsrChunkProcessor implements ChunkProcessor { + + private final Log logger = LogFactory.getLog(getClass()); + private ItemReader itemReader; + private final MulticasterBatchListener listener = new MulticasterBatchListener<>(); + private RepeatOperations repeatTemplate; + private ItemProcessor itemProcessor; + private ItemWriter itemWriter; + + public JsrChunkProcessor() { + this(null, null, null, null); + } + + public JsrChunkProcessor(ItemReader reader, ItemProcessor processor, ItemWriter writer, RepeatOperations repeatTemplate) { + this.itemReader = reader; + this.itemProcessor = processor; + this.itemWriter = writer; + this.repeatTemplate = repeatTemplate; + } + + protected MulticasterBatchListener getListener() { + return listener; + } + + /** + * Loops through reading (via {@link #provide(StepContribution, Chunk)} and + * processing (via {@link #transform(StepContribution, Object)}) until the chunk + * is complete. Once the chunk is complete, the results are written (via + * {@link #persist(StepContribution, Chunk)}. + * + * @see ChunkProcessor#process(StepContribution, Chunk) + * @param contribution a {@link StepContribution} + * @param chunk a {@link Chunk} + */ + @Override + public void process(final StepContribution contribution, final Chunk chunk) + throws Exception { + + final AtomicInteger filterCount = new AtomicInteger(0); + final Chunk output = new Chunk<>(); + + repeatTemplate.iterate(new RepeatCallback() { + + @Override + public RepeatStatus doInIteration(RepeatContext context) throws Exception { + I item = provide(contribution, chunk); + + if(item != null) { + contribution.incrementReadCount(); + } else { + return RepeatStatus.FINISHED; + } + + O processedItem = transform(contribution, item); + + if(processedItem == null) { + filterCount.incrementAndGet(); + } else { + output.add(processedItem); + } + + return RepeatStatus.CONTINUABLE; + } + }); + + contribution.incrementFilterCount(filterCount.get()); + if(output.size() > 0) { + persist(contribution, output); + } + } + + /** + * Register some {@link StepListener}s with the handler. Each will get the + * callbacks in the order specified at the correct stage. + * + * @param listeners list of listeners to be used within this step + */ + public void setListeners(List listeners) { + for (StepListener listener : listeners) { + registerListener(listener); + } + } + + /** + * Register a listener for callbacks at the appropriate stages in a process. + * + * @param listener a {@link StepListener} + */ + public void registerListener(StepListener listener) { + this.listener.register(listener); + } + + /** + * Responsible for the reading portion of the chunking loop. In this implementation, delegates + * to {@link #doProvide(StepContribution, Chunk)} + * + * @param contribution a {@link StepContribution} + * @param chunk a {@link Chunk} + * @return an item + * @throws Exception thrown if error occurs during the reading portion of the chunking loop. + */ + protected I provide(final StepContribution contribution, final Chunk chunk) throws Exception { + return doProvide(contribution, chunk); + } + + /** + * Implements reading as well as any related listener calls required. + * + * @param contribution a {@link StepContribution} + * @param chunk a {@link Chunk} + * @return an item + * @throws Exception thrown if error occurs during reading or listener calls. + */ + protected final I doProvide(final StepContribution contribution, final Chunk chunk) throws Exception { + try { + listener.beforeRead(); + I item = itemReader.read(); + if(item != null) { + listener.afterRead(item); + } else { + chunk.setEnd(); + } + + return item; + } + catch (Exception e) { + if (logger.isDebugEnabled()) { + logger.debug(e.getMessage() + " : " + e.getClass().getName()); + } + listener.onReadError(e); + throw e; + } + } + + /** + * Responsible for the processing portion of the chunking loop. In this implementation, delegates to the + * {@link #doTransform(Object)} if a processor is available (returns the item unmodified if it is not) + * + * @param contribution a {@link StepContribution} + * @param item an item + * @return a processed item if a processor is present (the unmodified item if it is not) + * @throws Exception thrown if error occurs during the processing portion of the chunking loop. + */ + protected O transform(final StepContribution contribution, final I item) throws Exception { + if (itemProcessor == null) { + @SuppressWarnings("unchecked") + O result = (O) item; + return result; + } + + return doTransform(item); + } + + /** + * Implements processing and all related listener calls. + * + * @param item the item to be processed + * @return the processed item + * @throws Exception thrown if error occurs during processing. + */ + protected final O doTransform(I item) throws Exception { + try { + listener.beforeProcess(item); + O result = itemProcessor.process(item); + listener.afterProcess(item, result); + return result; + } + catch (Exception e) { + listener.onProcessError(item, e); + throw e; + } + } + + /** + * Responsible for the writing portion of the chunking loop. In this implementation, delegates to the + * {{@link #doPersist(StepContribution, Chunk)}. + * + * @param contribution a {@link StepContribution} + * @param chunk a {@link Chunk} + * @throws Exception thrown if error occurs during the writing portion of the chunking loop. + */ + protected void persist(final StepContribution contribution, final Chunk chunk) throws Exception { + doPersist(contribution, chunk); + + contribution.incrementWriteCount(chunk.getItems().size()); + } + + /** + * Implements writing and all related listener calls + * + * @param contribution a {@link StepContribution} + * @param chunk a {@link Chunk} + * @throws Exception thrown if error occurs during the writing portion of the chunking loop. + */ + protected final void doPersist(final StepContribution contribution, final Chunk chunk) throws Exception { + try { + List items = chunk.getItems(); + listener.beforeWrite(items); + itemWriter.write(items); + listener.afterWrite(items); + } + catch (Exception e) { + listener.onWriteError(e, chunk.getItems()); + throw e; + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/item/JsrChunkProvider.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/item/JsrChunkProvider.java new file mode 100644 index 0000000000..2772468d91 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/item/JsrChunkProvider.java @@ -0,0 +1,47 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.item; + +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.step.item.Chunk; +import org.springframework.batch.core.step.item.ChunkProvider; + +/** + * A no-op {@link ChunkProvider}. The JSR-352 chunking model does not cache the + * input as the regular Spring Batch implementations do so this component is not + * needed within a chunking loop. + * + * @author Michael Minella + * + * @param The type of input for the step + */ +public class JsrChunkProvider implements ChunkProvider { + + /* (non-Javadoc) + * @see org.springframework.batch.core.step.item.ChunkProvider#provide(org.springframework.batch.core.StepContribution) + */ + @Override + public Chunk provide(StepContribution contribution) throws Exception { + return new Chunk<>(); + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.step.item.ChunkProvider#postProcess(org.springframework.batch.core.StepContribution, org.springframework.batch.core.step.item.Chunk) + */ + @Override + public void postProcess(StepContribution contribution, Chunk chunk) { + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/item/JsrFaultTolerantChunkProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/item/JsrFaultTolerantChunkProcessor.java new file mode 100644 index 0000000000..82637ecdaa --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/item/JsrFaultTolerantChunkProcessor.java @@ -0,0 +1,350 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.item; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.StepListener; +import org.springframework.batch.core.listener.MulticasterBatchListener; +import org.springframework.batch.core.step.item.BatchRetryTemplate; +import org.springframework.batch.core.step.item.Chunk; +import org.springframework.batch.core.step.item.ChunkMonitor; +import org.springframework.batch.core.step.item.ForceRollbackForWriteSkipException; +import org.springframework.batch.core.step.skip.LimitCheckingItemSkipPolicy; +import org.springframework.batch.core.step.skip.SkipException; +import org.springframework.batch.core.step.skip.SkipPolicy; +import org.springframework.batch.core.step.skip.SkipPolicyFailedException; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.repeat.RepeatOperations; +import org.springframework.classify.BinaryExceptionClassifier; +import org.springframework.classify.Classifier; +import org.springframework.retry.RecoveryCallback; +import org.springframework.retry.RetryCallback; +import org.springframework.retry.RetryContext; +import org.springframework.retry.RetryException; +import org.springframework.util.Assert; + +import javax.batch.operations.BatchRuntimeException; +import java.util.List; + +/** + * Extension of the {@link JsrChunkProcessor} that adds skip and retry functionality. + * + * @author Michael Minella + * @author Chris Schaefer + * + * @param input type for the step + * @param output type for the step + */ +public class JsrFaultTolerantChunkProcessor extends JsrChunkProcessor { + protected final Log logger = LogFactory.getLog(getClass()); + private SkipPolicy skipPolicy = new LimitCheckingItemSkipPolicy(); + private Classifier rollbackClassifier = new BinaryExceptionClassifier(true); + private final BatchRetryTemplate batchRetryTemplate; + private ChunkMonitor chunkMonitor = new ChunkMonitor(); + private boolean hasProcessor = false; + + public JsrFaultTolerantChunkProcessor(ItemReader reader, ItemProcessor processor, ItemWriter writer, RepeatOperations repeatTemplate, BatchRetryTemplate batchRetryTemplate) { + super(reader, processor, writer, repeatTemplate); + hasProcessor = processor != null; + this.batchRetryTemplate = batchRetryTemplate; + } + + /** + * @param skipPolicy a {@link SkipPolicy} + */ + public void setSkipPolicy(SkipPolicy skipPolicy) { + Assert.notNull(skipPolicy, "A skip policy is required"); + + this.skipPolicy = skipPolicy; + } + + /** + * @param rollbackClassifier a {@link Classifier} + */ + public void setRollbackClassifier(Classifier rollbackClassifier) { + Assert.notNull(rollbackClassifier, "A rollbackClassifier is required"); + + this.rollbackClassifier = rollbackClassifier; + } + + /** + * @param chunkMonitor a {@link ChunkMonitor} + */ + public void setChunkMonitor(ChunkMonitor chunkMonitor) { + Assert.notNull(chunkMonitor, "A chunkMonitor is required"); + + this.chunkMonitor = chunkMonitor; + } + + /** + * Register some {@link StepListener}s with the handler. Each will get the + * callbacks in the order specified at the correct stage. + * + * @param listeners listeners to be registered + */ + @Override + public void setListeners(List listeners) { + for (StepListener listener : listeners) { + registerListener(listener); + } + } + + /** + * Register a listener for callbacks at the appropriate stages in a process. + * + * @param listener a {@link StepListener} + */ + @Override + public void registerListener(StepListener listener) { + getListener().register(listener); + } + + /** + * Adds retry and skip logic to the reading phase of the chunk loop. + * + * @param contribution a {@link StepContribution} + * @param chunk a {@link Chunk} + * @return I an item + * @throws Exception thrown if error occurs. + */ + @Override + protected I provide(final StepContribution contribution, final Chunk chunk) throws Exception { + RetryCallback retryCallback = new RetryCallback() { + + @Override + public I doWithRetry(RetryContext arg0) throws Exception { + while (true) { + try { + return doProvide(contribution, chunk); + } + catch (Exception e) { + if (shouldSkip(skipPolicy, e, contribution.getStepSkipCount())) { + + // increment skip count and try again + contribution.incrementReadSkipCount(); + chunk.skip(e); + + getListener().onSkipInRead(e); + + logger.debug("Skipping failed input", e); + } + else { + getListener().onRetryReadException(e); + + if(rollbackClassifier.classify(e)) { + throw e; + } + else { + throw e; + } + } + } + } + } + }; + + RecoveryCallback recoveryCallback = new RecoveryCallback() { + + @Override + public I recover(RetryContext context) throws Exception { + Throwable e = context.getLastThrowable(); + if (shouldSkip(skipPolicy, e, contribution.getStepSkipCount())) { + contribution.incrementReadSkipCount(); + logger.debug("Skipping after failed process", e); + return null; + } + else { + if (rollbackClassifier.classify(e)) { + // Default is to rollback unless the classifier + // allows us to continue + throw new RetryException("Non-skippable exception in recoverer while reading", e); + } + + throw new BatchRuntimeException(e); + } + } + + }; + + return batchRetryTemplate.execute(retryCallback, recoveryCallback); + } + + /** + * Convenience method for calling process skip policy. + * + * @param policy the skip policy + * @param e the cause of the skip + * @param skipCount the current skip count + */ + private boolean shouldSkip(SkipPolicy policy, Throwable e, int skipCount) { + try { + return policy.shouldSkip(e, skipCount); + } + catch (SkipException ex) { + throw ex; + } + catch (RuntimeException ex) { + throw new SkipPolicyFailedException("Fatal exception in SkipPolicy.", ex, e); + } + } + + /** + * Adds retry and skip logic to the process phase of the chunk loop. + * + * @param contribution a {@link StepContribution} + * @param item an item to be processed + * @return O an item that has been processed if a processor is available + * @throws Exception thrown if error occurs. + */ + @Override + @SuppressWarnings("unchecked") + protected O transform(final StepContribution contribution, final I item) throws Exception { + if (!hasProcessor) { + return (O) item; + } + + RetryCallback retryCallback = new RetryCallback() { + + @Override + public O doWithRetry(RetryContext context) throws Exception { + try { + return doTransform(item); + } + catch (Exception e) { + if (shouldSkip(skipPolicy, e, contribution.getStepSkipCount())) { + // If we are not re-throwing then we should check if + // this is skippable + contribution.incrementProcessSkipCount(); + logger.debug("Skipping after failed process with no rollback", e); + // If not re-throwing then the listener will not be + // called in next chunk. + getListener().onSkipInProcess(item, e); + } else { + getListener().onRetryProcessException(item, e); + + if (rollbackClassifier.classify(e)) { + // Default is to rollback unless the classifier + // allows us to continue + throw e; + } + else { + throw e; + } + } + } + return null; + } + + }; + + RecoveryCallback recoveryCallback = new RecoveryCallback() { + @Override + public O recover(RetryContext context) throws Exception { + Throwable e = context.getLastThrowable(); + if (shouldSkip(skipPolicy, e, contribution.getStepSkipCount())) { + contribution.incrementProcessSkipCount(); + logger.debug("Skipping after failed process", e); + return null; + } + else { + if (rollbackClassifier.classify(e)) { + // Default is to rollback unless the classifier + // allows us to continue + throw new RetryException("Non-skippable exception in recoverer while processing", e); + } + + throw new BatchRuntimeException(e); + } + } + }; + + return batchRetryTemplate.execute(retryCallback, recoveryCallback); + } + + /** + * Adds retry and skip logic to the write phase of the chunk loop. + * + * @param contribution a {@link StepContribution} + * @param chunk a {@link Chunk} + * @throws Exception thrown if error occurs. + */ + @Override + protected void persist(final StepContribution contribution, final Chunk chunk) throws Exception { + + RetryCallback retryCallback = new RetryCallback() { + @Override + @SuppressWarnings({ "unchecked", "rawtypes" }) + public Object doWithRetry(RetryContext context) throws Exception { + + chunkMonitor.setChunkSize(chunk.size()); + try { + doPersist(contribution, chunk); + } + catch (Exception e) { + if (shouldSkip(skipPolicy, e, contribution.getStepSkipCount())) { + // Per section 9.2.7 of JSR-352, the SkipListener receives all the items within the chunk + ((MulticasterBatchListener) getListener()).onSkipInWrite(chunk.getItems(), e); + } else { + getListener().onRetryWriteException((List) chunk.getItems(), e); + + if (rollbackClassifier.classify(e)) { + throw e; + } + } + /* + * If the exception is marked as no-rollback, we need to + * override that, otherwise there's no way to write the + * rest of the chunk or to honour the skip listener + * contract. + */ + throw new ForceRollbackForWriteSkipException( + "Force rollback on skippable exception so that skipped item can be located.", e); + } + contribution.incrementWriteCount(chunk.size()); + return null; + + } + }; + + RecoveryCallback recoveryCallback = new RecoveryCallback() { + + @Override + public O recover(RetryContext context) throws Exception { + Throwable e = context.getLastThrowable(); + if (shouldSkip(skipPolicy, e, contribution.getStepSkipCount())) { + contribution.incrementWriteSkipCount(); + logger.debug("Skipping after failed write", e); + return null; + } + else { + if (rollbackClassifier.classify(e)) { + // Default is to rollback unless the classifier + // allows us to continue + throw new RetryException("Non-skippable exception in recoverer while write", e); + } + return null; + } + } + + }; + + batchRetryTemplate.execute(retryCallback, recoveryCallback); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/item/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/item/package-info.java new file mode 100644 index 0000000000..90ad4517a7 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/item/package-info.java @@ -0,0 +1,10 @@ +/** + * JSR-352 specific components for implementing item based processing including fault tolerance. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.step.item; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/package-info.java new file mode 100644 index 0000000000..0fba98d0ce --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/jsr/step/package-info.java @@ -0,0 +1,10 @@ +/** + * JSR-352 extensions of existing batch {@link org.springframework.batch.core.Step} types. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.jsr.step; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobExecutionNotFailedException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobExecutionNotFailedException.java index f972d7ce1e..31fc309e1d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobExecutionNotFailedException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobExecutionNotFailedException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,17 +24,20 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class JobExecutionNotFailedException extends JobExecutionException { /** * Create an exception with the given message. + * + * @param msg the error message. */ public JobExecutionNotFailedException(String msg) { super(msg); } /** - * @param msg The message to send to caller + * @param msg the error message * @param e the cause of the exception */ public JobExecutionNotFailedException(String msg, Throwable e) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobExecutionNotRunningException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobExecutionNotRunningException.java index d64367a8a1..a068b5a2ee 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobExecutionNotRunningException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobExecutionNotRunningException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,6 +24,7 @@ * @author Dave Syer * @since 2.0 */ +@SuppressWarnings("serial") public class JobExecutionNotRunningException extends JobExecutionException { /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobExecutionNotStoppedException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobExecutionNotStoppedException.java index 2afba836d2..4ec94b45b9 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobExecutionNotStoppedException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobExecutionNotStoppedException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,10 +24,13 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class JobExecutionNotStoppedException extends JobExecutionException { /** * Create an exception with the given message. + * + * @param msg the message. */ public JobExecutionNotStoppedException(String msg) { super(msg); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobInstanceAlreadyExistsException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobInstanceAlreadyExistsException.java index 10d5a251b5..3558abba13 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobInstanceAlreadyExistsException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobInstanceAlreadyExistsException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,18 +26,21 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class JobInstanceAlreadyExistsException extends JobExecutionException { /** * Create an exception with the given message. + * + * @param msg the error message. */ public JobInstanceAlreadyExistsException(String msg) { super(msg); } /** - * @param msg The message to send to caller - * @param e the cause of the exception + * @param msg the error message. + * @param e the cause of the exception. */ public JobInstanceAlreadyExistsException(String msg, Throwable e) { super(msg, e); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobLauncher.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobLauncher.java index 59fdd3cbdf..ae2c258597 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobLauncher.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobLauncher.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -46,7 +46,9 @@ public interface JobLauncher { * one created. A exception will only be thrown if there is a failure to * start the job. If the job encounters some error while processing, the * JobExecution will be returned, and the status will need to be inspected. - * + * + * @param job the job to be executed. + * @param jobParameters the parameters passed to this execution of the job. * @return the {@link JobExecution} if it returns synchronously. If the * implementation is asynchronous, the status might well be unknown. * diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobOperator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobOperator.java index 129203c47b..59ef08e6ef 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobOperator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobOperator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -50,7 +50,8 @@ public interface JobOperator { * @param instanceId the id of a {@link JobInstance} * @return the id values of all the {@link JobExecution JobExecutions} * associated with this instance - * @throws NoSuchJobInstanceException + * @throws NoSuchJobInstanceException if the {@link JobInstance} associated with the + * {@code instanceId} cannot be found. */ List getExecutions(long instanceId) throws NoSuchJobInstanceException; @@ -62,7 +63,7 @@ public interface JobOperator { * @param start the start index of the instances * @param count the maximum number of values to return * @return the id values of the {@link JobInstance JobInstances} - * @throws NoSuchJobException + * @throws NoSuchJobException is thrown if no {@link JobInstance}s for the jobName exist. */ List getJobInstances(String jobName, int start, int count) throws NoSuchJobException; @@ -99,7 +100,7 @@ public interface JobOperator { * name * @throws JobInstanceAlreadyExistsException if a job instance with this * name and parameters already exists - * @throws JobParametersInvalidException + * @throws JobParametersInvalidException thrown if any of the job parameters are invalid. */ Long start(String jobName, String parameters) throws NoSuchJobException, JobInstanceAlreadyExistsException, JobParametersInvalidException; @@ -130,8 +131,8 @@ Long restart(long executionId) throws JobInstanceAlreadyCompleteException, NoSuc * {@link JobParametersIncrementer} attached to the specified job. If the * previous instance is still in a failed state, this method should still * create a new instance and run it with different parameters (as long as - * the {@link JobParametersIncrementer} is working).
      - *
      + * the {@link JobParametersIncrementer} is working).
      + *
      * * The last three exception described below should be extremely unlikely, * but cannot be ruled out entirely. It points to some other thread or @@ -140,11 +141,14 @@ Long restart(long executionId) throws JobInstanceAlreadyCompleteException, NoSuc * @param jobName the name of the job to launch * @return the {@link JobExecution} id of the execution created when the job * is launched + * * @throws NoSuchJobException if there is no such job definition available * @throws JobParametersNotFoundException if the parameters cannot be found - * @throws JobParametersInvalidException - * @throws UnexpectedJobExecutionException + * @throws JobParametersInvalidException thrown if some of the job parameters are invalid. * @throws UnexpectedJobExecutionException if an unexpected condition arises + * @throws JobRestartException thrown if a job is restarted illegally. + * @throws JobExecutionAlreadyRunningException thrown if attempting to restart a job that is already executing. + * @throws JobInstanceAlreadyCompleteException thrown if attempting to restart a completed job. */ Long startNextInstance(String jobName) throws NoSuchJobException, JobParametersNotFoundException, JobRestartException, JobExecutionAlreadyRunningException, JobInstanceAlreadyCompleteException, UnexpectedJobExecutionException, JobParametersInvalidException; @@ -197,17 +201,17 @@ Long startNextInstance(String jobName) throws NoSuchJobException, JobParametersN */ Set getJobNames(); - /** - * Mark the {@link JobExecution} as ABANDONED. If a stop signal is ignored - * because the process died this is the best way to mark a job as finished - * with (as opposed to STOPPED). An abandoned job execution can be - * restarted, but a stopping one cannot. - * - * @param jobExecutionId the job execution id to abort - * @return the {@link JobExecution} that was aborted - * @throws NoSuchJobExecutionException - * @throws JobExecutionAlreadyRunningException if the job is running (it - * should be stopped first) - */ - JobExecution abandon(long jobExecutionId) throws NoSuchJobExecutionException, JobExecutionAlreadyRunningException; + /** + * Mark the {@link JobExecution} as ABANDONED. If a stop signal is ignored + * because the process died this is the best way to mark a job as finished + * with (as opposed to STOPPED). An abandoned job execution cannot be + * restarted by the framework. + * + * @param jobExecutionId the job execution id to abort + * @return the {@link JobExecution} that was aborted + * @throws NoSuchJobExecutionException thrown if there is no job execution for the jobExecutionId. + * @throws JobExecutionAlreadyRunningException if the job is running (it + * should be stopped first) + */ + JobExecution abandon(long jobExecutionId) throws NoSuchJobExecutionException, JobExecutionAlreadyRunningException; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobParametersNotFoundException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobParametersNotFoundException.java index aa80e52ab3..6f4b978ac1 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobParametersNotFoundException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/JobParametersNotFoundException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,18 +26,21 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class JobParametersNotFoundException extends JobExecutionException { /** * Create an exception with the given message. + * + * @param msg the error message. */ public JobParametersNotFoundException(String msg) { super(msg); } /** - * @param msg The message to send to caller - * @param e the cause of the exception + * @param msg the error message. + * @param e the cause of the exception. */ public JobParametersNotFoundException(String msg, Throwable e) { super(msg, e); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/NoSuchJobException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/NoSuchJobException.java index 44d39abd22..0713c135d0 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/NoSuchJobException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/NoSuchJobException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,18 +26,21 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class NoSuchJobException extends JobExecutionException { /** * Create an exception with the given message. + * + * @param msg the error message. */ public NoSuchJobException(String msg) { super(msg); } /** - * @param msg The message to send to caller - * @param e the cause of the exception + * @param msg the error message. + * @param e the cause of the exception. */ public NoSuchJobException(String msg, Throwable e) { super(msg, e); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/NoSuchJobExecutionException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/NoSuchJobExecutionException.java index 63e863bf2c..96a48412b7 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/NoSuchJobExecutionException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/NoSuchJobExecutionException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,18 +25,21 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class NoSuchJobExecutionException extends JobExecutionException { /** * Create an exception with the given message. + * + * @param msg the error message. */ public NoSuchJobExecutionException(String msg) { super(msg); } /** - * @param msg The message to send to caller - * @param e the cause of the exception + * @param msg the error message. + * @param e the cause of the exception. */ public NoSuchJobExecutionException(String msg, Throwable e) { super(msg, e); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/NoSuchJobInstanceException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/NoSuchJobInstanceException.java index 01629991fb..589bc30ebf 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/NoSuchJobInstanceException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/NoSuchJobInstanceException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,18 +25,21 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class NoSuchJobInstanceException extends JobExecutionException { /** * Create an exception with the given message. + * + * @param msg the error message. */ public NoSuchJobInstanceException(String msg) { super(msg); } /** - * @param msg The message to send to caller - * @param e the cause of the exception + * @param msg the error message. + * @param e the cause of the exception. */ public NoSuchJobInstanceException(String msg, Throwable e) { super(msg, e); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/package-info.java new file mode 100644 index 0000000000..0b629cc889 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/package-info.java @@ -0,0 +1,10 @@ +/** + * Interfaces and simple implementations of launch concerns. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.launch; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/package.html deleted file mode 100644 index b302ca258e..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Interfaces and simple implementations of launch concerns. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/CommandLineJobRunner.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/CommandLineJobRunner.java index b8c9e71cbf..3d3dbba5d1 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/CommandLineJobRunner.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/CommandLineJobRunner.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,22 +21,21 @@ import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; -import java.util.HashMap; -import java.util.HashSet; +import java.util.LinkedHashSet; import java.util.List; -import java.util.Map; import java.util.Properties; import java.util.Set; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobInstance; -import org.springframework.batch.core.JobParameter; import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; import org.springframework.batch.core.JobParametersIncrementer; import org.springframework.batch.core.configuration.JobLocator; import org.springframework.batch.core.converter.DefaultJobParametersConverter; @@ -46,11 +45,12 @@ import org.springframework.batch.core.launch.JobExecutionNotRunningException; import org.springframework.batch.core.launch.JobExecutionNotStoppedException; import org.springframework.batch.core.launch.JobLauncher; -import org.springframework.batch.core.launch.JobParametersNotFoundException; +import org.springframework.batch.core.launch.NoSuchJobException; import org.springframework.batch.core.repository.JobRepository; import org.springframework.beans.factory.BeanDefinitionStoreException; import org.springframework.beans.factory.config.AutowireCapableBeanFactory; import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; import org.springframework.util.Assert; import org.springframework.util.StringUtils; @@ -98,11 +98,12 @@ *

      * * - * jobPath jobIdentifier (jobParameters)* + * jobPath <options> jobIdentifier (jobParameters)* * * *

      * The command line options are as follows + *

      *
        *
      • jobPath: the xml application context containing a {@link Job} *
      • -restart: (optional) to restart the last failed execution
      • @@ -115,7 +116,6 @@ *
      • jobParameters: 0 to many parameters that will be used to launch a job * specified in the form of key=value pairs. *
      - *

      * *

      * If the -next option is used the parameters on the command line @@ -132,18 +132,18 @@ * {@link JobParameters} using a {@link JobParametersConverter} from the * application context (if there is one, or a * {@link DefaultJobParametersConverter} otherwise). Below is an example - * arguments list: " + * arguments list: "

      * *

      * * java org.springframework.batch.core.launch.support.CommandLineJobRunner testJob.xml * testJob schedule.date=2008/01/24 vendor.id=3902483920 - * + * *

      * *

      * Once arguments have been successfully parsed, autowiring will be used to set - * various dependencies. The {@JobLauncher} for example, will be + * various dependencies. The {@link JobLauncher} for example, will be * loaded this way. If none is contained in the bean factory (it searches by * type) then a {@link BeanDefinitionStoreException} will be thrown. The same * exception will also be thrown if there is more than one present. Assuming the @@ -155,6 +155,7 @@ * * @author Dave Syer * @author Lucas Ward + * @author Mahmoud Ben Hassine * @since 1.0 */ public class CommandLineJobRunner { @@ -219,7 +220,7 @@ public void setExitCodeMapper(ExitCodeMapper exitCodeMapper) { * dependency injection. Typically overridden by * {@link #setSystemExiter(SystemExiter)}. * - * @param systemExiter + * @param systemExiter {@link SystemExiter} instance to be used by CommandLineJobRunner instance. */ public static void presetSystemExiter(SystemExiter systemExiter) { CommandLineJobRunner.systemExiter = systemExiter; @@ -239,7 +240,7 @@ public static String getErrorMessage() { /** * Injection setter for the {@link SystemExiter}. * - * @param systemExiter + * @param systemExiter {@link SystemExiter} instance to be used by CommandLineJobRunner instance. */ public void setSystemExiter(SystemExiter systemExiter) { CommandLineJobRunner.systemExiter = systemExiter; @@ -248,7 +249,8 @@ public void setSystemExiter(SystemExiter systemExiter) { /** * Injection setter for {@link JobParametersConverter}. * - * @param jobParametersConverter + * @param jobParametersConverter instance of {@link JobParametersConverter} + * to be used by the CommandLineJobRunner instance. */ public void setJobParametersConverter(JobParametersConverter jobParametersConverter) { this.jobParametersConverter = jobParametersConverter; @@ -257,7 +259,7 @@ public void setJobParametersConverter(JobParametersConverter jobParametersConver /** * Delegate to the exiter to (possibly) exit the VM gracefully. * - * @param status + * @param status int exit code that should be reported. */ public void exit(int status) { systemExiter.exit(status); @@ -276,12 +278,18 @@ public void setJobLocator(JobLocator jobLocator) { * job paths. If a JobLocator has been set, then use it to obtain an actual * job, if not ask the context for it. */ + @SuppressWarnings("resource") int start(String jobPath, String jobIdentifier, String[] parameters, Set opts) { ConfigurableApplicationContext context = null; try { - context = new ClassPathXmlApplicationContext(jobPath); + try { + context = new AnnotationConfigApplicationContext(Class.forName(jobPath)); + } catch (ClassNotFoundException cnfe) { + context = new ClassPathXmlApplicationContext(jobPath); + } + context.getAutowireCapableBeanFactory().autowireBeanProperties(this, AutowireCapableBeanFactory.AUTOWIRE_BY_TYPE, false); @@ -292,7 +300,7 @@ int start(String jobPath, String jobIdentifier, String[] parameters, Set } String jobName = jobIdentifier; - + JobParameters jobParameters = jobParametersConverter.getJobParameters(StringUtils .splitArrayElementsIntoProperties(parameters, "=")); Assert.isTrue(parameters == null || parameters.length == 0 || !jobParameters.isEmpty(), @@ -333,19 +341,21 @@ int start(String jobPath, String jobIdentifier, String[] parameters, Set jobName = jobExecution.getJobInstance().getJobName(); } - Job job; + Job job = null; if (jobLocator != null) { - job = jobLocator.getJob(jobName); + try { + job = jobLocator.getJob(jobName); + } catch (NoSuchJobException e) { + } } - else { + if (job == null) { job = (Job) context.getBean(jobName); } if (opts.contains("-next")) { - JobParameters nextParameters = getNextJobParameters(job); - Map map = new HashMap(nextParameters.getParameters()); - map.putAll(jobParameters.getParameters()); - jobParameters = new JobParameters(map); + jobParameters = new JobParametersBuilder(jobParameters, jobExplorer) + .getNextJobParameters(job) + .toJobParameters(); } JobExecution jobExecution = launcher.run(job, jobParameters); @@ -383,7 +393,7 @@ private List getJobExecutionsWithStatusGreaterThan(String jobIdent int start = 0; int count = 100; - List executions = new ArrayList(); + List executions = new ArrayList<>(); List lastInstances = jobExplorer.getJobInstances(jobIdentifier, start, count); while (!lastInstances.isEmpty()) { @@ -422,7 +432,7 @@ private List getStoppedJobExecutions(String jobIdentifier) { if (jobExecutions.isEmpty()) { return null; } - List result = new ArrayList(); + List result = new ArrayList<>(); for (JobExecution jobExecution : jobExecutions) { if (jobExecution.getStatus() != BatchStatus.ABANDONED) { result.add(jobExecution); @@ -432,16 +442,17 @@ private List getStoppedJobExecutions(String jobIdentifier) { } private List getRunningJobExecutions(String jobIdentifier) { - List jobExecutions = getJobExecutionsWithStatusGreaterThan(jobIdentifier, BatchStatus.COMPLETED); - if (jobExecutions.isEmpty()) { - return null; - } - List result = new ArrayList(); - for (JobExecution jobExecution : jobExecutions) { - if (jobExecution.isRunning()) { + Long executionId = getLongIdentifier(jobIdentifier); + List result = new ArrayList<>(); + if (executionId != null) { + JobExecution jobExecution = jobExplorer.getJobExecution(executionId); + if (jobExecution != null && jobExecution.isRunning()) { result.add(jobExecution); } } + else { + result.addAll(jobExplorer.findRunningJobExecutions(jobIdentifier)); + } return result.isEmpty() ? null : result; } @@ -455,76 +466,52 @@ private Long getLongIdentifier(String jobIdentifier) { } } - /** - * @param job the job that we need to find the next parameters for - * @return the next job parameters if they can be located - * @throws JobParametersNotFoundException if there is a problem - */ - private JobParameters getNextJobParameters(Job job) throws JobParametersNotFoundException { - String jobIdentifier = job.getName(); - JobParameters jobParameters; - List lastInstances = jobExplorer.getJobInstances(jobIdentifier, 0, 1); - - JobParametersIncrementer incrementer = job.getJobParametersIncrementer(); - if (incrementer == null) { - throw new JobParametersNotFoundException("No job parameters incrementer found for job=" + jobIdentifier); - } - - if (lastInstances.isEmpty()) { - jobParameters = incrementer.getNext(new JobParameters()); - if (jobParameters == null) { - throw new JobParametersNotFoundException("No bootstrap parameters found from incrementer for job=" - + jobIdentifier); - } - } - else { - List lastExecutions = jobExplorer.getJobExecutions(lastInstances.get(0)); - jobParameters = incrementer.getNext(lastExecutions.get(0).getJobParameters()); - } - return jobParameters; - } - /** * Launch a batch job using a {@link CommandLineJobRunner}. Creates a new * Spring context for the job execution, and uses a common parent for all * such contexts. No exception are thrown from this method, rather * exceptions are logged and an integer returned through the exit status in * a {@link JvmSystemExiter} (which can be overridden by defining one in the - * Spring context).
      + * Spring context).
      * Parameters can be provided in the form key=value, and will be converted * using the injected {@link JobParametersConverter}. * - * @param args

      + * @param args *

        *
      • -restart: (optional) if the job has failed or stopped and the most * should be restarted. If specified then the jobIdentifier parameter can be - * interpreted either as the name of the job or the id of teh job execution + * interpreted either as the name of the job or the id of the job execution * that failed.
      • *
      • -next: (optional) if the job has a {@link JobParametersIncrementer} - * that can be used to launch the next in a sequence
      • + * that can be used to launch the next instance in a sequence *
      • jobPath: the xml application context containing a {@link Job} *
      • jobIdentifier: the bean id of the job or id of the failed execution * in the case of a restart. *
      • jobParameters: 0 to many parameters that will be used to launch a * job. *
      + *

      * The options (-restart, -next) can occur anywhere in the * command line. *

      + * + * @throws Exception is thrown if error occurs. */ public static void main(String[] args) throws Exception { CommandLineJobRunner command = new CommandLineJobRunner(); - List newargs = new ArrayList(Arrays.asList(args)); + List newargs = new ArrayList<>(Arrays.asList(args)); try { if (System.in.available() > 0) { BufferedReader reader = new BufferedReader(new InputStreamReader(System.in)); String line = " "; - while (StringUtils.hasLength(line)) { + while (line != null) { if (!line.startsWith("#") && StringUtils.hasText(line)) { - logger.debug("Stdin arg: " + line); + if (logger.isDebugEnabled()) { + logger.debug("Stdin arg: " + line); + } newargs.add(line); } line = reader.readLine(); @@ -538,8 +525,8 @@ public static void main(String[] args) throws Exception { } } - Set opts = new HashSet(); - List params = new ArrayList(); + Set opts = new LinkedHashSet<>(); + List params = new ArrayList<>(); int count = 0; String jobPath = null; @@ -566,10 +553,11 @@ public static void main(String[] args) throws Exception { } if (jobPath == null || jobIdentifier == null) { - String message = "At least 2 arguments are required: JobPath and jobIdentifier."; + String message = "At least 2 arguments are required: JobPath/JobClass and jobIdentifier."; logger.error(message); CommandLineJobRunner.message = message; command.exit(1); + return; } String[] parameters = params.toArray(new String[params.size()]); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/ExitCodeMapper.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/ExitCodeMapper.java index f1b22c45af..4b15bba20b 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/ExitCodeMapper.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/ExitCodeMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/JobRegistryBackgroundJobRunner.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/JobRegistryBackgroundJobRunner.java index 0cffcdfd79..692512d0a0 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/JobRegistryBackgroundJobRunner.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/JobRegistryBackgroundJobRunner.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -81,10 +81,10 @@ public class JobRegistryBackgroundJobRunner { private JobRegistry jobRegistry; - private static List errors = Collections.synchronizedList(new ArrayList()); + private static List errors = Collections.synchronizedList(new ArrayList<>()); /** - * @param parentContextPath + * @param parentContextPath the parentContextPath to be used by the JobRegistryBackgroundJobRunner. */ public JobRegistryBackgroundJobRunner(String parentContextPath) { super(); @@ -116,7 +116,7 @@ public void setJobRegistry(JobRegistry jobRegistry) { */ public static List getErrors() { synchronized (errors) { - return new ArrayList(errors); + return new ArrayList<>(errors); } } @@ -211,7 +211,7 @@ public void run() { errors.add(e); throw e; } - }; + } }).start(); logger.info("Waiting for parent context to start."); @@ -249,7 +249,7 @@ public void run() { } /** - * De-register all the {@link Job} instances that were regsistered by this + * Unregister all the {@link Job} instances that were registered by this * post processor. * @see org.springframework.beans.factory.DisposableBean#destroy() */ diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/JvmSystemExiter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/JvmSystemExiter.java index 733d15b04a..41828979ef 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/JvmSystemExiter.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/JvmSystemExiter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/RunIdIncrementer.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/RunIdIncrementer.java index 25abace0ad..9b6cff5d02 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/RunIdIncrementer.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/RunIdIncrementer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,9 +18,11 @@ import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.JobParametersBuilder; import org.springframework.batch.core.JobParametersIncrementer; +import org.springframework.lang.Nullable; /** * @author Dave Syer + * @author Mahmoud Ben Hassine */ public class RunIdIncrementer implements JobParametersIncrementer { @@ -41,11 +43,11 @@ public void setKey(String key) { * Increment the run.id parameter (starting with 1). */ @Override - public JobParameters getNext(JobParameters parameters) { + public JobParameters getNext(@Nullable JobParameters parameters) { JobParameters params = (parameters == null) ? new JobParameters() : parameters; - long id = params.getLong(key, 0L) + 1; + long id = params.getLong(key, new Long(0)) + 1; return new JobParametersBuilder(params).addLong(key, id).toJobParameters(); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/RuntimeExceptionTranslator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/RuntimeExceptionTranslator.java index 3b30b8bbf6..988be9bfb5 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/RuntimeExceptionTranslator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/RuntimeExceptionTranslator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/ScheduledJobParametersFactory.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/ScheduledJobParametersFactory.java index 520990817b..cf8d5a272e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/ScheduledJobParametersFactory.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/ScheduledJobParametersFactory.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,9 +27,11 @@ import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.JobParametersBuilder; import org.springframework.batch.core.converter.JobParametersConverter; +import org.springframework.lang.Nullable; /** * @author Lucas Ward + * @author Mahmoud Ben Hassine * */ public class ScheduledJobParametersFactory implements JobParametersConverter { @@ -44,7 +46,7 @@ public class ScheduledJobParametersFactory implements JobParametersConverter { * @see org.springframework.batch.core.runtime.JobParametersFactory#getJobParameters(java.util.Properties) */ @Override - public JobParameters getJobParameters(Properties props) { + public JobParameters getJobParameters(@Nullable Properties props) { if (props == null || props.isEmpty()) { return new JobParameters(); @@ -75,7 +77,7 @@ public JobParameters getJobParameters(Properties props) { * @see org.springframework.batch.core.converter.JobParametersConverter#getProperties(org.springframework.batch.core.JobParameters) */ @Override - public Properties getProperties(JobParameters params) { + public Properties getProperties(@Nullable JobParameters params) { if (params == null || params.isEmpty()) { return new Properties(); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SimpleJobLauncher.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SimpleJobLauncher.java index 41dd76ac06..fdd14fff81 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SimpleJobLauncher.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SimpleJobLauncher.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,6 +15,8 @@ */ package org.springframework.batch.core.launch.support; +import java.time.Duration; + import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.core.BatchStatus; @@ -26,6 +28,7 @@ import org.springframework.batch.core.JobParametersInvalidException; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.metrics.BatchMetrics; import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; import org.springframework.batch.core.repository.JobRepository; @@ -53,8 +56,10 @@ * Repository can reliably recreate it. * * @author Lucas Ward - * @Author Dave Syer + * @author Dave Syer * @author Will Schipp + * @author Michael Minella + * @author Mahmoud Ben Hassine * * @since 1.0 * @@ -77,13 +82,15 @@ public class SimpleJobLauncher implements JobLauncher, InitializingBean { * @param job the job to be run. * @param jobParameters the {@link JobParameters} for this particular * execution. - * @return JobExecutionAlreadyRunningException if the JobInstance already + * @return the {@link JobExecution} if it returns synchronously. If the + * implementation is asynchronous, the status might well be unknown. + * @throws JobExecutionAlreadyRunningException if the JobInstance already * exists and has an execution already running. * @throws JobRestartException if the execution would be a re-start, but a * re-start is either not allowed or not needed. * @throws JobInstanceAlreadyCompleteException if this instance has already * completed successfully - * @throws JobParametersInvalidException + * @throws JobParametersInvalidException thrown if jobParameters is invalid. */ @Override public JobExecution run(final Job job, final JobParameters jobParameters) @@ -100,21 +107,27 @@ public JobExecution run(final Job job, final JobParameters jobParameters) throw new JobRestartException("JobInstance already exists and is not restartable"); } /* - * validate here if it has stepExecutions that are UNKNOWN + * validate here if it has stepExecutions that are UNKNOWN, STARTING, STARTED and STOPPING * retrieve the previous execution and check */ for (StepExecution execution : lastExecution.getStepExecutions()) { - if (execution.getStatus() == BatchStatus.UNKNOWN) { - //throw - throw new JobRestartException("Step [" + execution.getStepName() + "] is of status UNKNOWN"); - }//end if - }//end for + BatchStatus status = execution.getStatus(); + if (status.isRunning() || status == BatchStatus.STOPPING) { + throw new JobExecutionAlreadyRunningException("A job execution for this job is already running: " + + lastExecution); + } else if (status == BatchStatus.UNKNOWN) { + throw new JobRestartException( + "Cannot restart step [" + execution.getStepName() + "] from UNKNOWN status. " + + "The last execution ended with a failure that could not be rolled back, " + + "so it may be dangerous to proceed. Manual intervention is probably necessary."); + } + } } // Check the validity of the parameters before doing creating anything // in the repository... job.getJobParametersValidator().validate(jobParameters); - + /* * There is a very small probability that a non-restartable job can be * restarted, but only if another process or thread manages to launch @@ -132,8 +145,10 @@ public void run() { logger.info("Job: [" + job + "] launched with the following parameters: [" + jobParameters + "]"); job.execute(jobExecution); + Duration jobExecutionDuration = BatchMetrics.calculateDuration(jobExecution.getStartTime(), jobExecution.getEndTime()); logger.info("Job: [" + job + "] completed with the following parameters: [" + jobParameters - + "] and the following status: [" + jobExecution.getStatus() + "]"); + + "] and the following status: [" + jobExecution.getStatus() + "]" + + (jobExecutionDuration == null ? "" : " in " + BatchMetrics.formatDuration(jobExecutionDuration))); } catch (Throwable t) { logger.info("Job: [" + job @@ -166,9 +181,9 @@ else if (t instanceof Error) { } /** - * Set the JobRepsitory. + * Set the JobRepository. * - * @param jobRepository + * @param jobRepository instance of {@link JobRepository}. */ public void setJobRepository(JobRepository jobRepository) { this.jobRepository = jobRepository; @@ -177,7 +192,7 @@ public void setJobRepository(JobRepository jobRepository) { /** * Set the TaskExecutor. (Optional) * - * @param taskExecutor + * @param taskExecutor instance of {@link TaskExecutor}. */ public void setTaskExecutor(TaskExecutor taskExecutor) { this.taskExecutor = taskExecutor; @@ -195,5 +210,4 @@ public void afterPropertiesSet() throws Exception { taskExecutor = new SyncTaskExecutor(); } } - } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SimpleJobOperator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SimpleJobOperator.java index 0735e7f2e7..5fc0d35822 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SimpleJobOperator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SimpleJobOperator.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,8 +31,9 @@ import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.JobParameters; -import org.springframework.batch.core.JobParametersIncrementer; +import org.springframework.batch.core.JobParametersBuilder; import org.springframework.batch.core.JobParametersInvalidException; +import org.springframework.batch.core.Step; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.UnexpectedJobExecutionException; import org.springframework.batch.core.configuration.JobRegistry; @@ -44,7 +45,6 @@ import org.springframework.batch.core.launch.JobInstanceAlreadyExistsException; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.launch.JobOperator; -import org.springframework.batch.core.launch.JobParametersNotFoundException; import org.springframework.batch.core.launch.NoSuchJobException; import org.springframework.batch.core.launch.NoSuchJobExecutionException; import org.springframework.batch.core.launch.NoSuchJobInstanceException; @@ -52,6 +52,12 @@ import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.batch.core.step.NoSuchStepException; +import org.springframework.batch.core.step.StepLocator; +import org.springframework.batch.core.step.tasklet.StoppableTasklet; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.core.step.tasklet.TaskletStep; import org.springframework.batch.support.PropertiesConverter; import org.springframework.beans.factory.InitializingBean; import org.springframework.transaction.annotation.Transactional; @@ -71,13 +77,12 @@ * * @author Dave Syer * @author Lucas Ward + * @author Will Schipp + * @author Mahmoud Ben Hassine * @since 2.0 */ public class SimpleJobOperator implements JobOperator, InitializingBean { - /** - * - */ private static final String ILLEGAL_STATE_MSG = "Illegal state (only happens on a race condition): " + "%s with name=%s and parameters=%s"; @@ -153,7 +158,7 @@ public List getExecutions(long instanceId) throws NoSuchJobInstanceExcepti if (jobInstance == null) { throw new NoSuchJobInstanceException(String.format("No job instance with id=%d", instanceId)); } - List list = new ArrayList(); + List list = new ArrayList<>(); for (JobExecution jobExecution : jobExplorer.getJobExecutions(jobInstance)) { list.add(jobExecution.getId()); } @@ -167,7 +172,7 @@ public List getExecutions(long instanceId) throws NoSuchJobInstanceExcepti */ @Override public Set getJobNames() { - return new TreeSet(jobRegistry.getJobNames()); + return new TreeSet<>(jobRegistry.getJobNames()); } /* @@ -177,8 +182,9 @@ public Set getJobNames() { */ @Override public List getJobInstances(String jobName, int start, int count) throws NoSuchJobException { - List list = new ArrayList(); - for (JobInstance jobInstance : jobExplorer.getJobInstances(jobName, start, count)) { + List list = new ArrayList<>(); + List jobInstances = jobExplorer.getJobInstances(jobName, start, count); + for (JobInstance jobInstance : jobInstances) { list.add(jobInstance.getId()); } if (list.isEmpty() && !jobRegistry.getJobNames().contains(jobName)) { @@ -211,7 +217,7 @@ public String getParameters(long executionId) throws NoSuchJobExecutionException */ @Override public Set getRunningExecutions(String jobName) throws NoSuchJobException { - Set set = new LinkedHashSet(); + Set set = new LinkedHashSet<>(); for (JobExecution jobExecution : jobExplorer.findRunningJobExecutions(jobName)) { set.add(jobExecution.getId()); } @@ -232,7 +238,7 @@ public Set getRunningExecutions(String jobName) throws NoSuchJobException public Map getStepExecutionSummaries(long executionId) throws NoSuchJobExecutionException { JobExecution jobExecution = findExecutionById(executionId); - Map map = new LinkedHashMap(); + Map map = new LinkedHashMap<>(); for (StepExecution stepExecution : jobExecution.getStepExecutions()) { map.put(stepExecution.getId(), stepExecution.toString()); } @@ -328,30 +334,15 @@ public Long start(String jobName, String parameters) throws NoSuchJobException, * @see JobOperator#startNextInstance(String ) */ @Override - public Long startNextInstance(String jobName) throws NoSuchJobException, JobParametersNotFoundException, + public Long startNextInstance(String jobName) throws NoSuchJobException, UnexpectedJobExecutionException, JobParametersInvalidException { logger.info("Locating parameters for next instance of job with name=" + jobName); Job job = jobRegistry.getJob(jobName); - List lastInstances = jobExplorer.getJobInstances(jobName, 0, 1); - - JobParametersIncrementer incrementer = job.getJobParametersIncrementer(); - if (incrementer == null) { - throw new JobParametersNotFoundException("No job parameters incrementer found for job=" + jobName); - } - - JobParameters parameters; - if (lastInstances.isEmpty()) { - parameters = incrementer.getNext(new JobParameters()); - if (parameters == null) { - throw new JobParametersNotFoundException("No bootstrap parameters found for job=" + jobName); - } - } - else { - List lastExecutions = jobExplorer.getJobExecutions(lastInstances.get(0)); - parameters = incrementer.getNext(lastExecutions.get(0).getJobParameters()); - } + JobParameters parameters = new JobParametersBuilder(jobExplorer) + .getNextJobParameters(job) + .toJobParameters(); logger.info(String.format("Attempting to launch job with name=%s and parameters=%s", jobName, parameters)); try { @@ -393,6 +384,35 @@ public boolean stop(long executionId) throws NoSuchJobExecutionException, JobExe jobExecution.setStatus(BatchStatus.STOPPING); jobRepository.update(jobExecution); + try { + Job job = jobRegistry.getJob(jobExecution.getJobInstance().getJobName()); + if (job instanceof StepLocator) {//can only process as StepLocator is the only way to get the step object + //get the current stepExecution + for (StepExecution stepExecution : jobExecution.getStepExecutions()) { + if (stepExecution.getStatus().isRunning()) { + try { + //have the step execution that's running -> need to 'stop' it + Step step = ((StepLocator)job).getStep(stepExecution.getStepName()); + if (step instanceof TaskletStep) { + Tasklet tasklet = ((TaskletStep)step).getTasklet(); + if (tasklet instanceof StoppableTasklet) { + StepSynchronizationManager.register(stepExecution); + ((StoppableTasklet)tasklet).stop(); + StepSynchronizationManager.release(); + } + } + } + catch (NoSuchStepException e) { + logger.warn("Step not found",e); + } + } + } + } + } + catch (NoSuchJobException e) { + logger.warn("Cannot find Job object in the job registry. StoppableTasklet#stop() will not be called",e); + } + return true; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SimpleJvmExitCodeMapper.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SimpleJvmExitCodeMapper.java index e97776b246..a5db659603 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SimpleJvmExitCodeMapper.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SimpleJvmExitCodeMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,7 +40,7 @@ public class SimpleJvmExitCodeMapper implements ExitCodeMapper { private Map mapping; public SimpleJvmExitCodeMapper() { - mapping = new HashMap(); + mapping = new HashMap<>(); mapping.put(ExitStatus.COMPLETED.getExitCode(), JVM_EXITCODE_COMPLETED); mapping.put(ExitStatus.FAILED.getExitCode(), JVM_EXITCODE_GENERIC_ERROR); mapping.put(ExitCodeMapper.JOB_NOT_PROVIDED, JVM_EXITCODE_JOB_ERROR); @@ -62,8 +62,8 @@ public void setMapping(Map exitCodeMap) { /** * Get the operating system exit status that matches a certain Batch - * Framework Exitcode - * @param exitCode The exitcode of the Batch Job as known by the Batch + * Framework exit code + * @param exitCode The exit code of the Batch Job as known by the Batch * Framework * @return The exitCode of the Batch Job as known by the JVM */ diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SystemExiter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SystemExiter.java index 778ed3f1e1..a49efa3fd0 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SystemExiter.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/SystemExiter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/package-info.java new file mode 100644 index 0000000000..8a468e015a --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/package-info.java @@ -0,0 +1,10 @@ +/** + * Support classes for use in bootstrap and launch implementations or configurations. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.launch.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/package.html deleted file mode 100644 index 0401779d5a..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/launch/support/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Support classes for use in bootstrap and launch implementations or configurations. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/AbstractListenerFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/AbstractListenerFactoryBean.java index 23604e4e45..39a8568c47 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/AbstractListenerFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/AbstractListenerFactoryBean.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,15 +15,15 @@ */ package org.springframework.batch.core.listener; -import static org.springframework.batch.support.MethodInvokerUtils.getMethodInvokerByAnnotation; -import static org.springframework.batch.support.MethodInvokerUtils.getMethodInvokerForInterface; - import java.util.HashMap; import java.util.HashSet; import java.util.Map; import java.util.Map.Entry; import java.util.Set; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + import org.springframework.aop.TargetSource; import org.springframework.aop.framework.Advised; import org.springframework.aop.framework.ProxyFactory; @@ -35,6 +35,9 @@ import org.springframework.core.Ordered; import org.springframework.util.Assert; +import static org.springframework.batch.support.MethodInvokerUtils.getMethodInvokerByAnnotation; +import static org.springframework.batch.support.MethodInvokerUtils.getMethodInvokerForInterface; + /** * {@link FactoryBean} implementation that builds a listener based on the * various lifecycle methods or annotations that are provided. There are three @@ -62,8 +65,9 @@ * @since 2.0 * @see ListenerMetaData */ -@SuppressWarnings("rawtypes") -public abstract class AbstractListenerFactoryBean implements FactoryBean, InitializingBean { +public abstract class AbstractListenerFactoryBean implements FactoryBean, InitializingBean { + + private static final Log logger = LogFactory.getLog(AbstractListenerFactoryBean.class); private Object delegate; @@ -71,9 +75,8 @@ public abstract class AbstractListenerFactoryBean implements FactoryBean, Initia @Override public Object getObject() { - if (metaDataMap == null) { - metaDataMap = new HashMap(); + metaDataMap = new HashMap<>(); } // Because all annotations and interfaces should be checked for, make // sure that each meta data @@ -85,19 +88,17 @@ public Object getObject() { } } - Set> listenerInterfaces = new HashSet>(); + Set> listenerInterfaces = new HashSet<>(); // For every entry in the map, try and find a method by interface, name, // or annotation. If the same - Map> invokerMap = new HashMap>(); + Map> invokerMap = new HashMap<>(); boolean synthetic = false; for (Entry entry : metaDataMap.entrySet()) { - final ListenerMetaData metaData = this.getMetaDataFromPropertyName(entry.getKey()); - Set invokers = new HashSet(); + Set invokers = new HashSet<>(); MethodInvoker invoker; - invoker = getMethodInvokerForInterface(metaData.getListenerInterface(), metaData.getMethodName(), delegate, metaData.getParamTypes()); if (invoker != null) { @@ -110,10 +111,12 @@ public Object getObject() { synthetic = true; } - invoker = getMethodInvokerByAnnotation(metaData.getAnnotation(), delegate, metaData.getParamTypes()); - if (invoker != null) { - invokers.add(invoker); - synthetic = true; + if(metaData.getAnnotation() != null) { + invoker = getMethodInvokerByAnnotation(metaData.getAnnotation(), delegate, metaData.getParamTypes()); + if (invoker != null) { + invokers.add(invoker); + synthetic = true; + } } if (!invokers.isEmpty()) { @@ -155,7 +158,10 @@ public Object getObject() { else { proxyFactory.setTarget(delegate); } - proxyFactory.setInterfaces(listenerInterfaces.toArray(new Class[0])); + @SuppressWarnings("rawtypes") + Class[] a = new Class[0]; + + proxyFactory.setInterfaces(listenerInterfaces.toArray(a)); proxyFactory.addAdvisor(new DefaultPointcutAdvisor(new MethodInvokerMethodInterceptor(invokerMap, ordered))); return proxyFactory.getProxy(); @@ -199,6 +205,8 @@ public void afterPropertiesSet() throws Exception { * into a listener. * * @param target the object to check + * @param listenerType the class of the listener. + * @param metaDataValues array of {@link ListenerMetaData}. * @return true if the delegate is an instance of any of the listener * interface, or contains the marker annotations */ @@ -215,6 +223,10 @@ public static boolean isListener(Object target, Class listenerType, ListenerM && listenerType.isAssignableFrom(targetSource.getTargetClass())) { return true; } + + if(targetSource != null && targetSource.getTargetClass() != null && targetSource.getTargetClass().isInterface()) { + logger.warn(String.format("%s is an interface. The implementing class will not be queried for annotation based listener configurations. If using @StepScope on a @Bean method, be sure to return the implementing class so listener annotations can be used.", targetSource.getTargetClass().getName())); + } } for (ListenerMetaData metaData : metaDataValues) { if (MethodInvokerUtils.getMethodInvokerByAnnotation(metaData.getAnnotation(), target) != null) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ChunkListenerSupport.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ChunkListenerSupport.java index 65d00cc1f2..555340a9e1 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ChunkListenerSupport.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ChunkListenerSupport.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeChunkListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeChunkListener.java index f6982aa4c2..66e882cc2d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeChunkListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeChunkListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -28,12 +28,12 @@ */ public class CompositeChunkListener implements ChunkListener { - private OrderedComposite listeners = new OrderedComposite(); + private OrderedComposite listeners = new OrderedComposite<>(); /** * Public setter for the listeners. * - * @param listeners + * @param listeners list of {@link ChunkListener}. */ public void setListeners(List listeners) { this.listeners.setItems(listeners); @@ -42,42 +42,47 @@ public void setListeners(List listeners) { /** * Register additional listener. * - * @param chunkListener + * @param chunkListener instance of {@link ChunkListener}. */ public void register(ChunkListener chunkListener) { listeners.add(chunkListener); } /** - * Call the registered listeners in order, respecting and prioritizing those - * that implement {@link Ordered}. + * Call the registered listeners in reverse order. * * @see org.springframework.batch.core.ChunkListener#afterChunk(ChunkContext context) */ @Override public void afterChunk(ChunkContext context) { - for (Iterator iterator = listeners.iterator(); iterator.hasNext();) { + for (Iterator iterator = listeners.reverse(); iterator.hasNext();) { ChunkListener listener = iterator.next(); listener.afterChunk(context); } } /** - * Call the registered listeners in reverse order. + * Call the registered listeners in order, respecting and prioritizing those + * that implement {@link Ordered}. * * @see org.springframework.batch.core.ChunkListener#beforeChunk(ChunkContext context) */ @Override public void beforeChunk(ChunkContext context) { - for (Iterator iterator = listeners.reverse(); iterator.hasNext();) { + for (Iterator iterator = listeners.iterator(); iterator.hasNext();) { ChunkListener listener = iterator.next(); listener.beforeChunk(context); } } + /** + * Call the registered listeners in reverse order. + * + * @see org.springframework.batch.core.ChunkListener#afterChunkError(ChunkContext context) + */ @Override public void afterChunkError(ChunkContext context) { - for (Iterator iterator = listeners.iterator(); iterator.hasNext();) { + for (Iterator iterator = listeners.reverse(); iterator.hasNext();) { ChunkListener listener = iterator.next(); listener.afterChunkError(context); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeItemProcessListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeItemProcessListener.java index 8a85ac1fb7..944972fa6d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeItemProcessListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeItemProcessListener.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,31 +20,33 @@ import org.springframework.batch.core.ItemProcessListener; import org.springframework.core.Ordered; +import org.springframework.lang.Nullable; /** * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public class CompositeItemProcessListener implements ItemProcessListener { - private OrderedComposite> listeners = new OrderedComposite>(); + private OrderedComposite> listeners = new OrderedComposite<>(); /** * Public setter for the listeners. * - * @param itemReadListeners + * @param itemProcessorListeners list of {@link ItemProcessListener}s to be called when process events occur. */ - public void setListeners(List> itemReadListeners) { - this.listeners.setItems(itemReadListeners); + public void setListeners(List> itemProcessorListeners) { + this.listeners.setItems(itemProcessorListeners); } /** * Register additional listener. * - * @param itemReaderListener + * @param itemProcessorListener instance of {@link ItemProcessListener} to be registered. */ - public void register(ItemProcessListener itemReaderListener) { - listeners.add(itemReaderListener); + public void register(ItemProcessListener itemProcessorListener) { + listeners.add(itemProcessorListener); } /** @@ -54,7 +56,7 @@ public void register(ItemProcessListener itemReaderListene * java.lang.Object) */ @Override - public void afterProcess(T item, S result) { + public void afterProcess(T item, @Nullable S result) { for (Iterator> iterator = listeners.reverse(); iterator.hasNext();) { ItemProcessListener listener = iterator.next(); listener.afterProcess(item, result); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeItemReadListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeItemReadListener.java index 18c782bb78..616cf6c05d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeItemReadListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeItemReadListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -28,12 +28,12 @@ */ public class CompositeItemReadListener implements ItemReadListener { - private OrderedComposite> listeners = new OrderedComposite>(); + private OrderedComposite> listeners = new OrderedComposite<>(); /** * Public setter for the listeners. * - * @param itemReadListeners + * @param itemReadListeners list of {@link ItemReadListener}s to be called when read events occur. */ public void setListeners(List> itemReadListeners) { this.listeners.setItems(itemReadListeners); @@ -42,7 +42,7 @@ public void setListeners(List> itemReadLis /** * Register additional listener. * - * @param itemReaderListener + * @param itemReaderListener instance of {@link ItemReadListener} to be registered. */ public void register(ItemReadListener itemReaderListener) { listeners.add(itemReaderListener); @@ -81,7 +81,7 @@ public void beforeRead() { */ @Override public void onReadError(Exception ex) { - for (Iterator> iterator = listeners.iterator(); iterator.hasNext();) { + for (Iterator> iterator = listeners.reverse(); iterator.hasNext();) { ItemReadListener listener = iterator.next(); listener.onReadError(ex); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeItemWriteListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeItemWriteListener.java index d7ecb635e1..67dd84fea6 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeItemWriteListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeItemWriteListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -28,12 +28,12 @@ */ public class CompositeItemWriteListener implements ItemWriteListener { - private OrderedComposite> listeners = new OrderedComposite>(); + private OrderedComposite> listeners = new OrderedComposite<>(); /** * Public setter for the listeners. * - * @param itemWriteListeners + * @param itemWriteListeners list of {@link ItemWriteListener}s to be called when write events occur. */ public void setListeners(List> itemWriteListeners) { this.listeners.setItems(itemWriteListeners); @@ -42,7 +42,7 @@ public void setListeners(List> itemWriteL /** * Register additional listener. * - * @param itemWriteListener + * @param itemWriteListener list of {@link ItemWriteListener}s to be registered. */ public void register(ItemWriteListener itemWriteListener) { listeners.add(itemWriteListener); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeJobExecutionListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeJobExecutionListener.java index 699073ec41..0495fd0c02 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeJobExecutionListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeJobExecutionListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -28,12 +28,12 @@ */ public class CompositeJobExecutionListener implements JobExecutionListener { - private OrderedComposite listeners = new OrderedComposite(); + private OrderedComposite listeners = new OrderedComposite<>(); /** * Public setter for the listeners. * - * @param listeners + * @param listeners list of {@link JobExecutionListener}s to be called when job execution events occur. */ public void setListeners(List listeners) { this.listeners.setItems(listeners); @@ -42,7 +42,7 @@ public void setListeners(List listeners) { /** * Register additional listener. * - * @param jobExecutionListener + * @param jobExecutionListener instance {@link JobExecutionListener} to be registered. */ public void register(JobExecutionListener jobExecutionListener) { listeners.add(jobExecutionListener); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeRetryProcessListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeRetryProcessListener.java new file mode 100644 index 0000000000..d5e0debaf2 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeRetryProcessListener.java @@ -0,0 +1,62 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.listener; + +import java.util.Iterator; +import java.util.List; +import javax.batch.api.chunk.listener.RetryProcessListener; + +/** + *

      + * Composite class holding {@link RetryProcessListener}'s. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class CompositeRetryProcessListener implements RetryProcessListener { + private OrderedComposite listeners = new OrderedComposite<>(); + + /** + *

      + * Public setter for the {@link RetryProcessListener}'s. + *

      + * + * @param listeners the {@link RetryProcessListener}'s to set + */ + public void setListeners(List listeners) { + this.listeners.setItems(listeners); + } + + /** + *

      + * Register an additional {@link RetryProcessListener}. + *

      + * + * @param listener the {@link RetryProcessListener} to register + */ + public void register(RetryProcessListener listener) { + listeners.add(listener); + } + + @Override + public void onRetryProcessException(Object item, Exception ex) throws Exception { + for (Iterator iterator = listeners.reverse(); iterator.hasNext();) { + RetryProcessListener listener = iterator.next(); + listener.onRetryProcessException(item, ex); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeRetryReadListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeRetryReadListener.java new file mode 100644 index 0000000000..d26564bb3d --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeRetryReadListener.java @@ -0,0 +1,62 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.listener; + +import java.util.Iterator; +import java.util.List; +import javax.batch.api.chunk.listener.RetryReadListener; + +/** + *

      + * Composite class holding {@link RetryReadListener}'s. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class CompositeRetryReadListener implements RetryReadListener { + private OrderedComposite listeners = new OrderedComposite<>(); + + /** + *

      + * Public setter for the {@link RetryReadListener}'s. + *

      + * + * @param listeners the {@link RetryReadListener}'s to set + */ + public void setListeners(List listeners) { + this.listeners.setItems(listeners); + } + + /** + *

      + * Register an additional {@link RetryReadListener}. + *

      + * + * @param listener the {@link RetryReadListener} to register + */ + public void register(RetryReadListener listener) { + listeners.add(listener); + } + + @Override + public void onRetryReadException(Exception ex) throws Exception { + for (Iterator iterator = listeners.reverse(); iterator.hasNext();) { + RetryReadListener listener = iterator.next(); + listener.onRetryReadException(ex); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeRetryWriteListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeRetryWriteListener.java new file mode 100644 index 0000000000..70f013fa60 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeRetryWriteListener.java @@ -0,0 +1,62 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.listener; + +import java.util.Iterator; +import java.util.List; +import javax.batch.api.chunk.listener.RetryWriteListener; + +/** + *

      + * Composite class holding {@link RetryWriteListener}'s. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class CompositeRetryWriteListener implements RetryWriteListener { + private OrderedComposite listeners = new OrderedComposite<>(); + + /** + *

      + * Public setter for the {@link RetryWriteListener}'s. + *

      + * + * @param listeners the {@link RetryWriteListener}'s to set + */ + public void setListeners(List listeners) { + this.listeners.setItems(listeners); + } + + /** + *

      + * Register an additional {@link RetryWriteListener}. + *

      + * + * @param listener the {@link RetryWriteListener} to register + */ + public void register(RetryWriteListener listener) { + listeners.add(listener); + } + + @Override + public void onRetryWriteException(List items, Exception ex) throws Exception { + for (Iterator iterator = listeners.reverse(); iterator.hasNext();) { + RetryWriteListener listener = iterator.next(); + listener.onRetryWriteException(items, ex); + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeSkipListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeSkipListener.java index 2e39c4e6fd..488d057dad 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeSkipListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeSkipListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,12 +27,12 @@ */ public class CompositeSkipListener implements SkipListener { - private OrderedComposite> listeners = new OrderedComposite>(); + private OrderedComposite> listeners = new OrderedComposite<>(); /** * Public setter for the listeners. * - * @param listeners + * @param listeners list of {@link SkipListener}s to be called when skip events occur. */ public void setListeners(List> listeners) { this.listeners.setItems(listeners); @@ -41,7 +41,7 @@ public void setListeners(List> liste /** * Register additional listener. * - * @param listener + * @param listener instance of {@link SkipListener} to be registered. */ public void register(SkipListener listener) { listeners.add(listener); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeStepExecutionListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeStepExecutionListener.java index b08cb6302f..64fc10717b 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeStepExecutionListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/CompositeStepExecutionListener.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.StepExecutionListener; import org.springframework.core.Ordered; +import org.springframework.lang.Nullable; /** * @author Lucas Ward @@ -30,12 +31,12 @@ */ public class CompositeStepExecutionListener implements StepExecutionListener { - private OrderedComposite list = new OrderedComposite(); + private OrderedComposite list = new OrderedComposite<>(); /** * Public setter for the listeners. * - * @param listeners + * @param listeners list of {@link StepExecutionListener}s to be called when step execution events occur. */ public void setListeners(StepExecutionListener[] listeners) { list.setItems(Arrays.asList(listeners)); @@ -44,7 +45,7 @@ public void setListeners(StepExecutionListener[] listeners) { /** * Register additional listener. * - * @param stepExecutionListener + * @param stepExecutionListener instance of {@link StepExecutionListener} to be registered. */ public void register(StepExecutionListener stepExecutionListener) { list.add(stepExecutionListener); @@ -55,6 +56,7 @@ public void register(StepExecutionListener stepExecutionListener) { * prioritizing those that implement {@link Ordered}. * @see org.springframework.batch.core.StepExecutionListener#afterStep(StepExecution) */ + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { for (Iterator iterator = list.reverse(); iterator.hasNext();) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ExecutionContextPromotionListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ExecutionContextPromotionListener.java index 0f55912ae3..da6af4529f 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ExecutionContextPromotionListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ExecutionContextPromotionListener.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.support.PatternMatcher; import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -46,6 +47,7 @@ public class ExecutionContextPromotionListener extends StepExecutionListenerSupp private boolean strict = false; + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { ExecutionContext stepContext = stepExecution.getExecutionContext(); @@ -99,7 +101,7 @@ public void setStatuses(String[] statuses) { * If set to TRUE, the listener will throw an exception if any 'key' is not * found in the Step {@link ExecutionContext}. FALSE by default. * - * @param strict + * @param strict boolean the value of the flag. */ public void setStrict(boolean strict) { this.strict = strict; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ItemListenerSupport.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ItemListenerSupport.java index 4b305a5b59..556e224be1 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ItemListenerSupport.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ItemListenerSupport.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ import org.springframework.batch.core.ItemProcessListener; import org.springframework.batch.core.ItemReadListener; import org.springframework.batch.core.ItemWriteListener; +import org.springframework.lang.Nullable; /** * Basic no-op implementation of the {@link ItemReadListener}, @@ -28,6 +29,7 @@ * at once. * * @author Lucas Ward + * @author Mahmoud Ben Hassine * */ public class ItemListenerSupport implements ItemReadListener, ItemProcessListener, ItemWriteListener { @@ -66,7 +68,7 @@ public void onReadError(Exception ex) { * java.lang.Object) */ @Override - public void afterProcess(I item, O result) { + public void afterProcess(I item, @Nullable O result) { } /* diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobExecutionListenerSupport.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobExecutionListenerSupport.java index 94ea489842..e6e8dd1fa5 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobExecutionListenerSupport.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobExecutionListenerSupport.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobListenerFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobListenerFactoryBean.java index bb39640fcd..bea210d1a9 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobListenerFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobListenerFactoryBean.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,7 +27,7 @@ * @see AbstractListenerFactoryBean * @see JobListenerMetaData */ -public class JobListenerFactoryBean extends AbstractListenerFactoryBean { +public class JobListenerFactoryBean extends AbstractListenerFactoryBean { @Override protected ListenerMetaData getMetaDataFromPropertyName(String propertyName) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobListenerMetaData.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobListenerMetaData.java index 14d0bd7373..bd71b9de71 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobListenerMetaData.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobListenerMetaData.java @@ -1,11 +1,11 @@ /* - * Copyright 2002-2013 the original author or authors. + * Copyright 2002-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,12 +23,14 @@ import org.springframework.batch.core.JobExecutionListener; import org.springframework.batch.core.annotation.AfterJob; import org.springframework.batch.core.annotation.BeforeJob; +import org.springframework.lang.Nullable; /** * Enumeration for {@link JobExecutionListener} meta data, which ties together the names * of methods, their interfaces, annotation, and expected arguments. * * @author Lucas Ward + * @author Mahmoud Ben Hassine * @since 2.0 * @see JobListenerFactoryBean */ @@ -50,7 +52,7 @@ public enum JobListenerMetaData implements ListenerMetaData { } static{ - propertyMap = new HashMap(); + propertyMap = new HashMap<>(); for(JobListenerMetaData metaData : values()){ propertyMap.put(metaData.getPropertyName(), metaData); } @@ -84,9 +86,10 @@ public Class[] getParamTypes() { /** * Return the relevant meta data for the provided property name. * - * @param propertyName - * @return meta data with supplied property name, null if none exists. + * @param propertyName name of the property to retrieve. + * @return meta data with supplied property name, {@code null} if none exists. */ + @Nullable public static JobListenerMetaData fromPropertyName(String propertyName){ return propertyMap.get(propertyName); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobParameterExecutionContextCopyListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobParameterExecutionContextCopyListener.java index a0794d38a6..4a7bbf9187 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobParameterExecutionContextCopyListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/JobParameterExecutionContextCopyListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ListenerMetaData.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ListenerMetaData.java index 73e80600a7..56f14b1b31 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ListenerMetaData.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/ListenerMetaData.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/MethodInvokerMethodInterceptor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/MethodInvokerMethodInterceptor.java index ef26737306..4f49189742 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/MethodInvokerMethodInterceptor.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/MethodInvokerMethodInterceptor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/MulticasterBatchListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/MulticasterBatchListener.java index c9dde84327..4866d517ab 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/MulticasterBatchListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/MulticasterBatchListener.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,8 +15,13 @@ */ package org.springframework.batch.core.listener; +import java.lang.reflect.InvocationTargetException; import java.util.List; +import javax.batch.api.chunk.listener.RetryProcessListener; +import javax.batch.api.chunk.listener.RetryReadListener; +import javax.batch.api.chunk.listener.RetryWriteListener; +import javax.batch.operations.BatchRuntimeException; import org.springframework.batch.core.ChunkListener; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.ItemProcessListener; @@ -28,28 +33,37 @@ import org.springframework.batch.core.StepListener; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.item.ItemStream; +import org.springframework.lang.Nullable; /** * @author Dave Syer * @author Michael Minella + * @author Chris Schaefer + * @author Mahmoud Ben Hassine */ public class MulticasterBatchListener implements StepExecutionListener, ChunkListener, ItemReadListener, -ItemProcessListener, ItemWriteListener, SkipListener { +ItemProcessListener, ItemWriteListener, SkipListener, RetryReadListener, RetryProcessListener, RetryWriteListener { private CompositeStepExecutionListener stepListener = new CompositeStepExecutionListener(); private CompositeChunkListener chunkListener = new CompositeChunkListener(); - private CompositeItemReadListener itemReadListener = new CompositeItemReadListener(); + private CompositeItemReadListener itemReadListener = new CompositeItemReadListener<>(); - private CompositeItemProcessListener itemProcessListener = new CompositeItemProcessListener(); + private CompositeItemProcessListener itemProcessListener = new CompositeItemProcessListener<>(); - private CompositeItemWriteListener itemWriteListener = new CompositeItemWriteListener(); + private CompositeItemWriteListener itemWriteListener = new CompositeItemWriteListener<>(); - private CompositeSkipListener skipListener = new CompositeSkipListener(); + private CompositeSkipListener skipListener = new CompositeSkipListener<>(); + + private CompositeRetryReadListener retryReadListener = new CompositeRetryReadListener(); + + private CompositeRetryProcessListener retryProcessListener = new CompositeRetryProcessListener(); + + private CompositeRetryWriteListener retryWriteListener = new CompositeRetryWriteListener(); /** - * Initialise the listener instance. + * Initialize the listener instance. */ public MulticasterBatchListener() { super(); @@ -71,6 +85,8 @@ public void setListeners(List listeners) { * Register the listener for callbacks on the appropriate interfaces * implemented. Any {@link StepListener} can be provided, or an * {@link ItemStream}. Other types will be ignored. + * + * @param listener the {@link StepListener} instance to be registered. */ public void register(StepListener listener) { if (listener instanceof StepExecutionListener) { @@ -99,26 +115,32 @@ public void register(StepListener listener) { SkipListener skipListener = (SkipListener) listener; this.skipListener.register(skipListener); } + if(listener instanceof RetryReadListener) { + this.retryReadListener.register((RetryReadListener) listener); + } + if(listener instanceof RetryProcessListener) { + this.retryProcessListener.register((RetryProcessListener) listener); + } + if(listener instanceof RetryWriteListener) { + this.retryWriteListener.register((RetryWriteListener) listener); + } } /** - * @param item - * @param result * @see org.springframework.batch.core.listener.CompositeItemProcessListener#afterProcess(java.lang.Object, * java.lang.Object) */ @Override - public void afterProcess(T item, S result) { + public void afterProcess(T item, @Nullable S result) { try { itemProcessListener.afterProcess(item, result); } catch (RuntimeException e) { - throw new StepListenerFailedException("Error in afterProcess.", e); + throw new StepListenerFailedException("Error in afterProcess.", getTargetException(e)); } } /** - * @param item * @see org.springframework.batch.core.listener.CompositeItemProcessListener#beforeProcess(java.lang.Object) */ @Override @@ -127,13 +149,11 @@ public void beforeProcess(T item) { itemProcessListener.beforeProcess(item); } catch (RuntimeException e) { - throw new StepListenerFailedException("Error in beforeProcess.", e); + throw new StepListenerFailedException("Error in beforeProcess.", getTargetException(e)); } } /** - * @param item - * @param ex * @see org.springframework.batch.core.listener.CompositeItemProcessListener#onProcessError(java.lang.Object, * java.lang.Exception) */ @@ -150,6 +170,7 @@ public void onProcessError(T item, Exception ex) { /** * @see org.springframework.batch.core.listener.CompositeStepExecutionListener#afterStep(StepExecution) */ + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { try { @@ -161,7 +182,6 @@ public ExitStatus afterStep(StepExecution stepExecution) { } /** - * @param stepExecution * @see org.springframework.batch.core.listener.CompositeStepExecutionListener#beforeStep(org.springframework.batch.core.StepExecution) */ @Override @@ -175,7 +195,6 @@ public void beforeStep(StepExecution stepExecution) { } /** - * * @see org.springframework.batch.core.listener.CompositeChunkListener#afterChunk(ChunkContext context) */ @Override @@ -184,12 +203,11 @@ public void afterChunk(ChunkContext context) { chunkListener.afterChunk(context); } catch (RuntimeException e) { - throw new StepListenerFailedException("Error in afterChunk.", e); + throw new StepListenerFailedException("Error in afterChunk.", getTargetException(e)); } } /** - * * @see org.springframework.batch.core.listener.CompositeChunkListener#beforeChunk(ChunkContext context) */ @Override @@ -198,12 +216,11 @@ public void beforeChunk(ChunkContext context) { chunkListener.beforeChunk(context); } catch (RuntimeException e) { - throw new StepListenerFailedException("Error in beforeChunk.", e); + throw new StepListenerFailedException("Error in beforeChunk.", getTargetException(e)); } } /** - * @param item * @see org.springframework.batch.core.listener.CompositeItemReadListener#afterRead(java.lang.Object) */ @Override @@ -212,12 +229,11 @@ public void afterRead(T item) { itemReadListener.afterRead(item); } catch (RuntimeException e) { - throw new StepListenerFailedException("Error in afterRead.", e); + throw new StepListenerFailedException("Error in afterRead.", getTargetException(e)); } } /** - * * @see org.springframework.batch.core.listener.CompositeItemReadListener#beforeRead() */ @Override @@ -226,12 +242,11 @@ public void beforeRead() { itemReadListener.beforeRead(); } catch (RuntimeException e) { - throw new StepListenerFailedException("Error in beforeRead.", e); + throw new StepListenerFailedException("Error in beforeRead.", getTargetException(e)); } } /** - * @param ex * @see org.springframework.batch.core.listener.CompositeItemReadListener#onReadError(java.lang.Exception) */ @Override @@ -245,7 +260,6 @@ public void onReadError(Exception ex) { } /** - * * @see ItemWriteListener#afterWrite(List) */ @Override @@ -254,12 +268,11 @@ public void afterWrite(List items) { itemWriteListener.afterWrite(items); } catch (RuntimeException e) { - throw new StepListenerFailedException("Error in afterWrite.", e); + throw new StepListenerFailedException("Error in afterWrite.", getTargetException(e)); } } /** - * @param items * @see ItemWriteListener#beforeWrite(List) */ @Override @@ -268,13 +281,11 @@ public void beforeWrite(List items) { itemWriteListener.beforeWrite(items); } catch (RuntimeException e) { - throw new StepListenerFailedException("Error in beforeWrite.", e); + throw new StepListenerFailedException("Error in beforeWrite.", getTargetException(e)); } } /** - * @param ex - * @param items * @see ItemWriteListener#onWriteError(Exception, List) */ @Override @@ -288,7 +299,6 @@ public void onWriteError(Exception ex, List items) { } /** - * @param t * @see org.springframework.batch.core.listener.CompositeSkipListener#onSkipInRead(java.lang.Throwable) */ @Override @@ -297,8 +307,6 @@ public void onSkipInRead(Throwable t) { } /** - * @param item - * @param t * @see org.springframework.batch.core.listener.CompositeSkipListener#onSkipInWrite(java.lang.Object, * java.lang.Throwable) */ @@ -308,8 +316,6 @@ public void onSkipInWrite(S item, Throwable t) { } /** - * @param item - * @param t * @see org.springframework.batch.core.listener.CompositeSkipListener#onSkipInProcess(Object, * Throwable) */ @@ -327,4 +333,44 @@ public void afterChunkError(ChunkContext context) { throw new StepListenerFailedException("Error in afterFailedChunk.", e); } } + + @Override + public void onRetryReadException(Exception ex) throws Exception { + try { + retryReadListener.onRetryReadException(ex); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + @Override + public void onRetryProcessException(Object item, Exception ex) throws Exception { + try { + retryProcessListener.onRetryProcessException(item, ex); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + @Override + public void onRetryWriteException(List items, Exception ex) throws Exception { + try { + retryWriteListener.onRetryWriteException(items, ex); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + /** + * Unwrap the target exception from a wrapped {@link InvocationTargetException}. + * @param e the exception to introspect + * @return the target exception if any + */ + private Throwable getTargetException(RuntimeException e) { + Throwable cause = e.getCause(); + if (cause != null && cause instanceof InvocationTargetException) { + return ((InvocationTargetException) cause).getTargetException(); + } + return e; + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/OrderedComposite.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/OrderedComposite.java index 19377dcd17..f8cf88e5db 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/OrderedComposite.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/OrderedComposite.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,14 +32,13 @@ */ class OrderedComposite { - private List unordered = new ArrayList(); + private List unordered = new ArrayList<>(); - private List ordered = new ArrayList(); + private List ordered = new ArrayList<>(); - @SuppressWarnings("unchecked") private Comparator comparator = new AnnotationAwareOrderComparator(); - private List list = new ArrayList(); + private List list = new ArrayList<>(); /** * Public setter for the listeners. @@ -85,7 +84,7 @@ else if (!unordered.contains(item)) { * @return an iterator over the list of items */ public Iterator iterator() { - return new ArrayList(list).iterator(); + return new ArrayList<>(list).iterator(); } /** @@ -94,7 +93,7 @@ public Iterator iterator() { * @return an iterator over the list of items */ public Iterator reverse() { - ArrayList result = new ArrayList(list); + ArrayList result = new ArrayList<>(list); Collections.reverse(result); return result.iterator(); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/SkipListenerSupport.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/SkipListenerSupport.java index 4afc57d9d2..69d54856d6 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/SkipListenerSupport.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/SkipListenerSupport.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepExecutionListenerSupport.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepExecutionListenerSupport.java index d87b01c129..49bd5a282d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepExecutionListenerSupport.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepExecutionListenerSupport.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,6 +18,7 @@ import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.StepExecutionListener; +import org.springframework.lang.Nullable; /** * @author Dave Syer @@ -28,6 +29,7 @@ public class StepExecutionListenerSupport implements StepExecutionListener { /* (non-Javadoc) * @see org.springframework.batch.core.domain.StepListener#afterStep(StepExecution stepExecution) */ + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { return null; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerFactoryBean.java index e77ef8a602..bd96859460 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerFactoryBean.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,7 +27,7 @@ * @see AbstractListenerFactoryBean * @see StepListenerMetaData */ -public class StepListenerFactoryBean extends AbstractListenerFactoryBean { +public class StepListenerFactoryBean extends AbstractListenerFactoryBean { @Override protected ListenerMetaData getMetaDataFromPropertyName(String propertyName) { @@ -45,8 +45,7 @@ protected Class getDefaultListenerClass() { } @Override - @SuppressWarnings("rawtypes") - public Class getObjectType() { + public Class getObjectType() { return StepListener.class; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerFailedException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerFailedException.java index 0bd597f7e9..0691a1c723 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerFailedException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerFailedException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,6 +23,7 @@ * @author Michael Minella * */ +@SuppressWarnings("serial") public class StepListenerFailedException extends RuntimeException { /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerMetaData.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerMetaData.java index 31d46e66ec..1c5472e939 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerMetaData.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerMetaData.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -91,7 +91,7 @@ public enum StepListenerMetaData implements ListenerMetaData { } static{ - propertyMap = new HashMap(); + propertyMap = new HashMap<>(); for(StepListenerMetaData metaData : values()){ propertyMap.put(metaData.getPropertyName(), metaData); } @@ -125,7 +125,7 @@ public String getPropertyName() { /** * Return the relevant meta data for the provided property name. * - * @param propertyName + * @param propertyName property name to retrieve data for. * @return meta data with supplied property name, null if none exists. */ public static StepListenerMetaData fromPropertyName(String propertyName){ diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerSupport.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerSupport.java index 567ce3075f..9482c9b7f3 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerSupport.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/StepListenerSupport.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,12 +27,14 @@ import org.springframework.batch.core.StepExecutionListener; import org.springframework.batch.core.StepListener; import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.lang.Nullable; /** * Basic no-op implementations of all {@link StepListener} interfaces. * * @author Lucas Ward * @author Robert Kasanicky + * @author Mahmoud Ben Hassine */ public class StepListenerSupport implements StepExecutionListener, ChunkListener, ItemReadListener, ItemProcessListener, ItemWriteListener, SkipListener { @@ -40,6 +42,7 @@ public class StepListenerSupport implements StepExecutionListener, ChunkLis /* (non-Javadoc) * @see org.springframework.batch.core.StepExecutionListener#afterStep(org.springframework.batch.core.StepExecution) */ + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { return null; @@ -112,7 +115,7 @@ public void onWriteError(Exception exception, List items) { * @see org.springframework.batch.core.ItemProcessListener#afterProcess(java.lang.Object, java.lang.Object) */ @Override - public void afterProcess(T item, S result) { + public void afterProcess(T item, @Nullable S result) { } /* (non-Javadoc) diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/package-info.java new file mode 100644 index 0000000000..0f23a4ab87 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/package-info.java @@ -0,0 +1,10 @@ +/** + * Generic implementations of core batch listener interfaces. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.listener; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/listener/package.html deleted file mode 100644 index c8f398aa3c..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/listener/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Generic implementations of core batch listener interfaces. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/metrics/BatchMetrics.java b/spring-batch-core/src/main/java/org/springframework/batch/core/metrics/BatchMetrics.java new file mode 100644 index 0000000000..e00eeb8c7f --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/metrics/BatchMetrics.java @@ -0,0 +1,134 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.metrics; + +import java.time.Duration; +import java.util.Arrays; +import java.util.Date; +import java.util.concurrent.TimeUnit; + +import io.micrometer.core.instrument.LongTaskTimer; +import io.micrometer.core.instrument.Metrics; +import io.micrometer.core.instrument.Tag; +import io.micrometer.core.instrument.Timer; + +import org.springframework.lang.Nullable; + +/** + * Central class for batch metrics. It provides: + * + *
        + *
      • the main entry point to interact with Micrometer's {@link Metrics#globalRegistry} + * with common metrics such as {@link Timer} and {@link LongTaskTimer}.
      • + *
      • Some utility methods like calculating durations and formatting them in + * a human readable format.
      • + *
      + * + * Only intended for internal use. + * + * @author Mahmoud Ben Hassine + * @since 4.2 + */ +public final class BatchMetrics { + + private static final String METRICS_PREFIX = "spring.batch."; + + public static final String STATUS_SUCCESS = "SUCCESS"; + + public static final String STATUS_FAILURE = "FAILURE"; + + private BatchMetrics() {} + + /** + * Create a {@link Timer}. + * @param name of the timer. Will be prefixed with {@link BatchMetrics#METRICS_PREFIX}. + * @param description of the timer + * @param tags of the timer + * @return a new timer instance + */ + public static Timer createTimer(String name, String description, Tag... tags) { + return Timer.builder(METRICS_PREFIX + name) + .description(description) + .tags(Arrays.asList(tags)) + .register(Metrics.globalRegistry); + } + + /** + * Create a new {@link Timer.Sample}. + * @return a new timer sample instance + */ + public static Timer.Sample createTimerSample() { + return Timer.start(Metrics.globalRegistry); + } + + /** + * Create a new {@link LongTaskTimer}. + * @param name of the long task timer. Will be prefixed with {@link BatchMetrics#METRICS_PREFIX}. + * @param description of the long task timer. + * @param tags of the timer + * @return a new long task timer instance + */ + public static LongTaskTimer createLongTaskTimer(String name, String description, Tag... tags) { + return LongTaskTimer.builder(METRICS_PREFIX + name) + .description(description) + .tags(Arrays.asList(tags)) + .register(Metrics.globalRegistry); + } + + /** + * Calculate the duration between two dates. + * @param startTime the start time + * @param endTime the end time + * @return the duration between start time and end time + */ + @Nullable + public static Duration calculateDuration(@Nullable Date startTime, @Nullable Date endTime) { + if (startTime == null || endTime == null) { + return null; + } + return Duration.between(startTime.toInstant(), endTime.toInstant()); + } + + /** + * Format a duration in a human readable format like: 2h32m15s10ms. + * @param duration to format + * @return A human readable duration + */ + public static String formatDuration(@Nullable Duration duration) { + if (duration == null || duration.isZero() || duration.isNegative()) { + return ""; + } + StringBuilder formattedDuration = new StringBuilder(); + long hours = duration.toHours(); + long minutes = duration.toMinutes(); + long seconds = duration.getSeconds(); + long millis = duration.toMillis(); + if (hours != 0) { + formattedDuration.append(hours).append("h"); + } + if (minutes != 0) { + formattedDuration.append(minutes - TimeUnit.HOURS.toMinutes(hours)).append("m"); + } + if (seconds != 0) { + formattedDuration.append(seconds - TimeUnit.MINUTES.toSeconds(minutes)).append("s"); + } + if (millis != 0) { + formattedDuration.append(millis - TimeUnit.SECONDS.toMillis(seconds)).append("ms"); + } + return formattedDuration.toString(); + } + +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/package-info.java new file mode 100644 index 0000000000..b68e233a73 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/package-info.java @@ -0,0 +1,12 @@ +/** + * Core domain context for Spring Batch covering jobs, steps, configuration and execution abstractions. Most classes + * here are interfaces with implementations saved for specific applications. This is the public API of Spring Batch. + * There is a reference implementation of the core interfaces in the execution module. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/package.html deleted file mode 100644 index 51b136d976..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/package.html +++ /dev/null @@ -1,11 +0,0 @@ - - -

      -Core domain context for Spring Batch covering jobs, steps, -configuration and execution abstractions. Most classes here are -interfaces with implementations saved for specific applications. This -is the public API of Spring Batch. There is a reference -implementation of the core interfaces in the execution module. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/PartitionHandler.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/PartitionHandler.java index bb59e9472c..b25325bbcf 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/PartitionHandler.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/PartitionHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/StepExecutionSplitter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/StepExecutionSplitter.java index da70b143e2..f39db71759 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/StepExecutionSplitter.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/StepExecutionSplitter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,12 +16,12 @@ package org.springframework.batch.core.partition; -import java.util.Set; - import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobExecutionException; import org.springframework.batch.core.StepExecution; +import java.util.Set; + /** * Strategy interface for generating input contexts for a partitioned step * execution independent from the fabric they are going to run on. @@ -44,8 +44,8 @@ public interface StepExecutionSplitter { * executable instances with the same parent {@link JobExecution}. The grid * size will be treated as a hint for the size of the collection to be * returned. It may or may not correspond to the physical size of an - * execution grid.
      - *
      + * execution grid.
      + *
      * * On a restart clients of the {@link StepExecutionSplitter} should expect * it to reconstitute the state of the last failed execution and only return diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/package-info.java new file mode 100644 index 0000000000..c6e4a484b6 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/package-info.java @@ -0,0 +1,10 @@ +/** + * Interfaces for partitioning components. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.partition; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/AbstractPartitionHandler.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/AbstractPartitionHandler.java index e75585efaf..b461e2508c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/AbstractPartitionHandler.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/AbstractPartitionHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/DefaultStepExecutionAggregator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/DefaultStepExecutionAggregator.java index a293d46732..2f9eb3d48b 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/DefaultStepExecutionAggregator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/DefaultStepExecutionAggregator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,13 +16,13 @@ package org.springframework.batch.core.partition.support; -import java.util.Collection; - import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.StepExecution; import org.springframework.util.Assert; +import java.util.Collection; + /** * Convenience class for aggregating a set of {@link StepExecution} instances * into a single result. @@ -53,6 +53,8 @@ public void aggregate(StepExecution result, Collection executions BatchStatus status = stepExecution.getStatus(); result.setStatus(BatchStatus.max(result.getStatus(), status)); result.setExitStatus(result.getExitStatus().and(stepExecution.getExitStatus())); + result.setFilterCount(result.getFilterCount() + stepExecution.getFilterCount()); + result.setProcessSkipCount(result.getProcessSkipCount() + stepExecution.getProcessSkipCount()); result.setCommitCount(result.getCommitCount() + stepExecution.getCommitCount()); result.setRollbackCount(result.getRollbackCount() + stepExecution.getRollbackCount()); result.setReadCount(result.getReadCount() + stepExecution.getReadCount()); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/MultiResourcePartitioner.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/MultiResourcePartitioner.java index 03f3eef097..28086b994c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/MultiResourcePartitioner.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/MultiResourcePartitioner.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -70,7 +70,7 @@ public void setKeyName(String keyName) { */ @Override public Map partition(int gridSize) { - Map map = new HashMap(gridSize); + Map map = new HashMap<>(gridSize); int i = 0; for (Resource resource : resources) { ExecutionContext context = new ExecutionContext(); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/PartitionNameProvider.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/PartitionNameProvider.java index 19571d404d..88d05fb31d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/PartitionNameProvider.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/PartitionNameProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/PartitionStep.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/PartitionStep.java index 6eed9e759c..118a3e4a2a 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/PartitionStep.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/PartitionStep.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,8 +16,6 @@ package org.springframework.batch.core.partition.support; -import java.util.Collection; - import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.JobExecutionException; import org.springframework.batch.core.Step; @@ -28,6 +26,8 @@ import org.springframework.batch.item.ExecutionContext; import org.springframework.util.Assert; +import java.util.Collection; + /** * Implementation of {@link Step} which partitions the execution and spreads the * load using a {@link PartitionHandler}. @@ -100,6 +100,7 @@ public void afterPropertiesSet() throws Exception { */ @Override protected void doExecute(StepExecution stepExecution) throws Exception { + stepExecution.getExecutionContext().put(STEP_TYPE_KEY, this.getClass().getName()); // Wait for task completion and then aggregate the results Collection executions = partitionHandler.handle(stepExecutionSplitter, stepExecution); @@ -110,7 +111,13 @@ protected void doExecute(StepExecution stepExecution) throws Exception { if (stepExecution.getStatus().isUnsuccessful()) { throw new JobExecutionException("Partition handler returned an unsuccessful step"); } + } + protected StepExecutionSplitter getStepExecutionSplitter() { + return stepExecutionSplitter; } + protected PartitionHandler getPartitionHandler() { + return partitionHandler; + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/Partitioner.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/Partitioner.java index a9788b647c..2e711d780e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/Partitioner.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/Partitioner.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/RemoteStepExecutionAggregator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/RemoteStepExecutionAggregator.java index f49ddf4f50..dd8b60d732 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/RemoteStepExecutionAggregator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/RemoteStepExecutionAggregator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -90,7 +90,7 @@ public void aggregate(StepExecution result, Collection executions if (executions == null) { return; } - Collection updates = new ArrayList(); + Collection updates = new ArrayList<>(); for (StepExecution stepExecution : executions) { Long id = stepExecution.getId(); Assert.state(id != null, "StepExecution has null id. It must be saved first: " + stepExecution); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/SimplePartitioner.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/SimplePartitioner.java index 7b9c3848e5..c1de4f8688 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/SimplePartitioner.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/SimplePartitioner.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,7 +36,7 @@ public class SimplePartitioner implements Partitioner { @Override public Map partition(int gridSize) { - Map map = new HashMap(gridSize); + Map map = new HashMap<>(gridSize); for (int i = 0; i < gridSize; i++) { map.put(PARTITION_KEY + i, new ExecutionContext()); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/SimpleStepExecutionSplitter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/SimpleStepExecutionSplitter.java index f46f140fb1..522cfb2fbb 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/SimpleStepExecutionSplitter.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/SimpleStepExecutionSplitter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -46,6 +46,7 @@ * {step1:partition0, step1:partition1, ...}. * * @author Dave Syer + * @author Mahmoud Ben Hassine * @since 2.0 */ public class SimpleStepExecutionSplitter implements StepExecutionSplitter, InitializingBean { @@ -175,7 +176,7 @@ public Set split(StepExecution stepExecution, int gridSize) throw JobExecution jobExecution = stepExecution.getJobExecution(); Map contexts = getContexts(stepExecution, gridSize); - Set set = new HashSet(contexts.size()); + Set set = new HashSet<>(contexts.size()); for (Entry context : contexts.entrySet()) { @@ -184,16 +185,19 @@ public Set split(StepExecution stepExecution, int gridSize) throw StepExecution currentStepExecution = jobExecution.createStepExecution(stepName); - boolean startable = getStartable(currentStepExecution, context.getValue()); + boolean startable = isStartable(currentStepExecution, context.getValue()); if (startable) { set.add(currentStepExecution); } } - + jobRepository.addAll(set); - return set; + Set executions = new HashSet<>(set.size()); + executions.addAll(set); + + return executions; } @@ -215,7 +219,7 @@ private Map getContexts(StepExecution stepExecution, i } else { if (partitioner instanceof PartitionNameProvider) { - result = new HashMap(); + result = new HashMap<>(); Collection names = ((PartitionNameProvider) partitioner).getPartitionNames(splitSize); for (String name : names) { /* @@ -235,7 +239,29 @@ private Map getContexts(StepExecution stepExecution, i return result; } - private boolean getStartable(StepExecution stepExecution, ExecutionContext context) throws JobExecutionException { + /** + * Check if a step execution is startable. + * @param stepExecution the step execution to check + * @param context the execution context of the step + * @return true if the step execution is startable, false otherwise + * @throws JobExecutionException if unable to check if the step execution is startable + */ + protected boolean isStartable(StepExecution stepExecution, ExecutionContext context) throws JobExecutionException { + return getStartable(stepExecution, context); + } + + /** + * Check if a step execution is startable. + * @param stepExecution the step execution to check + * @param context the execution context of the step + * @return true if the step execution is startable, false otherwise + * @throws JobExecutionException if unable to check if the step execution is startable + * @deprecated This method is deprecated in favor of + * {@link SimpleStepExecutionSplitter#isStartable} and will be removed in a + * future version. + */ + @Deprecated + protected boolean getStartable(StepExecution stepExecution, ExecutionContext context) throws JobExecutionException { JobInstance jobInstance = stepExecution.getJobExecution().getJobInstance(); String stepName = stepExecution.getStepName(); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/StepExecutionAggregator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/StepExecutionAggregator.java index 570ce5879f..bcd5e1ab0c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/StepExecutionAggregator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/StepExecutionAggregator.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.partition.support; import java.util.Collection; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/TaskExecutorPartitionHandler.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/TaskExecutorPartitionHandler.java index 083274862e..16400e1e01 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/TaskExecutorPartitionHandler.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/TaskExecutorPartitionHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,7 +29,6 @@ import org.springframework.batch.core.partition.PartitionHandler; import org.springframework.batch.core.step.StepHolder; import org.springframework.beans.factory.InitializingBean; -import org.springframework.beans.factory.annotation.Required; import org.springframework.core.task.SyncTaskExecutor; import org.springframework.core.task.TaskExecutor; import org.springframework.core.task.TaskRejectedException; @@ -40,7 +39,7 @@ * partitioned {@link Step} locally in multiple threads. This can be an * effective approach for scaling batch steps that are IO intensive, like * directory and filesystem scanning and copying. - *

      + *
      * By default, the thread pool is synchronous. * * @author Sebastien Gerard @@ -55,6 +54,7 @@ public class TaskExecutorPartitionHandler extends AbstractPartitionHandler imple @Override public void afterPropertiesSet() throws Exception { + Assert.state(step != null, "A Step must be provided."); } /** @@ -74,7 +74,6 @@ public void setTaskExecutor(TaskExecutor taskExecutor) { * * @param step the {@link Step} instance to use to execute business logic */ - @Required public void setStep(Step step) { this.step = step; } @@ -91,11 +90,11 @@ public Step getStep() { } @Override - protected Set doHandle(StepExecution masterStepExecution, + protected Set doHandle(StepExecution managerStepExecution, Set partitionStepExecutions) throws Exception { Assert.notNull(step, "A Step must be provided."); - final Set> tasks = new HashSet>(getGridSize()); - final Set result = new HashSet(); + final Set> tasks = new HashSet<>(getGridSize()); + final Set result = new HashSet<>(); for (final StepExecution stepExecution : partitionStepExecutions) { final FutureTask task = createTask(step, stepExecution); @@ -133,7 +132,7 @@ protected Set doHandle(StepExecution masterStepExecution, */ protected FutureTask createTask(final Step step, final StepExecution stepExecution) { - return new FutureTask(new Callable() { + return new FutureTask<>(new Callable() { @Override public StepExecution call() throws Exception { step.execute(stepExecution); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/package-info.java new file mode 100644 index 0000000000..2e8601b4fc --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/partition/support/package-info.java @@ -0,0 +1,10 @@ +/** + * Implementation of common partition components. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.partition.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/ExecutionContextSerializer.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/ExecutionContextSerializer.java index c17f806d66..cce7d69d9e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/ExecutionContextSerializer.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/ExecutionContextSerializer.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,6 +15,8 @@ */ package org.springframework.batch.core.repository; +import java.util.Map; + import org.springframework.core.serializer.Deserializer; import org.springframework.core.serializer.Serializer; @@ -29,6 +31,6 @@ * @see Serializer * @see Deserializer */ -public interface ExecutionContextSerializer extends Serializer, Deserializer { +public interface ExecutionContextSerializer extends Serializer>, Deserializer> { } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobExecutionAlreadyRunningException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobExecutionAlreadyRunningException.java index 48390971d1..ab1b6f091a 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobExecutionAlreadyRunningException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobExecutionAlreadyRunningException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,18 +21,19 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class JobExecutionAlreadyRunningException extends JobExecutionException { /** - * @param msg + * @param msg the exception message. */ public JobExecutionAlreadyRunningException(String msg) { super(msg); } /** - * @param msg - * @param cause + * @param msg the exception message. + * @param cause the cause of the exception. */ public JobExecutionAlreadyRunningException(String msg, Throwable cause) { super(msg, cause); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobInstanceAlreadyCompleteException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobInstanceAlreadyCompleteException.java index d921329877..b60a15a19c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobInstanceAlreadyCompleteException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobInstanceAlreadyCompleteException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,6 +24,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class JobInstanceAlreadyCompleteException extends JobExecutionException { /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobRepository.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobRepository.java index 688c7c8c51..a65a7a51b9 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobRepository.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobRepository.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,8 +16,6 @@ package org.springframework.batch.core.repository; -import java.util.Collection; - import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobInstance; @@ -27,28 +25,33 @@ import org.springframework.batch.core.repository.dao.JobExecutionDao; import org.springframework.batch.core.repository.dao.JobInstanceDao; import org.springframework.batch.item.ExecutionContext; +import org.springframework.lang.Nullable; import org.springframework.transaction.annotation.Isolation; +import java.util.Collection; + /** *

      * Repository responsible for persistence of batch meta-data entities. *

      - * + * * @see JobInstance * @see JobExecution * @see StepExecution - * + * * @author Lucas Ward * @author Dave Syer * @author Robert Kasanicky * @author David Turanski + * @author Michael Minella + * @author Mahmoud Ben Hassine */ public interface JobRepository { /** * Check if an instance of this job already exists with the parameters * provided. - * + * * @param jobName the name of the job * @param jobParameters the parameters to match * @return true if a {@link JobInstance} already exists for this job name @@ -56,6 +59,27 @@ public interface JobRepository { */ boolean isJobInstanceExists(String jobName, JobParameters jobParameters); + /** + * Create a new {@link JobInstance} with the name and job parameters provided. + * + * @param jobName logical name of the job + * @param jobParameters parameters used to execute the job + * @return the new {@link JobInstance} + */ + JobInstance createJobInstance(String jobName, JobParameters jobParameters); + + /** + * Create a new {@link JobExecution} based upon the {@link JobInstance} it's associated + * with, the {@link JobParameters} used to execute it with and the location of the configuration + * file that defines the job. + * + * @param jobInstance {@link JobInstance} instance to initialize the new JobExecution. + * @param jobParameters {@link JobParameters} instance to initialize the new JobExecution. + * @param jobConfigurationLocation {@link String} instance to initialize the new JobExecution. + * @return the new {@link JobExecution}. + */ + JobExecution createJobExecution(JobInstance jobInstance, JobParameters jobParameters, String jobConfigurationLocation); + /** *

      * Create a {@link JobExecution} for a given {@link Job} and @@ -64,7 +88,7 @@ public interface JobRepository { * completed. If matching {@link JobInstance} does not exist yet it will be * created. *

      - * + * *

      * If this method is run in a transaction (as it normally would be) with * isolation level at {@link Isolation#REPEATABLE_READ} or better, then this @@ -77,12 +101,13 @@ public interface JobRepository { * (e.g. if using a non-relational data-store, or if the platform does not * support the higher isolation levels). *

      - * - * @param jobName the name of the job that is to be executed

      - * + * + * @param jobName the name of the job that is to be executed + * * @param jobParameters the runtime parameters for the job - * + * * @return a valid {@link JobExecution} for the arguments provided + * * @throws JobExecutionAlreadyRunningException if there is a * {@link JobExecution} already running for the job instance with the * provided job and parameters. @@ -91,18 +116,18 @@ public interface JobRepository { * false. * @throws JobInstanceAlreadyCompleteException if a {@link JobInstance} is * found and was already completed successfully. - * + * */ JobExecution createJobExecution(String jobName, JobParameters jobParameters) throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException; /** * Update the {@link JobExecution} (but not its {@link ExecutionContext}). - * + * * Preconditions: {@link JobExecution} must contain a valid * {@link JobInstance} and be saved (have an id assigned). - * - * @param jobExecution + * + * @param jobExecution {@link JobExecution} instance to be updated in the repo. */ void update(JobExecution jobExecution); @@ -111,10 +136,10 @@ JobExecution createJobExecution(String jobName, JobParameters jobParameters) * be assigned - it is not permitted that an ID be assigned before calling * this method. Instead, it should be left blank, to be assigned by a * {@link JobRepository}. - * + * * Preconditions: {@link StepExecution} must have a valid {@link Step}. - * - * @param stepExecution + * + * @param stepExecution {@link StepExecution} instance to be added to the repo. */ void add(StepExecution stepExecution); @@ -122,44 +147,47 @@ JobExecution createJobExecution(String jobName, JobParameters jobParameters) * Save a collection of {@link StepExecution}s and each {@link ExecutionContext}. The * StepExecution ID will be assigned - it is not permitted that an ID be assigned before calling * this method. Instead, it should be left blank, to be assigned by {@link JobRepository}. - * + * * Preconditions: {@link StepExecution} must have a valid {@link Step}. - * - * @param stepExecution + * + * @param stepExecutions collection of {@link StepExecution} instances to be added to the repo. */ void addAll(Collection stepExecutions); /** * Update the {@link StepExecution} (but not its {@link ExecutionContext}). - * + * * Preconditions: {@link StepExecution} must be saved (have an id assigned). - * - * @param stepExecution + * + * @param stepExecution {@link StepExecution} instance to be updated in the repo. */ void update(StepExecution stepExecution); /** * Persist the updated {@link ExecutionContext}s of the given * {@link StepExecution}. - * - * @param stepExecution + * + * @param stepExecution {@link StepExecution} instance to be used to update the context. */ void updateExecutionContext(StepExecution stepExecution); /** * Persist the updated {@link ExecutionContext} of the given * {@link JobExecution}. - * @param jobExecution + * @param jobExecution {@link JobExecution} instance to be used to update the context. */ void updateExecutionContext(JobExecution jobExecution); /** + * @param jobInstance {@link JobInstance} instance containing the step executions. * @param stepName the name of the step execution that might have run. * @return the last execution of step for the given job instance. */ + @Nullable StepExecution getLastStepExecution(JobInstance jobInstance, String stepName); /** + * @param jobInstance {@link JobInstance} instance containing the step executions. * @param stepName the name of the step execution that might have run. * @return the execution count of the step within the given job instance. */ @@ -170,6 +198,7 @@ JobExecution createJobExecution(String jobName, JobParameters jobParameters) * @param jobParameters parameters identifying the {@link JobInstance} * @return the last execution of job if exists, null otherwise */ + @Nullable JobExecution getLastJobExecution(String jobName, JobParameters jobParameters); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobRestartException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobRestartException.java index 778367c394..07b7edbb9c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobRestartException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/JobRestartException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,6 +23,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class JobRestartException extends JobExecutionException { /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/AbstractJdbcBatchMetadataDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/AbstractJdbcBatchMetadataDao.java index a6e3936865..be76f7e197 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/AbstractJdbcBatchMetadataDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/AbstractJdbcBatchMetadataDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -81,7 +81,7 @@ public void setClobTypeToUse(int clobTypeToUse) { @Override public void afterPropertiesSet() throws Exception { - Assert.notNull(jdbcTemplate); + Assert.notNull(jdbcTemplate, "JdbcOperations is required"); } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/DefaultExecutionContextSerializer.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/DefaultExecutionContextSerializer.java index 93323eb8aa..dcec66fc21 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/DefaultExecutionContextSerializer.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/DefaultExecutionContextSerializer.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,6 +18,8 @@ import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; +import java.io.Serializable; +import java.util.Map; import org.springframework.batch.core.repository.ExecutionContextSerializer; import org.springframework.core.serializer.DefaultDeserializer; @@ -44,27 +46,38 @@ public class DefaultExecutionContextSerializer implements ExecutionContextSerial * Serializes an execution context to the provided {@link OutputStream}. The * stream is not closed prior to it's return. * - * @param context - * @param out + * @param context {@link Map} contents of the {@code ExecutionContext}. + * @param out {@link OutputStream} where the serialized context information + * will be written. */ @Override @SuppressWarnings("unchecked") - public void serialize(Object context, OutputStream out) throws IOException { - Assert.notNull(context); - Assert.notNull(out); + public void serialize(Map context, OutputStream out) throws IOException { + Assert.notNull(context, "context is required"); + Assert.notNull(out, "OutputStream is required"); + for(Object value : context.values()) { + Assert.notNull(value, "A null value was found"); + if (!(value instanceof Serializable)) { + throw new IllegalArgumentException( + "Value: [ " + value + "must be serializable." + + "Object of class [" + value.getClass().getName() + + "] must be an instance of " + Serializable.class); + } + } serializer.serialize(context, out); } /** * Deserializes an execution context from the provided {@link InputStream}. * - * @param inputStream + * @param inputStream {@link InputStream} containing the information to be deserialized. * @return the object serialized in the provided {@link InputStream} */ + @SuppressWarnings("unchecked") @Override - public Object deserialize(InputStream inputStream) throws IOException { - return deserializer.deserialize(inputStream); + public Map deserialize(InputStream inputStream) throws IOException { + return (Map) deserializer.deserialize(inputStream); } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/ExecutionContextDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/ExecutionContextDao.java index 601777120f..a6e236bc8a 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/ExecutionContextDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/ExecutionContextDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,13 +31,13 @@ public interface ExecutionContextDao { /** - * @param jobExecution + * @param jobExecution {@link JobExecution} instance that contains the context. * @return execution context associated with the given jobExecution */ ExecutionContext getExecutionContext(JobExecution jobExecution); /** - * @param stepExecution + * @param stepExecution {@link StepExecution} instance that contains the context. * @return execution context associated with the given stepExecution */ ExecutionContext getExecutionContext(StepExecution stepExecution); @@ -45,35 +45,41 @@ public interface ExecutionContextDao { /** * Persist the execution context associated with the given jobExecution, * persistent entry for the context should not exist yet. - * @param jobExecution + * + * @param jobExecution {@link JobExecution} instance that contains the context. */ void saveExecutionContext(final JobExecution jobExecution); /** * Persist the execution context associated with the given stepExecution, * persistent entry for the context should not exist yet. - * @param stepExecution + * + * @param stepExecution {@link StepExecution} instance that contains the context. */ void saveExecutionContext(final StepExecution stepExecution); /** * Persist the execution context associated with each stepExecution in a given collection, * persistent entry for the context should not exist yet. - * @param stepExecution + * + * @param stepExecutions a collection of {@link StepExecution}s that contain + * the contexts. */ void saveExecutionContexts(final Collection stepExecutions); /** * Persist the updates of execution context associated with the given * jobExecution. Persistent entry should already exist for this context. - * @param jobExecution + * + * @param jobExecution {@link JobExecution} instance that contains the context. */ void updateExecutionContext(final JobExecution jobExecution); /** * Persist the updates of execution context associated with the given * stepExecution. Persistent entry should already exist for this context. - * @param stepExecution + * + * @param stepExecution {@link StepExecution} instance that contains the context. */ void updateExecutionContext(final StepExecution stepExecution); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/Jackson2ExecutionContextStringSerializer.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/Jackson2ExecutionContextStringSerializer.java new file mode 100644 index 0000000000..f20136f427 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/Jackson2ExecutionContextStringSerializer.java @@ -0,0 +1,144 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.repository.dao; + +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.util.Date; +import java.util.HashMap; +import java.util.Map; + +import com.fasterxml.jackson.annotation.JsonIgnore; +import com.fasterxml.jackson.core.JsonParser; +import com.fasterxml.jackson.core.type.TypeReference; +import com.fasterxml.jackson.databind.DeserializationContext; +import com.fasterxml.jackson.databind.DeserializationFeature; +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.MapperFeature; +import com.fasterxml.jackson.databind.ObjectMapper; + +import com.fasterxml.jackson.databind.deser.std.StdDeserializer; +import com.fasterxml.jackson.databind.module.SimpleModule; +import org.springframework.batch.core.JobParameter; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.repository.ExecutionContextSerializer; +import org.springframework.util.Assert; + +/** + * Implementation that uses Jackson2 to provide (de)serialization. + * + * @author Marten Deinum + * @author Mahmoud Ben Hassine + * @since 3.0.7 + * + * @see ExecutionContextSerializer + */ +public class Jackson2ExecutionContextStringSerializer implements ExecutionContextSerializer { + + private ObjectMapper objectMapper; + + public Jackson2ExecutionContextStringSerializer() { + this.objectMapper = new ObjectMapper(); + this.objectMapper.configure(MapperFeature.DEFAULT_VIEW_INCLUSION, false); + this.objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, true); + this.objectMapper.enableDefaultTyping(); + this.objectMapper.registerModule(new JobParametersModule()); + } + + public void setObjectMapper(ObjectMapper objectMapper) { + Assert.notNull(objectMapper, "ObjectMapper must not be null"); + this.objectMapper = objectMapper.copy(); + this.objectMapper.registerModule(new JobParametersModule()); + } + + public Map deserialize(InputStream in) throws IOException { + + TypeReference> typeRef = new TypeReference>() {}; + return objectMapper.readValue(in, typeRef); + } + + public void serialize(Map context, OutputStream out) throws IOException { + + Assert.notNull(context, "A context is required"); + Assert.notNull(out, "An OutputStream is required"); + + objectMapper.writeValue(out, context); + } + + // BATCH-2680 + /** + * Custom Jackson module to support {@link JobParameter} and {@link JobParameters} + * deserialization. + */ + private class JobParametersModule extends SimpleModule { + + private static final long serialVersionUID = 1L; + + private JobParametersModule() { + super("Job parameters module"); + setMixInAnnotation(JobParameters.class, JobParametersMixIn.class); + addDeserializer(JobParameter.class, new JobParameterDeserializer()); + } + + private abstract class JobParametersMixIn { + @JsonIgnore + abstract boolean isEmpty(); + } + + private class JobParameterDeserializer extends StdDeserializer { + + private static final long serialVersionUID = 1L; + private static final String IDENTIFYING_KEY_NAME = "identifying"; + private static final String TYPE_KEY_NAME = "type"; + private static final String VALUE_KEY_NAME = "value"; + + JobParameterDeserializer() { + super(JobParameter.class); + } + + @Override + public JobParameter deserialize(JsonParser parser, DeserializationContext context) throws IOException { + JsonNode node = parser.readValueAsTree(); + boolean identifying = node.get(IDENTIFYING_KEY_NAME).asBoolean(); + String type = node.get(TYPE_KEY_NAME).asText(); + JsonNode value = node.get(VALUE_KEY_NAME); + Object parameterValue; + switch (JobParameter.ParameterType.valueOf(type)) { + case STRING: { + parameterValue = value.asText(); + return new JobParameter((String) parameterValue, identifying); + } + case DATE: { + parameterValue = new Date(value.get(1).asLong()); + return new JobParameter((Date) parameterValue, identifying); + } + case LONG: { + parameterValue = value.get(1).asLong(); + return new JobParameter((Long) parameterValue, identifying); + } + case DOUBLE: { + parameterValue = value.asDouble(); + return new JobParameter((Double) parameterValue, identifying); + } + } + return null; + } + } + + } + +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcExecutionContextDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcExecutionContextDao.java index f69a2a21cd..1e12749396 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcExecutionContextDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcExecutionContextDao.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,7 +36,7 @@ import org.springframework.core.serializer.Serializer; import org.springframework.jdbc.core.BatchPreparedStatementSetter; import org.springframework.jdbc.core.PreparedStatementSetter; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; import org.springframework.jdbc.support.lob.DefaultLobHandler; import org.springframework.jdbc.support.lob.LobHandler; import org.springframework.util.Assert; @@ -52,6 +52,7 @@ * @author Thomas Risberg * @author Michael Minella * @author David Turanski + * @author Mahmoud Ben Hassine */ public class JdbcExecutionContextDao extends AbstractJdbcBatchMetadataDao implements ExecutionContextDao { @@ -84,9 +85,10 @@ public class JdbcExecutionContextDao extends AbstractJdbcBatchMetadataDao implem /** * Setter for {@link Serializer} implementation * - * @param serializer + * @param serializer {@link ExecutionContextSerializer} instance to use. */ public void setSerializer(ExecutionContextSerializer serializer) { + Assert.notNull(serializer, "Serializer must not be null"); this.serializer = serializer; } @@ -98,7 +100,7 @@ public void setSerializer(ExecutionContextSerializer serializer) { * Default value is 2500. Clients using multi-bytes charsets on the database * server may need to reduce this value to as little as half the value of * the column size. - * @param shortContextLength + * @param shortContextLength int max length of the short context. */ public void setShortContextLength(int shortContextLength) { this.shortContextLength = shortContextLength; @@ -148,15 +150,18 @@ public void updateExecutionContext(final JobExecution jobExecution) { @Override public void updateExecutionContext(final StepExecution stepExecution) { + // Attempt to prevent concurrent modification errors by blocking here if + // someone is already trying to do it. + synchronized (stepExecution) { + Long executionId = stepExecution.getId(); + ExecutionContext executionContext = stepExecution.getExecutionContext(); + Assert.notNull(executionId, "ExecutionId must not be null."); + Assert.notNull(executionContext, "The ExecutionContext must not be null."); - Long executionId = stepExecution.getId(); - ExecutionContext executionContext = stepExecution.getExecutionContext(); - Assert.notNull(executionId, "ExecutionId must not be null."); - Assert.notNull(executionContext, "The ExecutionContext must not be null."); - - String serializedContext = serializeContext(executionContext); + String serializedContext = serializeContext(executionContext); - persistSerializedContext(executionId, serializedContext, UPDATE_STEP_EXECUTION_CONTEXT); + persistSerializedContext(executionId, serializedContext, UPDATE_STEP_EXECUTION_CONTEXT); + } } @Override @@ -187,7 +192,7 @@ public void saveExecutionContext(StepExecution stepExecution) { @Override public void saveExecutionContexts(Collection stepExecutions) { Assert.notNull(stepExecutions, "Attempt to save an null collection of step executions"); - Map serializedContexts = new HashMap(stepExecutions.size()); + Map serializedContexts = new HashMap<>(stepExecutions.size()); for (StepExecution stepExecution : stepExecutions) { Long executionId = stepExecution.getId(); ExecutionContext executionContext = stepExecution.getExecutionContext(); @@ -205,6 +210,7 @@ public void setLobHandler(LobHandler lobHandler) { @Override public void afterPropertiesSet() throws Exception { super.afterPropertiesSet(); + Assert.state(serializer != null, "ExecutionContextSerializer is required"); } /** @@ -243,8 +249,7 @@ public void setValues(PreparedStatement ps) throws SQLException { } /** - * @param executionId - * @param serializedContext + * @param serializedContexts * @param sql with parameters (shortContext, longContext, executionId) */ private void persistSerializedContexts(final Map serializedContexts, String sql) { @@ -284,9 +289,8 @@ public int getBatchSize() { } } - @SuppressWarnings("unchecked") private String serializeContext(ExecutionContext ctx) { - Map m = new HashMap(); + Map m = new HashMap<>(); for (Entry me : ctx.entrySet()) { m.put(me.getKey(), me.getValue()); } @@ -305,8 +309,7 @@ private String serializeContext(ExecutionContext ctx) { return results; } - @SuppressWarnings("unchecked") - private class ExecutionContextRowMapper implements ParameterizedRowMapper { + private class ExecutionContextRowMapper implements RowMapper { @Override public ExecutionContext mapRow(ResultSet rs, int i) throws SQLException { @@ -319,7 +322,7 @@ public ExecutionContext mapRow(ResultSet rs, int i) throws SQLException { Map map; try { ByteArrayInputStream in = new ByteArrayInputStream(serializedContext.getBytes("ISO-8859-1")); - map = (Map) serializer.deserialize(in); + map = serializer.deserialize(in); } catch (IOException ioe) { throw new IllegalArgumentException("Unable to deserialize the execution context", ioe); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcJobExecutionDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcJobExecutionDao.java index 1bfda6996d..327d5a1311 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcJobExecutionDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcJobExecutionDao.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.JobExecution; @@ -40,8 +41,9 @@ import org.springframework.dao.EmptyResultDataAccessException; import org.springframework.dao.OptimisticLockingFailureException; import org.springframework.jdbc.core.RowCallbackHandler; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; import org.springframework.jdbc.support.incrementer.DataFieldMaxValueIncrementer; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -57,13 +59,15 @@ * @author Dave Syer * @author Robert Kasanicky * @author Michael Minella + * @author Mahmoud Ben Hassine + * @author Dimitrios Liapis */ public class JdbcJobExecutionDao extends AbstractJdbcBatchMetadataDao implements JobExecutionDao, InitializingBean { private static final Log logger = LogFactory.getLog(JdbcJobExecutionDao.class); private static final String SAVE_JOB_EXECUTION = "INSERT into %PREFIX%JOB_EXECUTION(JOB_EXECUTION_ID, JOB_INSTANCE_ID, START_TIME, " - + "END_TIME, STATUS, EXIT_CODE, EXIT_MESSAGE, VERSION, CREATE_TIME, LAST_UPDATED) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"; + + "END_TIME, STATUS, EXIT_CODE, EXIT_MESSAGE, VERSION, CREATE_TIME, LAST_UPDATED, JOB_CONFIGURATION_LOCATION) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"; private static final String CHECK_JOB_EXECUTION_EXISTS = "SELECT COUNT(*) FROM %PREFIX%JOB_EXECUTION WHERE JOB_EXECUTION_ID = ?"; @@ -72,17 +76,17 @@ public class JdbcJobExecutionDao extends AbstractJdbcBatchMetadataDao implements private static final String UPDATE_JOB_EXECUTION = "UPDATE %PREFIX%JOB_EXECUTION set START_TIME = ?, END_TIME = ?, " + " STATUS = ?, EXIT_CODE = ?, EXIT_MESSAGE = ?, VERSION = ?, CREATE_TIME = ?, LAST_UPDATED = ? where JOB_EXECUTION_ID = ? and VERSION = ?"; - private static final String FIND_JOB_EXECUTIONS = "SELECT JOB_EXECUTION_ID, START_TIME, END_TIME, STATUS, EXIT_CODE, EXIT_MESSAGE, CREATE_TIME, LAST_UPDATED, VERSION" + private static final String FIND_JOB_EXECUTIONS = "SELECT JOB_EXECUTION_ID, START_TIME, END_TIME, STATUS, EXIT_CODE, EXIT_MESSAGE, CREATE_TIME, LAST_UPDATED, VERSION, JOB_CONFIGURATION_LOCATION" + " from %PREFIX%JOB_EXECUTION where JOB_INSTANCE_ID = ? order by JOB_EXECUTION_ID desc"; - private static final String GET_LAST_EXECUTION = "SELECT JOB_EXECUTION_ID, START_TIME, END_TIME, STATUS, EXIT_CODE, EXIT_MESSAGE, CREATE_TIME, LAST_UPDATED, VERSION " + private static final String GET_LAST_EXECUTION = "SELECT JOB_EXECUTION_ID, START_TIME, END_TIME, STATUS, EXIT_CODE, EXIT_MESSAGE, CREATE_TIME, LAST_UPDATED, VERSION, JOB_CONFIGURATION_LOCATION " + "from %PREFIX%JOB_EXECUTION E where JOB_INSTANCE_ID = ? and JOB_EXECUTION_ID in (SELECT max(JOB_EXECUTION_ID) from %PREFIX%JOB_EXECUTION E2 where E2.JOB_INSTANCE_ID = ?)"; - private static final String GET_EXECUTION_BY_ID = "SELECT JOB_EXECUTION_ID, START_TIME, END_TIME, STATUS, EXIT_CODE, EXIT_MESSAGE, CREATE_TIME, LAST_UPDATED, VERSION" + private static final String GET_EXECUTION_BY_ID = "SELECT JOB_EXECUTION_ID, START_TIME, END_TIME, STATUS, EXIT_CODE, EXIT_MESSAGE, CREATE_TIME, LAST_UPDATED, VERSION, JOB_CONFIGURATION_LOCATION" + " from %PREFIX%JOB_EXECUTION where JOB_EXECUTION_ID = ?"; private static final String GET_RUNNING_EXECUTIONS = "SELECT E.JOB_EXECUTION_ID, E.START_TIME, E.END_TIME, E.STATUS, E.EXIT_CODE, E.EXIT_MESSAGE, E.CREATE_TIME, E.LAST_UPDATED, E.VERSION, " - + "E.JOB_INSTANCE_ID from %PREFIX%JOB_EXECUTION E, %PREFIX%JOB_INSTANCE I where E.JOB_INSTANCE_ID=I.JOB_INSTANCE_ID and I.JOB_NAME=? and E.END_TIME is NULL order by E.JOB_EXECUTION_ID desc"; + + "E.JOB_INSTANCE_ID, E.JOB_CONFIGURATION_LOCATION from %PREFIX%JOB_EXECUTION E, %PREFIX%JOB_INSTANCE I where E.JOB_INSTANCE_ID=I.JOB_INSTANCE_ID and I.JOB_NAME=? and E.START_TIME is not NULL and E.END_TIME is NULL order by E.JOB_EXECUTION_ID desc"; private static final String CURRENT_VERSION_JOB_EXECUTION = "SELECT VERSION FROM %PREFIX%JOB_EXECUTION WHERE JOB_EXECUTION_ID=?"; @@ -151,12 +155,13 @@ public void saveJobExecution(JobExecution jobExecution) { Object[] parameters = new Object[] { jobExecution.getId(), jobExecution.getJobId(), jobExecution.getStartTime(), jobExecution.getEndTime(), jobExecution.getStatus().toString(), jobExecution.getExitStatus().getExitCode(), jobExecution.getExitStatus().getExitDescription(), - jobExecution.getVersion(), jobExecution.getCreateTime(), jobExecution.getLastUpdated() }; + jobExecution.getVersion(), jobExecution.getCreateTime(), jobExecution.getLastUpdated(), + jobExecution.getJobConfigurationName() }; getJdbcTemplate().update( getQuery(SAVE_JOB_EXECUTION), parameters, new int[] { Types.BIGINT, Types.BIGINT, Types.TIMESTAMP, Types.TIMESTAMP, Types.VARCHAR, - Types.VARCHAR, Types.VARCHAR, Types.INTEGER, Types.TIMESTAMP, Types.TIMESTAMP }); + Types.VARCHAR, Types.VARCHAR, Types.INTEGER, Types.TIMESTAMP, Types.TIMESTAMP, Types.VARCHAR }); insertJobParameters(jobExecution.getId(), jobExecution.getJobParameters()); } @@ -170,7 +175,7 @@ public void saveJobExecution(JobExecution jobExecution) { */ private void validateJobExecution(JobExecution jobExecution) { - Assert.notNull(jobExecution); + Assert.notNull(jobExecution, "jobExecution cannot be null"); Assert.notNull(jobExecution.getJobId(), "JobExecution Job-Id cannot be null."); Assert.notNull(jobExecution.getStatus(), "JobExecution status cannot be null."); Assert.notNull(jobExecution.getCreateTime(), "JobExecution create time cannot be null"); @@ -201,7 +206,9 @@ public void updateJobExecution(JobExecution jobExecution) { String exitDescription = jobExecution.getExitStatus().getExitDescription(); if (exitDescription != null && exitDescription.length() > exitMessageLength) { exitDescription = exitDescription.substring(0, exitMessageLength); - logger.debug("Truncating long message before update of JobExecution: " + jobExecution); + if (logger.isDebugEnabled()) { + logger.debug("Truncating long message before update of JobExecution: " + jobExecution); + } } Object[] parameters = new Object[] { jobExecution.getStartTime(), jobExecution.getEndTime(), jobExecution.getStatus().toString(), jobExecution.getExitStatus().getExitCode(), exitDescription, @@ -212,7 +219,7 @@ public void updateJobExecution(JobExecution jobExecution) { // it // is invalid and // an exception should be thrown. - if (getJdbcTemplate().queryForInt(getQuery(CHECK_JOB_EXECUTION_EXISTS), + if (getJdbcTemplate().queryForObject(getQuery(CHECK_JOB_EXECUTION_EXISTS), Integer.class, new Object[] { jobExecution.getId() }) != 1) { throw new NoSuchObjectException("Invalid JobExecution, ID " + jobExecution.getId() + " not found."); } @@ -225,17 +232,18 @@ public void updateJobExecution(JobExecution jobExecution) { // Avoid concurrent modifications... if (count == 0) { - int curentVersion = getJdbcTemplate().queryForInt(getQuery(CURRENT_VERSION_JOB_EXECUTION), + int currentVersion = getJdbcTemplate().queryForObject(getQuery(CURRENT_VERSION_JOB_EXECUTION), Integer.class, new Object[] { jobExecution.getId() }); throw new OptimisticLockingFailureException("Attempt to update job execution id=" + jobExecution.getId() + " with wrong version (" + jobExecution.getVersion() - + "), where current version is " + curentVersion); + + "), where current version is " + currentVersion); } jobExecution.incrementVersion(); } } + @Nullable @Override public JobExecution getLastJobExecution(JobInstance jobInstance) { @@ -257,10 +265,11 @@ public JobExecution getLastJobExecution(JobInstance jobInstance) { /* * (non-Javadoc) * - * @seeorg.springframework.batch.core.repository.dao.JobExecutionDao# + * @see org.springframework.batch.core.repository.dao.JobExecutionDao# * getLastJobExecution(java.lang.String) */ @Override + @Nullable public JobExecution getJobExecution(Long executionId) { try { JobExecution jobExecution = getJdbcTemplate().queryForObject(getQuery(GET_EXECUTION_BY_ID), @@ -275,13 +284,13 @@ public JobExecution getJobExecution(Long executionId) { /* * (non-Javadoc) * - * @seeorg.springframework.batch.core.repository.dao.JobExecutionDao# + * @see org.springframework.batch.core.repository.dao.JobExecutionDao# * findRunningJobExecutions(java.lang.String) */ @Override public Set findRunningJobExecutions(String jobName) { - final Set result = new HashSet(); + final Set result = new HashSet<>(); RowCallbackHandler handler = new RowCallbackHandler() { @Override public void processRow(ResultSet rs) throws SQLException { @@ -296,7 +305,7 @@ public void processRow(ResultSet rs) throws SQLException { @Override public void synchronizeStatus(JobExecution jobExecution) { - int currentVersion = getJdbcTemplate().queryForInt(getQuery(CURRENT_VERSION_JOB_EXECUTION), + int currentVersion = getJdbcTemplate().queryForObject(getQuery(CURRENT_VERSION_JOB_EXECUTION), Integer.class, jobExecution.getId()); if (currentVersion != jobExecution.getVersion().intValue()) { @@ -352,11 +361,11 @@ private void insertParameter(Long executionId, ParameterType type, String key, } /** - * @param executionId - * @return + * @param executionId {@link Long} containing the id for the execution. + * @return job parameters for the requested execution id */ - private JobParameters getJobParameters(Long executionId) { - final Map map = new HashMap(); + protected JobParameters getJobParameters(Long executionId) { + final Map map = new HashMap<>(); RowCallbackHandler handler = new RowCallbackHandler() { @Override public void processRow(ResultSet rs) throws SQLException { @@ -389,7 +398,7 @@ public void processRow(ResultSet rs) throws SQLException { * @author Dave Syer * */ - private final class JobExecutionRowMapper implements ParameterizedRowMapper { + private final class JobExecutionRowMapper implements RowMapper { private JobInstance jobInstance; @@ -405,16 +414,17 @@ public JobExecutionRowMapper(JobInstance jobInstance) { @Override public JobExecution mapRow(ResultSet rs, int rowNum) throws SQLException { Long id = rs.getLong(1); + String jobConfigurationLocation = rs.getString(10); JobExecution jobExecution; if (jobParameters == null) { jobParameters = getJobParameters(id); } if (jobInstance == null) { - jobExecution = new JobExecution(id, jobParameters); + jobExecution = new JobExecution(id, jobParameters, jobConfigurationLocation); } else { - jobExecution = new JobExecution(jobInstance, id, jobParameters); + jobExecution = new JobExecution(jobInstance, id, jobParameters, jobConfigurationLocation); } jobExecution.setStartTime(rs.getTimestamp(2)); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcJobInstanceDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcJobInstanceDao.java index 8740423419..d9de3e1a58 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcJobInstanceDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcJobInstanceDao.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,12 +27,14 @@ import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.JobKeyGenerator; import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.NoSuchJobException; import org.springframework.beans.factory.InitializingBean; import org.springframework.dao.DataAccessException; import org.springframework.dao.EmptyResultDataAccessException; import org.springframework.jdbc.core.ResultSetExtractor; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; import org.springframework.jdbc.support.incrementer.DataFieldMaxValueIncrementer; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.StringUtils; @@ -49,10 +51,16 @@ * @author Dave Syer * @author Robert Kasanicky * @author Michael Minella + * @author Will Schipp + * @author Mahmoud Ben Hassine */ public class JdbcJobInstanceDao extends AbstractJdbcBatchMetadataDao implements JobInstanceDao, InitializingBean { + private static final String STAR_WILDCARD = "*"; + + private static final String SQL_WILDCARD = "%"; + private static final String CREATE_JOB_INSTANCE = "INSERT into %PREFIX%JOB_INSTANCE(JOB_INSTANCE_ID, JOB_NAME, JOB_KEY, VERSION)" + " values (?, ?, ?, ?)"; @@ -61,6 +69,8 @@ public class JdbcJobInstanceDao extends AbstractJdbcBatchMetadataDao implements private static final String FIND_JOBS_WITH_KEY = FIND_JOBS_WITH_NAME + " and JOB_KEY = ?"; + private static final String COUNT_JOBS_WITH_NAME = "SELECT COUNT(*) from %PREFIX%JOB_INSTANCE where JOB_NAME = ?"; + private static final String FIND_JOBS_WITH_EMPTY_KEY = "SELECT JOB_INSTANCE_ID, JOB_NAME from %PREFIX%JOB_INSTANCE where JOB_NAME = ? and (JOB_KEY = ? OR JOB_KEY is NULL)"; private static final String GET_JOB_FROM_ID = "SELECT JOB_INSTANCE_ID, JOB_NAME, JOB_KEY, VERSION from %PREFIX%JOB_INSTANCE where JOB_INSTANCE_ID = ?"; @@ -72,12 +82,17 @@ public class JdbcJobInstanceDao extends AbstractJdbcBatchMetadataDao implements private static final String FIND_LAST_JOBS_BY_NAME = "SELECT JOB_INSTANCE_ID, JOB_NAME from %PREFIX%JOB_INSTANCE where JOB_NAME = ? order by JOB_INSTANCE_ID desc"; + private static final String FIND_LAST_JOB_INSTANCE_BY_JOB_NAME = "SELECT JOB_INSTANCE_ID, JOB_NAME from %PREFIX%JOB_INSTANCE I1 where" + + " I1.JOB_NAME = ? and I1.JOB_INSTANCE_ID in (SELECT max(I2.JOB_INSTANCE_ID) from %PREFIX%JOB_INSTANCE I2 where I2.JOB_NAME = ?)"; + + private static final String FIND_LAST_JOBS_LIKE_NAME = "SELECT JOB_INSTANCE_ID, JOB_NAME from %PREFIX%JOB_INSTANCE where JOB_NAME like ? order by JOB_INSTANCE_ID desc"; + private DataFieldMaxValueIncrementer jobIncrementer; private JobKeyGenerator jobKeyGenerator = new DefaultJobKeyGenerator(); /** - * In this jdbc implementation a job id is obtained by asking the + * In this JDBC implementation a job id is obtained by asking the * jobIncrementer (which is likely a sequence) for the next long value, and * then passing the Id and parameter values into an INSERT statement. * @@ -120,6 +135,7 @@ public JobInstance createJobInstance(String jobName, * if any {@link JobParameters} fields are null. */ @Override + @Nullable public JobInstance getJobInstance(final String jobName, final JobParameters jobParameters) { @@ -128,7 +144,7 @@ public JobInstance getJobInstance(final String jobName, String jobKey = jobKeyGenerator.generateKey(jobParameters); - ParameterizedRowMapper rowMapper = new JobInstanceRowMapper(); + RowMapper rowMapper = new JobInstanceRowMapper(); List instances; if (StringUtils.hasLength(jobKey)) { @@ -143,7 +159,7 @@ public JobInstance getJobInstance(final String jobName, if (instances.isEmpty()) { return null; } else { - Assert.state(instances.size() == 1); + Assert.state(instances.size() == 1, "instance count must be 1 but was " + instances.size()); return instances.get(0); } } @@ -156,7 +172,8 @@ public JobInstance getJobInstance(final String jobName, * (java.lang.Long) */ @Override - public JobInstance getJobInstance(Long instanceId) { + @Nullable + public JobInstance getJobInstance(@Nullable Long instanceId) { try { return getJdbcTemplate().queryForObject(getQuery(GET_JOB_FROM_ID), @@ -177,7 +194,7 @@ public JobInstance getJobInstance(Long instanceId) { @Override public List getJobNames() { return getJdbcTemplate().query(getQuery(FIND_JOB_NAMES), - new ParameterizedRowMapper() { + new RowMapper() { @Override public String mapRow(ResultSet rs, int rowNum) throws SQLException { @@ -189,27 +206,26 @@ public String mapRow(ResultSet rs, int rowNum) /* * (non-Javadoc) * - * @seeorg.springframework.batch.core.repository.dao.JobInstanceDao# + * @see org.springframework.batch.core.repository.dao.JobInstanceDao# * getLastJobInstances(java.lang.String, int) */ @Override - @SuppressWarnings("rawtypes") public List getJobInstances(String jobName, final int start, final int count) { - ResultSetExtractor extractor = new ResultSetExtractor() { + ResultSetExtractor> extractor = new ResultSetExtractor>() { - private List list = new ArrayList(); + private List list = new ArrayList<>(); @Override - public Object extractData(ResultSet rs) throws SQLException, + public List extractData(ResultSet rs) throws SQLException, DataAccessException { int rowNum = 0; while (rowNum < start && rs.next()) { rowNum++; } while (rowNum < start + count && rs.next()) { - ParameterizedRowMapper rowMapper = new JobInstanceRowMapper(); + RowMapper rowMapper = new JobInstanceRowMapper(); list.add(rowMapper.mapRow(rs, rowNum)); rowNum++; } @@ -218,13 +234,31 @@ public Object extractData(ResultSet rs) throws SQLException, }; - @SuppressWarnings("unchecked") - List result = (List) getJdbcTemplate().query(getQuery(FIND_LAST_JOBS_BY_NAME), + List result = getJdbcTemplate().query(getQuery(FIND_LAST_JOBS_BY_NAME), new Object[] { jobName }, extractor); return result; } + /* + * (non-Javadoc) + * + * @see org.springframework.batch.core.repository.dao.JobInstanceDao# + * getLastJobInstance(java.lang.String) + */ + @Override + @Nullable + public JobInstance getLastJobInstance(String jobName) { + try { + return getJdbcTemplate().queryForObject( + getQuery(FIND_LAST_JOB_INSTANCE_BY_JOB_NAME), + new Object[] { jobName, jobName }, + new JobInstanceRowMapper()); + } catch (EmptyResultDataAccessException e) { + return null; + } + } + /* * (non-Javadoc) * @@ -233,6 +267,7 @@ public Object extractData(ResultSet rs) throws SQLException, * (org.springframework.batch.core.JobExecution) */ @Override + @Nullable public JobInstance getJobInstance(JobExecution jobExecution) { try { @@ -244,6 +279,22 @@ public JobInstance getJobInstance(JobExecution jobExecution) { } } + /* (non-Javadoc) + * @see org.springframework.batch.core.repository.dao.JobInstanceDao#getJobInstanceCount(java.lang.String) + */ + @Override + public int getJobInstanceCount(@Nullable String jobName) throws NoSuchJobException { + + try { + return getJdbcTemplate().queryForObject( + getQuery(COUNT_JOBS_WITH_NAME), + Integer.class, + jobName); + } catch (EmptyResultDataAccessException e) { + throw new NoSuchJobException("No job instances were found for job name " + jobName); + } + } + /** * Setter for {@link DataFieldMaxValueIncrementer} to be used when * generating primary keys for {@link JobInstance} instances. @@ -258,15 +309,14 @@ public void setJobIncrementer(DataFieldMaxValueIncrementer jobIncrementer) { @Override public void afterPropertiesSet() throws Exception { super.afterPropertiesSet(); - Assert.notNull(jobIncrementer); + Assert.notNull(jobIncrementer, "JobIncrementer is required"); } /** * @author Dave Syer * */ - private final class JobInstanceRowMapper implements - ParameterizedRowMapper { + private final class JobInstanceRowMapper implements RowMapper { public JobInstanceRowMapper() { } @@ -279,4 +329,37 @@ public JobInstance mapRow(ResultSet rs, int rowNum) throws SQLException { return jobInstance; } } + + @Override + public List findJobInstancesByName(String jobName, final int start, final int count) { + @SuppressWarnings("rawtypes") + ResultSetExtractor extractor = new ResultSetExtractor() { + private List list = new ArrayList<>(); + + @Override + public Object extractData(ResultSet rs) throws SQLException, + DataAccessException { + int rowNum = 0; + while (rowNum < start && rs.next()) { + rowNum++; + } + while (rowNum < start + count && rs.next()) { + RowMapper rowMapper = new JobInstanceRowMapper(); + list.add(rowMapper.mapRow(rs, rowNum)); + rowNum++; + } + return list; + } + }; + + if (jobName.contains(STAR_WILDCARD)) { + jobName = jobName.replaceAll("\\" + STAR_WILDCARD, SQL_WILDCARD); + } + + @SuppressWarnings("unchecked") + List result = (List) getJdbcTemplate().query(getQuery(FIND_LAST_JOBS_LIKE_NAME), + new Object[] { jobName }, extractor); + + return result; + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcStepExecutionDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcStepExecutionDao.java index 84af2cd0b6..cc268adc6b 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcStepExecutionDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JdbcStepExecutionDao.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,34 +29,38 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.StepExecution; import org.springframework.beans.factory.InitializingBean; import org.springframework.dao.OptimisticLockingFailureException; import org.springframework.jdbc.core.BatchPreparedStatementSetter; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; import org.springframework.jdbc.support.incrementer.DataFieldMaxValueIncrementer; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** - * JDBC implementation of {@link StepExecutionDao}.
      + * JDBC implementation of {@link StepExecutionDao}.
      * * Allows customization of the tables names used by Spring Batch for step meta - * data via a prefix property.
      + * data via a prefix property.
      * * Uses sequences or tables (via Spring's {@link DataFieldMaxValueIncrementer} * abstraction) to create all primary keys before inserting a new row. All * objects are checked to ensure all fields to be stored are not null. If any * are found to be null, an IllegalArgumentException will be thrown. This could * be left to JdbcTemplate, however, the exception will be fairly vague, and - * fails to highlight which field caused the exception.
      + * fails to highlight which field caused the exception.
      * * @author Lucas Ward * @author Dave Syer * @author Robert Kasanicky * @author David Turanski + * @author Mahmoud Ben Hassine * * @see StepExecutionDao */ @@ -64,23 +68,42 @@ public class JdbcStepExecutionDao extends AbstractJdbcBatchMetadataDao implement private static final Log logger = LogFactory.getLog(JdbcStepExecutionDao.class); - private static final String SAVE_STEP_EXECUTION = "INSERT into %PREFIX%STEP_EXECUTION(STEP_EXECUTION_ID, VERSION, STEP_NAME, JOB_EXECUTION_ID, START_TIME, " - + "END_TIME, STATUS, COMMIT_COUNT, READ_COUNT, FILTER_COUNT, WRITE_COUNT, EXIT_CODE, EXIT_MESSAGE, READ_SKIP_COUNT, WRITE_SKIP_COUNT, PROCESS_SKIP_COUNT, ROLLBACK_COUNT, LAST_UPDATED) " - + "values(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"; + private static final String SAVE_STEP_EXECUTION = "INSERT into %PREFIX%STEP_EXECUTION(STEP_EXECUTION_ID, VERSION, " + + "STEP_NAME, JOB_EXECUTION_ID, START_TIME, END_TIME, STATUS, COMMIT_COUNT, READ_COUNT, FILTER_COUNT, " + + "WRITE_COUNT, EXIT_CODE, EXIT_MESSAGE, READ_SKIP_COUNT, WRITE_SKIP_COUNT, PROCESS_SKIP_COUNT, " + + "ROLLBACK_COUNT, LAST_UPDATED) values(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"; private static final String UPDATE_STEP_EXECUTION = "UPDATE %PREFIX%STEP_EXECUTION set START_TIME = ?, END_TIME = ?, " + "STATUS = ?, COMMIT_COUNT = ?, READ_COUNT = ?, FILTER_COUNT = ?, WRITE_COUNT = ?, EXIT_CODE = ?, " - + "EXIT_MESSAGE = ?, VERSION = ?, READ_SKIP_COUNT = ?, PROCESS_SKIP_COUNT = ?, WRITE_SKIP_COUNT = ?, ROLLBACK_COUNT = ?, LAST_UPDATED = ?" + + "EXIT_MESSAGE = ?, VERSION = ?, READ_SKIP_COUNT = ?, PROCESS_SKIP_COUNT = ?, WRITE_SKIP_COUNT = ?, " + + "ROLLBACK_COUNT = ?, LAST_UPDATED = ?" + " where STEP_EXECUTION_ID = ? and VERSION = ?"; - private static final String GET_RAW_STEP_EXECUTIONS = "SELECT STEP_EXECUTION_ID, STEP_NAME, START_TIME, END_TIME, STATUS, COMMIT_COUNT," - + " READ_COUNT, FILTER_COUNT, WRITE_COUNT, EXIT_CODE, EXIT_MESSAGE, READ_SKIP_COUNT, WRITE_SKIP_COUNT, PROCESS_SKIP_COUNT, ROLLBACK_COUNT, LAST_UPDATED, VERSION from %PREFIX%STEP_EXECUTION where JOB_EXECUTION_ID = ?"; + private static final String GET_RAW_STEP_EXECUTIONS = "SELECT STEP_EXECUTION_ID, STEP_NAME, START_TIME, END_TIME, " + + "STATUS, COMMIT_COUNT, READ_COUNT, FILTER_COUNT, WRITE_COUNT, EXIT_CODE, EXIT_MESSAGE, READ_SKIP_COUNT, " + + "WRITE_SKIP_COUNT, PROCESS_SKIP_COUNT, ROLLBACK_COUNT, LAST_UPDATED, VERSION from %PREFIX%STEP_EXECUTION " + + "where JOB_EXECUTION_ID = ?"; private static final String GET_STEP_EXECUTIONS = GET_RAW_STEP_EXECUTIONS + " order by STEP_EXECUTION_ID"; private static final String GET_STEP_EXECUTION = GET_RAW_STEP_EXECUTIONS + " and STEP_EXECUTION_ID = ?"; - private static final String CURRENT_VERSION_STEP_EXECUTION = "SELECT VERSION FROM %PREFIX%STEP_EXECUTION WHERE STEP_EXECUTION_ID=?"; + private static final String GET_LAST_STEP_EXECUTION = "SELECT " + + " SE.STEP_EXECUTION_ID, SE.STEP_NAME, SE.START_TIME, SE.END_TIME, SE.STATUS, SE.COMMIT_COUNT, " + + "SE.READ_COUNT, SE.FILTER_COUNT, SE.WRITE_COUNT, SE.EXIT_CODE, SE.EXIT_MESSAGE, SE.READ_SKIP_COUNT, " + + "SE.WRITE_SKIP_COUNT, SE.PROCESS_SKIP_COUNT, SE.ROLLBACK_COUNT, SE.LAST_UPDATED, SE.VERSION," + + " JE.JOB_EXECUTION_ID, JE.START_TIME, JE.END_TIME, JE.STATUS, JE.EXIT_CODE, JE.EXIT_MESSAGE, " + + "JE.CREATE_TIME, JE.LAST_UPDATED, JE.VERSION" + + " from %PREFIX%JOB_EXECUTION JE, %PREFIX%STEP_EXECUTION SE" + + " where " + + " SE.JOB_EXECUTION_ID in (SELECT JOB_EXECUTION_ID from %PREFIX%JOB_EXECUTION " + + "where JE.JOB_INSTANCE_ID = ?)" + + " and SE.JOB_EXECUTION_ID = JE.JOB_EXECUTION_ID " + + " and SE.STEP_NAME = ?" + + " order by SE.START_TIME desc, SE.STEP_EXECUTION_ID desc"; + + private static final String CURRENT_VERSION_STEP_EXECUTION = "SELECT VERSION FROM %PREFIX%STEP_EXECUTION WHERE " + + "STEP_EXECUTION_ID=?"; private int exitMessageLength = DEFAULT_EXIT_MESSAGE_LENGTH; @@ -107,7 +130,7 @@ public void afterPropertiesSet() throws Exception { /** * Save a StepExecution. A unique id will be generated by the - * stepExecutionIncrementor, and then set in the StepExecution. All values + * stepExecutionIncrementer, and then set in the StepExecution. All values * will then be stored via an INSERT statement. * * @see StepExecutionDao#saveStepExecution(StepExecution) @@ -115,7 +138,7 @@ public void afterPropertiesSet() throws Exception { @Override public void saveStepExecution(StepExecution stepExecution) { List parameters = buildStepExecutionParameters(stepExecution); - Object[] parameterValues = (Object[])parameters.get(0); + Object[] parameterValues = parameters.get(0); //Template expects an int array fails with Integer int[] parameterTypes = new int[parameters.get(1).length]; @@ -147,7 +170,7 @@ public int getBatchSize() { public void setValues(PreparedStatement ps, int i) throws SQLException { StepExecution stepExecution = iterator.next(); List parameters = buildStepExecutionParameters(stepExecution); - Object[] parameterValues = (Object[]) parameters.get(0); + Object[] parameterValues = parameters.get(0); Integer[] parameterTypes = (Integer[]) parameters.get(1); for (int indx = 0; indx < parameterValues.length; indx++) { switch (parameterTypes[indx]) { @@ -185,7 +208,7 @@ private List buildStepExecutionParameters(StepExecution stepExecution) validateStepExecution(stepExecution); stepExecution.setId(stepExecutionIncrementer.nextLongValue()); stepExecution.incrementVersion(); //Should be 0 - List parameters = new ArrayList(); + List parameters = new ArrayList<>(); String exitDescription = truncateExitDescription(stepExecution.getExitStatus().getExitDescription()); Object[] parameterValues = new Object[] { stepExecution.getId(), stepExecution.getVersion(), stepExecution.getStepName(), stepExecution.getJobExecutionId(), stepExecution.getStartTime(), @@ -210,7 +233,7 @@ private List buildStepExecutionParameters(StepExecution stepExecution) * @throws IllegalArgumentException */ private void validateStepExecution(StepExecution stepExecution) { - Assert.notNull(stepExecution); + Assert.notNull(stepExecution, "stepExecution is required"); Assert.notNull(stepExecution.getStepName(), "StepExecution step name cannot be null."); Assert.notNull(stepExecution.getStartTime(), "StepExecution start time cannot be null."); Assert.notNull(stepExecution.getStatus(), "StepExecution status cannot be null."); @@ -250,11 +273,11 @@ public void updateStepExecution(StepExecution stepExecution) { // Avoid concurrent modifications... if (count == 0) { - int curentVersion = getJdbcTemplate().queryForInt(getQuery(CURRENT_VERSION_STEP_EXECUTION), - new Object[] { stepExecution.getId() }); + int currentVersion = getJdbcTemplate().queryForObject(getQuery(CURRENT_VERSION_STEP_EXECUTION), + new Object[] { stepExecution.getId() }, Integer.class); throw new OptimisticLockingFailureException("Attempt to update step execution id=" + stepExecution.getId() + " with wrong version (" + stepExecution.getVersion() - + "), where current version is " + curentVersion); + + "), where current version is " + currentVersion); } stepExecution.incrementVersion(); @@ -270,7 +293,9 @@ public void updateStepExecution(StepExecution stepExecution) { */ private String truncateExitDescription(String description) { if (description != null && description.length() > exitMessageLength) { - logger.debug("Truncating long message before update of StepExecution, original message is: " + description); + if (logger.isDebugEnabled()) { + logger.debug("Truncating long message before update of StepExecution, original message is: " + description); + } return description.substring(0, exitMessageLength); } else { return description; @@ -278,6 +303,7 @@ private String truncateExitDescription(String description) { } @Override + @Nullable public StepExecution getStepExecution(JobExecution jobExecution, Long stepExecutionId) { List executions = getJdbcTemplate().query(getQuery(GET_STEP_EXECUTION), new StepExecutionRowMapper(jobExecution), jobExecution.getId(), stepExecutionId); @@ -291,13 +317,37 @@ public StepExecution getStepExecution(JobExecution jobExecution, Long stepExecut } } + @Override + public StepExecution getLastStepExecution(JobInstance jobInstance, String stepName) { + List executions = getJdbcTemplate().query( + getQuery(GET_LAST_STEP_EXECUTION), + (rs, rowNum) -> { + Long jobExecutionId = rs.getLong(18); + JobExecution jobExecution = new JobExecution(jobExecutionId); + jobExecution.setStartTime(rs.getTimestamp(19)); + jobExecution.setEndTime(rs.getTimestamp(20)); + jobExecution.setStatus(BatchStatus.valueOf(rs.getString(21))); + jobExecution.setExitStatus(new ExitStatus(rs.getString(22), rs.getString(23))); + jobExecution.setCreateTime(rs.getTimestamp(24)); + jobExecution.setLastUpdated(rs.getTimestamp(25)); + jobExecution.setVersion(rs.getInt(26)); + return new StepExecutionRowMapper(jobExecution).mapRow(rs, rowNum); + }, + jobInstance.getInstanceId(), stepName); + if (executions.isEmpty()) { + return null; + } else { + return executions.get(0); + } + } + @Override public void addStepExecutions(JobExecution jobExecution) { getJdbcTemplate().query(getQuery(GET_STEP_EXECUTIONS), new StepExecutionRowMapper(jobExecution), jobExecution.getId()); } - private static class StepExecutionRowMapper implements ParameterizedRowMapper { + private static class StepExecutionRowMapper implements RowMapper { private final JobExecution jobExecution; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JobExecutionDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JobExecutionDao.java index 3a1514663e..3b86d47af1 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JobExecutionDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JobExecutionDao.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,12 +21,14 @@ import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobInstance; +import org.springframework.lang.Nullable; /** * Data Access Object for job executions. * * @author Lucas Ward * @author Robert Kasanicky + * @author Mahmoud Ben Hassine */ public interface JobExecutionDao { @@ -36,7 +38,7 @@ public interface JobExecutionDao { * Preconditions: jobInstance the jobExecution belongs to must have a * jobInstanceId. * - * @param jobExecution + * @param jobExecution {@link JobExecution} instance to be saved. */ void saveJobExecution(JobExecution jobExecution); @@ -46,13 +48,16 @@ public interface JobExecutionDao { * Preconditions: jobExecution must have an Id (which can be obtained by the * save method) and a jobInstanceId. * - * @param jobExecution + * @param jobExecution {@link JobExecution} instance to be updated. */ void updateJobExecution(JobExecution jobExecution); /** - * Return all {@link JobExecution} for given {@link JobInstance}, sorted + * Return all {@link JobExecution}s for given {@link JobInstance}, sorted * backwards by creation order (so the first element is the most recent). + * + * @param jobInstance parent {@link JobInstance} of the {@link JobExecution}s to find. + * @return {@link List} containing JobExecutions for the jobInstance. */ List findJobExecutions(JobInstance jobInstance); @@ -60,19 +65,24 @@ public interface JobExecutionDao { * Find the last {@link JobExecution} to have been created for a given * {@link JobInstance}. * @param jobInstance the {@link JobInstance} - * @return the last {@link JobExecution} to execute for this instance + * @return the last {@link JobExecution} to execute for this instance or + * {@code null} if no job execution is found for the given job instance. */ + @Nullable JobExecution getLastJobExecution(JobInstance jobInstance); /** + * @param jobName {@link String} containing the name of the job. * @return all {@link JobExecution} that are still running (or indeterminate * state), i.e. having null end date, for the specified job name. */ Set findRunningJobExecutions(String jobName); /** + * @param executionId {@link Long} containing the id of the execution. * @return the {@link JobExecution} for given identifier. */ + @Nullable JobExecution getJobExecution(Long executionId); /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JobInstanceDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JobInstanceDao.java index e1e3f70335..92af7964ea 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JobInstanceDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/JobInstanceDao.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,63 +21,74 @@ import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.NoSuchJobException; +import org.springframework.lang.Nullable; /** * Data Access Object for job instances. - * + * * @author Lucas Ward * @author Robert Kasanicky - * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * */ public interface JobInstanceDao { /** * Create a JobInstance with given name and parameters. - * + * * PreConditions: JobInstance for given name and parameters must not already * exist - * + * * PostConditions: A valid job instance will be returned which has been * persisted and contains an unique Id. - * - * @param jobName - * @param jobParameters - * @return JobInstance + * + * @param jobName {@link String} containing the name of the job. + * @param jobParameters {@link JobParameters} containing the parameters for + * the JobInstance. + * @return JobInstance {@link JobInstance} instance that was created. */ JobInstance createJobInstance(String jobName, JobParameters jobParameters); /** * Find the job instance that matches the given name and parameters. If no * matching job instances are found, then returns null. - * + * * @param jobName the name of the job * @param jobParameters the parameters with which the job was executed * @return {@link JobInstance} object matching the job name and - * {@link JobParameters} or null + * {@link JobParameters} or {@code null} */ + @Nullable JobInstance getJobInstance(String jobName, JobParameters jobParameters); /** * Fetch the job instance with the provided identifier. - * + * * @param instanceId the job identifier - * @return the job instance with this identifier or null if it doesn't exist + * @return the job instance with this identifier or {@code null} if it doesn't exist */ - JobInstance getJobInstance(Long instanceId); + @Nullable + JobInstance getJobInstance(@Nullable Long instanceId); /** * Fetch the JobInstance for the provided JobExecution. - * + * * @param jobExecution the JobExecution - * @return the JobInstance for the provided execution or null if it doesn't exist. + * @return the JobInstance for the provided execution or {@code null} if it doesn't exist. */ + @Nullable JobInstance getJobInstance(JobExecution jobExecution); - + /** * Fetch the last job instances with the provided name, sorted backwards by * primary key. - * - * + * + * if using the JdbcJobInstance, you can provide the jobName with a wildcard + * (e.g. *Job) to return 'like' job names. (e.g. *Job will return 'someJob' + * and 'otherJob') + * * @param jobName the job name * @param start the start index of the instances to return * @param count the maximum number of objects to return @@ -85,11 +96,49 @@ public interface JobInstanceDao { */ List getJobInstances(String jobName, int start, int count); + /** + * Fetch the last job instance by Id for the given job. + * @param jobName name of the job + * @return the last job instance by Id if any or null otherwise + * + * @since 4.2 + */ + @Nullable + default JobInstance getLastJobInstance(String jobName) { + throw new UnsupportedOperationException(); + } + /** * Retrieve the names of all job instances sorted alphabetically - i.e. jobs * that have ever been executed. + * * @return the names of all job instances */ List getJobNames(); + + /** + * Fetch the last job instances with the provided name, sorted backwards by + * primary key, using a 'like' criteria + * + * @param jobName {@link String} containing the name of the job. + * @param start int containing the offset of where list of job instances + * results should begin. + * @param count int containing the number of job instances to return. + * @return a list of {@link JobInstance} for the job name requested. + */ + List findJobInstancesByName(String jobName, int start, int count); + + + /** + * Query the repository for the number of unique {@link JobInstance}s + * associated with the supplied job name. + * + * @param jobName the name of the job to query for + * @return the number of {@link JobInstance}s that exist within the + * associated job repository + * + * @throws NoSuchJobException thrown if no Job has the jobName specified. + */ + int getJobInstanceCount(@Nullable String jobName) throws NoSuchJobException; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapExecutionContextDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapExecutionContextDao.java index 9a136a098c..1af5ed5a95 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapExecutionContextDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapExecutionContextDao.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,9 +23,9 @@ import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.StepExecution; import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.support.SerializationUtils; import org.springframework.batch.support.transaction.TransactionAwareProxyFactory; import org.springframework.util.Assert; +import org.springframework.util.SerializationUtils; /** * In-memory implementation of {@link ExecutionContextDao} backed by maps. @@ -33,6 +33,7 @@ * @author Robert Kasanicky * @author Dave Syer * @author David Turanski + * @author Mahmoud Ben Hassine */ @SuppressWarnings("serial") public class MapExecutionContextDao implements ExecutionContextDao { @@ -151,7 +152,7 @@ public void saveExecutionContext(StepExecution stepExecution) { @Override public void saveExecutionContexts(Collection stepExecutions) { - Assert.notNull(stepExecutions,"Attempt to save a nulk collection of step executions"); + Assert.notNull(stepExecutions,"Attempt to save a null collection of step executions"); for (StepExecution stepExecution: stepExecutions) { saveExecutionContext(stepExecution); saveExecutionContext(stepExecution.getJobExecution()); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapJobExecutionDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapJobExecutionDao.java index 7c30b8ac2f..066c97184a 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapJobExecutionDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapJobExecutionDao.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -28,9 +28,10 @@ import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobInstance; -import org.springframework.batch.support.SerializationUtils; import org.springframework.dao.OptimisticLockingFailureException; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; +import org.springframework.util.SerializationUtils; /** * In-memory implementation of {@link JobExecutionDao}. @@ -38,7 +39,7 @@ public class MapJobExecutionDao implements JobExecutionDao { // JDK6 Make this into a ConcurrentSkipListMap: adds and removes tend to be very near the front or back - private final ConcurrentMap executionsById = new ConcurrentHashMap(); + private final ConcurrentMap executionsById = new ConcurrentHashMap<>(); private final AtomicLong currentId = new AtomicLong(0L); @@ -53,7 +54,7 @@ private static JobExecution copy(JobExecution original) { @Override public void saveJobExecution(JobExecution jobExecution) { - Assert.isTrue(jobExecution.getId() == null); + Assert.isTrue(jobExecution.getId() == null, "jobExecution id is not null"); Long newId = currentId.getAndIncrement(); jobExecution.setId(newId); jobExecution.incrementVersion(); @@ -62,7 +63,7 @@ public void saveJobExecution(JobExecution jobExecution) { @Override public List findJobExecutions(JobInstance jobInstance) { - List executions = new ArrayList(); + List executions = new ArrayList<>(); for (JobExecution exec : executionsById.values()) { if (exec.getJobInstance().equals(jobInstance)) { executions.add(copy(exec)); @@ -96,7 +97,7 @@ public void updateJobExecution(JobExecution jobExecution) { synchronized (jobExecution) { if (!persistedExecution.getVersion().equals(jobExecution.getVersion())) { - throw new OptimisticLockingFailureException("Attempt to update step execution id=" + id + throw new OptimisticLockingFailureException("Attempt to update job execution id=" + id + " with wrong version (" + jobExecution.getVersion() + "), where current version is " + persistedExecution.getVersion()); } @@ -105,8 +106,9 @@ public void updateJobExecution(JobExecution jobExecution) { } } + @Nullable @Override - public JobExecution getLastJobExecution(JobInstance jobInstance) { + public JobExecution getLastJobExecution(@Nullable JobInstance jobInstance) { JobExecution lastExec = null; for (JobExecution exec : executionsById.values()) { if (!exec.getJobInstance().equals(jobInstance)) { @@ -125,12 +127,12 @@ public JobExecution getLastJobExecution(JobInstance jobInstance) { /* * (non-Javadoc) * - * @seeorg.springframework.batch.core.repository.dao.JobExecutionDao# + * @see org.springframework.batch.core.repository.dao.JobExecutionDao# * findRunningJobExecutions(java.lang.String) */ @Override public Set findRunningJobExecutions(String jobName) { - Set result = new HashSet(); + Set result = new HashSet<>(); for (JobExecution exec : executionsById.values()) { if (!exec.getJobInstance().getJobName().equals(jobName) || !exec.isRunning()) { continue; @@ -148,6 +150,7 @@ public Set findRunningJobExecutions(String jobName) { * (java.lang.Long) */ @Override + @Nullable public JobExecution getJobExecution(Long executionId) { return copy(executionsById.get(executionId)); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapJobInstanceDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapJobInstanceDao.java index 6848de662d..fa41750845 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapJobInstanceDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapJobInstanceDao.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,16 +29,19 @@ import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.JobKeyGenerator; import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.NoSuchJobException; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** * In-memory implementation of {@link JobInstanceDao}. */ public class MapJobInstanceDao implements JobInstanceDao { + private static final String STAR_WILDCARD = "\\*"; + private static final String STAR_WILDCARD_PATTERN = ".*"; // JDK6 Make a ConcurrentSkipListSet: tends to add on end - private final Map jobInstances = new ConcurrentHashMap(); - // private final Set jobInstances = new CopyOnWriteArraySet(); + private final Map jobInstances = new ConcurrentHashMap<>(); private JobKeyGenerator jobKeyGenerator = new DefaultJobKeyGenerator(); @@ -55,18 +58,20 @@ public JobInstance createJobInstance(String jobName, JobParameters jobParameters JobInstance jobInstance = new JobInstance(currentId.getAndIncrement(), jobName); jobInstance.incrementVersion(); - jobInstances.put(jobName + jobKeyGenerator.generateKey(jobParameters), jobInstance); + jobInstances.put(jobName + "|" + jobKeyGenerator.generateKey(jobParameters), jobInstance); return jobInstance; } @Override + @Nullable public JobInstance getJobInstance(String jobName, JobParameters jobParameters) { - return jobInstances.get(jobName + jobKeyGenerator.generateKey(jobParameters)); + return jobInstances.get(jobName + "|" + jobKeyGenerator.generateKey(jobParameters)); } @Override - public JobInstance getJobInstance(Long instanceId) { + @Nullable + public JobInstance getJobInstance(@Nullable Long instanceId) { for (Map.Entry instanceEntry : jobInstances.entrySet()) { JobInstance instance = instanceEntry.getValue(); if (instance.getId().equals(instanceId)) { @@ -78,7 +83,7 @@ public JobInstance getJobInstance(Long instanceId) { @Override public List getJobNames() { - List result = new ArrayList(); + List result = new ArrayList<>(); for (Map.Entry instanceEntry : jobInstances.entrySet()) { result.add(instanceEntry.getValue().getJobName()); } @@ -88,29 +93,83 @@ public List getJobNames() { @Override public List getJobInstances(String jobName, int start, int count) { - List result = new ArrayList(); + List result = new ArrayList<>(); for (Map.Entry instanceEntry : jobInstances.entrySet()) { JobInstance instance = instanceEntry.getValue(); if (instance.getJobName().equals(jobName)) { result.add(instance); } } + + sortDescending(result); + + return subset(result, start, count); + } + + @Override + @Nullable + public JobInstance getLastJobInstance(String jobName) { + List jobInstances = getJobInstances(jobName, 0, 1); + return jobInstances.isEmpty() ? null : jobInstances.get(0); + } + + @Override + @Nullable + public JobInstance getJobInstance(JobExecution jobExecution) { + return jobExecution.getJobInstance(); + } + + @Override + public int getJobInstanceCount(@Nullable String jobName) throws NoSuchJobException { + int count = 0; + + for (Map.Entry instanceEntry : jobInstances.entrySet()) { + String key = instanceEntry.getKey(); + String curJobName = key.substring(0, key.lastIndexOf("|")); + + if(curJobName.equals(jobName)) { + count++; + } + } + + if(count == 0) { + throw new NoSuchJobException("No job instances for job name " + jobName + " were found"); + } else { + return count; + } + } + + @Override + public List findJobInstancesByName(String jobName, int start, int count) { + List result = new ArrayList<>(); + String convertedJobName = jobName.replaceAll(STAR_WILDCARD, STAR_WILDCARD_PATTERN); + + for (Map.Entry instanceEntry : jobInstances.entrySet()) { + JobInstance instance = instanceEntry.getValue(); + + if(instance.getJobName().matches(convertedJobName)) { + result.add(instance); + } + } + + sortDescending(result); + + return subset(result, start, count); + } + + private void sortDescending(List result) { Collections.sort(result, new Comparator() { - // sort by ID descending @Override public int compare(JobInstance o1, JobInstance o2) { return Long.signum(o2.getId() - o1.getId()); } }); - - int startIndex = Math.min(start, result.size()); - int endIndex = Math.min(start + count, result.size()); - return result.subList(startIndex, endIndex); } - @Override - public JobInstance getJobInstance(JobExecution jobExecution) { - return jobExecution.getJobInstance(); - } + private List subset(List jobInstances, int start, int count) { + int startIndex = Math.min(start, jobInstances.size()); + int endIndex = Math.min(start + count, jobInstances.size()); + return jobInstances.subList(startIndex, endIndex); + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapStepExecutionDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapStepExecutionDao.java index 69f69d5925..5b3154a5fb 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapStepExecutionDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/MapStepExecutionDao.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,20 +27,22 @@ import org.springframework.batch.core.Entity; import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.StepExecution; -import org.springframework.batch.support.SerializationUtils; import org.springframework.dao.OptimisticLockingFailureException; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.ReflectionUtils; +import org.springframework.util.SerializationUtils; /** * In-memory implementation of {@link StepExecutionDao}. */ public class MapStepExecutionDao implements StepExecutionDao { - private Map> executionsByJobExecutionId = new ConcurrentHashMap>(); + private Map> executionsByJobExecutionId = new ConcurrentHashMap<>(); - private Map executionsByStepExecutionId = new ConcurrentHashMap(); + private Map executionsByStepExecutionId = new ConcurrentHashMap<>(); private AtomicLong currentId = new AtomicLong(); @@ -62,19 +64,19 @@ public void doWith(Field field) throws IllegalArgumentException, IllegalAccessEx field.setAccessible(true); field.set(targetExecution, field.get(sourceExecution)); } - }); + }, ReflectionUtils.COPYABLE_FIELDS); } @Override public void saveStepExecution(StepExecution stepExecution) { - Assert.isTrue(stepExecution.getId() == null); - Assert.isTrue(stepExecution.getVersion() == null); + Assert.isTrue(stepExecution.getId() == null, "stepExecution id was not null"); + Assert.isTrue(stepExecution.getVersion() == null, "stepExecution version was not null"); Assert.notNull(stepExecution.getJobExecutionId(), "JobExecution must be saved already."); Map executions = executionsByJobExecutionId.get(stepExecution.getJobExecutionId()); if (executions == null) { - executions = new ConcurrentHashMap(); + executions = new ConcurrentHashMap<>(); executionsByJobExecutionId.put(stepExecution.getJobExecutionId(), executions); } @@ -89,7 +91,7 @@ public void saveStepExecution(StepExecution stepExecution) { @Override public void updateStepExecution(StepExecution stepExecution) { - Assert.notNull(stepExecution.getJobExecutionId()); + Assert.notNull(stepExecution.getJobExecutionId(), "jobExecution id is null"); Map executions = executionsByJobExecutionId.get(stepExecution.getJobExecutionId()); Assert.notNull(executions, "step executions for given job execution are expected to be already saved"); @@ -113,17 +115,41 @@ public void updateStepExecution(StepExecution stepExecution) { } @Override + @Nullable public StepExecution getStepExecution(JobExecution jobExecution, Long stepExecutionId) { return executionsByStepExecutionId.get(stepExecutionId); } + @Override + public StepExecution getLastStepExecution(JobInstance jobInstance, String stepName) { + StepExecution latest = null; + for (StepExecution stepExecution : executionsByStepExecutionId.values()) { + if (!stepExecution.getStepName().equals(stepName) + || stepExecution.getJobExecution().getJobInstance().getInstanceId() != jobInstance.getInstanceId()) { + continue; + } + if (latest == null) { + latest = stepExecution; + } + if (latest.getStartTime().getTime() < stepExecution.getStartTime().getTime()) { + latest = stepExecution; + } + // Use step execution ID as the tie breaker if start time is identical + if (latest.getStartTime().getTime() == stepExecution.getStartTime().getTime() && + latest.getId() < stepExecution.getId()) { + latest = stepExecution; + } + } + return latest; + } + @Override public void addStepExecutions(JobExecution jobExecution) { Map executions = executionsByJobExecutionId.get(jobExecution.getId()); if (executions == null || executions.isEmpty()) { return; } - List result = new ArrayList(executions.values()); + List result = new ArrayList<>(executions.values()); Collections.sort(result, new Comparator() { @Override @@ -132,7 +158,7 @@ public int compare(Entity o1, Entity o2) { } }); - List copy = new ArrayList(result.size()); + List copy = new ArrayList<>(result.size()); for (StepExecution exec : result) { copy.add(copy(exec)); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/NoSuchObjectException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/NoSuchObjectException.java index 98dcdbecf9..80fad9ce6b 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/NoSuchObjectException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/NoSuchObjectException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/StepExecutionDao.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/StepExecutionDao.java index f1e46e5493..cedcb82372 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/StepExecutionDao.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/StepExecutionDao.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,7 +19,9 @@ import java.util.Collection; import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.StepExecution; +import org.springframework.lang.Nullable; public interface StepExecutionDao { @@ -30,7 +32,7 @@ public interface StepExecutionDao { * * Postconditions: Id will be set to a unique Long. * - * @param stepExecution + * @param stepExecution {@link StepExecution} instance to be saved. */ void saveStepExecution(StepExecution stepExecution); @@ -41,7 +43,7 @@ public interface StepExecutionDao { * * Postconditions: StepExecution Id will be set to a unique Long. * - * @param stepExecutions + * @param stepExecutions a collection of {@link JobExecution} instances to be saved. */ void saveStepExecutions(Collection stepExecutions); @@ -50,7 +52,7 @@ public interface StepExecutionDao { * * Preconditions: Id must not be null. * - * @param stepExecution + * @param stepExecution {@link StepExecution} instance to be updated. */ void updateStepExecution(StepExecution stepExecution); @@ -61,8 +63,22 @@ public interface StepExecutionDao { * @param stepExecutionId the step execution id * @return a {@link StepExecution} */ + @Nullable StepExecution getStepExecution(JobExecution jobExecution, Long stepExecutionId); + /** + * Retrieve the last {@link StepExecution} for a given {@link JobInstance} + * ordered by starting time and then id. + * + * @param jobInstance the parent {@link JobInstance} + * @param stepName the name of the step + * @return a {@link StepExecution} + */ + @Nullable + default StepExecution getLastStepExecution(JobInstance jobInstance, String stepName) { + throw new UnsupportedOperationException(); + } + /** * Retrieve all the {@link StepExecution} for the parent {@link JobExecution}. * diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/XStreamExecutionContextStringSerializer.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/XStreamExecutionContextStringSerializer.java index 4f050ebb61..f911d86e59 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/XStreamExecutionContextStringSerializer.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/XStreamExecutionContextStringSerializer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,12 @@ import java.io.InputStream; import java.io.InputStreamReader; import java.io.OutputStream; +import java.util.Map; + +import com.thoughtworks.xstream.XStream; +import com.thoughtworks.xstream.converters.reflection.ReflectionProvider; +import com.thoughtworks.xstream.io.HierarchicalStreamDriver; +import com.thoughtworks.xstream.io.json.JettisonMappedXmlDriver; import org.springframework.batch.core.repository.ExecutionContextSerializer; import org.springframework.beans.factory.InitializingBean; @@ -28,19 +34,19 @@ import org.springframework.core.serializer.Serializer; import org.springframework.util.Assert; -import com.thoughtworks.xstream.XStream; -import com.thoughtworks.xstream.converters.reflection.ReflectionProvider; -import com.thoughtworks.xstream.io.HierarchicalStreamDriver; -import com.thoughtworks.xstream.io.json.JettisonMappedXmlDriver; - /** * Implementation that uses XStream and Jettison to provide serialization. * * @author Thomas Risberg * @author Michael Minella + * @author Mahmoud Ben Hassine * @since 2.0 * @see ExecutionContextSerializer + * @deprecated Due to the incompatibilities between current Jettison versions and XStream + * versions, this serializer is deprecated in favor of + * {@link Jackson2ExecutionContextStringSerializer} */ +@Deprecated public class XStreamExecutionContextStringSerializer implements ExecutionContextSerializer, InitializingBean { private ReflectionProvider reflectionProvider = null; @@ -77,14 +83,16 @@ public synchronized void init() throws Exception { /** * Serializes the passed execution context to the supplied OutputStream. * - * @param context - * @param out + * @param context {@link Map} containing the context information. + * @param out {@link OutputStream} where the serialized context information + * will be written. + * * @see Serializer#serialize(Object, OutputStream) */ @Override - public void serialize(Object context, OutputStream out) throws IOException { - Assert.notNull(context); - Assert.notNull(out); + public void serialize(Map context, OutputStream out) throws IOException { + Assert.notNull(context, "context is required"); + Assert.notNull(out, "An OutputStream is required"); out.write(xstream.toXML(context).getBytes()); } @@ -92,12 +100,14 @@ public void serialize(Object context, OutputStream out) throws IOException { /** * Deserializes the supplied input stream into a new execution context. * - * @param in + * @param in {@link InputStream} containing the information to be deserialized. + * @return a reconstructed execution context * @see Deserializer#deserialize(InputStream) */ + @SuppressWarnings("unchecked") @Override - public Object deserialize(InputStream in) throws IOException { + public Map deserialize(InputStream in) throws IOException { BufferedReader br = new BufferedReader(new InputStreamReader(in)); StringBuilder sb = new StringBuilder(); @@ -107,6 +117,6 @@ public Object deserialize(InputStream in) throws IOException { sb.append(line); } - return xstream.fromXML(sb.toString()); + return (Map) xstream.fromXML(sb.toString()); } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/package-info.java new file mode 100644 index 0000000000..88e0759c1e --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/package-info.java @@ -0,0 +1,10 @@ +/** + * Specific implementations of dao concerns. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.repository.dao; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/package.html deleted file mode 100644 index 480b83c4ff..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/dao/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Specific implementations of dao concerns. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/package-info.java new file mode 100644 index 0000000000..46105e337d --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/package-info.java @@ -0,0 +1,10 @@ +/** + * Interfaces and generic implementations of repository concerns. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.repository; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/package.html deleted file mode 100644 index 88ab2c6da8..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Interfaces and generic implementations of repository concerns. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/AbstractJobRepositoryFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/AbstractJobRepositoryFactoryBean.java index f6a10fb11b..778e466044 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/AbstractJobRepositoryFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/AbstractJobRepositoryFactoryBean.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -46,8 +46,7 @@ * @author Lucas Ward * @author Robert Kasanicky */ -@SuppressWarnings("rawtypes") -public abstract class AbstractJobRepositoryFactoryBean implements FactoryBean, InitializingBean { +public abstract class AbstractJobRepositoryFactoryBean implements FactoryBean, InitializingBean { private PlatformTransactionManager transactionManager; @@ -64,21 +63,29 @@ public abstract class AbstractJobRepositoryFactoryBean implements FactoryBean, I /** * @return fully configured {@link JobInstanceDao} implementation. + * + * @throws Exception thrown if error occurs creating JobInstanceDao. */ protected abstract JobInstanceDao createJobInstanceDao() throws Exception; /** * @return fully configured {@link JobExecutionDao} implementation. + * + * @throws Exception thrown if error occurs creating JobExecutionDao. */ protected abstract JobExecutionDao createJobExecutionDao() throws Exception; /** * @return fully configured {@link StepExecutionDao} implementation. + * + * @throws Exception thrown if error occurs creating StepExecutionDao. */ protected abstract StepExecutionDao createStepExecutionDao() throws Exception; /** * @return fully configured {@link ExecutionContextDao} implementation. + * + * @throws Exception thrown if error occurs creating ExecutionContextDao. */ protected abstract ExecutionContextDao createExecutionContextDao() throws Exception; @@ -148,9 +155,11 @@ public PlatformTransactionManager getTransactionManager() { * a cast. * @return the {@link JobRepository} from {@link #getObject()} * @throws Exception if the repository could not be created + * @deprecated use {@link #getObject()} instead */ + @Deprecated public JobRepository getJobRepository() throws Exception { - return (JobRepository) getObject(); + return getObject(); } private void initializeProxy() throws Exception { @@ -197,11 +206,11 @@ private Object getTarget() throws Exception { } @Override - public Object getObject() throws Exception { + public JobRepository getObject() throws Exception { if (proxyFactory == null) { afterPropertiesSet(); } - return proxyFactory.getProxy(); + return (JobRepository) proxyFactory.getProxy(getClass().getClassLoader()); } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/JobRepositoryFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/JobRepositoryFactoryBean.java index 1407429b2b..3fa06d23dd 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/JobRepositoryFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/JobRepositoryFactoryBean.java @@ -1,11 +1,11 @@ /* - * Copyright 2002-2007 the original author or authors. + * Copyright 2002-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,17 +16,17 @@ package org.springframework.batch.core.repository.support; -import static org.springframework.batch.support.DatabaseType.SYBASE; - +import java.lang.reflect.Field; import java.sql.Types; - import javax.sql.DataSource; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.batch.core.repository.ExecutionContextSerializer; import org.springframework.batch.core.repository.dao.AbstractJdbcBatchMetadataDao; import org.springframework.batch.core.repository.dao.ExecutionContextDao; +import org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer; import org.springframework.batch.core.repository.dao.JdbcExecutionContextDao; import org.springframework.batch.core.repository.dao.JdbcJobExecutionDao; import org.springframework.batch.core.repository.dao.JdbcJobInstanceDao; @@ -34,7 +34,6 @@ import org.springframework.batch.core.repository.dao.JobExecutionDao; import org.springframework.batch.core.repository.dao.JobInstanceDao; import org.springframework.batch.core.repository.dao.StepExecutionDao; -import org.springframework.batch.core.repository.dao.XStreamExecutionContextStringSerializer; import org.springframework.batch.item.database.support.DataFieldMaxValueIncrementerFactory; import org.springframework.batch.item.database.support.DefaultDataFieldMaxValueIncrementerFactory; import org.springframework.batch.support.DatabaseType; @@ -42,11 +41,13 @@ import org.springframework.beans.factory.InitializingBean; import org.springframework.jdbc.core.JdbcOperations; import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.jdbc.support.lob.DefaultLobHandler; import org.springframework.jdbc.support.lob.LobHandler; -import org.springframework.jdbc.support.lob.OracleLobHandler; import org.springframework.util.Assert; import org.springframework.util.StringUtils; +import static org.springframework.batch.support.DatabaseType.SYBASE; + /** * A {@link FactoryBean} that automates the creation of a * {@link SimpleJobRepository} using JDBC DAO implementations which persist @@ -64,7 +65,7 @@ public class JobRepositoryFactoryBean extends AbstractJobRepositoryFactoryBean i private DataSource dataSource; - private JdbcOperations jdbcTemplate; + private JdbcOperations jdbcOperations; private String databaseType; @@ -78,11 +79,20 @@ public class JobRepositoryFactoryBean extends AbstractJobRepositoryFactoryBean i private ExecutionContextSerializer serializer; + private Integer lobType; + + /** + * @param type a value from the {@link java.sql.Types} class to indicate the type to use for a CLOB + */ + public void setClobType(int type) { + this.lobType = type; + } + /** * A custom implementation of the {@link ExecutionContextSerializer}. - * The default, if not injected, is the {@link XStreamExecutionContextStringSerializer}. + * The default, if not injected, is the {@link Jackson2ExecutionContextStringSerializer}. * - * @param serializer + * @param serializer used to serialize/deserialize {@link org.springframework.batch.item.ExecutionContext} * @see ExecutionContextSerializer */ public void setSerializer(ExecutionContextSerializer serializer) { @@ -125,6 +135,15 @@ public void setMaxVarCharLength(int maxVarCharLength) { public void setDataSource(DataSource dataSource) { this.dataSource = dataSource; } + + /** + * Public setter for the {@link JdbcOperations}. If this property is not set explicitly, + * a new {@link JdbcTemplate} will be created for the configured DataSource by default. + * @param jdbcOperations a {@link JdbcOperations} + */ + public void setJdbcOperations(JdbcOperations jdbcOperations) { + this.jdbcOperations = jdbcOperations; + } /** * Sets the database type. @@ -137,7 +156,7 @@ public void setDatabaseType(String dbType) { /** * Sets the table prefix for all the batch meta-data tables. - * @param tablePrefix + * @param tablePrefix prefix prepended to batch meta-data tables */ public void setTablePrefix(String tablePrefix) { this.tablePrefix = tablePrefix; @@ -152,7 +171,9 @@ public void afterPropertiesSet() throws Exception { Assert.notNull(dataSource, "DataSource must not be null."); - jdbcTemplate = new JdbcTemplate(dataSource); + if (jdbcOperations == null) { + jdbcOperations = new JdbcTemplate(dataSource); + } if (incrementerFactory == null) { incrementerFactory = new DefaultDataFieldMaxValueIncrementerFactory(dataSource); @@ -164,12 +185,11 @@ public void afterPropertiesSet() throws Exception { } if (lobHandler == null && databaseType.equalsIgnoreCase(DatabaseType.ORACLE.toString())) { - lobHandler = new OracleLobHandler(); + lobHandler = new DefaultLobHandler(); } if(serializer == null) { - XStreamExecutionContextStringSerializer defaultSerializer = new XStreamExecutionContextStringSerializer(); - defaultSerializer.afterPropertiesSet(); + Jackson2ExecutionContextStringSerializer defaultSerializer = new Jackson2ExecutionContextStringSerializer(); serializer = defaultSerializer; } @@ -178,13 +198,17 @@ public void afterPropertiesSet() throws Exception { + "' is an unsupported database type. The supported database types are " + StringUtils.arrayToCommaDelimitedString(incrementerFactory.getSupportedIncrementerTypes())); + if(lobType != null) { + Assert.isTrue(isValidTypes(lobType), "lobType must be a value from the java.sql.Types class"); + } + super.afterPropertiesSet(); } @Override protected JobInstanceDao createJobInstanceDao() throws Exception { JdbcJobInstanceDao dao = new JdbcJobInstanceDao(); - dao.setJdbcTemplate(jdbcTemplate); + dao.setJdbcTemplate(jdbcOperations); dao.setJobIncrementer(incrementerFactory.getIncrementer(databaseType, tablePrefix + "JOB_SEQ")); dao.setTablePrefix(tablePrefix); dao.afterPropertiesSet(); @@ -194,7 +218,7 @@ protected JobInstanceDao createJobInstanceDao() throws Exception { @Override protected JobExecutionDao createJobExecutionDao() throws Exception { JdbcJobExecutionDao dao = new JdbcJobExecutionDao(); - dao.setJdbcTemplate(jdbcTemplate); + dao.setJdbcTemplate(jdbcOperations); dao.setJobExecutionIncrementer(incrementerFactory.getIncrementer(databaseType, tablePrefix + "JOB_EXECUTION_SEQ")); dao.setTablePrefix(tablePrefix); @@ -207,7 +231,7 @@ protected JobExecutionDao createJobExecutionDao() throws Exception { @Override protected StepExecutionDao createStepExecutionDao() throws Exception { JdbcStepExecutionDao dao = new JdbcStepExecutionDao(); - dao.setJdbcTemplate(jdbcTemplate); + dao.setJdbcTemplate(jdbcOperations); dao.setStepExecutionIncrementer(incrementerFactory.getIncrementer(databaseType, tablePrefix + "STEP_EXECUTION_SEQ")); dao.setTablePrefix(tablePrefix); @@ -220,7 +244,7 @@ protected StepExecutionDao createStepExecutionDao() throws Exception { @Override protected ExecutionContextDao createExecutionContextDao() throws Exception { JdbcExecutionContextDao dao = new JdbcExecutionContextDao(); - dao.setJdbcTemplate(jdbcTemplate); + dao.setJdbcTemplate(jdbcOperations); dao.setTablePrefix(tablePrefix); dao.setClobTypeToUse(determineClobTypeToUse(this.databaseType)); dao.setSerializer(serializer); @@ -235,13 +259,31 @@ protected ExecutionContextDao createExecutionContextDao() throws Exception { return dao; } - private int determineClobTypeToUse(String databaseType) { - if (SYBASE == DatabaseType.valueOf(databaseType.toUpperCase())) { - return Types.LONGVARCHAR; + private int determineClobTypeToUse(String databaseType) throws Exception { + if(lobType != null) { + return lobType; + } else { + if (SYBASE == DatabaseType.valueOf(databaseType.toUpperCase())) { + return Types.LONGVARCHAR; + } + else { + return Types.CLOB; + } } - else { - return Types.CLOB; + } + + private boolean isValidTypes(int value) throws Exception { + boolean result = false; + + for (Field field : Types.class.getFields()) { + int curValue = field.getInt(null); + if(curValue == value) { + result = true; + break; + } } + + return result; } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/MapJobRepositoryFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/MapJobRepositoryFactoryBean.java index 2d15845404..1e7870d763 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/MapJobRepositoryFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/MapJobRepositoryFactoryBean.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -60,7 +60,7 @@ public MapJobRepositoryFactoryBean() { /** * Create a new instance with the provided transaction manager. * - * @param transactionManager + * @param transactionManager {@link org.springframework.transaction.PlatformTransactionManager} */ public MapJobRepositoryFactoryBean(PlatformTransactionManager transactionManager) { setTransactionManager(transactionManager); @@ -83,7 +83,7 @@ public ExecutionContextDao getExecutionContextDao() { } /** - * Convenience method to clear all the map daos globally, removing all + * Convenience method to clear all the map DAOs globally, removing all * entities. */ public void clear() { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/SimpleJobRepository.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/SimpleJobRepository.java index a54a952550..be7effa317 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/SimpleJobRepository.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/SimpleJobRepository.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,11 +16,6 @@ package org.springframework.batch.core.repository.support; -import java.util.ArrayList; -import java.util.Collection; -import java.util.Date; -import java.util.List; - import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.core.BatchStatus; @@ -37,8 +32,13 @@ import org.springframework.batch.core.repository.dao.JobInstanceDao; import org.springframework.batch.core.repository.dao.StepExecutionDao; import org.springframework.batch.item.ExecutionContext; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; +import java.util.Collection; +import java.util.Date; +import java.util.List; + /** * *

      @@ -50,6 +50,7 @@ * @author Dave Syer * @author Robert Kasanicky * @author David Turanski + * @author Mahmoud Ben Hassine * * @see JobRepository * @see JobInstanceDao @@ -116,13 +117,17 @@ public JobExecution createJobExecution(String jobName, JobParameters jobParamete // check for running executions and find the last started for (JobExecution execution : executions) { - if (execution.isRunning()) { + if (execution.isRunning() || execution.isStopping()) { throw new JobExecutionAlreadyRunningException("A job execution for this job is already running: " + jobInstance); } - BatchStatus status = execution.getStatus(); - if (status == BatchStatus.COMPLETED || status == BatchStatus.ABANDONED) { + if (status == BatchStatus.UNKNOWN) { + throw new JobRestartException("Cannot restart job from UNKNOWN status. " + + "The last execution ended with a failure that could not be rolled back, " + + "so it may be dangerous to proceed. Manual intervention is probably necessary."); + } + if (execution.getJobParameters().getParameters().size() > 0 && (status == BatchStatus.COMPLETED || status == BatchStatus.ABANDONED)) { throw new JobInstanceAlreadyCompleteException( "A job instance already exists and is complete for parameters=" + jobParameters + ". If you want to run this job again, change the parameters."); @@ -136,7 +141,7 @@ public JobExecution createJobExecution(String jobName, JobParameters jobParamete executionContext = new ExecutionContext(); } - JobExecution jobExecution = new JobExecution(jobInstance, jobParameters); + JobExecution jobExecution = new JobExecution(jobInstance, jobParameters, null); jobExecution.setExecutionContext(executionContext); jobExecution.setLastUpdated(new Date(System.currentTimeMillis())); @@ -157,6 +162,8 @@ public void update(JobExecution jobExecution) { Assert.notNull(jobExecution.getId(), "JobExecution must be already saved (have an id assigned)."); jobExecution.setLastUpdated(new Date(System.currentTimeMillis())); + + jobExecutionDao.synchronizeStatus(jobExecution); jobExecutionDao.updateJobExecution(jobExecution); } @@ -209,32 +216,15 @@ public void updateExecutionContext(JobExecution jobExecution) { } @Override + @Nullable public StepExecution getLastStepExecution(JobInstance jobInstance, String stepName) { - List jobExecutions = jobExecutionDao.findJobExecutions(jobInstance); - List stepExecutions = new ArrayList(jobExecutions.size()); - - for (JobExecution jobExecution : jobExecutions) { - stepExecutionDao.addStepExecutions(jobExecution); - for (StepExecution stepExecution : jobExecution.getStepExecutions()) { - if (stepName.equals(stepExecution.getStepName())) { - stepExecutions.add(stepExecution); - } - } - } - - StepExecution latest = null; - for (StepExecution stepExecution : stepExecutions) { - if (latest == null) { - latest = stepExecution; - } - if (latest.getStartTime().getTime() < stepExecution.getStartTime().getTime()) { - latest = stepExecution; - } - } + StepExecution latest = stepExecutionDao.getLastStepExecution(jobInstance, stepName); if (latest != null) { - ExecutionContext executionContext = ecDao.getExecutionContext(latest); - latest.setExecutionContext(executionContext); + ExecutionContext stepExecutionContext = ecDao.getExecutionContext(latest); + latest.setExecutionContext(stepExecutionContext); + ExecutionContext jobExecutionContext = ecDao.getExecutionContext(latest.getJobExecution()); + latest.getJobExecution().setExecutionContext(jobExecutionContext); } return latest; @@ -258,7 +248,7 @@ public int getStepExecutionCount(JobInstance jobInstance, String stepName) { return count; } - /* + /** * Check to determine whether or not the JobExecution that is the parent of * the provided StepExecution has been interrupted. If, after synchronizing * the status with the database, the status has been updated to STOPPING, @@ -276,6 +266,7 @@ private void checkForInterruption(StepExecution stepExecution) { } @Override + @Nullable public JobExecution getLastJobExecution(String jobName, JobParameters jobParameters) { JobInstance jobInstance = jobInstanceDao.getJobInstance(jobName, jobParameters); if (jobInstance == null) { @@ -285,8 +276,39 @@ public JobExecution getLastJobExecution(String jobName, JobParameters jobParamet if (jobExecution != null) { jobExecution.setExecutionContext(ecDao.getExecutionContext(jobExecution)); + stepExecutionDao.addStepExecutions(jobExecution); } return jobExecution; } + + @Override + public JobInstance createJobInstance(String jobName, JobParameters jobParameters) { + Assert.notNull(jobName, "A job name is required to create a JobInstance"); + Assert.notNull(jobParameters, "Job parameters are required to create a JobInstance"); + + JobInstance jobInstance = jobInstanceDao.createJobInstance(jobName, jobParameters); + + return jobInstance; + } + + @Override + public JobExecution createJobExecution(JobInstance jobInstance, + JobParameters jobParameters, String jobConfigurationLocation) { + + Assert.notNull(jobInstance, "A JobInstance is required to associate the JobExecution with"); + Assert.notNull(jobParameters, "A JobParameters object is required to create a JobExecution"); + + JobExecution jobExecution = new JobExecution(jobInstance, jobParameters, jobConfigurationLocation); + ExecutionContext executionContext = new ExecutionContext(); + jobExecution.setExecutionContext(executionContext); + jobExecution.setLastUpdated(new Date(System.currentTimeMillis())); + + // Save the JobExecution so that it picks up an ID (useful for clients + // monitoring asynchronous executions): + jobExecutionDao.saveJobExecution(jobExecution); + ecDao.saveExecutionContext(jobExecution); + + return jobExecution; + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/package-info.java new file mode 100644 index 0000000000..434f78772b --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/package-info.java @@ -0,0 +1,10 @@ +/** + * Specific implementations of repository concerns. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.repository.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/package.html deleted file mode 100644 index d8f255db5c..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Specific implementations of repository concerns. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/resource/StepExecutionSimpleCompletionPolicy.java b/spring-batch-core/src/main/java/org/springframework/batch/core/resource/StepExecutionSimpleCompletionPolicy.java index ddcaa90a09..ccbdb833de 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/resource/StepExecutionSimpleCompletionPolicy.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/resource/StepExecutionSimpleCompletionPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -80,8 +80,6 @@ public void beforeStep(StepExecution stepExecution) { } /** - * @param context - * @param result * @return true if the commit interval has been reached or the result * indicates completion * @see CompletionPolicy#isComplete(RepeatContext, RepeatStatus) @@ -94,7 +92,6 @@ public boolean isComplete(RepeatContext context, RepeatStatus result) { } /** - * @param context * @return if the commit interval has been reached * @see org.springframework.batch.repeat.CompletionPolicy#isComplete(org.springframework.batch.repeat.RepeatContext) */ @@ -106,7 +103,6 @@ public boolean isComplete(RepeatContext context) { } /** - * @param parent * @return a new {@link RepeatContext} * @see org.springframework.batch.repeat.CompletionPolicy#start(org.springframework.batch.repeat.RepeatContext) */ @@ -118,7 +114,6 @@ public RepeatContext start(RepeatContext parent) { } /** - * @param context * @see org.springframework.batch.repeat.CompletionPolicy#update(org.springframework.batch.repeat.RepeatContext) */ @Override diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/BatchScopeSupport.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/BatchScopeSupport.java new file mode 100644 index 0000000000..e08f310ea0 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/BatchScopeSupport.java @@ -0,0 +1,226 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import org.springframework.aop.scope.ScopedProxyUtils; +import org.springframework.batch.core.scope.context.StepContext; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.config.BeanDefinitionHolder; +import org.springframework.beans.factory.config.BeanDefinitionVisitor; +import org.springframework.beans.factory.config.BeanFactoryPostProcessor; +import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; +import org.springframework.beans.factory.config.Scope; +import org.springframework.beans.factory.support.BeanDefinitionReaderUtils; +import org.springframework.beans.factory.support.BeanDefinitionRegistry; +import org.springframework.core.Ordered; +import org.springframework.util.Assert; +import org.springframework.util.StringValueResolver; + +/** + * ScopeSupport. + * + * @author Michael Minella + * @since 3.0 + */ +public abstract class BatchScopeSupport implements Scope, BeanFactoryPostProcessor, Ordered { + + private boolean autoProxy = true; + + private boolean proxyTargetClass = false; + + private String name; + + private int order = Ordered.LOWEST_PRECEDENCE; + + /** + * @param order the order value to set priority of callback execution for + * the {@link BeanFactoryPostProcessor} part of this scope bean. + */ + public void setOrder(int order) { + this.order = order; + } + + @Override + public int getOrder() { + return order; + } + + public String getName() { + return this.name; + } + + /** + * Public setter for the name property. This can then be used as a bean + * definition attribute, e.g. scope="job". + * + * @param name the name to set for this scope. + */ + public void setName(String name) { + this.name = name; + } + + /** + * Flag to indicate that proxies should use dynamic subclassing. This allows + * classes with no interface to be proxied. Defaults to false. + * + * @param proxyTargetClass set to true to have proxies created using dynamic + * subclasses + */ + public void setProxyTargetClass(boolean proxyTargetClass) { + this.proxyTargetClass = proxyTargetClass; + } + + /** + * Flag to indicate that bean definitions need not be auto proxied. This gives control back to the declarer of the + * bean definition (e.g. in an @Configuration class). + * + * @param autoProxy the flag value to set (default true) + */ + public void setAutoProxy(boolean autoProxy) { + this.autoProxy = autoProxy; + } + + public abstract String getTargetNamePrefix(); + + /** + * Register this scope with the enclosing BeanFactory. + * + * @see BeanFactoryPostProcessor#postProcessBeanFactory(ConfigurableListableBeanFactory) + * + * @param beanFactory the BeanFactory to register with + * @throws BeansException if there is a problem. + */ + @Override + public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException { + + beanFactory.registerScope(name, this); + + if(!autoProxy) { + return; + } + + Assert.state(beanFactory instanceof BeanDefinitionRegistry, + "BeanFactory was not a BeanDefinitionRegistry, so JobScope cannot be used."); + BeanDefinitionRegistry registry = (BeanDefinitionRegistry) beanFactory; + + for (String beanName : beanFactory.getBeanDefinitionNames()) { + if (!beanName.startsWith(getTargetNamePrefix())) { + BeanDefinition definition = beanFactory.getBeanDefinition(beanName); + // Replace this or any of its inner beans with scoped proxy if it + // has this scope + boolean scoped = name.equals(definition.getScope()); + Scopifier scopifier = new Scopifier(registry, name, proxyTargetClass, scoped); + scopifier.visitBeanDefinition(definition); + + if (scoped && !definition.isAbstract()) { + createScopedProxy(beanName, definition, registry, proxyTargetClass); + } + } + } + + } + + /** + * Wrap a target bean definition in a proxy that defers initialization until + * after the {@link StepContext} is available. Amounts to adding + * <aop-auto-proxy/> to a step scoped bean. + * + * @param beanName the bean name to replace + * @param definition the bean definition to replace + * @param registry the enclosing {@link BeanDefinitionRegistry} + * @param proxyTargetClass true if we need to force use of dynamic + * subclasses + * @return a {@link BeanDefinitionHolder} for the new representation of the + * target. Caller should register it if needed to be visible at top level in + * bean factory. + */ + protected static BeanDefinitionHolder createScopedProxy(String beanName, BeanDefinition definition, + BeanDefinitionRegistry registry, boolean proxyTargetClass) { + + BeanDefinitionHolder proxyHolder; + + proxyHolder = ScopedProxyUtils.createScopedProxy(new BeanDefinitionHolder(definition, beanName), registry, + proxyTargetClass); + + registry.registerBeanDefinition(beanName, proxyHolder.getBeanDefinition()); + + return proxyHolder; + + } + + /** + * Helper class to scan a bean definition hierarchy and force the use of + * auto-proxy for step scoped beans. + * + * @author Dave Syer + * + */ + protected static class Scopifier extends BeanDefinitionVisitor { + + private final boolean proxyTargetClass; + + private final BeanDefinitionRegistry registry; + + private final String scope; + + private final boolean scoped; + + public Scopifier(BeanDefinitionRegistry registry, String scope, boolean proxyTargetClass, boolean scoped) { + super(new StringValueResolver() { + @Override + public String resolveStringValue(String value) { + return value; + } + }); + this.registry = registry; + this.proxyTargetClass = proxyTargetClass; + this.scope = scope; + this.scoped = scoped; + } + + @Override + protected Object resolveValue(Object value) { + + BeanDefinition definition = null; + String beanName = null; + if (value instanceof BeanDefinition) { + definition = (BeanDefinition) value; + beanName = BeanDefinitionReaderUtils.generateBeanName(definition, registry); + } + else if (value instanceof BeanDefinitionHolder) { + BeanDefinitionHolder holder = (BeanDefinitionHolder) value; + definition = holder.getBeanDefinition(); + beanName = holder.getBeanName(); + } + + if (definition != null) { + boolean nestedScoped = scope.equals(definition.getScope()); + boolean scopeChangeRequiresProxy = !scoped && nestedScoped; + if (scopeChangeRequiresProxy) { + // Exit here so that nested inner bean definitions are not + // analysed + return createScopedProxy(beanName, definition, registry, proxyTargetClass); + } + } + + // Nested inner bean definitions are recursively analysed here + value = super.resolveValue(value); + return value; + + } + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/JobScope.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/JobScope.java new file mode 100644 index 0000000000..6a591ef566 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/JobScope.java @@ -0,0 +1,168 @@ +/* + * Copyright 2006-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.core.scope.context.JobContext; +import org.springframework.batch.core.scope.context.JobSynchronizationManager; +import org.springframework.beans.BeanWrapper; +import org.springframework.beans.BeanWrapperImpl; +import org.springframework.beans.factory.ObjectFactory; +import org.springframework.beans.factory.config.Scope; + +/** + * Scope for job context. Objects in this scope use the Spring container as an + * object factory, so there is only one instance of such a bean per executing + * job. All objects in this scope are <aop:scoped-proxy/> (no need to + * decorate the bean definitions).
      + *
      + * + * In addition, support is provided for late binding of references accessible + * from the {@link JobContext} using #{..} placeholders. Using this feature, + * bean properties can be pulled from the job or job execution context and the + * job parameters. E.g. + * + *
      + * <bean id="..." class="..." scope="job">
      + * 	<property name="name" value="#{jobParameters[input]}" />
      + * </bean>
      + *
      + * <bean id="..." class="..." scope="job">
      + * 	<property name="name" value="#{jobExecutionContext['input.stem']}.txt" />
      + * </bean>
      + * 
      + * + * The {@link JobContext} is referenced using standard bean property paths (as + * per {@link BeanWrapper}). The examples above all show the use of the Map + * accessors provided as a convenience for job attributes. + * + * @author Dave Syer + * @author Jimmy Praet (create JobScope based on {@link StepScope}) + * @author Michael Minella + * @since 3.0 + */ +public class JobScope extends BatchScopeSupport { + + private static final String TARGET_NAME_PREFIX = "jobScopedTarget."; + + private Log logger = LogFactory.getLog(getClass()); + + private final Object mutex = new Object(); + + /** + * Context key for clients to use for conversation identifier. + */ + public static final String ID_KEY = "JOB_IDENTIFIER"; + + public JobScope() { + super(); + setName("job"); + } + + /** + * This will be used to resolve expressions in job-scoped beans. + */ + @Override + public Object resolveContextualObject(String key) { + JobContext context = getContext(); + // TODO: support for attributes as well maybe (setters not exposed yet + // so not urgent). + return new BeanWrapperImpl(context).getPropertyValue(key); + } + + /** + * @see Scope#get(String, ObjectFactory) + */ + @Override + public Object get(String name, ObjectFactory objectFactory) { + JobContext context = getContext(); + Object scopedObject = context.getAttribute(name); + + if (scopedObject == null) { + + synchronized (mutex) { + scopedObject = context.getAttribute(name); + if (scopedObject == null) { + + if (logger.isDebugEnabled()) { + logger.debug(String.format("Creating object in scope=%s, name=%s", this.getName(), name)); + } + + scopedObject = objectFactory.getObject(); + context.setAttribute(name, scopedObject); + + } + + } + + } + return scopedObject; + } + + /** + * @see Scope#getConversationId() + */ + @Override + public String getConversationId() { + JobContext context = getContext(); + return context.getId(); + } + + /** + * @see Scope#registerDestructionCallback(String, Runnable) + */ + @Override + public void registerDestructionCallback(String name, Runnable callback) { + JobContext context = getContext(); + if (logger.isDebugEnabled()) { + logger.debug(String.format("Registered destruction callback in scope=%s, name=%s", this.getName(), name)); + } + context.registerDestructionCallback(name, callback); + } + + /** + * @see Scope#remove(String) + */ + @Override + public Object remove(String name) { + JobContext context = getContext(); + if (logger.isDebugEnabled()) { + logger.debug(String.format("Removing from scope=%s, name=%s", this.getName(), name)); + } + return context.removeAttribute(name); + } + + /** + * Get an attribute accessor in the form of a {@link JobContext} that can + * be used to store scoped bean instances. + * + * @return the current job context which we can use as a scope storage + * medium + */ + private JobContext getContext() { + JobContext context = JobSynchronizationManager.getContext(); + if (context == null) { + throw new IllegalStateException("No context holder available for job scope"); + } + return context; + } + + @Override + public String getTargetNamePrefix() { + return TARGET_NAME_PREFIX; + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/StepScope.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/StepScope.java index 644b3617af..f014bf10eb 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/StepScope.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/StepScope.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,31 +17,19 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; -import org.springframework.aop.scope.ScopedProxyUtils; import org.springframework.batch.core.scope.context.StepContext; import org.springframework.batch.core.scope.context.StepSynchronizationManager; import org.springframework.beans.BeanWrapper; import org.springframework.beans.BeanWrapperImpl; -import org.springframework.beans.BeansException; import org.springframework.beans.factory.ObjectFactory; -import org.springframework.beans.factory.config.BeanDefinition; -import org.springframework.beans.factory.config.BeanDefinitionHolder; -import org.springframework.beans.factory.config.BeanDefinitionVisitor; -import org.springframework.beans.factory.config.BeanFactoryPostProcessor; -import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; import org.springframework.beans.factory.config.Scope; -import org.springframework.beans.factory.support.BeanDefinitionReaderUtils; -import org.springframework.beans.factory.support.BeanDefinitionRegistry; -import org.springframework.core.Ordered; -import org.springframework.util.Assert; -import org.springframework.util.StringValueResolver; /** * Scope for step context. Objects in this scope use the Spring container as an * object factory, so there is only one instance of such a bean per executing * step. All objects in this scope are <aop:scoped-proxy/> (no need to - * decorate the bean definitions).
      - *
      + * decorate the bean definitions).
      + *
      * * In addition, support is provided for late binding of references accessible * from the {@link StepContext} using #{..} placeholders. Using this feature, @@ -74,57 +62,22 @@ * @author Michael Minella * @since 2.0 */ -public class StepScope implements Scope, BeanFactoryPostProcessor, Ordered { +public class StepScope extends BatchScopeSupport { - private Log logger = LogFactory.getLog(getClass()); - - private int order = Ordered.LOWEST_PRECEDENCE; + private static final String TARGET_NAME_PREFIX = "stepScopedTarget."; - private boolean autoProxy = true; + private Log logger = LogFactory.getLog(getClass()); private final Object mutex = new Object(); - /** - * @param order the order value to set priority of callback execution for - * the {@link BeanFactoryPostProcessor} part of this scope bean. - */ - public void setOrder(int order) { - this.order = order; - } - - @Override - public int getOrder() { - return order; - } - /** * Context key for clients to use for conversation identifier. */ public static final String ID_KEY = "STEP_IDENTIFIER"; - private String name = "step"; - - private boolean proxyTargetClass = false; - - /** - * Flag to indicate that proxies should use dynamic subclassing. This allows - * classes with no interface to be proxied. Defaults to false. - * - * @param proxyTargetClass set to true to have proxies created using dynamic - * subclasses - */ - public void setProxyTargetClass(boolean proxyTargetClass) { - this.proxyTargetClass = proxyTargetClass; - } - - /** - * Flag to indicate that bean definitions need not be auto proxied. This gives control back to the declarer of the - * bean definition (e.g. in an @Configuration class). - * - * @param autoProxy the flag value to set (default true) - */ - public void setAutoProxy(boolean autoProxy) { - this.autoProxy = autoProxy; + public StepScope() { + super(); + setName("step"); } /** @@ -141,9 +94,8 @@ public Object resolveContextualObject(String key) { /** * @see Scope#get(String, ObjectFactory) */ - @SuppressWarnings("rawtypes") @Override - public Object get(String name, ObjectFactory objectFactory) { + public Object get(String name, ObjectFactory objectFactory) { StepContext context = getContext(); Object scopedObject = context.getAttribute(name); @@ -153,7 +105,10 @@ public Object get(String name, ObjectFactory objectFactory) { scopedObject = context.getAttribute(name); if (scopedObject == null) { - logger.debug(String.format("Creating object in scope=%s, name=%s", this.name, name)); + if (logger.isDebugEnabled()) { + logger.debug(String.format("Creating object in scope=%s, name=%s", this.getName(), name)); + } + scopedObject = objectFactory.getObject(); context.setAttribute(name, scopedObject); @@ -181,7 +136,9 @@ public String getConversationId() { @Override public void registerDestructionCallback(String name, Runnable callback) { StepContext context = getContext(); - logger.debug(String.format("Registered destruction callback in scope=%s, name=%s", this.name, name)); + if (logger.isDebugEnabled()) { + logger.debug(String.format("Registered destruction callback in scope=%s, name=%s", this.getName(), name)); + } context.registerDestructionCallback(name, callback); } @@ -191,7 +148,9 @@ public void registerDestructionCallback(String name, Runnable callback) { @Override public Object remove(String name) { StepContext context = getContext(); - logger.debug(String.format("Removing from scope=%s, name=%s", this.name, name)); + if (logger.isDebugEnabled()) { + logger.debug(String.format("Removing from scope=%s, name=%s", this.getName(), name)); + } return context.removeAttribute(name); } @@ -210,140 +169,8 @@ private StepContext getContext() { return context; } - /** - * Register this scope with the enclosing BeanFactory. - * - * @see BeanFactoryPostProcessor#postProcessBeanFactory(ConfigurableListableBeanFactory) - * - * @param beanFactory the BeanFactory to register with - * @throws BeansException if there is a problem. - */ @Override - public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException { - - beanFactory.registerScope(name, this); - - if(!autoProxy) { - return; - } - - Assert.state(beanFactory instanceof BeanDefinitionRegistry, - "BeanFactory was not a BeanDefinitionRegistry, so StepScope cannot be used."); - BeanDefinitionRegistry registry = (BeanDefinitionRegistry) beanFactory; - - for (String beanName : beanFactory.getBeanDefinitionNames()) { - BeanDefinition definition = beanFactory.getBeanDefinition(beanName); - // Replace this or any of its inner beans with scoped proxy if it - // has this scope - boolean scoped = name.equals(definition.getScope()); - Scopifier scopifier = new Scopifier(registry, name, proxyTargetClass, scoped); - scopifier.visitBeanDefinition(definition); - - if (scoped && !definition.isAbstract()) { - createScopedProxy(beanName, definition, registry, proxyTargetClass); - } - } - - } - - /** - * Public setter for the name property. This can then be used as a bean - * definition attribute, e.g. scope="step". Defaults to "step". - * - * @param name the name to set for this scope. - */ - public void setName(String name) { - this.name = name; - } - - /** - * Wrap a target bean definition in a proxy that defers initialization until - * after the {@link StepContext} is available. Amounts to adding - * <aop-auto-proxy/> to a step scoped bean. - * - * @param beanName the bean name to replace - * @param definition the bean definition to replace - * @param registry the enclosing {@link BeanDefinitionRegistry} - * @param proxyTargetClass true if we need to force use of dynamic - * subclasses - * @return a {@link BeanDefinitionHolder} for the new representation of the - * target. Caller should register it if needed to be visible at top level in - * bean factory. - */ - private static BeanDefinitionHolder createScopedProxy(String beanName, BeanDefinition definition, - BeanDefinitionRegistry registry, boolean proxyTargetClass) { - - BeanDefinitionHolder proxyHolder; - - proxyHolder = ScopedProxyUtils.createScopedProxy(new BeanDefinitionHolder(definition, beanName), registry, - proxyTargetClass); - - registry.registerBeanDefinition(beanName, proxyHolder.getBeanDefinition()); - - return proxyHolder; - - } - - /** - * Helper class to scan a bean definition hierarchy and force the use of - * auto-proxy for step scoped beans. - * - * @author Dave Syer - * - */ - private static class Scopifier extends BeanDefinitionVisitor { - - private final boolean proxyTargetClass; - - private final BeanDefinitionRegistry registry; - - private final String scope; - - private final boolean scoped; - - public Scopifier(BeanDefinitionRegistry registry, String scope, boolean proxyTargetClass, boolean scoped) { - super(new StringValueResolver() { - @Override - public String resolveStringValue(String value) { - return value; - } - }); - this.registry = registry; - this.proxyTargetClass = proxyTargetClass; - this.scope = scope; - this.scoped = scoped; - } - - @Override - protected Object resolveValue(Object value) { - - BeanDefinition definition = null; - String beanName = null; - if (value instanceof BeanDefinition) { - definition = (BeanDefinition) value; - beanName = BeanDefinitionReaderUtils.generateBeanName(definition, registry); - } - else if (value instanceof BeanDefinitionHolder) { - BeanDefinitionHolder holder = (BeanDefinitionHolder) value; - definition = holder.getBeanDefinition(); - beanName = holder.getBeanName(); - } - - if (definition != null) { - boolean nestedScoped = scope.equals(definition.getScope()); - boolean scopeChangeRequiresProxy = !scoped && nestedScoped; - if (scopeChangeRequiresProxy) { - // Exit here so that nested inner bean definitions are not - // analysed - return createScopedProxy(beanName, definition, registry, proxyTargetClass); - } - } - - // Nested inner bean definitions are recursively analysed here - value = super.resolveValue(value); - return value; - - } - + public String getTargetNamePrefix() { + return TARGET_NAME_PREFIX; } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/ChunkContext.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/ChunkContext.java index 26831dfb66..0aac72ec6f 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/ChunkContext.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/ChunkContext.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class ChunkContext extends AttributeAccessorSupport { private final StepContext stepContext; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/JobContext.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/JobContext.java new file mode 100644 index 0000000000..0384327026 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/JobContext.java @@ -0,0 +1,239 @@ +/* + * Copyright 2006-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope.context; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Map.Entry; +import java.util.Properties; +import java.util.Set; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameter; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.UnexpectedJobExecutionException; +import org.springframework.batch.core.scope.StepScope; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.repeat.context.SynchronizedAttributeAccessor; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * A context object that can be used to interrogate the current {@link JobExecution} and some of its associated + * properties using expressions + * based on bean paths. Has public getters for the job execution and + * convenience methods for accessing commonly used properties like the {@link ExecutionContext} associated with the job + * execution. + * + * @author Dave Syer + * @author Jimmy Praet (create JobContext based on {@link StepContext}) + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class JobContext extends SynchronizedAttributeAccessor { + + private JobExecution jobExecution; + + private Map> callbacks = new HashMap<>(); + + public JobContext(JobExecution jobExecution) { + super(); + Assert.notNull(jobExecution, "A JobContext must have a non-null JobExecution"); + this.jobExecution = jobExecution; + } + + /** + * Convenient accessor for current job name identifier. + * + * @return the job name identifier of the enclosing {@link JobInstance} associated with the current + * {@link JobExecution} + */ + public String getJobName() { + Assert.state(jobExecution.getJobInstance() != null, "JobExecution does not have a JobInstance"); + return jobExecution.getJobInstance().getJobName(); + } + + /** + * Convenient accessor for System properties to make it easy to access them + * from placeholder expressions. + * + * @return the current System properties + */ + public Properties getSystemProperties() { + return System.getProperties(); + } + + /** + * @return a map containing the items from the job {@link ExecutionContext} + */ + public Map getJobExecutionContext() { + Map result = new HashMap<>(); + for (Entry entry : jobExecution.getExecutionContext().entrySet()) { + result.put(entry.getKey(), entry.getValue()); + } + return Collections.unmodifiableMap(result); + } + + /** + * @return a map containing the items from the {@link JobParameters} + */ + public Map getJobParameters() { + Map result = new HashMap<>(); + for (Entry entry : jobExecution.getJobParameters().getParameters() + .entrySet()) { + result.put(entry.getKey(), entry.getValue().getValue()); + } + return Collections.unmodifiableMap(result); + } + + /** + * Allow clients to register callbacks for clean up on close. + * + * @param name + * the callback id (unique attribute key in this context) + * @param callback + * a callback to execute on close + */ + public void registerDestructionCallback(String name, Runnable callback) { + synchronized (callbacks) { + Set set = callbacks.get(name); + if (set == null) { + set = new HashSet<>(); + callbacks.put(name, set); + } + set.add(callback); + } + } + + private void unregisterDestructionCallbacks(String name) { + synchronized (callbacks) { + callbacks.remove(name); + } + } + + /** + * Override base class behaviour to ensure destruction callbacks are + * unregistered as well as the default behaviour. + * + * @see SynchronizedAttributeAccessor#removeAttribute(String) + */ + @Override + @Nullable + public Object removeAttribute(String name) { + unregisterDestructionCallbacks(name); + return super.removeAttribute(name); + } + + /** + * Clean up the context at the end of a step execution. Must be called once + * at the end of a step execution to honour the destruction callback + * contract from the {@link StepScope}. + */ + public void close() { + + List errors = new ArrayList<>(); + + Map> copy = Collections.unmodifiableMap(callbacks); + + for (Entry> entry : copy.entrySet()) { + Set set = entry.getValue(); + for (Runnable callback : set) { + if (callback != null) { + /* + * The documentation of the interface says that these + * callbacks must not throw exceptions, but we don't trust + * them necessarily... + */ + try { + callback.run(); + } catch (RuntimeException t) { + errors.add(t); + } + } + } + } + + if (errors.isEmpty()) { + return; + } + + Exception error = errors.get(0); + if (error instanceof RuntimeException) { + throw (RuntimeException) error; + } else { + throw new UnexpectedJobExecutionException("Could not close step context, rethrowing first of " + + errors.size() + " exceptions.", error); + } + } + + /** + * The current {@link JobExecution} that is active in this context. + * + * @return the current {@link JobExecution} + */ + public JobExecution getJobExecution() { + return jobExecution; + } + + /** + * @return unique identifier for this context based on the step execution + */ + public String getId() { + Assert.state(jobExecution.getId() != null, "JobExecution has no id. " + + "It must be saved before it can be used in job scope."); + return "jobExecution#" + jobExecution.getId(); + } + + /** + * Extend the base class method to include the job execution itself as a key + * (i.e. two contexts are only equal if their job executions are the same). + */ + @Override + public boolean equals(Object other) { + if (!(other instanceof JobContext)) { + return false; + } + if (other == this) { + return true; + } + JobContext context = (JobContext) other; + if (context.jobExecution == jobExecution) { + return true; + } + return jobExecution.equals(context.jobExecution); + } + + /** + * Overrides the default behaviour to provide a hash code based only on the + * job execution. + */ + @Override + public int hashCode() { + return jobExecution.hashCode(); + } + + @Override + public String toString() { + return super.toString() + ", jobExecutionContext=" + getJobExecutionContext() + ", jobParameters=" + + getJobParameters(); + } + +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/JobScopeManager.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/JobScopeManager.java new file mode 100644 index 0000000000..467ebafad7 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/JobScopeManager.java @@ -0,0 +1,47 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.core.scope.context; + +import org.aspectj.lang.annotation.Around; +import org.aspectj.lang.annotation.Aspect; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; + +/** + * Convenient aspect to wrap a single threaded job execution, where the + * implementation of the {@link Job} is not job scope aware (i.e. not the ones + * provided by the framework). + * + * @author Dave Syer + * @author Jimmy Praet + * @since 3.0 + */ +@Aspect +public class JobScopeManager { + + @Around("execution(void org.springframework.batch.core.Job+.execute(*)) && target(job) && args(jobExecution)") + public void execute(Job job, JobExecution jobExecution) { + JobSynchronizationManager.register(jobExecution); + try { + job.execute(jobExecution); + } + finally { + JobSynchronizationManager.release(); + } + } + +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/JobSynchronizationManager.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/JobSynchronizationManager.java new file mode 100644 index 0000000000..1870e9191a --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/JobSynchronizationManager.java @@ -0,0 +1,95 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope.context; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.lang.Nullable; + +/** + * Central convenience class for framework use in managing the job scope + * context. Generally only to be used by implementations of {@link Job}. N.B. + * it is the responsibility of every {@link Job} implementation to ensure that + * a {@link JobContext} is available on every thread that might be involved in + * a job execution, including worker threads from a pool. + * + * @author Dave Syer + * @author Jimmy Praet + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class JobSynchronizationManager { + + private static final SynchronizationManagerSupport manager = new SynchronizationManagerSupport() { + + @Override + protected JobContext createNewContext(JobExecution execution, @Nullable BatchPropertyContext args) { + return new JobContext(execution); + } + + @Override + protected void close(JobContext context) { + context.close(); + } + }; + + /** + * Getter for the current context if there is one, otherwise returns {@code null}. + * + * @return the current {@link JobContext} or {@code null} if there is none (if one + * has not been registered for this thread). + */ + @Nullable + public static JobContext getContext() { + return manager.getContext(); + } + + /** + * Register a context with the current thread - always put a matching + * {@link #close()} call in a finally block to ensure that the correct + * context is available in the enclosing block. + * + * @param JobExecution the step context to register + * @return a new {@link JobContext} or the current one if it has the same + * {@link JobExecution} + */ + public static JobContext register(JobExecution JobExecution) { + return manager.register(JobExecution); + } + + /** + * Method for unregistering the current context - should always and only be + * used by in conjunction with a matching {@link #register(JobExecution)} + * to ensure that {@link #getContext()} always returns the correct value. + * Does not call {@link JobContext#close()} - that is left up to the caller + * because he has a reference to the context (having registered it) and only + * he has knowledge of when the step actually ended. + */ + public static void close() { + manager.close(); + } + + /** + * A convenient "deep" close operation. Call this instead of + * {@link #close()} if the step execution for the current context is ending. + * Delegates to {@link JobContext#close()} and then ensures that + * {@link #close()} is also called in a finally block. + */ + public static void release() { + manager.release(); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepContext.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepContext.java index 1884604aef..99c35c66dc 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepContext.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepContext.java @@ -1,268 +1,310 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.core.scope.context; - -import java.util.ArrayList; -import java.util.Collections; -import java.util.HashMap; -import java.util.HashSet; -import java.util.List; -import java.util.Map; -import java.util.Properties; -import java.util.Set; -import java.util.Map.Entry; - -import org.springframework.batch.core.JobInstance; -import org.springframework.batch.core.JobParameter; -import org.springframework.batch.core.JobParameters; -import org.springframework.batch.core.StepExecution; -import org.springframework.batch.core.UnexpectedJobExecutionException; -import org.springframework.batch.core.scope.StepScope; -import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.repeat.context.SynchronizedAttributeAccessor; -import org.springframework.util.Assert; - -/** - * A context object that can be used to interrogate the current - * {@link StepExecution} and some of its associated properties using expressions - * based on bean paths. Has public getters for the step execution and - * convenience methods for accessing commonly used properties like the - * {@link ExecutionContext} associated with the step or its enclosing job - * execution. - * - * @author Dave Syer - * - */ -public class StepContext extends SynchronizedAttributeAccessor { - - private StepExecution stepExecution; - - private Map> callbacks = new HashMap>(); - - /** - * Create a new instance of {@link StepContext} for this - * {@link StepExecution}. - * - * @param stepExecution a step execution - */ - public StepContext(StepExecution stepExecution) { - super(); - Assert.notNull(stepExecution, "A StepContext must have a non-null StepExecution"); - this.stepExecution = stepExecution; - } - - /** - * Convenient accessor for current step name identifier. Usually this is the - * same as the bean name of the step that is executing (but might not be - * e.g. in a partition). - * - * @return the step name identifier of the current {@link StepExecution} - */ - public String getStepName() { - return stepExecution.getStepName(); - } - - /** - * Convenient accessor for current job name identifier. - * - * @return the job name identifier of the enclosing {@link JobInstance} - * associated with the current {@link StepExecution} - */ - public String getJobName() { - Assert.state(stepExecution.getJobExecution() != null, "StepExecution does not have a JobExecution"); - Assert.state(stepExecution.getJobExecution().getJobInstance() != null, - "StepExecution does not have a JobInstance"); - return stepExecution.getJobExecution().getJobInstance().getJobName(); - } - - /** - * Convenient accessor for System properties to make it easy to access them - * from placeholder expressions. - * - * @return the current System properties - */ - public Properties getSystemProperties() { - return System.getProperties(); - } - - /** - * @return a map containing the items from the step {@link ExecutionContext} - */ - public Map getStepExecutionContext() { - Map result = new HashMap(); - for (Entry entry : stepExecution.getExecutionContext().entrySet()) { - result.put(entry.getKey(), entry.getValue()); - } - return Collections.unmodifiableMap(result); - } - - /** - * @return a map containing the items from the job {@link ExecutionContext} - */ - public Map getJobExecutionContext() { - Map result = new HashMap(); - for (Entry entry : stepExecution.getJobExecution().getExecutionContext().entrySet()) { - result.put(entry.getKey(), entry.getValue()); - } - return Collections.unmodifiableMap(result); - } - - /** - * @return a map containing the items from the {@link JobParameters} - */ - public Map getJobParameters() { - Map result = new HashMap(); - for (Entry entry : stepExecution.getJobParameters().getParameters().entrySet()) { - result.put(entry.getKey(), entry.getValue().getValue()); - } - return Collections.unmodifiableMap(result); - } - - /** - * Allow clients to register callbacks for clean up on close. - * - * @param name the callback id (unique attribute key in this context) - * @param callback a callback to execute on close - */ - public void registerDestructionCallback(String name, Runnable callback) { - synchronized (callbacks) { - Set set = callbacks.get(name); - if (set == null) { - set = new HashSet(); - callbacks.put(name, set); - } - set.add(callback); - } - } - - private void unregisterDestructionCallbacks(String name) { - synchronized (callbacks) { - callbacks.remove(name); - } - } - - /** - * Override base class behaviour to ensure destruction callbacks are - * unregistered as well as the default behaviour. - * - * @see SynchronizedAttributeAccessor#removeAttribute(String) - */ - @Override - public Object removeAttribute(String name) { - unregisterDestructionCallbacks(name); - return super.removeAttribute(name); - } - - /** - * Clean up the context at the end of a step execution. Must be called once - * at the end of a step execution to honour the destruction callback - * contract from the {@link StepScope}. - */ - public void close() { - - List errors = new ArrayList(); - - Map> copy = Collections.unmodifiableMap(callbacks); - - for (Entry> entry : copy.entrySet()) { - Set set = entry.getValue(); - for (Runnable callback : set) { - if (callback != null) { - /* - * The documentation of the interface says that these - * callbacks must not throw exceptions, but we don't trust - * them necessarily... - */ - try { - callback.run(); - } - catch (RuntimeException t) { - errors.add(t); - } - } - } - } - - if (errors.isEmpty()) { - return; - } - - Exception error = errors.get(0); - if (error instanceof RuntimeException) { - throw (RuntimeException) error; - } - else { - throw new UnexpectedJobExecutionException("Could not close step context, rethrowing first of " - + errors.size() + " exceptions.", error); - } - } - - /** - * The current {@link StepExecution} that is active in this context. - * - * @return the current {@link StepExecution} - */ - public StepExecution getStepExecution() { - return stepExecution; - } - - /** - * @return unique identifier for this context based on the step execution - */ - public String getId() { - Assert.state(stepExecution.getId() != null, "StepExecution has no id. " - + "It must be saved before it can be used in step scope."); - return "execution#" + stepExecution.getId(); - } - - /** - * Extend the base class method to include the step execution itself as a - * key (i.e. two contexts are only equal if their step executions are the - * same). - * - * @see SynchronizedAttributeAccessor#equals(Object) - */ - @Override - public boolean equals(Object other) { - if (!(other instanceof StepContext)) - return false; - if (other == this) - return true; - StepContext context = (StepContext) other; - if (context.stepExecution == stepExecution) { - return true; - } - return stepExecution.equals(context.stepExecution); - } - - /** - * Overrides the default behaviour to provide a hash code based only on the - * step execution. - * - * @see SynchronizedAttributeAccessor#hashCode() - */ - @Override - public int hashCode() { - return stepExecution.hashCode(); - } - - @Override - public String toString() { - return super.toString() + ", stepExecutionContext=" + getStepExecutionContext() + ", jobExecutionContext=" - + getJobExecutionContext() + ", jobParameters=" + getJobParameters(); - } - -} +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope.context; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Map.Entry; +import java.util.Properties; +import java.util.Set; + +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameter; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.UnexpectedJobExecutionException; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.scope.StepScope; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.repeat.context.SynchronizedAttributeAccessor; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * A context object that can be used to interrogate the current + * {@link StepExecution} and some of its associated properties using expressions + * based on bean paths. Has public getters for the step execution and + * convenience methods for accessing commonly used properties like the + * {@link ExecutionContext} associated with the step or its enclosing job + * execution. + * + * @author Dave Syer + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @author Nicolas Widart + * + */ +public class StepContext extends SynchronizedAttributeAccessor { + + private StepExecution stepExecution; + + private Map> callbacks = new HashMap<>(); + + private BatchPropertyContext propertyContext = null; + + /** + * Create a new instance of {@link StepContext} for this + * {@link StepExecution}. + * + * @param stepExecution a step execution + */ + public StepContext(StepExecution stepExecution) { + super(); + Assert.notNull(stepExecution, "A StepContext must have a non-null StepExecution"); + this.stepExecution = stepExecution; + } + + public StepContext(StepExecution stepExecution, BatchPropertyContext propertyContext) { + super(); + Assert.notNull(stepExecution, "A StepContext must have a non-null StepExecution"); + this.stepExecution = stepExecution; + this.propertyContext = propertyContext; + } + + /** + * Convenient accessor for current step name identifier. Usually this is the + * same as the bean name of the step that is executing (but might not be + * e.g. in a partition). + * + * @return the step name identifier of the current {@link StepExecution} + */ + public String getStepName() { + return stepExecution.getStepName(); + } + + /** + * Convenient accessor for current job name identifier. + * + * @return the job name identifier of the enclosing {@link JobInstance} + * associated with the current {@link StepExecution} + */ + public String getJobName() { + Assert.state(stepExecution.getJobExecution() != null, "StepExecution does not have a JobExecution"); + Assert.state(stepExecution.getJobExecution().getJobInstance() != null, + "StepExecution does not have a JobInstance"); + return stepExecution.getJobExecution().getJobInstance().getJobName(); + } + + /** + * Convenient accessor for current {@link JobInstance} identifier. + * + * @return the identifier of the enclosing {@link JobInstance} + * associated with the current {@link StepExecution} + */ + public Long getJobInstanceId() { + Assert.state(stepExecution.getJobExecution() != null, "StepExecution does not have a JobExecution"); + Assert.state(stepExecution.getJobExecution().getJobInstance() != null, + "StepExecution does not have a JobInstance"); + return stepExecution.getJobExecution().getJobInstance().getInstanceId(); + } + + /** + * Convenient accessor for System properties to make it easy to access them + * from placeholder expressions. + * + * @return the current System properties + */ + public Properties getSystemProperties() { + return System.getProperties(); + } + + /** + * @return a map containing the items from the step {@link ExecutionContext} + */ + public Map getStepExecutionContext() { + Map result = new HashMap<>(); + for (Entry entry : stepExecution.getExecutionContext().entrySet()) { + result.put(entry.getKey(), entry.getValue()); + } + return Collections.unmodifiableMap(result); + } + + /** + * @return a map containing the items from the job {@link ExecutionContext} + */ + public Map getJobExecutionContext() { + Map result = new HashMap<>(); + for (Entry entry : stepExecution.getJobExecution().getExecutionContext().entrySet()) { + result.put(entry.getKey(), entry.getValue()); + } + return Collections.unmodifiableMap(result); + } + + /** + * @return a map containing the items from the {@link JobParameters} + */ + public Map getJobParameters() { + Map result = new HashMap<>(); + for (Entry entry : stepExecution.getJobParameters().getParameters().entrySet()) { + result.put(entry.getKey(), entry.getValue().getValue()); + } + return Collections.unmodifiableMap(result); + } + + @SuppressWarnings({"rawtypes", "unchecked"}) + public Map getPartitionPlan() { + Map partitionPlanProperties = new HashMap<>(); + + if(propertyContext != null) { + Map partitionProperties = propertyContext.getStepProperties(getStepName()); + partitionPlanProperties = partitionProperties; + } + + return Collections.unmodifiableMap(partitionPlanProperties); + } + + /** + * Allow clients to register callbacks for clean up on close. + * + * @param name the callback id (unique attribute key in this context) + * @param callback a callback to execute on close + */ + public void registerDestructionCallback(String name, Runnable callback) { + synchronized (callbacks) { + Set set = callbacks.get(name); + if (set == null) { + set = new HashSet<>(); + callbacks.put(name, set); + } + set.add(callback); + } + } + + private void unregisterDestructionCallbacks(String name) { + synchronized (callbacks) { + callbacks.remove(name); + } + } + + /** + * Override base class behaviour to ensure destruction callbacks are + * unregistered as well as the default behaviour. + * + * @see SynchronizedAttributeAccessor#removeAttribute(String) + */ + @Override + @Nullable + public Object removeAttribute(String name) { + unregisterDestructionCallbacks(name); + return super.removeAttribute(name); + } + + /** + * Clean up the context at the end of a step execution. Must be called once + * at the end of a step execution to honour the destruction callback + * contract from the {@link StepScope}. + */ + public void close() { + + List errors = new ArrayList<>(); + + Map> copy = Collections.unmodifiableMap(callbacks); + + for (Entry> entry : copy.entrySet()) { + Set set = entry.getValue(); + for (Runnable callback : set) { + if (callback != null) { + /* + * The documentation of the interface says that these + * callbacks must not throw exceptions, but we don't trust + * them necessarily... + */ + try { + callback.run(); + } + catch (RuntimeException t) { + errors.add(t); + } + } + } + } + + if (errors.isEmpty()) { + return; + } + + Exception error = errors.get(0); + if (error instanceof RuntimeException) { + throw (RuntimeException) error; + } + else { + throw new UnexpectedJobExecutionException("Could not close step context, rethrowing first of " + + errors.size() + " exceptions.", error); + } + } + + /** + * The current {@link StepExecution} that is active in this context. + * + * @return the current {@link StepExecution} + */ + public StepExecution getStepExecution() { + return stepExecution; + } + + /** + * @return unique identifier for this context based on the step execution + */ + public String getId() { + Assert.state(stepExecution.getId() != null, "StepExecution has no id. " + + "It must be saved before it can be used in step scope."); + return "execution#" + stepExecution.getId(); + } + + /** + * Extend the base class method to include the step execution itself as a + * key (i.e. two contexts are only equal if their step executions are the + * same). + * + * @see SynchronizedAttributeAccessor#equals(Object) + */ + @Override + public boolean equals(Object other) { + if (!(other instanceof StepContext)) { + return false; + } + if (other == this) { + return true; + } + StepContext context = (StepContext) other; + if (context.stepExecution == stepExecution) { + return true; + } + return stepExecution.equals(context.stepExecution); + } + + /** + * Overrides the default behaviour to provide a hash code based only on the + * step execution. + * + * @see SynchronizedAttributeAccessor#hashCode() + */ + @Override + public int hashCode() { + return stepExecution.hashCode(); + } + + @Override + public String toString() { + return super.toString() + ", stepExecutionContext=" + getStepExecutionContext() + ", jobExecutionContext=" + + getJobExecutionContext() + ", jobParameters=" + getJobParameters(); + } + +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepContextRepeatCallback.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepContextRepeatCallback.java index b4a383e815..7ca7fae71a 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepContextRepeatCallback.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepContextRepeatCallback.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,18 +32,19 @@ * callback inside a {@link Step}. * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public abstract class StepContextRepeatCallback implements RepeatCallback { - private final Queue attributeQueue = new LinkedBlockingQueue(); + private final Queue attributeQueue = new LinkedBlockingQueue<>(); private final StepExecution stepExecution; private final Log logger = LogFactory.getLog(StepContextRepeatCallback.class); /** - * @param stepExecution + * @param stepExecution instance of {@link StepExecution} to be used by StepContextRepeatCallback. */ public StepContextRepeatCallback(StepExecution stepExecution) { this.stepExecution = stepExecution; @@ -54,7 +55,7 @@ public StepContextRepeatCallback(StepExecution stepExecution) { * delegated to {@link #doInChunkContext(RepeatContext, ChunkContext)}. This * is to ensure that the current thread has a reference to the context, even * if the callback is executed in a pooled thread. Handles the registration - * and de-registration of the step context, so clients should not duplicate + * and unregistration of the step context, so clients should not duplicate * those calls. * * @see RepeatCallback#doInIteration(RepeatContext) @@ -65,7 +66,9 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { // The StepContext has to be the same for all chunks, // otherwise step-scoped beans will be re-initialised for each chunk. StepContext stepContext = StepSynchronizationManager.register(stepExecution); - logger.debug("Preparing chunk execution for StepContext: "+ObjectUtils.identityToString(stepContext)); + if (logger.isDebugEnabled()) { + logger.debug("Preparing chunk execution for StepContext: "+ObjectUtils.identityToString(stepContext)); + } ChunkContext chunkContext = attributeQueue.poll(); if (chunkContext == null) { @@ -73,7 +76,9 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { } try { - logger.debug("Chunk execution starting: queue size="+attributeQueue.size()); + if (logger.isDebugEnabled()) { + logger.debug("Chunk execution starting: queue size="+attributeQueue.size()); + } return doInChunkContext(context, chunkContext); } finally { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepScopeManager.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepScopeManager.java index 701cafb4b9..f9f35a1f1d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepScopeManager.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepScopeManager.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepSynchronizationManager.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepSynchronizationManager.java index 50c556a89c..72907c137c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepSynchronizationManager.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/StepSynchronizationManager.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,13 +15,10 @@ */ package org.springframework.batch.core.scope.context; -import java.util.HashMap; -import java.util.Map; -import java.util.Stack; -import java.util.concurrent.atomic.AtomicInteger; - import org.springframework.batch.core.Step; import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.lang.Nullable; /** * Central convenience class for framework use in managing the step scope @@ -29,82 +26,77 @@ * it is the responsibility of every {@link Step} implementation to ensure that * a {@link StepContext} is available on every thread that might be involved in * a step execution, including worker threads from a pool. - * + * * @author Dave Syer - * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * */ public class StepSynchronizationManager { - /* - * We have to deal with single and multi-threaded execution, with a single - * and with multiple step execution instances. That's 2x2 = 4 scenarios. - */ + private static final SynchronizationManagerSupport manager = + new SynchronizationManagerSupport() { - /** - * Storage for the current step execution; has to be ThreadLocal because it - * is needed to locate a StepContext in components that are not part of a - * Step (like when re-hydrating a scoped proxy). Doesn't use - * InheritableThreadLocal because there are side effects if a step is trying - * to run multiple child steps (e.g. with partitioning). The Stack is used - * to cover the single threaded case, so that the API is the same as - * multi-threaded. - */ - private static final ThreadLocal> executionHolder = new ThreadLocal>(); + @Override + protected StepContext createNewContext(StepExecution execution, @Nullable BatchPropertyContext propertyContext) { + StepContext context; - /** - * Reference counter for each step execution: how many threads are using the - * same one? - */ - private static final Map counts = new HashMap(); + if(propertyContext != null) { + context = new StepContext(execution, propertyContext); + } else { + context = new StepContext(execution); + } - /** - * Simple map from a running step execution to the associated context. - */ - private static final Map contexts = new HashMap(); + return context; + } + + @Override + protected void close(StepContext context) { + context.close(); + } + }; /** - * Getter for the current context if there is one, otherwise returns null. - * - * @return the current {@link StepContext} or null if there is none (if one + * Getter for the current context if there is one, otherwise returns {@code null}. + * + * @return the current {@link StepContext} or {@code null} if there is none (if one * has not been registered for this thread). */ + @Nullable public static StepContext getContext() { - if (getCurrent().isEmpty()) { - return null; - } - synchronized (contexts) { - return contexts.get(getCurrent().peek()); - } + return manager.getContext(); } /** * Register a context with the current thread - always put a matching * {@link #close()} call in a finally block to ensure that the correct * context is available in the enclosing block. - * + * * @param stepExecution the step context to register * @return a new {@link StepContext} or the current one if it has the same * {@link StepExecution} */ public static StepContext register(StepExecution stepExecution) { - if (stepExecution == null) { - return null; - } - getCurrent().push(stepExecution); - StepContext context; - synchronized (contexts) { - context = contexts.get(stepExecution); - if (context == null) { - context = new StepContext(stepExecution); - contexts.put(stepExecution, context); - } - } - increment(); - return context; + return manager.register(stepExecution); } /** - * Method for de-registering the current context - should always and only be + * Register a context with the current thread - always put a matching + * {@link #close()} call in a finally block to ensure that the correct + * context is available in the enclosing block. + * + * @param stepExecution the step context to register + * @param propertyContext an instance of {@link BatchPropertyContext} to be + * used by the StepSynchronizationManager. + * @return a new {@link StepContext} or the current one if it has the same + * {@link StepExecution} + */ + public static StepContext register(StepExecution stepExecution, BatchPropertyContext propertyContext) { + return manager.register(stepExecution, propertyContext); + } + + /** + * Method for unregistering the current context - should always and only be * used by in conjunction with a matching {@link #register(StepExecution)} * to ensure that {@link #getContext()} always returns the correct value. * Does not call {@link StepContext#close()} - that is left up to the caller @@ -112,46 +104,7 @@ public static StepContext register(StepExecution stepExecution) { * he has knowledge of when the step actually ended. */ public static void close() { - StepContext oldSession = getContext(); - if (oldSession == null) { - return; - } - decrement(); - } - - private static void decrement() { - StepExecution current = getCurrent().pop(); - if (current != null) { - int remaining = counts.get(current).decrementAndGet(); - if (remaining <= 0) { - synchronized (contexts) { - contexts.remove(current); - counts.remove(current); - } - } - } - } - - private static void increment() { - StepExecution current = getCurrent().peek(); - if (current != null) { - AtomicInteger count; - synchronized (counts) { - count = counts.get(current); - if (count == null) { - count = new AtomicInteger(); - counts.put(current, count); - } - } - count.incrementAndGet(); - } - } - - private static Stack getCurrent() { - if (executionHolder.get() == null) { - executionHolder.set(new Stack()); - } - return executionHolder.get(); + manager.close(); } /** @@ -161,15 +114,6 @@ private static Stack getCurrent() { * {@link #close()} is also called in a finally block. */ public static void release() { - StepContext context = getContext(); - try { - if (context != null) { - context.close(); - } - } - finally { - close(); - } + manager.release(); } - } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/SynchronizationManagerSupport.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/SynchronizationManagerSupport.java new file mode 100644 index 0000000000..ba9e695d88 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/SynchronizationManagerSupport.java @@ -0,0 +1,207 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope.context; + +import java.util.Map; +import java.util.Stack; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.atomic.AtomicInteger; + +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.lang.Nullable; + + +/** + * Central convenience class for framework use in managing the scope + * context. + * + * @author Dave Syer + * @author Jimmy Praet + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public abstract class SynchronizationManagerSupport { + + /* + * We have to deal with single and multi-threaded execution, with a single + * and with multiple step execution instances. That's 2x2 = 4 scenarios. + */ + + /** + * Storage for the current execution; has to be ThreadLocal because it + * is needed to locate a context in components that are not part of a + * step/job (like when re-hydrating a scoped proxy). Doesn't use + * InheritableThreadLocal because there are side effects if a step is trying + * to run multiple child steps (e.g. with partitioning). The Stack is used + * to cover the single threaded case, so that the API is the same as + * multi-threaded. + */ + private final ThreadLocal> executionHolder = new ThreadLocal<>(); + + /** + * Reference counter for each execution: how many threads are using the + * same one? + */ + private final Map counts = new ConcurrentHashMap<>(); + + /** + * Simple map from a running execution to the associated context. + */ + private final Map contexts = new ConcurrentHashMap<>(); + + /** + * Getter for the current context if there is one, otherwise returns {@code null}. + * + * @return the current context or {@code null} if there is none (if one + * has not been registered for this thread). + */ + @Nullable + public C getContext() { + if (getCurrent().isEmpty()) { + return null; + } + synchronized (contexts) { + return contexts.get(getCurrent().peek()); + } + } + + /** + * Register a context with the current thread - always put a matching {@link #close()} call in a finally block to + * ensure that the correct + * context is available in the enclosing block. + * + * @param execution the execution to register + * @return a new context or the current one if it has the same + * execution + */ + @Nullable + public C register(@Nullable E execution) { + if (execution == null) { + return null; + } + getCurrent().push(execution); + C context; + synchronized (contexts) { + context = contexts.get(execution); + if (context == null) { + context = createNewContext(execution, null); + contexts.put(execution, context); + } + } + increment(); + return context; + } + + /** + * Register a context with the current thread - always put a matching {@link #close()} call in a finally block to + * ensure that the correct + * context is available in the enclosing block. + * + * @param execution the execution to register + * @param propertyContext instance of {@link BatchPropertyContext} to be registered with this thread. + * @return a new context or the current one if it has the same + * execution + */ + @Nullable + public C register(@Nullable E execution, @Nullable BatchPropertyContext propertyContext) { + if (execution == null) { + return null; + } + getCurrent().push(execution); + C context; + synchronized (contexts) { + context = contexts.get(execution); + if (context == null) { + context = createNewContext(execution, propertyContext); + contexts.put(execution, context); + } + } + increment(); + return context; + } + + /** + * Method for unregistering the current context - should always and only be + * used by in conjunction with a matching {@link #register(Object)} to ensure that {@link #getContext()} always returns + * the correct value. + * Does not call close on the context - that is left up to the caller + * because he has a reference to the context (having registered it) and only + * he has knowledge of when the execution actually ended. + */ + public void close() { + C oldSession = getContext(); + if (oldSession == null) { + return; + } + decrement(); + } + + private void decrement() { + E current = getCurrent().pop(); + if (current != null) { + int remaining = counts.get(current).decrementAndGet(); + if (remaining <= 0) { + synchronized (contexts) { + contexts.remove(current); + counts.remove(current); + } + } + } + } + + public void increment() { + E current = getCurrent().peek(); + if (current != null) { + AtomicInteger count; + synchronized (counts) { + count = counts.get(current); + if (count == null) { + count = new AtomicInteger(); + counts.put(current, count); + } + } + count.incrementAndGet(); + } + } + + public Stack getCurrent() { + if (executionHolder.get() == null) { + executionHolder.set(new Stack<>()); + } + return executionHolder.get(); + } + + /** + * A convenient "deep" close operation. Call this instead of {@link #close()} if the execution for the current + * context is ending. + * Delegates to {@link #close(Object)} and then ensures that {@link #close()} is also called in a finally block. + */ + public void release() { + C context = getContext(); + try { + if (context != null) { + close(context); + } + } finally { + close(); + } + } + + protected abstract void close(C context); + + protected abstract C createNewContext(E execution, @Nullable BatchPropertyContext propertyContext); + +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/package-info.java new file mode 100644 index 0000000000..a12e79cc4c --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/context/package-info.java @@ -0,0 +1,10 @@ +/** + * Implementation of the contexts for each of the custom bean scopes in Spring Batch (Job and Step). + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.scope.context; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/scope/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/package-info.java new file mode 100644 index 0000000000..b79fafb67d --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/scope/package-info.java @@ -0,0 +1,10 @@ +/** + * Implementation of Spring Batch specific bean scopes (Job and Step). + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.scope; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/AbstractStep.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/AbstractStep.java index 91bf925204..08dd2f993f 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/AbstractStep.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/AbstractStep.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,8 +15,11 @@ */ package org.springframework.batch.core.step; +import java.time.Duration; import java.util.Date; +import io.micrometer.core.instrument.Tag; +import io.micrometer.core.instrument.Timer; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.core.BatchStatus; @@ -26,9 +29,11 @@ import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.StepExecutionListener; import org.springframework.batch.core.UnexpectedJobExecutionException; +import org.springframework.batch.core.configuration.annotation.StepScope; import org.springframework.batch.core.launch.NoSuchJobException; import org.springframework.batch.core.launch.support.ExitCodeMapper; import org.springframework.batch.core.listener.CompositeStepExecutionListener; +import org.springframework.batch.core.metrics.BatchMetrics; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.scope.context.StepSynchronizationManager; import org.springframework.batch.item.ExecutionContext; @@ -45,6 +50,9 @@ * @author Dave Syer * @author Ben Hale * @author Robert Kasanicky + * @author Michael Minella + * @author Chris Schaefer + * @author Mahmoud Ben Hassine */ public abstract class AbstractStep implements Step, InitializingBean, BeanNameAware { @@ -80,7 +88,7 @@ public String getName() { /** * Set the name property. Always overrides the default value if this object is a Spring bean. - * + * @param name the name of the {@link Step}. * @see #setBeanName(java.lang.String) */ public void setName(String name) { @@ -112,7 +120,7 @@ public int getStartLimit() { * @param startLimit the startLimit to set */ public void setStartLimit(int startLimit) { - this.startLimit = startLimit; + this.startLimit = startLimit == 0 ? Integer.MAX_VALUE : startLimit; } @Override @@ -133,7 +141,7 @@ public void setAllowStartIfComplete(boolean allowStartIfComplete) { /** * Convenient constructor for setting only the name property. * - * @param name + * @param name Name of the step */ public AbstractStep(String name) { this.name = name; @@ -144,7 +152,7 @@ public AbstractStep(String name) { * {@link StepExecution} before returning. * * @param stepExecution the current step context - * @throws Exception + * @throws Exception checked exception thrown by implementation */ protected abstract void doExecute(StepExecution stepExecution) throws Exception; @@ -153,7 +161,7 @@ public AbstractStep(String name) { * acquire resources. Does nothing by default. * * @param ctx the {@link ExecutionContext} to use - * @throws Exception + * @throws Exception checked exception thrown by implementation */ protected void open(ExecutionContext ctx) throws Exception { } @@ -163,7 +171,7 @@ protected void open(ExecutionContext ctx) throws Exception { * of the finally block), to close or release resources. Does nothing by default. * * @param ctx the {@link ExecutionContext} to use - * @throws Exception + * @throws Exception checked exception thrown by implementation */ protected void close(ExecutionContext ctx) throws Exception { } @@ -177,15 +185,20 @@ protected void close(ExecutionContext ctx) throws Exception { public final void execute(StepExecution stepExecution) throws JobInterruptedException, UnexpectedJobExecutionException { - logger.debug("Executing: id=" + stepExecution.getId()); + Assert.notNull(stepExecution, "stepExecution must not be null"); + + if (logger.isDebugEnabled()) { + logger.debug("Executing: id=" + stepExecution.getId()); + } stepExecution.setStartTime(new Date()); stepExecution.setStatus(BatchStatus.STARTED); + Timer.Sample sample = BatchMetrics.createTimerSample(); getJobRepository().update(stepExecution); // Start with a default value that will be trumped by anything ExitStatus exitStatus = ExitStatus.EXECUTING; - StepSynchronizationManager.register(stepExecution); + doExecutionRegistration(stepExecution); try { getCompositeListener().beforeStep(stepExecution); @@ -206,20 +219,22 @@ public final void execute(StepExecution stepExecution) throws JobInterruptedExce // Need to upgrade here not set, in case the execution was stopped stepExecution.upgradeStatus(BatchStatus.COMPLETED); - logger.debug("Step execution success: id=" + stepExecution.getId()); + if (logger.isDebugEnabled()) { + logger.debug("Step execution success: id=" + stepExecution.getId()); + } } catch (Throwable e) { stepExecution.upgradeStatus(determineBatchStatus(e)); exitStatus = exitStatus.and(getDefaultExitStatusForFailure(e)); stepExecution.addFailureException(e); if (stepExecution.getStatus() == BatchStatus.STOPPED) { - logger.info("Encountered interruption executing step: " + e.getMessage()); + logger.info(String.format("Encountered interruption executing step %s in job %s : %s", name, stepExecution.getJobExecution().getJobInstance().getJobName(), e.getMessage())); if (logger.isDebugEnabled()) { logger.debug("Full exception", e); } } else { - logger.error("Encountered an error executing the step", e); + logger.error(String.format("Encountered an error executing step %s in job %s", name, stepExecution.getJobExecution().getJobInstance().getJobName()), e); } } finally { @@ -232,7 +247,7 @@ public final void execute(StepExecution stepExecution) throws JobInterruptedExce exitStatus = exitStatus.and(getCompositeListener().afterStep(stepExecution)); } catch (Exception e) { - logger.error("Exception in afterStep callback", e); + logger.error(String.format("Exception in afterStep callback in step %s in job %s", name, stepExecution.getJobExecution().getJobInstance().getJobName()), e); } try { @@ -242,12 +257,19 @@ public final void execute(StepExecution stepExecution) throws JobInterruptedExce stepExecution.setStatus(BatchStatus.UNKNOWN); exitStatus = exitStatus.and(ExitStatus.UNKNOWN); stepExecution.addFailureException(e); - logger.error("Encountered an error saving batch meta data. " - + "This job is now in an unknown state and should not be restarted.", e); + logger.error(String.format("Encountered an error saving batch meta data for step %s in job %s. " + + "This job is now in an unknown state and should not be restarted.", name, stepExecution.getJobExecution().getJobInstance().getJobName()), e); } + sample.stop(BatchMetrics.createTimer("step", "Step duration", + Tag.of("job.name", stepExecution.getJobExecution().getJobInstance().getJobName()), + Tag.of("name", stepExecution.getStepName()), + Tag.of("status", stepExecution.getExitStatus().getExitCode()) + )); stepExecution.setEndTime(new Date()); stepExecution.setExitStatus(exitStatus); + Duration stepExecutionDuration = BatchMetrics.calculateDuration(stepExecution.getStartTime(), stepExecution.getEndTime()); + logger.info("Step: [" + stepExecution.getStepName() + "] executed in " + BatchMetrics.formatDuration(stepExecutionDuration)); try { getJobRepository().update(stepExecution); @@ -256,24 +278,42 @@ public final void execute(StepExecution stepExecution) throws JobInterruptedExce stepExecution.setStatus(BatchStatus.UNKNOWN); stepExecution.setExitStatus(exitStatus.and(ExitStatus.UNKNOWN)); stepExecution.addFailureException(e); - logger.error("Encountered an error saving batch meta data. " - + "This job is now in an unknown state and should not be restarted.", e); + logger.error(String.format("Encountered an error saving batch meta data for step %s in job %s. " + + "This job is now in an unknown state and should not be restarted.", name, stepExecution.getJobExecution().getJobInstance().getJobName()), e); } try { close(stepExecution.getExecutionContext()); } catch (Exception e) { - logger.error("Exception while closing step execution resources", e); + logger.error(String.format("Exception while closing step execution resources in step %s in job %s", name, stepExecution.getJobExecution().getJobInstance().getJobName()), e); stepExecution.addFailureException(e); } - StepSynchronizationManager.release(); + doExecutionRelease(); - logger.debug("Step execution complete: " + stepExecution.getSummary()); + if (logger.isDebugEnabled()) { + logger.debug("Step execution complete: " + stepExecution.getSummary()); + } } } + /** + * Releases the most recent {@link StepExecution} + */ + protected void doExecutionRelease() { + StepSynchronizationManager.release(); + } + + /** + * Registers the {@link StepExecution} for property resolution via {@link StepScope} + * + * @param stepExecution StepExecution to use when hydrating the StepScoped beans + */ + protected void doExecutionRegistration(StepExecution stepExecution) { + StepSynchronizationManager.register(stepExecution); + } + /** * Determine the step status based on the exception. */ diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/FatalStepExecutionException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/FatalStepExecutionException.java index 4aac34109c..1b5358bbcf 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/FatalStepExecutionException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/FatalStepExecutionException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class FatalStepExecutionException extends UnexpectedJobExecutionException { /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/NoSuchStepException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/NoSuchStepException.java index 7b8871fd04..3d6de212f7 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/NoSuchStepException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/NoSuchStepException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,11 +22,12 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class NoSuchStepException extends RuntimeException { /** * Create a new exception instance with the message provided. - * @param message + * @param message the message to be used for this exception. */ public NoSuchStepException(String message) { super(message); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/NoWorkFoundStepExecutionListener.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/NoWorkFoundStepExecutionListener.java index 8aced09990..ad1e941842 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/NoWorkFoundStepExecutionListener.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/NoWorkFoundStepExecutionListener.java @@ -1,11 +1,11 @@ /* - * Copyright 2002-2013 the original author or authors. + * Copyright 2002-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,6 +19,7 @@ import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.listener.StepExecutionListenerSupport; +import org.springframework.lang.Nullable; /** * Fails the step if no items have been processed ( item count is 0). @@ -27,6 +28,7 @@ */ public class NoWorkFoundStepExecutionListener extends StepExecutionListenerSupport { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { if (stepExecution.getReadCount() == 0) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepHolder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepHolder.java index 175c723fd3..2b94104816 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepHolder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepHolder.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step; import org.springframework.batch.core.Step; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepInterruptionPolicy.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepInterruptionPolicy.java index bc373b4b24..c9ab46fe3d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepInterruptionPolicy.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepInterruptionPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepLocator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepLocator.java index 7328cc84b9..1c5aba4f2e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepLocator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepLocator.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step; import java.util.Collection; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepLocatorStepFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepLocatorStepFactoryBean.java index 7345c6d542..25a10e38d4 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepLocatorStepFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/StepLocatorStepFactoryBean.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,14 +34,14 @@ public class StepLocatorStepFactoryBean implements FactoryBean { public String stepName; /** - * @param stepLocator + * @param stepLocator instance of {@link StepLocator} to be used by the factory bean. */ public void setStepLocator(StepLocator stepLocator) { this.stepLocator = stepLocator; } /** - * @param stepName + * @param stepName the name to be associated with the step. */ public void setStepName(String stepName) { this.stepName = stepName; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/ThreadStepInterruptionPolicy.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/ThreadStepInterruptionPolicy.java index daed5d7d19..c6a8ec7108 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/ThreadStepInterruptionPolicy.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/ThreadStepInterruptionPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/AbstractTaskletStepBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/AbstractTaskletStepBuilder.java index a946d142b6..d3288e8a8a 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/AbstractTaskletStepBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/AbstractTaskletStepBuilder.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,11 +15,18 @@ */ package org.springframework.batch.core.step.builder; +import java.lang.reflect.Method; +import java.util.HashSet; import java.util.LinkedHashSet; import java.util.Set; import org.springframework.batch.core.ChunkListener; import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecutionListener; +import org.springframework.batch.core.annotation.AfterChunk; +import org.springframework.batch.core.annotation.AfterChunkError; +import org.springframework.batch.core.annotation.BeforeChunk; +import org.springframework.batch.core.listener.StepListenerFactoryBean; import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.core.step.tasklet.TaskletStep; import org.springframework.batch.item.ItemStream; @@ -28,6 +35,7 @@ import org.springframework.batch.repeat.exception.ExceptionHandler; import org.springframework.batch.repeat.support.RepeatTemplate; import org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate; +import org.springframework.batch.support.ReflectionUtils; import org.springframework.core.task.SyncTaskExecutor; import org.springframework.core.task.TaskExecutor; import org.springframework.transaction.interceptor.TransactionAttribute; @@ -35,23 +43,25 @@ /** * Base class for step builders that want to build a {@link TaskletStep}. Handles common concerns across all tasklet * step variants, which are mostly to do with the type of tasklet they carry. - * + * * @author Dave Syer - * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * * @since 2.2 - * + * * @param the type of builder represented */ public abstract class AbstractTaskletStepBuilder> extends - StepBuilderHelper> { +StepBuilderHelper> { - private Set listeners = new LinkedHashSet(); + protected Set chunkListeners = new LinkedHashSet<>(); private RepeatOperations stepOperations; private TransactionAttribute transactionAttribute; - private Set streams = new LinkedHashSet(); + private Set streams = new LinkedHashSet<>(); private ExceptionHandler exceptionHandler = new DefaultExceptionHandler(); @@ -68,16 +78,18 @@ public AbstractTaskletStepBuilder(StepBuilderHelper parent) { /** * Build the step from the components collected by the fluent setters. Delegates first to {@link #enhance(Step)} and * then to {@link #createTasklet()} in subclasses to create the actual tasklet. - * - * @return a tasklet step fully configured and read to execute + * + * @return a tasklet step fully configured and ready to execute */ public TaskletStep build() { + registerStepListenerAsChunkListener(); + TaskletStep step = new TaskletStep(getName()); super.enhance(step); - step.setChunkListeners(listeners.toArray(new ChunkListener[0])); + step.setChunkListeners(chunkListeners.toArray(new ChunkListener[0])); if (transactionAttribute != null) { step.setTransactionAttribute(transactionAttribute); @@ -113,20 +125,54 @@ public TaskletStep build() { } + protected void registerStepListenerAsChunkListener() { + for (StepExecutionListener stepExecutionListener: properties.getStepExecutionListeners()){ + if (stepExecutionListener instanceof ChunkListener){ + listener((ChunkListener)stepExecutionListener); + } + } + } + /** * Register a chunk listener. - * + * * @param listener the listener to register * @return this for fluent chaining */ public AbstractTaskletStepBuilder listener(ChunkListener listener) { - listeners.add(listener); + chunkListeners.add(listener); return this; } + /** + * Registers objects using the annotation based listener configuration. + * + * @param listener the object that has a method configured with listener annotation + * @return this for fluent chaining + */ + @Override + public B listener(Object listener) { + super.listener(listener); + + Set chunkListenerMethods = new HashSet<>(); + chunkListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), BeforeChunk.class)); + chunkListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), AfterChunk.class)); + chunkListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), AfterChunkError.class)); + + if(chunkListenerMethods.size() > 0) { + StepListenerFactoryBean factory = new StepListenerFactoryBean(); + factory.setDelegate(listener); + this.listener((ChunkListener) factory.getObject()); + } + + @SuppressWarnings("unchecked") + B result = (B) this; + return result; + } + /** * Register a stream for callbacks that manage restart data. - * + * * @param stream the stream to register * @return this for fluent chaining */ @@ -138,7 +184,7 @@ public AbstractTaskletStepBuilder stream(ItemStream stream) { /** * Provide a task executor to use when executing the tasklet. Default is to use a single-threaded (synchronous) * executor. - * + * * @param taskExecutor the task executor to register * @return this for fluent chaining */ @@ -151,8 +197,8 @@ public AbstractTaskletStepBuilder taskExecutor(TaskExecutor taskExecutor) { * In the case of an asynchronous {@link #taskExecutor(TaskExecutor)} the number of concurrent tasklet executions * can be throttled (beyond any throttling provided by a thread pool). The throttle limit should be less than the * data source pool size used in the job repository for this step. - * - * @param throttleLimit maximium number of concurrent tasklet executions allowed + * + * @param throttleLimit maximum number of concurrent tasklet executions allowed * @return this for fluent chaining */ public AbstractTaskletStepBuilder throttleLimit(int throttleLimit) { @@ -162,7 +208,7 @@ public AbstractTaskletStepBuilder throttleLimit(int throttleLimit) { /** * Sets the exception handler to use in the case of tasklet failures. Default is to rethrow everything. - * + * * @param exceptionHandler the exception handler * @return this for fluent chaining */ @@ -174,7 +220,7 @@ public AbstractTaskletStepBuilder exceptionHandler(ExceptionHandler exception /** * Sets the repeat template used for iterating the tasklet execution. By default it will terminate only when the * tasklet returns FINISHED (or null). - * + * * @param repeatTemplate a repeat template with rules for iterating * @return this for fluent chaining */ @@ -186,7 +232,7 @@ public AbstractTaskletStepBuilder stepOperations(RepeatOperations repeatTempl /** * Sets the transaction attributes for the tasklet execution. Defaults to the default values for the transaction * manager, but can be manipulated to provide longer timeouts for instance. - * + * * @param transactionAttribute a transaction attribute set * @return this for fluent chaining */ @@ -197,7 +243,7 @@ public AbstractTaskletStepBuilder transactionAttribute(TransactionAttribute t /** * Convenience method for subclasses to access the step operations that were injected by user. - * + * * @return the repeat operations used to iterate the tasklet executions */ protected RepeatOperations getStepOperations() { @@ -206,7 +252,7 @@ protected RepeatOperations getStepOperations() { /** * Convenience method for subclasses to access the exception handler that was injected by user. - * + * * @return the exception handler */ protected ExceptionHandler getExceptionHandler() { @@ -215,7 +261,7 @@ protected ExceptionHandler getExceptionHandler() { /** * Convenience method for subclasses to determine if the step is concurrent. - * + * * @return true if the tasklet is going to be run in multiple threads */ protected boolean concurrent() { @@ -223,4 +269,20 @@ protected boolean concurrent() { return concurrent; } -} \ No newline at end of file + protected TaskExecutor getTaskExecutor() { + return taskExecutor; + } + + protected int getThrottleLimit() { + return throttleLimit; + } + + protected TransactionAttribute getTransactionAttribute() { + return transactionAttribute; + } + + protected Set getStreams() { + return this.streams; + } + +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/FaultTolerantStepBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/FaultTolerantStepBuilder.java index 05450eccf7..da8223f627 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/FaultTolerantStepBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/FaultTolerantStepBuilder.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,6 +15,7 @@ */ package org.springframework.batch.core.step.builder; +import java.lang.reflect.Method; import java.util.ArrayList; import java.util.Collection; import java.util.HashMap; @@ -24,16 +25,24 @@ import java.util.Map; import java.util.Set; +import javax.batch.operations.BatchRuntimeException; + import org.springframework.batch.core.ChunkListener; import org.springframework.batch.core.JobInterruptedException; import org.springframework.batch.core.SkipListener; +import org.springframework.batch.core.StepExecutionListener; import org.springframework.batch.core.StepListener; +import org.springframework.batch.core.annotation.OnSkipInProcess; +import org.springframework.batch.core.annotation.OnSkipInRead; +import org.springframework.batch.core.annotation.OnSkipInWrite; import org.springframework.batch.core.listener.StepListenerFactoryBean; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.step.FatalStepExecutionException; import org.springframework.batch.core.step.item.BatchRetryTemplate; import org.springframework.batch.core.step.item.ChunkMonitor; import org.springframework.batch.core.step.item.ChunkOrientedTasklet; +import org.springframework.batch.core.step.item.ChunkProcessor; +import org.springframework.batch.core.step.item.ChunkProvider; import org.springframework.batch.core.step.item.FaultTolerantChunkProcessor; import org.springframework.batch.core.step.item.FaultTolerantChunkProvider; import org.springframework.batch.core.step.item.ForceRollbackForWriteSkipException; @@ -49,10 +58,13 @@ import org.springframework.batch.core.step.skip.SkipPolicy; import org.springframework.batch.core.step.skip.SkipPolicyFailedException; import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.core.step.tasklet.TaskletStep; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ItemStream; import org.springframework.batch.repeat.RepeatOperations; import org.springframework.batch.repeat.support.RepeatTemplate; +import org.springframework.batch.support.ReflectionUtils; +import org.springframework.beans.factory.BeanCreationException; import org.springframework.classify.BinaryExceptionClassifier; import org.springframework.classify.Classifier; import org.springframework.classify.SubclassClassifier; @@ -76,6 +88,9 @@ * additional properties for retry and skip of failed items. * * @author Dave Syer + * @author Chris Schaefer + * @author Michael Minella + * @author Mahmoud Ben Hassine * * @since 2.2 */ @@ -89,7 +104,7 @@ public class FaultTolerantStepBuilder extends SimpleStepBuilder { private BackOffPolicy backOffPolicy; - private Set retryListeners = new LinkedHashSet(); + private Set retryListeners = new LinkedHashSet<>(); private RetryPolicy retryPolicy; @@ -97,17 +112,19 @@ public class FaultTolerantStepBuilder extends SimpleStepBuilder { private KeyGenerator keyGenerator; - private Collection> noRollbackExceptionClasses = new LinkedHashSet>(); + private Collection> noRollbackExceptionClasses = new LinkedHashSet<>(); + + private Map, Boolean> skippableExceptionClasses = new HashMap<>(); - private Map, Boolean> skippableExceptionClasses = new HashMap, Boolean>(); + private Collection> nonSkippableExceptionClasses = new HashSet<>(); - private Collection> nonSkippableExceptionClasses = new HashSet>(); + private Map, Boolean> retryableExceptionClasses = new HashMap<>(); - private Map, Boolean> retryableExceptionClasses = new HashMap, Boolean>(); + private Collection> nonRetryableExceptionClasses = new HashSet<>(); - private Collection> nonRetryableExceptionClasses = new HashSet>(); + private Set> skipListeners = new LinkedHashSet<>(); - private Set> skipListeners = new LinkedHashSet>(); + private Set jsrRetryListeners = new LinkedHashSet<>(); private int skipLimit = 0; @@ -133,6 +150,26 @@ protected FaultTolerantStepBuilder(SimpleStepBuilder parent) { super(parent); } + @Override + public TaskletStep build() { + registerStepListenerAsSkipListener(); + return super.build(); + } + + @SuppressWarnings("unchecked") + protected void registerStepListenerAsSkipListener() { + for (StepExecutionListener stepExecutionListener: properties.getStepExecutionListeners()){ + if (stepExecutionListener instanceof SkipListener){ + listener((SkipListener)stepExecutionListener); + } + } + for (ChunkListener chunkListener: this.chunkListeners){ + if (chunkListener instanceof SkipListener){ + listener((SkipListener)chunkListener); + } + } + } + /** * Create a new chunk oriented tasklet with reader, writer and processor as provided. * @@ -141,16 +178,44 @@ protected FaultTolerantStepBuilder(SimpleStepBuilder parent) { @Override protected Tasklet createTasklet() { Assert.state(getReader() != null, "ItemReader must be provided"); - Assert.state(getProcessor() != null || getWriter() != null, "ItemWriter or ItemProcessor must be provided"); + Assert.state(getWriter() != null, "ItemWriter must be provided"); addSpecialExceptions(); registerSkipListeners(); - FaultTolerantChunkProvider chunkProvider = createChunkProvider(); - FaultTolerantChunkProcessor chunkProcessor = createChunkProcessor(); - ChunkOrientedTasklet tasklet = new ChunkOrientedTasklet(chunkProvider, chunkProcessor); + ChunkProvider chunkProvider = createChunkProvider(); + ChunkProcessor chunkProcessor = createChunkProcessor(); + ChunkOrientedTasklet tasklet = new ChunkOrientedTasklet<>(chunkProvider, chunkProcessor); tasklet.setBuffering(!isReaderTransactionalQueue()); return tasklet; } + /** + * Registers objects using the annotation based listener configuration. + * + * @param listener the object that has a method configured with listener annotation + * @return this for fluent chaining + */ + @Override + @SuppressWarnings("unchecked") + public SimpleStepBuilder listener(Object listener) { + super.listener(listener); + + Set skipListenerMethods = new HashSet<>(); + skipListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), OnSkipInRead.class)); + skipListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), OnSkipInProcess.class)); + skipListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), OnSkipInWrite.class)); + + if(skipListenerMethods.size() > 0) { + StepListenerFactoryBean factory = new StepListenerFactoryBean(); + factory.setDelegate(listener); + skipListeners.add((SkipListener) factory.getObject()); + } + + @SuppressWarnings("unchecked") + SimpleStepBuilder result = this; + return result; + } + + /** * Register a skip listener. * @@ -162,6 +227,11 @@ public FaultTolerantStepBuilder listener(SkipListener listener(org.springframework.batch.core.jsr.RetryListener listener) { + jsrRetryListeners.add(listener); + return this; + } + @Override public FaultTolerantStepBuilder listener(ChunkListener listener) { super.listener(new TerminateOnExceptionChunkListenerDelegate(listener)); @@ -251,7 +321,7 @@ public FaultTolerantStepBuilder retryContextCache(RetryContextCache retryC } /** - * Sets the maximium number of failed items to skip before the step fails. Ignored if an explicit + * Sets the maximum number of failed items to skip before the step fails. Ignored if an explicit * {@link #skipPolicy(SkipPolicy)} is provided. * * @param skipLimit the skip limit to set @@ -276,7 +346,7 @@ public FaultTolerantStepBuilder noSkip(Class type) { /** * Explicitly request certain exceptions (and subclasses) to be skipped. * - * @param type + * @param type the exception type. * @return this for fluent chaining */ public FaultTolerantStepBuilder skip(Class type) { @@ -360,16 +430,16 @@ public AbstractTaskletStepBuilder> stream(ItemStream str return this; } - private FaultTolerantChunkProvider createChunkProvider() { + protected ChunkProvider createChunkProvider() { SkipPolicy readSkipPolicy = createSkipPolicy(); readSkipPolicy = getFatalExceptionAwareProxy(readSkipPolicy); - FaultTolerantChunkProvider chunkProvider = new FaultTolerantChunkProvider(getReader(), + FaultTolerantChunkProvider chunkProvider = new FaultTolerantChunkProvider<>(getReader(), createChunkOperations()); chunkProvider.setMaxSkipsOnRead(Math.max(getChunkSize(), FaultTolerantChunkProvider.DEFAULT_MAX_SKIPS_ON_READ)); chunkProvider.setSkipPolicy(readSkipPolicy); chunkProvider.setRollbackClassifier(getRollbackClassifier()); - ArrayList listeners = new ArrayList(getItemListeners()); + ArrayList listeners = new ArrayList<>(getItemListeners()); listeners.addAll(skipListeners); chunkProvider.setListeners(listeners); @@ -377,11 +447,11 @@ private FaultTolerantChunkProvider createChunkProvider() { } - private FaultTolerantChunkProcessor createChunkProcessor() { + protected ChunkProcessor createChunkProcessor() { BatchRetryTemplate batchRetryTemplate = createRetryOperations(); - FaultTolerantChunkProcessor chunkProcessor = new FaultTolerantChunkProcessor(getProcessor(), + FaultTolerantChunkProcessor chunkProcessor = new FaultTolerantChunkProcessor<>(getProcessor(), getWriter(), batchRetryTemplate); chunkProcessor.setBuffering(!isReaderTransactionalQueue()); chunkProcessor.setProcessorTransactional(processorTransactional); @@ -394,7 +464,7 @@ private FaultTolerantChunkProcessor createChunkProcessor() { chunkProcessor.setKeyGenerator(keyGenerator); detectStreamInReader(); - ArrayList listeners = new ArrayList(getItemListeners()); + ArrayList listeners = new ArrayList<>(getItemListeners()); listeners.addAll(skipListeners); chunkProcessor.setListeners(listeners); chunkProcessor.setChunkMonitor(chunkMonitor); @@ -407,13 +477,14 @@ private FaultTolerantChunkProcessor createChunkProcessor() { private void addSpecialExceptions() { addNonSkippableExceptionIfMissing(SkipLimitExceededException.class, NonSkippableReadException.class, SkipListenerFailedException.class, SkipPolicyFailedException.class, RetryException.class, - JobInterruptedException.class, Error.class); + JobInterruptedException.class, Error.class, BeanCreationException.class); addNonRetryableExceptionIfMissing(SkipLimitExceededException.class, NonSkippableReadException.class, TransactionException.class, FatalStepExecutionException.class, SkipListenerFailedException.class, - SkipPolicyFailedException.class, RetryException.class, JobInterruptedException.class, Error.class); + SkipPolicyFailedException.class, RetryException.class, JobInterruptedException.class, Error.class, + BatchRuntimeException.class, BeanCreationException.class); } - private void detectStreamInReader() { + protected void detectStreamInReader() { if (streamIsReader) { if (!concurrent()) { chunkMonitor.setItemReader(getReader()); @@ -429,10 +500,8 @@ private void detectStreamInReader() { * Register explicitly set item listeners and auto-register reader, processor and writer if applicable */ private void registerSkipListeners() { - // auto-register reader, processor and writer for (Object itemHandler : new Object[] { getReader(), getWriter(), getProcessor() }) { - if (StepListenerFactoryBean.isListener(itemHandler)) { StepListener listener = StepListenerFactoryBean.getListener(itemHandler); if (listener instanceof SkipListener) { @@ -450,7 +519,7 @@ private void registerSkipListeners() { * * @return an exception classifier: maps to true if an exception should cause rollback */ - private Classifier getRollbackClassifier() { + protected Classifier getRollbackClassifier() { Classifier classifier = new BinaryExceptionClassifier(noRollbackExceptionClasses, false); @@ -461,17 +530,14 @@ private Classifier getRollbackClassifier() { final Classifier binary = classifier; - Collection> types = new HashSet>(); + Collection> types = new HashSet<>(); types.add(ForceRollbackForWriteSkipException.class); types.add(ExhaustedRetryException.class); final Classifier panic = new BinaryExceptionClassifier(types, true); - classifier = new Classifier() { - @Override - public Boolean classify(Throwable classifiable) { - // Rollback if either the user's list or our own applies - return panic.classify(classifiable) || binary.classify(classifiable); - } + classifier = (Classifier) classifiable -> { + // Rollback if either the user's list or our own applies + return panic.classify(classifiable) || binary.classify(classifiable); }; } @@ -496,7 +562,7 @@ public boolean rollbackOn(Throwable ex) { protected SkipPolicy createSkipPolicy() { SkipPolicy skipPolicy = this.skipPolicy; - Map, Boolean> map = new HashMap, Boolean>( + Map, Boolean> map = new HashMap<>( skippableExceptionClasses); map.put(ForceRollbackForWriteSkipException.class, true); LimitCheckingItemSkipPolicy limitCheckingItemSkipPolicy = new LimitCheckingItemSkipPolicy(skipLimit, map); @@ -514,12 +580,12 @@ else if (limitCheckingItemSkipPolicy != null) { /** * @return fully configured retry template for item processing phase. */ - private BatchRetryTemplate createRetryOperations() { + protected BatchRetryTemplate createRetryOperations() { RetryPolicy retryPolicy = this.retryPolicy; SimpleRetryPolicy simpleRetryPolicy = null; - Map, Boolean> map = new HashMap, Boolean>( + Map, Boolean> map = new HashMap<>( retryableExceptionClasses); map.put(ForceRollbackForWriteSkipException.class, true); simpleRetryPolicy = new SimpleRetryPolicy(retryLimit, map); @@ -543,7 +609,7 @@ else if ((!retryableExceptionClasses.isEmpty() && retryLimit > 0)) { } batchRetryTemplate.setRetryPolicy(retryPolicyWrapper); - // Co-ordinate the retry policy with the exception handler: + // Coordinate the retry policy with the exception handler: RepeatOperations stepOperations = getStepOperations(); if (stepOperations instanceof RepeatTemplate) { SimpleRetryExceptionHandler exceptionHandler = new SimpleRetryExceptionHandler(retryPolicyWrapper, @@ -562,19 +628,31 @@ else if ((!retryableExceptionClasses.isEmpty() && retryLimit > 0)) { } + protected ChunkMonitor getChunkMonitor() { + return this.chunkMonitor; + } + + protected Set> getSkipListeners() { + return skipListeners; + } + + protected Set getJsrRetryListeners() { + return jsrRetryListeners; + } + /** - * Wrap the provided {@link #setRetryPolicy(RetryPolicy)} so that it never retries explicitly non-retryable + * Wrap the provided {@link org.springframework.retry.RetryPolicy} so that it never retries explicitly non-retryable * exceptions. */ private RetryPolicy getFatalExceptionAwareProxy(RetryPolicy retryPolicy) { NeverRetryPolicy neverRetryPolicy = new NeverRetryPolicy(); - Map, RetryPolicy> map = new HashMap, RetryPolicy>(); + Map, RetryPolicy> map = new HashMap<>(); for (Class fatal : nonRetryableExceptionClasses) { map.put(fatal, neverRetryPolicy); } - SubclassClassifier classifier = new SubclassClassifier( + SubclassClassifier classifier = new SubclassClassifier<>( retryPolicy); classifier.setTypeMap(map); @@ -590,15 +668,15 @@ private RetryPolicy getFatalExceptionAwareProxy(RetryPolicy retryPolicy) { * @param skipPolicy an existing skip policy * @return a skip policy that will not skip fatal exceptions */ - private SkipPolicy getFatalExceptionAwareProxy(SkipPolicy skipPolicy) { + protected SkipPolicy getFatalExceptionAwareProxy(SkipPolicy skipPolicy) { NeverSkipItemSkipPolicy neverSkipPolicy = new NeverSkipItemSkipPolicy(); - Map, SkipPolicy> map = new HashMap, SkipPolicy>(); + Map, SkipPolicy> map = new HashMap<>(); for (Class fatal : nonSkippableExceptionClasses) { map.put(fatal, neverSkipPolicy); } - SubclassClassifier classifier = new SubclassClassifier(skipPolicy); + SubclassClassifier classifier = new SubclassClassifier<>(skipPolicy); classifier.setTypeMap(map); ExceptionClassifierSkipPolicy skipPolicyWrapper = new ExceptionClassifierSkipPolicy(); @@ -606,8 +684,9 @@ private SkipPolicy getFatalExceptionAwareProxy(SkipPolicy skipPolicy) { return skipPolicyWrapper; } + @SuppressWarnings("unchecked") private void addNonSkippableExceptionIfMissing(Class... cls) { - List> exceptions = new ArrayList>(); + List> exceptions = new ArrayList<>(); for (Class exceptionClass : nonSkippableExceptionClasses) { exceptions.add(exceptionClass); } @@ -619,8 +698,9 @@ private void addNonSkippableExceptionIfMissing(Class... cls nonSkippableExceptionClasses = exceptions; } + @SuppressWarnings("unchecked") private void addNonRetryableExceptionIfMissing(Class... cls) { - List> exceptions = new ArrayList>(); + List> exceptions = new ArrayList<>(); for (Class exceptionClass : nonRetryableExceptionClasses) { exceptions.add(exceptionClass); } @@ -677,5 +757,21 @@ public void afterChunkError(ChunkContext context) { throw new FatalStepExecutionException("ChunkListener threw exception, rethrowing as fatal", t); } } + + @Override + public int hashCode() { + return chunkListener.hashCode(); + } + + @SuppressWarnings("unchecked") + @Override + public boolean equals(Object obj) { + if (obj instanceof FaultTolerantStepBuilder.TerminateOnExceptionChunkListenerDelegate){ + // unwrap the ChunkListener + obj = ((TerminateOnExceptionChunkListenerDelegate)obj).chunkListener; + } + return chunkListener.equals(obj); + } + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/FlowStepBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/FlowStepBuilder.java index 21ff668505..2e98516695 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/FlowStepBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/FlowStepBuilder.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/JobStepBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/JobStepBuilder.java index 9bbb30d9d7..2568508368 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/JobStepBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/JobStepBuilder.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/PartitionStepBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/PartitionStepBuilder.java index 99640ef790..fcf88cc62c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/PartitionStepBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/PartitionStepBuilder.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2011 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,9 +29,11 @@ /** * Step builder for {@link PartitionStep} instances. A partition step executes the same step (possibly remotely) * multiple times with different input parameters (in the form of execution context). Useful for parallelization. - * + * * @author Dave Syer - * + * @author Mahmoud Ben Hassine + * @author Dimitrios Liapis + * * @since 2.2 */ public class PartitionStepBuilder extends StepBuilderHelper { @@ -56,7 +58,7 @@ public class PartitionStepBuilder extends StepBuilderHelper parent) { @@ -66,13 +68,13 @@ public PartitionStepBuilder(StepBuilderHelper parent) { /** * Add a partitioner which can be used to create a {@link StepExecutionSplitter}. Use either this or an explicit * {@link #splitter(StepExecutionSplitter)} but not both. - * - * @param slaveStepName the name of the slave step (used to construct step execution names) + * + * @param workerStepName the name of the worker step (used to construct step execution names) * @param partitioner a partitioner to use * @return this for fluent chaining */ - public PartitionStepBuilder partitioner(String slaveStepName, Partitioner partitioner) { - this.stepName = slaveStepName; + public PartitionStepBuilder partitioner(String workerStepName, Partitioner partitioner) { + this.stepName = workerStepName; this.partitioner = partitioner; return this; } @@ -81,7 +83,7 @@ public PartitionStepBuilder partitioner(String slaveStepName, Partitioner partit * Provide an actual step instance to execute in parallel. If an explicit * {@link #partitionHandler(PartitionHandler)} is provided, the step is optional and is only used to extract * configuration data (name and other basic properties of a step). - * + * * @param step a step to execute in parallel * @return this for fluent chaining */ @@ -94,7 +96,7 @@ public PartitionStepBuilder step(Step step) { * Provide a task executor to use when constructing a {@link PartitionHandler} from the {@link #step(Step)}. Mainly * used for running a step locally in parallel, but can be used to execute remotely if the step is remote. Not used * if an explicit {@link #partitionHandler(PartitionHandler)} is provided. - * + * * @param taskExecutor a task executor to use when executing steps in parallel * @return this for fluent chaining */ @@ -107,9 +109,9 @@ public PartitionStepBuilder taskExecutor(TaskExecutor taskExecutor) { * Provide an explicit partition handler that will carry out the work of the partition step. The partition handler * is the main SPI for adapting a partition step to a specific distributed computation environment. Optional if you * only need local or remote processing through the Step interface. - * + * * @see #step(Step) for setting up a default handler that works with a local or remote Step - * + * * @param partitionHandler a partition handler * @return this for fluent chaining */ @@ -121,8 +123,8 @@ public PartitionStepBuilder partitionHandler(PartitionHandler partitionHandler) /** * A hint to the {@link #splitter(StepExecutionSplitter)} about how many step executions are required. If running * locally or remotely through a {@link #taskExecutor(TaskExecutor)} determines precisely the number of step - * execution sin the first attempt at a partition step execution. - * + * executions in the first attempt at a partition step execution. + * * @param gridSize the grid size * @return this for fluent chaining */ @@ -133,8 +135,8 @@ public PartitionStepBuilder gridSize(int gridSize) { /** * Provide an explicit {@link StepExecutionSplitter} instead of having one build from the - * {@link #partitioner(String, Partitioner)}. USeful if you need more control over the splitting. - * + * {@link #partitioner(String, Partitioner)}. Useful if you need more control over the splitting. + * * @param splitter a step execution splitter * @return this for fluent chaining */ @@ -146,7 +148,7 @@ public PartitionStepBuilder splitter(StepExecutionSplitter splitter) { /** * Provide a step execution aggregator for aggregating partitioned step executions into a single result for the * {@link PartitionStep} itself. Default is a simple implementation that works in most cases. - * + * * @param aggregator a step execution aggregator * @return this for fluent chaining */ @@ -156,7 +158,6 @@ public PartitionStepBuilder aggregator(StepExecutionAggregator aggregator) { } public Step build() { - PartitionStep step = new PartitionStep(); step.setName(getName()); super.enhance(step); @@ -217,4 +218,35 @@ public Step build() { } + protected TaskExecutor getTaskExecutor() { + return taskExecutor; + } + + protected Partitioner getPartitioner() { + return partitioner; + } + + protected Step getStep() { + return step; + } + + protected PartitionHandler getPartitionHandler() { + return partitionHandler; + } + + protected int getGridSize() { + return gridSize; + } + + protected StepExecutionSplitter getSplitter() { + return splitter; + } + + protected StepExecutionAggregator getAggregator() { + return aggregator; + } + + protected String getStepName() { + return stepName; + } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/SimpleStepBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/SimpleStepBuilder.java index 2b49a7f4b0..5eb78db585 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/SimpleStepBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/SimpleStepBuilder.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,9 +15,12 @@ */ package org.springframework.batch.core.step.builder; +import java.lang.reflect.Method; import java.util.ArrayList; +import java.util.HashSet; import java.util.LinkedHashSet; import java.util.Set; +import java.util.function.Function; import org.springframework.batch.core.ChunkListener; import org.springframework.batch.core.ItemProcessListener; @@ -25,6 +28,15 @@ import org.springframework.batch.core.ItemWriteListener; import org.springframework.batch.core.StepExecutionListener; import org.springframework.batch.core.StepListener; +import org.springframework.batch.core.annotation.AfterProcess; +import org.springframework.batch.core.annotation.AfterRead; +import org.springframework.batch.core.annotation.AfterWrite; +import org.springframework.batch.core.annotation.BeforeProcess; +import org.springframework.batch.core.annotation.BeforeRead; +import org.springframework.batch.core.annotation.BeforeWrite; +import org.springframework.batch.core.annotation.OnProcessError; +import org.springframework.batch.core.annotation.OnReadError; +import org.springframework.batch.core.annotation.OnWriteError; import org.springframework.batch.core.listener.StepListenerFactoryBean; import org.springframework.batch.core.step.item.ChunkOrientedTasklet; import org.springframework.batch.core.step.item.SimpleChunkProcessor; @@ -35,10 +47,12 @@ import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ItemStream; import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.function.FunctionItemProcessor; import org.springframework.batch.repeat.CompletionPolicy; import org.springframework.batch.repeat.RepeatOperations; import org.springframework.batch.repeat.policy.SimpleCompletionPolicy; import org.springframework.batch.repeat.support.RepeatTemplate; +import org.springframework.batch.support.ReflectionUtils; import org.springframework.util.Assert; /** @@ -49,6 +63,7 @@ * @see FaultTolerantStepBuilder for a step that handles retry and skip of failed items * * @author Dave Syer + * @author Mahmoud Ben Hassine * * @since 2.2 */ @@ -62,13 +77,15 @@ public class SimpleStepBuilder extends AbstractTaskletStepBuilder processor; + private Function itemProcessorFunction; + private int chunkSize = 0; private RepeatOperations chunkOperations; private CompletionPolicy completionPolicy; - private Set itemListeners = new LinkedHashSet(); + private Set itemListeners = new LinkedHashSet<>(); private boolean readerTransactionalQueue = false; @@ -99,8 +116,7 @@ protected SimpleStepBuilder(SimpleStepBuilder parent) { } public FaultTolerantStepBuilder faultTolerant() { - FaultTolerantStepBuilder builder = new FaultTolerantStepBuilder(this); - return builder; + return new FaultTolerantStepBuilder<>(this); } /** @@ -110,20 +126,44 @@ public FaultTolerantStepBuilder faultTolerant() { */ @Override public TaskletStep build() { + + registerStepListenerAsItemListener(); registerAsStreamsAndListeners(reader, processor, writer); return super.build(); } + protected void registerStepListenerAsItemListener() { + for (StepExecutionListener stepExecutionListener: properties.getStepExecutionListeners()){ + checkAndAddItemListener(stepExecutionListener); + } + for (ChunkListener chunkListener: this.chunkListeners){ + checkAndAddItemListener(chunkListener); + } + } + + @SuppressWarnings("unchecked") + private void checkAndAddItemListener(StepListener stepListener) { + if (stepListener instanceof ItemReadListener){ + listener((ItemReadListener)stepListener); + } + if (stepListener instanceof ItemProcessListener){ + listener((ItemProcessListener)stepListener); + } + if (stepListener instanceof ItemWriteListener){ + listener((ItemWriteListener)stepListener); + } + } + @Override protected Tasklet createTasklet() { Assert.state(reader != null, "ItemReader must be provided"); - Assert.state(processor != null || writer != null, "ItemWriter or ItemProcessor must be provided"); + Assert.state(writer != null, "ItemWriter must be provided"); RepeatOperations repeatOperations = createChunkOperations(); - SimpleChunkProvider chunkProvider = new SimpleChunkProvider(reader, repeatOperations); - SimpleChunkProcessor chunkProcessor = new SimpleChunkProcessor(processor, writer); - chunkProvider.setListeners(new ArrayList(itemListeners)); - chunkProcessor.setListeners(new ArrayList(itemListeners)); - ChunkOrientedTasklet tasklet = new ChunkOrientedTasklet(chunkProvider, chunkProcessor); + SimpleChunkProvider chunkProvider = new SimpleChunkProvider<>(getReader(), repeatOperations); + SimpleChunkProcessor chunkProcessor = new SimpleChunkProcessor<>(getProcessor(), getWriter()); + chunkProvider.setListeners(new ArrayList<>(itemListeners)); + chunkProcessor.setListeners(new ArrayList<>(itemListeners)); + ChunkOrientedTasklet tasklet = new ChunkOrientedTasklet<>(chunkProvider, chunkProcessor); tasklet.setBuffering(!readerTransactionalQueue); return tasklet; } @@ -194,6 +234,19 @@ public SimpleStepBuilder processor(ItemProcessor p return this; } + /** + * A {@link Function} to be delegated to as an {@link ItemProcessor}. If this is set, + * it will take precedence over any {@code ItemProcessor} configured via + * {@link #processor(ItemProcessor)}. + * + * @param function the function to delegate item processing to + * @return this for fluent chaining + */ + public SimpleStepBuilder processor(Function function) { + this.itemProcessorFunction = function; + return this; + } + /** * Sets a flag to say that the reader is transactional (usually a queue), which is to say that failed items might be * rolled back and re-presented in a subsequent transaction. Default is false, meaning that the items are read @@ -206,6 +259,40 @@ public SimpleStepBuilder readerIsTransactionalQueue() { return this; } + /** + * Registers objects using the annotation based listener configuration. + * + * @param listener the object that has a method configured with listener annotation + * @return this for fluent chaining + */ + @SuppressWarnings("unchecked") + @Override + public SimpleStepBuilder listener(Object listener) { + super.listener(listener); + + Set itemListenerMethods = new HashSet<>(); + itemListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), BeforeRead.class)); + itemListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), AfterRead.class)); + itemListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), BeforeProcess.class)); + itemListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), AfterProcess.class)); + itemListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), BeforeWrite.class)); + itemListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), AfterWrite.class)); + itemListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), OnReadError.class)); + itemListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), OnProcessError.class)); + itemListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), OnWriteError.class)); + + if(itemListenerMethods.size() > 0) { + StepListenerFactoryBean factory = new StepListenerFactoryBean(); + factory.setDelegate(listener); + itemListeners.add((StepListener) factory.getObject()); + } + + @SuppressWarnings("unchecked") + SimpleStepBuilder result = this; + return result; + } + + /** * Register an item reader listener. * @@ -243,7 +330,7 @@ public SimpleStepBuilder listener(ItemProcessListener chunkOperations(RepeatOperations repeatTemplate) { @@ -270,6 +357,10 @@ protected ItemWriter getWriter() { } protected ItemProcessor getProcessor() { + if(this.itemProcessorFunction != null) { + this.processor = new FunctionItemProcessor<>(this.itemProcessorFunction); + } + return processor; } @@ -288,7 +379,7 @@ protected Set getItemListeners() { /** * @return a {@link CompletionPolicy} consistent with the chunk size and injected policy (if present). */ - private CompletionPolicy getChunkCompletionPolicy() { + protected CompletionPolicy getChunkCompletionPolicy() { Assert.state(!(completionPolicy != null && chunkSize > 0), "You must specify either a chunkCompletionPolicy or a commitInterval but not both."); Assert.state(chunkSize >= 0, "The commitInterval must be positive or zero (for default value)."); @@ -303,7 +394,7 @@ private CompletionPolicy getChunkCompletionPolicy() { return new SimpleCompletionPolicy(chunkSize); } - private void registerAsStreamsAndListeners(ItemReader itemReader, + protected void registerAsStreamsAndListeners(ItemReader itemReader, ItemProcessor itemProcessor, ItemWriter itemWriter) { for (Object itemHandler : new Object[] { itemReader, itemWriter, itemProcessor }) { if (itemHandler instanceof ItemStream) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/StepBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/StepBuilder.java index d8a889c60d..efc8379451 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/StepBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/StepBuilder.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,16 +24,16 @@ /** * Convenient entry point for building all kinds of steps. Use this as a factory for fluent builders of any step. - * + * * @author Dave Syer - * + * * @since 2.2 */ public class StepBuilder extends StepBuilderHelper { /** * Initialize a step builder for a step with the given name. - * + * * @param name the name of the step */ public StepBuilder(String name) { @@ -42,7 +42,7 @@ public StepBuilder(String name) { /** * Build a step with a custom tasklet, not necessarily item processing. - * + * * @param tasklet a tasklet * @return a {@link TaskletStepBuilder} */ @@ -54,16 +54,16 @@ public TaskletStepBuilder tasklet(Tasklet tasklet) { * Build a step that processes items in chunks with the size provided. To extend the step to being fault tolerant, * call the {@link SimpleStepBuilder#faultTolerant()} method on the builder. In most cases you will want to * parameterize your call to this method, to preserve the type safety of your readers and writers, e.g. - * + * *
       	 * new StepBuilder("step1").<Order, Ledger> chunk(100).reader(new OrderReader()).writer(new LedgerWriter())
       	 * // ... etc.
       	 * 
      - * + * * @param chunkSize the chunk size (commit interval) * @return a {@link SimpleStepBuilder} - * @param I the type of item to be processed as input - * @param O the type of item to be output + * @param the type of item to be processed as input + * @param the type of item to be output */ public SimpleStepBuilder chunk(int chunkSize) { return new SimpleStepBuilder(this).chunk(chunkSize); @@ -73,16 +73,16 @@ public SimpleStepBuilder chunk(int chunkSize) { * Build a step that processes items in chunks with the completion policy provided. To extend the step to being * fault tolerant, call the {@link SimpleStepBuilder#faultTolerant()} method on the builder. In most cases you will * want to parameterize your call to this method, to preserve the type safety of your readers and writers, e.g. - * + * *
       	 * new StepBuilder("step1").<Order, Ledger> chunk(100).reader(new OrderReader()).writer(new LedgerWriter())
       	 * // ... etc.
       	 * 
      - * + * * @param completionPolicy the completion policy to use to control chunk processing * @return a {@link SimpleStepBuilder} - * @param I the type of item to be processed as input - * @param O the type of item to be output * + * @param the type of item to be processed as input + * @param the type of item to be output * */ public SimpleStepBuilder chunk(CompletionPolicy completionPolicy) { return new SimpleStepBuilder(this).chunk(completionPolicy); @@ -90,7 +90,7 @@ public SimpleStepBuilder chunk(CompletionPolicy completionPolicy) { /** * Create a partition step builder for a remote (or local) step. - * + * * @param stepName the name of the remote or delegate step * @param partitioner a partitioner to be used to construct new step executions * @return a {@link PartitionStepBuilder} @@ -101,7 +101,7 @@ public PartitionStepBuilder partitioner(String stepName, Partitioner partitioner /** * Create a partition step builder for a remote (or local) step. - * + * * @param step the step to execute in parallel * @return a PartitionStepBuilder */ @@ -111,7 +111,7 @@ public PartitionStepBuilder partitioner(Step step) { /** * Create a new step builder that will execute a job. - * + * * @param job a job to execute * @return a {@link JobStepBuilder} */ @@ -121,7 +121,7 @@ public JobStepBuilder job(Job job) { /** * Create a new step builder that will execute a flow. - * + * * @param flow a flow to execute * @return a {@link FlowStepBuilder} */ diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/StepBuilderException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/StepBuilderException.java index 1415817aa4..f6de1fe268 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/StepBuilderException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/StepBuilderException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ * * @since 2.2 */ +@SuppressWarnings("serial") public class StepBuilderException extends RuntimeException { public StepBuilderException(Exception e) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/StepBuilderHelper.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/StepBuilderHelper.java index 03f2e16ecc..5b6fb190e4 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/StepBuilderHelper.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/StepBuilderHelper.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2011 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,31 +15,39 @@ */ package org.springframework.batch.core.step.builder; -import java.util.ArrayList; -import java.util.List; - import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.core.Step; import org.springframework.batch.core.StepExecutionListener; +import org.springframework.batch.core.annotation.AfterStep; +import org.springframework.batch.core.annotation.BeforeStep; +import org.springframework.batch.core.listener.StepListenerFactoryBean; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.step.AbstractStep; import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.batch.support.ReflectionUtils; import org.springframework.transaction.PlatformTransactionManager; +import java.lang.reflect.Method; +import java.util.ArrayList; +import java.util.HashSet; +import java.util.List; +import java.util.Set; + /** * A base class and utility for other step builders providing access to common properties like job repository and * transaction manager. * * @author Dave Syer - * + * @author Michael Minella + * * @since 2.2 */ public abstract class StepBuilderHelper> { protected final Log logger = LogFactory.getLog(getClass()); - private final CommonStepProperties properties; + protected final CommonStepProperties properties; public StepBuilderHelper(String name) { this.properties = new CommonStepProperties(); @@ -76,6 +84,28 @@ public B startLimit(int startLimit) { return result; } + /** + * Registers objects using the annotation based listener configuration. + * + * @param listener the object that has a method configured with listener annotation + * @return this for fluent chaining + */ + public B listener(Object listener) { + Set stepExecutionListenerMethods = new HashSet<>(); + stepExecutionListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), BeforeStep.class)); + stepExecutionListenerMethods.addAll(ReflectionUtils.findMethod(listener.getClass(), AfterStep.class)); + + if(stepExecutionListenerMethods.size() > 0) { + StepListenerFactoryBean factory = new StepListenerFactoryBean(); + factory.setDelegate(listener); + properties.addStepExecutionListener((StepExecutionListener) factory.getObject()); + } + + @SuppressWarnings("unchecked") + B result = (B) this; + return result; + } + public B listener(StepExecutionListener listener) { properties.addStepExecutionListener(listener); @SuppressWarnings("unchecked") @@ -136,7 +166,7 @@ protected void enhance(Step target) { public static class CommonStepProperties { - private List stepExecutionListeners = new ArrayList(); + private List stepExecutionListeners = new ArrayList<>(); private int startLimit = Integer.MAX_VALUE; @@ -155,7 +185,7 @@ public CommonStepProperties(CommonStepProperties properties) { this.allowStartIfComplete = properties.allowStartIfComplete; this.jobRepository = properties.jobRepository; this.transactionManager = properties.transactionManager; - this.stepExecutionListeners = new ArrayList(properties.stepExecutionListeners); + this.stepExecutionListeners = new ArrayList<>(properties.stepExecutionListeners); } public JobRepository getJobRepository() { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/TaskletStepBuilder.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/TaskletStepBuilder.java index 81edc487be..68fb9d94ff 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/TaskletStepBuilder.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/TaskletStepBuilder.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -38,7 +38,7 @@ public TaskletStepBuilder(StepBuilderHelper parent) { } /** - * @param tasklet tehe tasklet to use + * @param tasklet the tasklet to use * @return this for fluent chaining */ public TaskletStepBuilder tasklet(Tasklet tasklet) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/package-info.java new file mode 100644 index 0000000000..e8d832a425 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/builder/package-info.java @@ -0,0 +1,10 @@ +/** + * Step level builders for java based job configuration. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.step.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/BatchListenerFactoryHelper.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/BatchListenerFactoryHelper.java index 72a5ebe1b6..901ca75b3e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/BatchListenerFactoryHelper.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/BatchListenerFactoryHelper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,7 +29,7 @@ abstract class BatchListenerFactoryHelper { public static List getListeners(StepListener[] listeners, Class cls) { - List list = new ArrayList(); + List list = new ArrayList<>(); for (StepListener stepListener : listeners) { if (cls.isAssignableFrom(stepListener.getClass())) { @SuppressWarnings("unchecked") diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/FaultTolerantStepFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/FaultTolerantStepFactoryBean.java index 566847214f..1626540700 100755 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/FaultTolerantStepFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/FaultTolerantStepFactoryBean.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -50,11 +50,11 @@ */ public class FaultTolerantStepFactoryBean extends SimpleStepFactoryBean { - private Map, Boolean> skippableExceptionClasses = new HashMap, Boolean>(); + private Map, Boolean> skippableExceptionClasses = new HashMap<>(); - private Collection> noRollbackExceptionClasses = new HashSet>(); + private Collection> noRollbackExceptionClasses = new HashSet<>(); - private Map, Boolean> retryableExceptionClasses = new HashMap, Boolean>(); + private Map, Boolean> retryableExceptionClasses = new HashMap<>(); private int cacheCapacity = 0; @@ -109,11 +109,11 @@ public void setRetryLimit(int retryLimit) { /** * Public setter for the capacity of the cache in the retry policy. If more items than this fail without being * skipped or recovered an exception will be thrown. This is to guard against inadvertent infinite loops generated - * by item identity problems.
      + * by item identity problems.
      * * The default value should be high enough and more for most purposes. To breach the limit in a single-threaded step * typically you have to have this many failures in a single transaction. Defaults to the value in the - * {@link MapRetryContextCache}.
      + * {@link MapRetryContextCache}.
      * * This property is ignored if the {@link #setRetryContextCache(RetryContextCache)} is set directly. * @@ -186,7 +186,7 @@ public void setSkipPolicy(SkipPolicy skipPolicy) { * Exception classes that when raised won't crash the job but will result in the item which handling caused the * exception being skipped. Any exception which is marked for "no rollback" is also skippable, but not vice versa. * Remember to set the {@link #setSkipLimit(int) skip limit} as well. - *

      + *
      * Defaults to all no exception. * * @param exceptionClasses defaults to Exception @@ -199,7 +199,7 @@ public void setSkippableExceptionClasses(Map, Boolean * Exception classes that are candidates for no rollback. The {@link Step} can not honour the no rollback hint in * all circumstances, but any exception on this list is counted as skippable, so even if there has to be a rollback, * then the step will not fail as long as the skip limit is not breached. - *

      + *
      * Defaults is empty. * * @param noRollbackExceptionClasses the exception classes to set @@ -209,7 +209,7 @@ public void setNoRollbackExceptionClasses(Collection> } /** - * @param processorTransactional + * @param processorTransactional boolean indicates if the {@code ItemProcessor} participates in the transaction. */ public void setProcessorTransactional(boolean processorTransactional) { this.processorTransactional = processorTransactional; @@ -217,7 +217,7 @@ public void setProcessorTransactional(boolean processorTransactional) { @Override protected SimpleStepBuilder createBuilder(String name) { - return new FaultTolerantStepBuilder(new StepBuilder(name)); + return new FaultTolerantStepBuilder<>(new StepBuilder(name)); } @Override diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/SimpleStepFactoryBean.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/SimpleStepFactoryBean.java index f17e3dcf99..f149337a4c 100755 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/SimpleStepFactoryBean.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/SimpleStepFactoryBean.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -60,8 +60,7 @@ * @author Robert Kasanicky * */ -@SuppressWarnings("rawtypes") -public class SimpleStepFactoryBean implements FactoryBean, BeanNameAware { +public class SimpleStepFactoryBean implements FactoryBean, BeanNameAware { private String name; @@ -317,7 +316,7 @@ public boolean rollbackOn(Throwable ex) { * @see FactoryBean#getObject() */ @Override - public final Object getObject() throws Exception { + public final Step getObject() throws Exception { SimpleStepBuilder builder = createBuilder(getName()); applyConfiguration(builder); TaskletStep step = builder.build(); @@ -325,7 +324,7 @@ public final Object getObject() throws Exception { } protected SimpleStepBuilder createBuilder(String name) { - return new SimpleStepBuilder(new StepBuilder(name)); + return new SimpleStepBuilder<>(new StepBuilder(name)); } @Override @@ -334,7 +333,7 @@ public Class getObjectType() { } /** - * Returns true by default, but in most cases a {@link Step} should not be treated as thread safe. Clients are + * Returns true by default, but in most cases a {@link Step} should not be treated as thread-safe. Clients are * recommended to create a new step for each job execution. * * @see org.springframework.beans.factory.FactoryBean#isSingleton() @@ -431,7 +430,7 @@ public void setTaskExecutor(TaskExecutor taskExecutor) { } /** - * Mkae the {@link TaskExecutor} available to subclasses + * Make the {@link TaskExecutor} available to subclasses * @return the taskExecutor to be used to execute chunks */ protected TaskExecutor getTaskExecutor() { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/package-info.java new file mode 100644 index 0000000000..030917bc84 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/factory/package-info.java @@ -0,0 +1,10 @@ +/** + * Factories for step level components. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.step.factory; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/BatchRetryTemplate.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/BatchRetryTemplate.java index 50b5c2f97d..1dd93e5153 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/BatchRetryTemplate.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/BatchRetryTemplate.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,11 +16,6 @@ package org.springframework.batch.core.step.item; -import java.util.ArrayList; -import java.util.Collection; -import java.util.Iterator; -import java.util.List; - import org.springframework.classify.Classifier; import org.springframework.retry.ExhaustedRetryException; import org.springframework.retry.RecoveryCallback; @@ -37,6 +32,11 @@ import org.springframework.retry.support.RetrySynchronizationManager; import org.springframework.retry.support.RetryTemplate; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Iterator; +import java.util.List; + /** * A special purpose retry template that deals specifically with multi-valued * stateful retry. This is useful in the case where the operation to be retried @@ -63,7 +63,7 @@ private class BatchRetryState extends DefaultRetryState { public BatchRetryState(Collection keys) { super(keys); - this.keys = new ArrayList(keys); + this.keys = new ArrayList<>(keys); } } @@ -114,7 +114,7 @@ protected RetryContext open(RetryPolicy retryPolicy, RetryState state) { BatchRetryState batchState = (BatchRetryState) state; - Collection contexts = new ArrayList(); + Collection contexts = new ArrayList<>(); for (RetryState retryState : batchState.keys) { contexts.add(super.open(retryPolicy, retryState)); } @@ -153,13 +153,13 @@ protected void close(RetryPolicy retryPolicy, RetryContext context, RetryState s @Override protected T handleRetryExhausted(RecoveryCallback recoveryCallback, RetryContext context, - RetryState state) throws Exception { + RetryState state) throws Throwable { BatchRetryState batchState = (BatchRetryState) state; BatchRetryContext batchContext = (BatchRetryContext) context; // Accumulate exceptions to be thrown so all the keys get a crack - Exception rethrowable = null; + Throwable rethrowable = null; ExhaustedRetryException exhausted = null; Iterator contextIterator = batchContext.contexts.iterator(); @@ -173,7 +173,7 @@ protected T handleRetryExhausted(RecoveryCallback recoveryCallback, Retry catch (ExhaustedRetryException e) { exhausted = e; } - catch (Exception e) { + catch (Throwable e) { rethrowable = e; } @@ -199,42 +199,42 @@ protected T handleRetryExhausted(RecoveryCallback recoveryCallback, Retry private RetryPolicy retryPolicy; - public T execute(RetryCallback retryCallback, Collection states) throws ExhaustedRetryException, + public T execute(RetryCallback retryCallback, Collection states) throws E, Exception { RetryState batchState = new BatchRetryState(states); return delegate.execute(retryCallback, batchState); } - public T execute(RetryCallback retryCallback, RecoveryCallback recoveryCallback, - Collection states) throws ExhaustedRetryException, Exception { + public T execute(RetryCallback retryCallback, RecoveryCallback recoveryCallback, + Collection states) throws E, Exception { RetryState batchState = new BatchRetryState(states); return delegate.execute(retryCallback, recoveryCallback, batchState); } @Override - public final T execute(RetryCallback retryCallback, RecoveryCallback recoveryCallback, - RetryState retryState) throws Exception, ExhaustedRetryException { + public final T execute(RetryCallback retryCallback, RecoveryCallback recoveryCallback, + RetryState retryState) throws E { return regular.execute(retryCallback, recoveryCallback, retryState); } @Override - public final T execute(RetryCallback retryCallback, RecoveryCallback recoveryCallback) throws Exception { + public final T execute(RetryCallback retryCallback, RecoveryCallback recoveryCallback) throws E { return regular.execute(retryCallback, recoveryCallback); } @Override - public final T execute(RetryCallback retryCallback, RetryState retryState) throws Exception, + public final T execute(RetryCallback retryCallback, RetryState retryState) throws E, ExhaustedRetryException { return regular.execute(retryCallback, retryState); } @Override - public final T execute(RetryCallback retryCallback) throws Exception { + public final T execute(RetryCallback retryCallback) throws E { return regular.execute(retryCallback); } public static List createState(List keys) { - List states = new ArrayList(); + List states = new ArrayList<>(); for (Object key : keys) { states.add(new DefaultRetryState(key)); } @@ -242,7 +242,7 @@ public static List createState(List keys) { } public static List createState(List keys, Classifier classifier) { - List states = new ArrayList(); + List states = new ArrayList<>(); for (Object key : keys) { states.add(new DefaultRetryState(key, classifier)); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/Chunk.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/Chunk.java index dfce3e0cd9..ae4e38639e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/Chunk.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/Chunk.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,11 +34,11 @@ */ public class Chunk implements Iterable { - private List items = new ArrayList(); + private List items = new ArrayList<>(); - private List> skips = new ArrayList>(); + private List> skips = new ArrayList<>(); - private List errors = new ArrayList(); + private List errors = new ArrayList<>(); private Object userData; @@ -57,16 +57,16 @@ public Chunk(Collection items) { public Chunk(Collection items, List> skips) { super(); if (items != null) { - this.items = new ArrayList(items); + this.items = new ArrayList<>(items); } if (skips != null) { - this.skips = new ArrayList>(skips); + this.skips = new ArrayList<>(skips); } } /** * Add the item to the chunk. - * @param item + * @param item the item to add */ public void add(W item) { items.add(item); @@ -85,7 +85,7 @@ public void clear() { * @return a copy of the items to be processed as an unmodifiable list */ public List getItems() { - return Collections.unmodifiableList(new ArrayList(items)); + return Collections.unmodifiableList(new ArrayList<>(items)); } /** @@ -96,7 +96,7 @@ public List> getSkips() { } /** - * @return a copy of the anonymous errros as an unmodifiable list + * @return a copy of the anonymous errors as an unmodifiable list */ public List getErrors() { return Collections.unmodifiableList(errors); @@ -227,7 +227,7 @@ public W next() { public void remove(Throwable e) { remove(); - skips.add(new SkipWrapper(next, e)); + skips.add(new SkipWrapper<>(next, e)); } @Override diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkMonitor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkMonitor.java index eb871790e9..c452c2a064 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkMonitor.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkMonitor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,12 +23,11 @@ import org.springframework.batch.item.ItemStreamException; import org.springframework.batch.item.ItemStreamSupport; import org.springframework.batch.item.support.CompositeItemStream; -import org.springframework.util.ClassUtils; /** * Manage the offset data between the last successful commit and updates made to * an input chunk. Only works with single threaded steps because it has to use a - * {@link ThreadLocal} to manage the state and co-ordinate between the caller + * {@link ThreadLocal} to manage the state and coordinate between the caller * and the wrapped {@link ItemStream}. * * @author Dave Syer @@ -55,14 +54,14 @@ public ChunkMonitorData(int offset, int chunkSize) { private CompositeItemStream stream = new CompositeItemStream(); - private ThreadLocal holder = new ThreadLocal(); + private ThreadLocal holder = new ThreadLocal<>(); private ItemReader reader; - public ChunkMonitor() { - this.setExecutionContextName(ChunkMonitor.class.getName()); - } - + public ChunkMonitor() { + this.setExecutionContextName(ChunkMonitor.class.getName()); + } + /** * @param stream the stream to set */ @@ -101,7 +100,7 @@ public void setChunkSize(int chunkSize) { @Override public void close() throws ItemStreamException { - super.close(); + super.close(); holder.set(null); if (streamsRegistered) { stream.close(); @@ -110,7 +109,7 @@ public void close() throws ItemStreamException { @Override public void open(ExecutionContext executionContext) throws ItemStreamException { - super.open(executionContext); + super.open(executionContext); if (streamsRegistered) { stream.open(executionContext); ChunkMonitorData data = new ChunkMonitorData(executionContext.getInt(getExecutionContextKey(OFFSET), 0), 0); @@ -127,18 +126,21 @@ public void open(ExecutionContext executionContext) throws ItemStreamException { throw new ItemStreamException("Could not position reader with offset: " + data.offset, e); } } + + resetOffset(); } } @Override public void update(ExecutionContext executionContext) throws ItemStreamException { - super.update(executionContext); + super.update(executionContext); if (streamsRegistered) { ChunkMonitorData data = getData(); if (data.offset == 0) { // Only call the underlying update method if we are on a chunk // boundary stream.update(executionContext); + executionContext.remove(getExecutionContextKey(OFFSET)); } else { executionContext.putInt(getExecutionContextKey(OFFSET), data.offset); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkOrientedTasklet.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkOrientedTasklet.java index 1fd2bcf8b7..5aabc201bc 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkOrientedTasklet.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkOrientedTasklet.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; /** * A {@link Tasklet} implementing variations on read-process-write item @@ -54,12 +55,13 @@ public ChunkOrientedTasklet(ChunkProvider chunkProvider, ChunkProcessor ch * readers. Main (or only) use case for setting this flag to false is a * transactional JMS item reader. * - * @param buffering + * @param buffering indicator */ public void setBuffering(boolean buffering) { this.buffering = buffering; } + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { @@ -85,7 +87,9 @@ public RepeatStatus execute(StepContribution contribution, ChunkContext chunkCon chunkContext.removeAttribute(INPUTS_KEY); chunkContext.setComplete(); - logger.debug("Inputs not busy, ended: " + inputs.isEnd()); + if (logger.isDebugEnabled()) { + logger.debug("Inputs not busy, ended: " + inputs.isEnd()); + } return RepeatStatus.continueIf(!inputs.isEnd()); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkProcessor.java index a99e13f3b0..76a7b7e35e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkProcessor.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkProcessor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkProvider.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkProvider.java index 6a8da53c3e..c87b225a03 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkProvider.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ChunkProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/DefaultItemFailureHandler.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/DefaultItemFailureHandler.java index 819f633549..936da3ed8a 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/DefaultItemFailureHandler.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/DefaultItemFailureHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/FaultTolerantChunkProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/FaultTolerantChunkProcessor.java index c3956ac49f..60bdee07f0 100755 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/FaultTolerantChunkProcessor.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/FaultTolerantChunkProcessor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,7 +25,9 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.listener.StepListenerFailedException; import org.springframework.batch.core.step.skip.LimitCheckingItemSkipPolicy; import org.springframework.batch.core.step.skip.NonSkippableProcessException; import org.springframework.batch.core.step.skip.SkipLimitExceededException; @@ -96,14 +98,14 @@ public void setWriteSkipPolicy(SkipPolicy SkipPolicy) { * A classifier that can distinguish between exceptions that cause rollback * (return true) or not (return false). * - * @param rollbackClassifier + * @param rollbackClassifier classifier */ public void setRollbackClassifier(Classifier rollbackClassifier) { this.rollbackClassifier = rollbackClassifier; } /** - * @param chunkMonitor + * @param chunkMonitor monitor */ public void setChunkMonitor(ChunkMonitor chunkMonitor) { this.chunkMonitor = chunkMonitor; @@ -115,7 +117,7 @@ public void setChunkMonitor(ChunkMonitor chunkMonitor) { * complicated because after a rollback the new chunk might or might not * contain items from the previous failed chunk. * - * @param buffering + * @param buffering true if items will be buffered */ public void setBuffering(boolean buffering) { this.buffering = buffering; @@ -143,9 +145,15 @@ protected void initializeUserData(Chunk inputs) { @SuppressWarnings("unchecked") UserData data = (UserData) inputs.getUserData(); if (data == null) { - data = new UserData(); + data = new UserData<>(); inputs.setUserData(data); - data.setOutputs(new Chunk()); + data.setOutputs(new Chunk<>()); + } + else { + // BATCH-2663: re-initialize filter count when scanning the chunk + if (data.scanning()) { + data.filterCount = 0; + } } } @@ -183,7 +191,7 @@ protected Chunk getAdjustedOutputs(Chunk inputs, Chunk outputs) { UserData data = (UserData) inputs.getUserData(); Chunk previous = data.getOutputs(); - Chunk next = new Chunk(outputs.getItems(), previous.getSkips()); + Chunk next = new Chunk<>(outputs.getItems(), previous.getSkips()); next.setBusy(previous.isBusy()); // Remember for next time if there are skips accumulating @@ -196,11 +204,11 @@ protected Chunk getAdjustedOutputs(Chunk inputs, Chunk outputs) { @Override protected Chunk transform(final StepContribution contribution, Chunk inputs) throws Exception { - Chunk outputs = new Chunk(); + Chunk outputs = new Chunk<>(); @SuppressWarnings("unchecked") final UserData data = (UserData) inputs.getUserData(); final Chunk cache = data.getOutputs(); - final Iterator cacheIterator = cache.isEmpty() ? null : new ArrayList(cache.getItems()).iterator(); + final Iterator cacheIterator = cache.isEmpty() ? null : new ArrayList<>(cache.getItems()).iterator(); final AtomicInteger count = new AtomicInteger(0); // final int scanLimit = processorTransactional && data.scanning() ? 1 : @@ -210,7 +218,7 @@ protected Chunk transform(final StepContribution contribution, Chunk input final I item = iterator.next(); - RetryCallback retryCallback = new RetryCallback() { + RetryCallback retryCallback = new RetryCallback() { @Override public O doWithRetry(RetryContext context) throws Exception { @@ -256,7 +264,7 @@ else if (shouldSkip(itemProcessSkipPolicy, e, contribution.getStepSkipCount())) } if (output == null) { // No need to re-process filtered items - iterator.remove(); + iterator.remove(); } return output; } @@ -313,15 +321,13 @@ public O recover(RetryContext context) throws Exception { @Override protected void write(final StepContribution contribution, final Chunk inputs, final Chunk outputs) throws Exception { - @SuppressWarnings("unchecked") final UserData data = (UserData) inputs.getUserData(); - final AtomicReference contextHolder = new AtomicReference(); + final AtomicReference contextHolder = new AtomicReference<>(); - RetryCallback retryCallback = new RetryCallback() { + RetryCallback retryCallback = new RetryCallback() { @Override public Object doWithRetry(RetryContext context) throws Exception { - contextHolder.set(context); if (!data.scanning()) { @@ -396,7 +402,6 @@ public Object recover(RetryContext context) throws Exception { @Override public Object recover(RetryContext context) throws Exception { - /* * If the last exception was not skippable we don't need to * do any scanning. We can just bomb out with a retry @@ -516,7 +521,7 @@ private List getInputKeys(final Chunk inputs) { if (keyGenerator == null) { return inputs.getItems(); } - List keys = new ArrayList(); + List keys = new ArrayList<>(); for (I item : inputs.getItems()) { keys.add(keyGenerator.getKey(item)); } @@ -565,15 +570,24 @@ private void scan(final StepContribution contribution, final Chunk inputs, fi logger.debug("Scanning for failed item on write: " + inputs); } } - if (outputs.isEmpty()) { + if (outputs.isEmpty() || inputs.isEmpty()) { data.scanning(false); inputs.setBusy(false); + chunkMonitor.resetOffset(); return; } Chunk.ChunkIterator inputIterator = inputs.iterator(); Chunk.ChunkIterator outputIterator = outputs.iterator(); + //BATCH-2442 : do not scan skipped items + if (!inputs.getSkips().isEmpty()) { + if (outputIterator.hasNext()) { + outputIterator.remove(); + return; + } + } + List items = Collections.singletonList(outputIterator.next()); inputIterator.next(); try { @@ -586,16 +600,25 @@ private void scan(final StepContribution contribution, final Chunk inputs, fi outputIterator.remove(); } catch (Exception e) { - doOnWriteError(e, items); - if (!shouldSkip(itemWriteSkipPolicy, e, -1) && !rollbackClassifier.classify(e)) { - inputIterator.remove(); - outputIterator.remove(); - } - else { - checkSkipPolicy(inputIterator, outputIterator, e, contribution, recovery); + try { + doOnWriteError(e, items); } - if (rollbackClassifier.classify(e)) { - throw e; + finally { + Throwable cause = e; + if(e instanceof StepListenerFailedException) { + cause = e.getCause(); + } + + if (!shouldSkip(itemWriteSkipPolicy, cause, -1) && !rollbackClassifier.classify(cause)) { + inputIterator.remove(); + outputIterator.remove(); + } + else { + checkSkipPolicy(inputIterator, outputIterator, cause, contribution, recovery); + } + if (rollbackClassifier.classify(cause)) { + throw (Exception) cause; + } } } chunkMonitor.incrementOffset(); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/FaultTolerantChunkProvider.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/FaultTolerantChunkProvider.java index e8fcf0e8a0..498d6d6344 100755 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/FaultTolerantChunkProvider.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/FaultTolerantChunkProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,8 +16,6 @@ package org.springframework.batch.core.step.item; -import org.springframework.classify.BinaryExceptionClassifier; -import org.springframework.classify.Classifier; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.step.skip.LimitCheckingItemSkipPolicy; import org.springframework.batch.core.step.skip.NonSkippableReadException; @@ -27,6 +25,8 @@ import org.springframework.batch.core.step.skip.SkipPolicyFailedException; import org.springframework.batch.item.ItemReader; import org.springframework.batch.repeat.RepeatOperations; +import org.springframework.classify.BinaryExceptionClassifier; +import org.springframework.classify.Classifier; /** * FaultTolerant implementation of the {@link ChunkProcessor} interface, that @@ -53,7 +53,7 @@ public class FaultTolerantChunkProvider extends SimpleChunkProvider { public FaultTolerantChunkProvider(ItemReader itemReader, RepeatOperations repeatOperations) { super(itemReader, repeatOperations); } - + /** * @param maxSkipsOnRead the maximum number of skips on read */ @@ -63,15 +63,15 @@ public void setMaxSkipsOnRead(int maxSkipsOnRead) { /** * The policy that determines whether exceptions can be skipped on read. - * @param SkipPolicy + * @param skipPolicy instance of {@link SkipPolicy} to be used by FaultTolerantChunkProvider. */ - public void setSkipPolicy(SkipPolicy SkipPolicy) { - this.skipPolicy = SkipPolicy; + public void setSkipPolicy(SkipPolicy skipPolicy) { + this.skipPolicy = skipPolicy; } /** * Classifier to determine whether exceptions have been marked as - * no-rollback (as opposed to skippable). If ecnounterd they are simply + * no-rollback (as opposed to skippable). If encountered they are simply * ignored, unless also skippable. * * @param rollbackClassifier the rollback classifier to set @@ -89,6 +89,7 @@ protected I read(StepContribution contribution, Chunk chunk) throws Exception catch (Exception e) { if (shouldSkip(skipPolicy, e, contribution.getStepSkipCount())) { + // increment skip count and try again contribution.incrementReadSkipCount(); chunk.skip(e); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ForceRollbackForWriteSkipException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ForceRollbackForWriteSkipException.java index bd602c5ed7..7c42de7ad1 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ForceRollbackForWriteSkipException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/ForceRollbackForWriteSkipException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,6 +23,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class ForceRollbackForWriteSkipException extends RuntimeException { public ForceRollbackForWriteSkipException(String msg, Throwable cause) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/KeyGenerator.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/KeyGenerator.java index f9cda2fabc..49da956d22 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/KeyGenerator.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/KeyGenerator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SimpleChunkProcessor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SimpleChunkProcessor.java index 485f89a7f5..b93ca0dad6 100755 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SimpleChunkProcessor.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SimpleChunkProcessor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,12 +18,18 @@ import java.util.List; +import io.micrometer.core.instrument.Tag; +import io.micrometer.core.instrument.Timer; + import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.StepListener; import org.springframework.batch.core.listener.MulticasterBatchListener; +import org.springframework.batch.core.metrics.BatchMetrics; import org.springframework.batch.item.ItemProcessor; import org.springframework.batch.item.ItemWriter; import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -39,22 +45,25 @@ public class SimpleChunkProcessor implements ChunkProcessor, Initializi private ItemWriter itemWriter; - private final MulticasterBatchListener listener = new MulticasterBatchListener(); + private final MulticasterBatchListener listener = new MulticasterBatchListener<>(); /** - * Default constructor for ease of configuration (both itemWriter and - * itemProcessor are mandatory). + * Default constructor for ease of configuration. */ @SuppressWarnings("unused") private SimpleChunkProcessor() { this(null, null); } - public SimpleChunkProcessor(ItemProcessor itemProcessor, ItemWriter itemWriter) { + public SimpleChunkProcessor(@Nullable ItemProcessor itemProcessor, ItemWriter itemWriter) { this.itemProcessor = itemProcessor; this.itemWriter = itemWriter; } + public SimpleChunkProcessor(ItemWriter itemWriter) { + this(null, itemWriter); + } + /** * @param itemProcessor the {@link ItemProcessor} to set */ @@ -77,14 +86,13 @@ public void setItemWriter(ItemWriter itemWriter) { @Override public void afterPropertiesSet() throws Exception { Assert.notNull(itemWriter, "ItemWriter must be set"); - Assert.notNull(itemProcessor, "ItemProcessor must be set"); } /** * Register some {@link StepListener}s with the handler. Each will get the * callbacks in the order specified at the correct stage. * - * @param listeners + * @param listeners list of {@link StepListener} instances. */ public void setListeners(List listeners) { for (StepListener listener : listeners) { @@ -111,7 +119,7 @@ protected MulticasterBatchListener getListener() { /** * @param item the input item * @return the result of the processing - * @throws Exception + * @throws Exception thrown if error occurs. */ protected final O doProcess(I item) throws Exception { @@ -137,8 +145,8 @@ protected final O doProcess(I item) throws Exception { /** * Surrounds the actual write call with listener callbacks. * - * @param items - * @throws Exception + * @param items list of items to be written. + * @throws Exception thrown if error occurs. */ protected final void doWrite(List items) throws Exception { @@ -161,15 +169,25 @@ protected final void doWrite(List items) throws Exception { /** * Call the listener's after write method. * - * @param items + * @param items list of items that were just written. */ protected final void doAfterWrite(List items) { listener.afterWrite(items); } + + /** + * Call listener's writerError method. + * @param e exception that occurred. + * @param items list of items that failed to be written. + */ protected final void doOnWriteError(Exception e, List items) { listener.onWriteError(e, items); } + /** + * @param items list of items to be written. + * @throws Exception thrown if error occurs. + */ protected void writeItems(List items) throws Exception { if (itemWriter != null) { itemWriter.write(items); @@ -265,11 +283,13 @@ protected Chunk getAdjustedOutputs(Chunk inputs, Chunk outputs) { * skipped they should be removed from the inputs as well. * * @param contribution the current step contribution - * @param inputs the inputs that gave rise to the ouputs + * @param inputs the inputs that gave rise to the outputs * @param outputs the outputs to write * @throws Exception if there is a problem */ protected void write(StepContribution contribution, Chunk inputs, Chunk outputs) throws Exception { + Timer.Sample sample = BatchMetrics.createTimerSample(); + String status = BatchMetrics.STATUS_SUCCESS; try { doWrite(outputs.getItems()); } @@ -279,16 +299,22 @@ protected void write(StepContribution contribution, Chunk inputs, Chunk ou * here, so prevent any more processing of these inputs. */ inputs.clear(); + status = BatchMetrics.STATUS_FAILURE; throw e; } + finally { + stopTimer(sample, contribution.getStepExecution(), "chunk.write", status, "Chunk writing"); + } contribution.incrementWriteCount(outputs.size()); } protected Chunk transform(StepContribution contribution, Chunk inputs) throws Exception { - Chunk outputs = new Chunk(); + Chunk outputs = new Chunk<>(); for (Chunk.ChunkIterator iterator = inputs.iterator(); iterator.hasNext();) { final I item = iterator.next(); O output; + Timer.Sample sample = BatchMetrics.createTimerSample(); + String status = BatchMetrics.STATUS_SUCCESS; try { output = doProcess(item); } @@ -298,8 +324,12 @@ protected Chunk transform(StepContribution contribution, Chunk inputs) thr * here, so prevent any more processing of these inputs. */ inputs.clear(); + status = BatchMetrics.STATUS_FAILURE; throw e; } + finally { + stopTimer(sample, contribution.getStepExecution(), "item.process", status, "Item processing"); + } if (output != null) { outputs.add(output); } @@ -310,4 +340,12 @@ protected Chunk transform(StepContribution contribution, Chunk inputs) thr return outputs; } + private void stopTimer(Timer.Sample sample, StepExecution stepExecution, String metricName, String status, String description) { + sample.stop(BatchMetrics.createTimer(metricName, description + " duration", + Tag.of("job.name", stepExecution.getJobExecution().getJobInstance().getJobName()), + Tag.of("step.name", stepExecution.getStepName()), + Tag.of("status", status) + )); + } + } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SimpleChunkProvider.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SimpleChunkProvider.java index 0a18103d1f..4a14220ae6 100755 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SimpleChunkProvider.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SimpleChunkProvider.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,16 +18,22 @@ import java.util.List; +import io.micrometer.core.instrument.Metrics; +import io.micrometer.core.instrument.Tag; +import io.micrometer.core.instrument.Timer; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.StepListener; import org.springframework.batch.core.listener.MulticasterBatchListener; +import org.springframework.batch.core.metrics.BatchMetrics; import org.springframework.batch.item.ItemReader; import org.springframework.batch.repeat.RepeatCallback; import org.springframework.batch.repeat.RepeatContext; import org.springframework.batch.repeat.RepeatOperations; import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; /** * Simple implementation of the ChunkProvider interface that does basic chunk @@ -35,6 +41,7 @@ * * @author Dave Syer * @author Michael Minella + * @author Mahmoud Ben Hassine * @see ChunkOrientedTasklet */ public class SimpleChunkProvider implements ChunkProvider { @@ -43,7 +50,7 @@ public class SimpleChunkProvider implements ChunkProvider { protected final ItemReader itemReader; - private final MulticasterBatchListener listener = new MulticasterBatchListener(); + private final MulticasterBatchListener listener = new MulticasterBatchListener<>(); private final RepeatOperations repeatOperations; @@ -56,7 +63,7 @@ public SimpleChunkProvider(ItemReader itemReader, RepeatOperations * Register some {@link StepListener}s with the handler. Each will get the * callbacks in the order specified at the correct stage. * - * @param listeners + * @param listeners list of {@link StepListener}s. */ public void setListeners(List listeners) { for (StepListener listener : listeners) { @@ -82,9 +89,10 @@ public void registerListener(StepListener listener) { /** * Surrounds the read call with listener callbacks. - * @return item - * @throws Exception + * @return the item or {@code null} if the data source is exhausted + * @throws Exception is thrown if error occurs during read. */ + @Nullable protected final I doRead() throws Exception { try { listener.beforeRead(); @@ -95,7 +103,9 @@ protected final I doRead() throws Exception { return item; } catch (Exception e) { - logger.debug(e.getMessage() + " : " + e.getClass().getName()); + if (logger.isDebugEnabled()) { + logger.debug(e.getMessage() + " : " + e.getClass().getName()); + } listener.onReadError(e); throw e; } @@ -104,20 +114,26 @@ protected final I doRead() throws Exception { @Override public Chunk provide(final StepContribution contribution) throws Exception { - final Chunk inputs = new Chunk(); + final Chunk inputs = new Chunk<>(); repeatOperations.iterate(new RepeatCallback() { @Override public RepeatStatus doInIteration(final RepeatContext context) throws Exception { I item = null; + Timer.Sample sample = Timer.start(Metrics.globalRegistry); + String status = BatchMetrics.STATUS_SUCCESS; try { item = read(contribution, inputs); } catch (SkipOverflowException e) { // read() tells us about an excess of skips by throwing an // exception + status = BatchMetrics.STATUS_FAILURE; return RepeatStatus.FINISHED; } + finally { + stopTimer(sample, contribution.getStepExecution(), status); + } if (item == null) { inputs.setEnd(); return RepeatStatus.FINISHED; @@ -133,6 +149,14 @@ public RepeatStatus doInIteration(final RepeatContext context) throws Exception } + private void stopTimer(Timer.Sample sample, StepExecution stepExecution, String status) { + sample.stop(BatchMetrics.createTimer("item.read", "Item reading duration", + Tag.of("job.name", stepExecution.getJobExecution().getJobInstance().getJobName()), + Tag.of("step.name", stepExecution.getStepName()), + Tag.of("status", status) + )); + } + @Override public void postProcess(StepContribution contribution, Chunk chunk) { // do nothing @@ -144,13 +168,14 @@ public void postProcess(StepContribution contribution, Chunk chunk) { * * @param contribution the current step execution contribution * @param chunk the current chunk - * @return a new item for processing + * @return a new item for processing or {@code null} if the data source is exhausted * * @throws SkipOverflowException if specifically the chunk is accumulating * too much data (e.g. skips) and it wants to force a commit. * * @throws Exception if there is a generic issue */ + @Nullable protected I read(StepContribution contribution, Chunk chunk) throws SkipOverflowException, Exception { return doRead(); } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SimpleRetryExceptionHandler.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SimpleRetryExceptionHandler.java index 7aedd7999c..5fa496813c 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SimpleRetryExceptionHandler.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SimpleRetryExceptionHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,8 +15,6 @@ */ package org.springframework.batch.core.step.item; -import java.util.Collection; - import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.repeat.RepeatContext; @@ -28,6 +26,8 @@ import org.springframework.retry.RetryPolicy; import org.springframework.retry.listener.RetryListenerSupport; +import java.util.Collection; + /** * An {@link ExceptionHandler} that is aware of the retry context so that it can * distinguish between a fatal exception and one that can be retried. Delegates @@ -58,7 +58,7 @@ public class SimpleRetryExceptionHandler extends RetryListenerSupport implements * exception is encountered * @param exceptionHandler the delegate to use if an exception actually * needs to be handled - * @param fatalExceptionClasses + * @param fatalExceptionClasses exceptions */ public SimpleRetryExceptionHandler(RetryPolicy retryPolicy, ExceptionHandler exceptionHandler, Collection> fatalExceptionClasses) { this.retryPolicy = retryPolicy; @@ -95,9 +95,11 @@ public void handleException(RepeatContext context, Throwable throwable) throws T * org.springframework.retry.RetryCallback, java.lang.Throwable) */ @Override - public void close(RetryContext context, RetryCallback callback, Throwable throwable) { + public void close(RetryContext context, RetryCallback callback, Throwable throwable) { if (!retryPolicy.canRetry(context)) { - logger.debug("Marking retry as exhausted: "+context); + if (logger.isDebugEnabled()) { + logger.debug("Marking retry as exhausted: "+context); + } getRepeatContext().setAttribute(EXHAUSTED, "true"); } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SkipOverflowException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SkipOverflowException.java index f818c47ab3..ba77616afb 100755 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SkipOverflowException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SkipOverflowException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class SkipOverflowException extends SkipException { /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SkipWrapper.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SkipWrapper.java index 7d678c0b53..f547f0ef94 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SkipWrapper.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/SkipWrapper.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,10 +16,13 @@ package org.springframework.batch.core.step.item; +import org.springframework.lang.Nullable; + /** * Wrapper for an item and its exception if it failed processing. * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public class SkipWrapper { @@ -29,21 +32,21 @@ public class SkipWrapper { final private T item; /** - * @param item + * @param item the item being wrapped. */ public SkipWrapper(T item) { this(item, null); } /** - * @param e + * @param e instance of {@link Throwable} that being wrapped. */ public SkipWrapper(Throwable e) { this(null, e); } - public SkipWrapper(T item, Throwable e) { + public SkipWrapper(T item, @Nullable Throwable e) { this.item = item; this.exception = e; } @@ -52,6 +55,7 @@ public SkipWrapper(T item, Throwable e) { * Public getter for the exception. * @return the exception */ + @Nullable public Throwable getException() { return exception; } @@ -69,4 +73,4 @@ public String toString() { return String.format("[exception=%s, item=%s]", exception, item); } -} \ No newline at end of file +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/package-info.java new file mode 100644 index 0000000000..3b75c6cf67 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/package-info.java @@ -0,0 +1,10 @@ +/** + * Specific implementations of step concerns for item-oriented approach. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.step.item; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/package.html deleted file mode 100644 index 7ef7634000..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/item/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Specific implementations of step concerns for item-oriented approach. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/DefaultJobParametersExtractor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/DefaultJobParametersExtractor.java index 8b778803a9..72a1f389c3 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/DefaultJobParametersExtractor.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/DefaultJobParametersExtractor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,11 +34,12 @@ * parameters of the surrounding job. * * @author Dave Syer + * @author Will Schipp * */ public class DefaultJobParametersExtractor implements JobParametersExtractor { - private Set keys = new HashSet(); + private Set keys = new HashSet<>(); private boolean useAllParentParameters = true; @@ -56,7 +57,7 @@ public class DefaultJobParametersExtractor implements JobParametersExtractor { * @param keys the keys to set */ public void setKeys(String[] keys) { - this.keys = new HashSet(Arrays.asList(keys)); + this.keys = new HashSet<>(Arrays.asList(keys)); } /** @@ -130,4 +131,15 @@ else if (jobParameters.containsKey(key)) { return builder.toJobParameters(); } + /** + * setter to support switching off all parent parameters + * + * @param useAllParentParameters if false do not include parent parameters. + * True if all parent parameters need to be included. + */ + public void setUseAllParentParameters(boolean useAllParentParameters) { + this.useAllParentParameters = useAllParentParameters; + } + + } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/JobParametersExtractor.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/JobParametersExtractor.java index 75caa1fc16..b53951cf81 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/JobParametersExtractor.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/JobParametersExtractor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/JobStep.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/JobStep.java index 77a931bb1d..6bd8033531 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/JobStep.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/JobStep.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,6 +15,8 @@ */ package org.springframework.batch.core.step.job; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; @@ -103,6 +105,8 @@ protected void doExecute(StepExecution stepExecution) throws Exception { ExecutionContext executionContext = stepExecution.getExecutionContext(); + executionContext.put(STEP_TYPE_KEY, this.getClass().getName()); + JobParameters jobParameters; if (executionContext.containsKey(JOB_PARAMETERS_KEY)) { jobParameters = (JobParameters) executionContext.get(JOB_PARAMETERS_KEY); @@ -113,11 +117,30 @@ protected void doExecute(StepExecution stepExecution) throws Exception { } JobExecution jobExecution = jobLauncher.run(job, jobParameters); + + stepExecution.setExitStatus(determineStepExitStatus(stepExecution, jobExecution)); + if (jobExecution.getStatus().isUnsuccessful()) { // AbstractStep will take care of the step execution status throw new UnexpectedJobExecutionException("Step failure: the delegate Job failed in JobStep."); } - + else if(jobExecution.getStatus().equals(BatchStatus.STOPPED)) { + stepExecution.setStatus(BatchStatus.STOPPED); + } + } + + /** + * Determines the {@link ExitStatus} taking into consideration the {@link ExitStatus} from + * the {@link StepExecution}, which invoked the {@link JobStep}, and the {@link JobExecution}. + * + * @param stepExecution the {@link StepExecution} which invoked the {@link JobExecution} + * @param jobExecution the {@link JobExecution} + * @return the final {@link ExitStatus} + */ + private ExitStatus determineStepExitStatus(StepExecution stepExecution, JobExecution jobExecution) { + ExitStatus exitStatus = stepExecution.getExitStatus() != null ? stepExecution.getExitStatus() : ExitStatus.COMPLETED; + + return exitStatus.and(jobExecution.getExitStatus()); } } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/package-info.java new file mode 100644 index 0000000000..e088a032d3 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/package-info.java @@ -0,0 +1,10 @@ +/** + * {@link org.springframework.batch.core.step.job.JobStep} and related components. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.step.job; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/package-info.java new file mode 100644 index 0000000000..08fe3c3b82 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/package-info.java @@ -0,0 +1,10 @@ +/** + * Specific implementations of step concerns. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.step; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/step/package.html deleted file mode 100644 index 25753d39db..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Specific implementations of step concerns. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/AlwaysSkipItemSkipPolicy.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/AlwaysSkipItemSkipPolicy.java index 9668dc5c8e..bc9f41d821 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/AlwaysSkipItemSkipPolicy.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/AlwaysSkipItemSkipPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/CompositeSkipPolicy.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/CompositeSkipPolicy.java index bb75f15087..be5a047296 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/CompositeSkipPolicy.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/CompositeSkipPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/ExceptionClassifierSkipPolicy.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/ExceptionClassifierSkipPolicy.java index a32be2db59..9f14cbb9f2 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/ExceptionClassifierSkipPolicy.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/ExceptionClassifierSkipPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -51,7 +51,7 @@ public void setExceptionClassifier(SubclassClassifier cla * to create a {@link Classifier} to locate a policy. */ public void setPolicyMap(Map, SkipPolicy> policyMap) { - SubclassClassifier subclassClassifier = new SubclassClassifier( + SubclassClassifier subclassClassifier = new SubclassClassifier<>( policyMap, new NeverSkipItemSkipPolicy()); this.classifier = subclassClassifier; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/LimitCheckingItemSkipPolicy.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/LimitCheckingItemSkipPolicy.java index e2024f34f1..a22ddcd61d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/LimitCheckingItemSkipPolicy.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/LimitCheckingItemSkipPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NeverSkipItemSkipPolicy.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NeverSkipItemSkipPolicy.java index bf1ff0399b..d9457b85ac 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NeverSkipItemSkipPolicy.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NeverSkipItemSkipPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NonSkippableProcessException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NonSkippableProcessException.java index 2b82b30b93..8544e46da9 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NonSkippableProcessException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NonSkippableProcessException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class NonSkippableProcessException extends SkipException { public NonSkippableProcessException(String msg, Throwable cause) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NonSkippableReadException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NonSkippableReadException.java index 7f02760b25..eaf15b05e7 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NonSkippableReadException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NonSkippableReadException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class NonSkippableReadException extends SkipException { public NonSkippableReadException(String msg, Throwable cause) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NonSkippableWriteException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NonSkippableWriteException.java index 039a8d79ff..8130b60a41 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NonSkippableWriteException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/NonSkippableWriteException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class NonSkippableWriteException extends SkipException { public NonSkippableWriteException(String msg, Throwable cause) { diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipException.java index bbfe3eca27..f760b327a9 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ * * @author Dave Syer */ +@SuppressWarnings("serial") public abstract class SkipException extends UnexpectedJobExecutionException { /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipLimitExceededException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipLimitExceededException.java index c3dd89ab24..3d7e4b59fa 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipLimitExceededException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipLimitExceededException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,13 +17,14 @@ /** - * Exception indicating that the skip limit for a particular {@Step} has + * Exception indicating that the skip limit for a particular {@link org.springframework.batch.core.Step} has * been exceeded. * * @author Ben Hale * @author Lucas Ward * @author Dave Syer */ +@SuppressWarnings("serial") public class SkipLimitExceededException extends SkipException { private final int skipLimit; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipListenerFailedException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipListenerFailedException.java index 0e7d8e7459..f7fda85cd2 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipListenerFailedException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipListenerFailedException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,6 +26,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class SkipListenerFailedException extends UnexpectedJobExecutionException { /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipPolicy.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipPolicy.java index c14e34edd6..8c81a5a5e6 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipPolicy.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,9 +26,9 @@ public interface SkipPolicy { /** * Returns true or false, indicating whether or not processing should * continue with the given throwable. Clients may use - * skipCount<0 to probe for exception types that are skippable, + * {@code skipCount<0} to probe for exception types that are skippable, * so implementations should be able to handle gracefully the case where - * skipCount<0. Implementations should avoid throwing any + * {@code skipCount<0}. Implementations should avoid throwing any * undeclared exceptions. * * @param t exception encountered while reading diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipPolicyFailedException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipPolicyFailedException.java index 25ccc75cc7..09d0cde3ca 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipPolicyFailedException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/SkipPolicyFailedException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,6 +25,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class SkipPolicyFailedException extends UnexpectedJobExecutionException { /** diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/package-info.java new file mode 100644 index 0000000000..5305064e22 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/package-info.java @@ -0,0 +1,10 @@ +/** + * Specific implementations of skip concerns for items in a step. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.step.skip; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/package.html deleted file mode 100644 index 6637a13718..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/skip/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Specific implementations of skip concerns for items in a step. -

      - - diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/CallableTaskletAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/CallableTaskletAdapter.java index 906af77532..65e7b22787 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/CallableTaskletAdapter.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/CallableTaskletAdapter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,7 @@ import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.repeat.RepeatStatus; import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -49,7 +50,7 @@ public void setCallable(Callable callable) { */ @Override public void afterPropertiesSet() throws Exception { - Assert.notNull(callable); + Assert.notNull(callable, "A Callable is required"); } /** @@ -57,6 +58,7 @@ public void afterPropertiesSet() throws Exception { * the {@link StepContribution} and the attributes. * @see Tasklet#execute(StepContribution, ChunkContext) */ + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { return callable.call(); diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/ConfigurableSystemProcessExitCodeMapper.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/ConfigurableSystemProcessExitCodeMapper.java index 0f4dca2900..4dabb44869 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/ConfigurableSystemProcessExitCodeMapper.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/ConfigurableSystemProcessExitCodeMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -50,7 +50,7 @@ public ExitStatus getExitStatus(int exitCode) { * {@link org.springframework.batch.core.ExitStatus} values. */ public void setMappings(Map mappings) { - Assert.notNull(mappings.get(ELSE_KEY)); + Assert.notNull(mappings.get(ELSE_KEY), "Missing value for " + ELSE_KEY); this.mappings = mappings; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/MethodInvokingTaskletAdapter.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/MethodInvokingTaskletAdapter.java index c713f76f65..a908784559 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/MethodInvokingTaskletAdapter.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/MethodInvokingTaskletAdapter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.item.adapter.AbstractMethodInvokingDelegator; import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; /** * A {@link Tasklet} that wraps a method in a POJO. By default the return @@ -31,6 +32,7 @@ * @see AbstractMethodInvokingDelegator * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public class MethodInvokingTaskletAdapter extends AbstractMethodInvokingDelegator implements Tasklet { @@ -42,8 +44,12 @@ public class MethodInvokingTaskletAdapter extends AbstractMethodInvokingDelegato * * @see Tasklet#execute(StepContribution, ChunkContext) */ + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + if (getArguments() == null) { + setArguments(new Object[]{contribution, chunkContext}); + } contribution.setExitStatus(mapResult(invokeDelegateMethod())); return RepeatStatus.FINISHED; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SimpleSystemProcessExitCodeMapper.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SimpleSystemProcessExitCodeMapper.java index ba27551f9d..81837a977f 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SimpleSystemProcessExitCodeMapper.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SimpleSystemProcessExitCodeMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,8 +21,8 @@ /** * Simple {@link SystemProcessExitCodeMapper} implementation that performs following mapping: * - * 0 -> ExitStatus.FINISHED - * else -> ExitStatus.FAILED + * 0 -> ExitStatus.FINISHED + * else -> ExitStatus.FAILED * * @author Robert Kasanicky */ diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/StoppableTasklet.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/StoppableTasklet.java new file mode 100644 index 0000000000..e7ec8e6a05 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/StoppableTasklet.java @@ -0,0 +1,41 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.step.tasklet; + +import org.springframework.batch.core.launch.JobOperator; + +/** + * An extension to the {@link Tasklet} interface to allow users to + * add logic for stopping a tasklet. It is up to each implementation + * as to how the stop will behave. The only guarantee provided by the + * framework is that a call to {@link JobOperator#stop(long)} will + * attempt to call the stop method on any currently running + * StoppableTasklet. The call to {@link StoppableTasklet#stop()} will + * be from a thread other than the thread executing {@link org.springframework.batch.core.step.tasklet.Tasklet#execute(org.springframework.batch.core.StepContribution, org.springframework.batch.core.scope.context.ChunkContext)} + * so the appropriate thread safety and visibility controls should be + * put in place. + * + * @author Will Schipp + * @since 3.0 + */ +public interface StoppableTasklet extends Tasklet { + + /** + * Used to signal that the job this {@link Tasklet} is executing + * within has been requested to stop. + */ + void stop(); +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SystemCommandException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SystemCommandException.java index 9a8e850bbe..646670c0c5 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SystemCommandException.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SystemCommandException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SystemCommandTasklet.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SystemCommandTasklet.java index 397ef75786..a1be7bcf6b 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SystemCommandTasklet.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SystemCommandTasklet.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,16 +22,20 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobInterruptedException; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.explore.JobExplorer; import org.springframework.batch.core.listener.StepExecutionListenerSupport; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.repeat.RepeatStatus; import org.springframework.beans.factory.InitializingBean; import org.springframework.core.task.SimpleAsyncTaskExecutor; import org.springframework.core.task.TaskExecutor; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -54,8 +58,9 @@ * still running when tasklet exits (abnormally). * * @author Robert Kasanicky + * @author Will Schipp */ -public class SystemCommandTasklet extends StepExecutionListenerSupport implements Tasklet, InitializingBean { +public class SystemCommandTasklet extends StepExecutionListenerSupport implements StoppableTasklet, InitializingBean { protected static final Log logger = LogFactory.getLog(SystemCommandTasklet.class); @@ -77,14 +82,21 @@ public class SystemCommandTasklet extends StepExecutionListenerSupport implement private boolean interruptOnCancel = false; + private volatile boolean stopped = false; + + private JobExplorer jobExplorer; + + private boolean stoppable = false; + /** * Execute system command and map its exit code to {@link ExitStatus} using * {@link SystemProcessExitCodeMapper}. */ + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { - FutureTask systemCommandTask = new FutureTask(new Callable() { + FutureTask systemCommandTask = new FutureTask<>(new Callable() { @Override public Integer call() throws Exception { @@ -99,7 +111,17 @@ public Integer call() throws Exception { taskExecutor.execute(systemCommandTask); while (true) { - Thread.sleep(checkInterval); + Thread.sleep(checkInterval);//moved to the end of the logic + + if(stoppable) { + JobExecution jobExecution = + jobExplorer.getJobExecution(chunkContext.getStepContext().getStepExecution().getJobExecutionId()); + + if(jobExecution.isStopping()) { + stopped = true; + } + } + if (systemCommandTask.isDone()) { contribution.setExitStatus(systemProcessExitCodeMapper.getExitStatus(systemCommandTask.get())); return RepeatStatus.FINISHED; @@ -112,8 +134,12 @@ else if (execution.isTerminateOnly()) { systemCommandTask.cancel(interruptOnCancel); throw new JobInterruptedException("Job interrupted while executing system command '" + command + "'"); } + else if (stopped) { + systemCommandTask.cancel(interruptOnCancel); + contribution.setExitStatus(ExitStatus.STOPPED); + return RepeatStatus.FINISHED; + } } - } /** @@ -152,6 +178,11 @@ public void afterPropertiesSet() throws Exception { Assert.notNull(systemProcessExitCodeMapper, "SystemProcessExitCodeMapper must be set"); Assert.isTrue(timeout > 0, "timeout value must be greater than zero"); Assert.notNull(taskExecutor, "taskExecutor is required"); + stoppable = jobExplorer != null; + } + + public void setJobExplorer(JobExplorer jobExplorer) { + this.jobExplorer = jobExplorer; } /** @@ -194,6 +225,8 @@ public void beforeStep(StepExecution stepExecution) { /** * Sets the task executor that will be used to execute the system command * NB! Avoid using a synchronous task executor + * + * @param taskExecutor instance of {@link TaskExecutor}. */ public void setTaskExecutor(TaskExecutor taskExecutor) { this.taskExecutor = taskExecutor; @@ -203,9 +236,25 @@ public void setTaskExecutor(TaskExecutor taskExecutor) { * If true tasklet will attempt to interrupt the thread * executing the system command if {@link #setTimeout(long)} has been * exceeded or user interrupts the job. false by default + * + * @param interruptOnCancel boolean determines if process should be interrupted */ public void setInterruptOnCancel(boolean interruptOnCancel) { this.interruptOnCancel = interruptOnCancel; } + /** + * Will interrupt the thread executing the system command only if + * {@link #setInterruptOnCancel(boolean)} has been set to true. Otherwise + * the underlying command will be allowed to finish before the tasklet + * ends. + * + * @since 3.0 + * @see StoppableTasklet#stop() + */ + @Override + public void stop() { + stopped = true; + } + } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SystemProcessExitCodeMapper.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SystemProcessExitCodeMapper.java index 1531aa2ac5..7546f3fb90 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SystemProcessExitCodeMapper.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/SystemProcessExitCodeMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/Tasklet.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/Tasklet.java index 743dcc188a..6c79ca3510 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/Tasklet.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/Tasklet.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,11 +18,13 @@ import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; /** * Strategy for processing in a step. * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public interface Tasklet { @@ -38,8 +40,11 @@ public interface Tasklet { * @param chunkContext attributes shared between invocations but not between * restarts * @return an {@link RepeatStatus} indicating whether processing is - * continuable. + * continuable. Returning {@code null} is interpreted as {@link RepeatStatus#FINISHED} + * + * @throws Exception thrown if error occurs during execution. */ + @Nullable RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception; } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/TaskletStep.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/TaskletStep.java index 8b98f9fe44..077a9c530d 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/TaskletStep.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/TaskletStep.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,8 +15,6 @@ */ package org.springframework.batch.core.step.tasklet; -import java.util.concurrent.Semaphore; - import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.core.BatchStatus; @@ -53,13 +51,15 @@ import org.springframework.transaction.support.TransactionTemplate; import org.springframework.util.Assert; +import java.util.concurrent.Semaphore; + /** * Simple implementation of executing the step as a call to a {@link Tasklet}, * possibly repeated, and each call surrounded by a transaction. The structure * is therefore that of a loop with transaction boundary inside the loop. The * loop is controlled by the step operations ( - * {@link #setStepOperations(RepeatOperations)}).
      - *
      + * {@link #setStepOperations(RepeatOperations)}).
      + *
      * * Clients can use interceptors in the step operations to intercept or listen to * the iteration on a step-wide basis, for instance to get a callback when the @@ -71,6 +71,8 @@ * @author Ben Hale * @author Robert Kasanicky * @author Michael Minella + * @author Will Schipp + * @author Mahmoud Ben Hassine */ @SuppressWarnings("serial") public class TaskletStep extends AbstractStep { @@ -99,6 +101,8 @@ public boolean rollbackOn(Throwable ex) { private Tasklet tasklet; + public static final String TASKLET_TYPE_KEY = "batch.taskletType"; + /** * Default constructor. */ @@ -107,7 +111,7 @@ public TaskletStep() { } /** - * @param name + * @param name the name for the {@link TaskletStep} */ public TaskletStep(String name) { super(name); @@ -197,7 +201,7 @@ public void setStreams(ItemStream[] streams) { * Register a single {@link ItemStream} for callbacks to the stream * interface. * - * @param stream + * @param stream instance of {@link ItemStream} */ public void registerStream(ItemStream stream) { this.stream.register(stream); @@ -232,7 +236,7 @@ public void setInterruptionPolicy(StepInterruptionPolicy interruptionPolicy) { * given an up to date {@link BatchStatus}, and the {@link JobRepository} is * used to store the result. Various reporting information are also added to * the current context governing the step execution, which would normally be - * available to the caller through the step's {@link ExecutionContext}.
      + * available to the caller through the step's {@link ExecutionContext}.
      * * @throws JobInterruptedException if the step or a chunk is interrupted * @throws RuntimeException if there is an exception during a chunk @@ -240,8 +244,9 @@ public void setInterruptionPolicy(StepInterruptionPolicy interruptionPolicy) { * */ @Override - @SuppressWarnings("unchecked") protected void doExecute(StepExecution stepExecution) throws Exception { + stepExecution.getExecutionContext().put(TASKLET_TYPE_KEY, tasklet.getClass().getName()); + stepExecution.getExecutionContext().put(STEP_TYPE_KEY, this.getClass().getName()); stream.update(stepExecution.getExecutionContext()); getJobRepository().updateExecutionContext(stepExecution); @@ -264,7 +269,7 @@ public RepeatStatus doInChunkContext(RepeatContext repeatContext, ChunkContext c RepeatStatus result; try { - result = (RepeatStatus) new TransactionTemplate(transactionManager, transactionAttribute) + result = new TransactionTemplate(transactionManager, transactionAttribute) .execute(new ChunkTransactionCallback(chunkContext, semaphore)); } catch (UncheckedTransactionException e) { @@ -279,7 +284,7 @@ public RepeatStatus doInChunkContext(RepeatContext repeatContext, ChunkContext c // caller interruptionPolicy.checkInterrupted(stepExecution); - return result; + return result == null ? RepeatStatus.FINISHED : result; } }); @@ -306,6 +311,14 @@ protected void open(ExecutionContext ctx) throws Exception { stream.open(ctx); } + /** + * retrieve the tasklet - helper method for JobOperator + * @return the {@link Tasklet} instance being executed within this step + */ + public Tasklet getTasklet() { + return tasklet; + } + /** * A callback for the transactional work inside a chunk. Also detects * failures in the transaction commit and rollback, only panicking if the @@ -315,8 +328,7 @@ protected void open(ExecutionContext ctx) throws Exception { * @author Dave Syer * */ - @SuppressWarnings("rawtypes") - private class ChunkTransactionCallback extends TransactionSynchronizationAdapter implements TransactionCallback { + private class ChunkTransactionCallback extends TransactionSynchronizationAdapter implements TransactionCallback { private final StepExecution stepExecution; @@ -374,7 +386,7 @@ public void afterCompletion(int status) { } @Override - public Object doInTransaction(TransactionStatus status) { + public RepeatStatus doInTransaction(TransactionStatus status) { TransactionSynchronizationManager.registerSynchronization(this); RepeatStatus result = RepeatStatus.CONTINUABLE; @@ -423,7 +435,9 @@ public Object doInTransaction(TransactionStatus status) { // Apply the contribution to the step // even if unsuccessful - logger.debug("Applying contribution: " + contribution); + if (logger.isDebugEnabled()) { + logger.debug("Applying contribution: " + contribution); + } stepExecution.apply(contribution); } @@ -437,31 +451,37 @@ public Object doInTransaction(TransactionStatus status) { // stay false and we can use that later. getJobRepository().updateExecutionContext(stepExecution); stepExecution.incrementCommitCount(); - logger.debug("Saving step execution before commit: " + stepExecution); + if (logger.isDebugEnabled()) { + logger.debug("Saving step execution before commit: " + stepExecution); + } getJobRepository().update(stepExecution); } catch (Exception e) { // If we get to here there was a problem saving the step // execution and we have to fail. - String msg = "JobRepository failure forcing exit with unknown status"; + String msg = "JobRepository failure forcing rollback"; logger.error(msg, e); - stepExecution.upgradeStatus(BatchStatus.UNKNOWN); - stepExecution.setTerminateOnly(); throw new FatalStepExecutionException(msg, e); } } catch (Error e) { - logger.debug("Rollback for Error: " + e.getClass().getName() + ": " + e.getMessage()); + if (logger.isDebugEnabled()) { + logger.debug("Rollback for Error: " + e.getClass().getName() + ": " + e.getMessage()); + } rollback(stepExecution); throw e; } catch (RuntimeException e) { - logger.debug("Rollback for RuntimeException: " + e.getClass().getName() + ": " + e.getMessage()); + if (logger.isDebugEnabled()) { + logger.debug("Rollback for RuntimeException: " + e.getClass().getName() + ": " + e.getMessage()); + } rollback(stepExecution); throw e; } catch (Exception e) { - logger.debug("Rollback for Exception: " + e.getClass().getName() + ": " + e.getMessage()); + if (logger.isDebugEnabled()) { + logger.debug("Rollback for Exception: " + e.getClass().getName() + ": " + e.getMessage()); + } rollback(stepExecution); // Allow checked exceptions throw new UncheckedTransactionException(e); @@ -487,20 +507,4 @@ private void copy(final StepExecution source, final StepExecution target) { } } - - /** - * Convenience wrapper for a checked exception so that it can cause a - * rollback and be extracted afterwards. - * - * @author Dave Syer - * - */ - private static class UncheckedTransactionException extends RuntimeException { - - public UncheckedTransactionException(Exception e) { - super(e); - } - - } - } diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/UncheckedTransactionException.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/UncheckedTransactionException.java new file mode 100644 index 0000000000..6ad49ea4e2 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/UncheckedTransactionException.java @@ -0,0 +1,31 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.step.tasklet; + +/** + * Convenience wrapper for a checked exception so that it can cause a + * rollback and be extracted afterwards. + * + * @author Dave Syer + * + */ +@SuppressWarnings("serial") +public class UncheckedTransactionException extends RuntimeException { + + public UncheckedTransactionException(Exception e) { + super(e); + } +} diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/package-info.java b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/package-info.java new file mode 100644 index 0000000000..ff6c2cd786 --- /dev/null +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/package-info.java @@ -0,0 +1,10 @@ +/** + * Interfaces and generic implementations of tasklet concerns. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.core.step.tasklet; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/package.html b/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/package.html deleted file mode 100644 index 5bf66aab7f..0000000000 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/step/tasklet/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Interfaces and generic implementations of tasklet concerns. -

      - - diff --git a/spring-batch-core/src/main/resources/META-INF/services/javax.batch.operations.JobOperator b/spring-batch-core/src/main/resources/META-INF/services/javax.batch.operations.JobOperator new file mode 100644 index 0000000000..7bb1f526d2 --- /dev/null +++ b/spring-batch-core/src/main/resources/META-INF/services/javax.batch.operations.JobOperator @@ -0,0 +1 @@ +org.springframework.batch.core.jsr.launch.JsrJobOperator \ No newline at end of file diff --git a/spring-batch-core/src/main/resources/META-INF/spring.handlers b/spring-batch-core/src/main/resources/META-INF/spring.handlers index fbdf08e4f2..2663f5208b 100644 --- a/spring-batch-core/src/main/resources/META-INF/spring.handlers +++ b/spring-batch-core/src/main/resources/META-INF/spring.handlers @@ -1 +1,2 @@ -http\://www.springframework.org/schema/batch=org.springframework.batch.core.configuration.xml.CoreNamespaceHandler \ No newline at end of file +http\://www.springframework.org/schema/batch=org.springframework.batch.core.configuration.xml.CoreNamespaceHandler +http\://xmlns.jcp.org/xml/ns/javaee=org.springframework.batch.core.jsr.configuration.xml.JsrNamespaceHandler \ No newline at end of file diff --git a/spring-batch-core/src/main/resources/META-INF/spring.schemas b/spring-batch-core/src/main/resources/META-INF/spring.schemas index a75ad57757..c42b201f59 100644 --- a/spring-batch-core/src/main/resources/META-INF/spring.schemas +++ b/spring-batch-core/src/main/resources/META-INF/spring.schemas @@ -1,4 +1,7 @@ -http\://www.springframework.org/schema/batch/spring-batch.xsd=/org/springframework/batch/core/configuration/xml/spring-batch-2.2.xsd +http\://www.springframework.org/schema/batch/spring-batch.xsd=/org/springframework/batch/core/configuration/xml/spring-batch-3.0.xsd +http\://www.springframework.org/schema/batch/spring-batch-3.0.xsd=/org/springframework/batch/core/configuration/xml/spring-batch-3.0.xsd http\://www.springframework.org/schema/batch/spring-batch-2.2.xsd=/org/springframework/batch/core/configuration/xml/spring-batch-2.2.xsd http\://www.springframework.org/schema/batch/spring-batch-2.1.xsd=/org/springframework/batch/core/configuration/xml/spring-batch-2.1.xsd -http\://www.springframework.org/schema/batch/spring-batch-2.0.xsd=/org/springframework/batch/core/configuration/xml/spring-batch-2.0.xsd \ No newline at end of file +http\://www.springframework.org/schema/batch/spring-batch-2.0.xsd=/org/springframework/batch/core/configuration/xml/spring-batch-2.0.xsd +http\://xmlns.jcp.org/xml/ns/javaee/jobXML_1_0.xsd=/org/springframework/batch/core/jsr/configuration/xml/jobXML_1_0.xsd +http\://xmlns.jcp.org/xml/ns/javaee/batchXML_1_0.xsd=/org/springframework/batch/core/jsr/configuration/xml/batchXML_1_0.xsd diff --git a/spring-batch-core/src/main/resources/batch-derby.properties b/spring-batch-core/src/main/resources/batch-derby.properties new file mode 100644 index 0000000000..0c44b0f96d --- /dev/null +++ b/spring-batch-core/src/main/resources/batch-derby.properties @@ -0,0 +1,17 @@ +# Placeholders batch.* +# for Derby: +batch.jdbc.driver=org.apache.derby.jdbc.EmbeddedDriver +batch.jdbc.url=jdbc:derby:derby-home/test;create=true +batch.jdbc.user=app +batch.jdbc.password= +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.DerbyMaxValueIncrementer +batch.schema.script=classpath:/org/springframework/batch/core/schema-derby.sql +batch.drop.script=classpath:/org/springframework/batch/core/schema-drop-derby.sql +batch.jdbc.testWhileIdle=true +batch.jdbc.validationQuery= + + +# Non-platform dependent settings that you might like to change +batch.data.source.init=true +batch.table.prefix=BATCH_ + diff --git a/spring-batch-core/src/main/resources/batch-h2.properties b/spring-batch-core/src/main/resources/batch-h2.properties new file mode 100644 index 0000000000..0c8b1c4c65 --- /dev/null +++ b/spring-batch-core/src/main/resources/batch-h2.properties @@ -0,0 +1,17 @@ +# Placeholders batch.* +# for H2: +batch.jdbc.driver=org.h2.Driver +batch.jdbc.url=jdbc:h2:file:build/data/h2 +batch.jdbc.user=sa +batch.jdbc.password= +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.H2SequenceMaxValueIncrementer +batch.schema.script=classpath:/org/springframework/batch/core/schema-h2.sql +batch.drop.script=classpath:/org/springframework/batch/core/schema-drop-h2.sql +batch.jdbc.testWhileIdle=true +batch.jdbc.validationQuery= + + +# Non-platform dependent settings that you might like to change +batch.data.source.init=true +batch.table.prefix=BATCH_ + diff --git a/spring-batch-core/src/main/resources/batch-hsql.properties b/spring-batch-core/src/main/resources/batch-hsql.properties new file mode 100644 index 0000000000..9a9641cfc7 --- /dev/null +++ b/spring-batch-core/src/main/resources/batch-hsql.properties @@ -0,0 +1,20 @@ +# Placeholders batch.* +# for HSQLDB: +batch.jdbc.driver=org.hsqldb.jdbcDriver +batch.jdbc.url=jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true;hsqldb.tx=mvcc +# Override and use this one in for a separate server process so you can inspect +# the results (or add it to system properties with -D to override at run time). +# batch.jdbc.url=jdbc:hsqldb:hsql://localhost:9005/samples +batch.jdbc.user=sa +batch.jdbc.password= +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.HsqlMaxValueIncrementer +batch.schema.script=classpath*:/org/springframework/batch/core/schema-hsqldb.sql +batch.drop.script=classpath*:/org/springframework/batch/core/schema-drop-hsqldb.sql +batch.jdbc.testWhileIdle=true +batch.jdbc.validationQuery= + + +# Non-platform dependent settings that you might like to change +batch.data.source.init=true +batch.table.prefix=BATCH_ + diff --git a/spring-batch-core/src/main/resources/batch-mysql.properties b/spring-batch-core/src/main/resources/batch-mysql.properties new file mode 100644 index 0000000000..e491937446 --- /dev/null +++ b/spring-batch-core/src/main/resources/batch-mysql.properties @@ -0,0 +1,17 @@ +# Placeholders batch.* +# for MySQL: +batch.jdbc.driver=com.mysql.jdbc.Driver +batch.jdbc.url=jdbc:mysql://localhost/test +batch.jdbc.user=test +batch.jdbc.password=test +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.MySQLMaxValueIncrementer +batch.schema.script=classpath:/org/springframework/batch/core/schema-mysql.sql +batch.drop.script=classpath:/org/springframework/batch/core/schema-drop-mysql.sql +batch.jdbc.testWhileIdle=true +batch.jdbc.validationQuery= + + +# Non-platform dependent settings that you might like to change +batch.data.source.init=true +batch.table.prefix=BATCH_ + diff --git a/spring-batch-core/src/main/resources/batch-oracle.properties b/spring-batch-core/src/main/resources/batch-oracle.properties new file mode 100644 index 0000000000..c8f157c5b9 --- /dev/null +++ b/spring-batch-core/src/main/resources/batch-oracle.properties @@ -0,0 +1,17 @@ +# Placeholders batch.* +# for Oracle: +batch.jdbc.driver=oracle.jdbc.OracleDriver +batch.jdbc.url=jdbc:oracle:thin:@oracle:1521:xe +batch.jdbc.user=spring +batch.jdbc.password=spring +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.OracleSequenceMaxValueIncrementer +batch.schema.script=classpath:/org/springframework/batch/core/schema-oracle10g.sql +batch.drop.script=classpath:/org/springframework/batch/core/schema-drop-oracle10g.sql +batch.jdbc.testWhileIdle=false +batch.jdbc.validationQuery= + + +# Non-platform dependent settings that you might like to change +batch.data.source.init=true +batch.table.prefix=BATCH_ + diff --git a/spring-batch-core/src/main/resources/batch-postgresql.properties b/spring-batch-core/src/main/resources/batch-postgresql.properties new file mode 100644 index 0000000000..a882cec072 --- /dev/null +++ b/spring-batch-core/src/main/resources/batch-postgresql.properties @@ -0,0 +1,17 @@ +# Placeholders batch.* +# for Postgres: +batch.jdbc.driver=org.postgresql.Driver +batch.jdbc.url=jdbc:postgresql://localhost/samples +batch.jdbc.user=postgres +batch.jdbc.password=dba +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.PostgresSequenceMaxValueIncrementer +batch.schema.script=classpath:/org/springframework/batch/core/schema-postgresql.sql +batch.drop.script=classpath:/org/springframework/batch/core/schema-drop-postgresql.sql +batch.jdbc.testWhileIdle=false +batch.jdbc.validationQuery= + + +# Non-platform dependent settings that you might like to change +batch.data.source.init=true +batch.table.prefix=BATCH_ + diff --git a/spring-batch-core/src/main/resources/batch-sqlf.properties b/spring-batch-core/src/main/resources/batch-sqlf.properties new file mode 100644 index 0000000000..6593892c58 --- /dev/null +++ b/spring-batch-core/src/main/resources/batch-sqlf.properties @@ -0,0 +1,17 @@ +# Placeholders batch.* +# for SQLFire: +batch.jdbc.driver=com.vmware.sqlfire.jdbc.ClientDriver +batch.jdbc.url=jdbc:sqlfire://localhost:1257/;update=true +batch.jdbc.user=SAMPLES +batch.jdbc.password=SAMPLES +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.DerbyMaxValueIncrementer +batch.schema.script=classpath:/org/springframework/batch/core/schema-sqlf.sql +batch.drop.script=classpath:/org/springframework/batch/core/schema-drop-sqlf.sql +batch.jdbc.testWhileIdle=false +batch.jdbc.validationQuery= + + +# Non-platform dependent settings that you might like to change +batch.data.source.init=true +batch.table.prefix=BATCH_ + diff --git a/spring-batch-core/src/main/resources/batch-sqlserver.properties b/spring-batch-core/src/main/resources/batch-sqlserver.properties new file mode 100644 index 0000000000..2036b1f757 --- /dev/null +++ b/spring-batch-core/src/main/resources/batch-sqlserver.properties @@ -0,0 +1,17 @@ +# Placeholders batch.* +# for MS SQLServer: +batch.jdbc.driver=net.sourceforge.jtds.jdbc.Driver +batch.jdbc.url=jdbc:jtds:sqlserver://localhost:1433;instance=SQLEXPRESS +batch.jdbc.user=sa +batch.jdbc.password=sa +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.SqlServerMaxValueIncrementer +batch.schema.script=classpath*:/org/springframework/batch/core/schema-sqlserver.sql +batch.drop.script=classpath*:/org/springframework/batch/core/schema-drop-sqlserver.sql +batch.jdbc.testWhileIdle=false +batch.jdbc.validationQuery= + + +# Non-platform dependent settings that you might like to change +batch.data.source.init=true +batch.table.prefix=BATCH_ + diff --git a/spring-batch-core/src/main/resources/batch-sybase.properties b/spring-batch-core/src/main/resources/batch-sybase.properties new file mode 100644 index 0000000000..4088228463 --- /dev/null +++ b/spring-batch-core/src/main/resources/batch-sybase.properties @@ -0,0 +1,17 @@ +# Placeholders batch.* +# for Sybase: +batch.jdbc.driver=net.sourceforge.jtds.jdbc.Driver +batch.jdbc.url=jdbc:jtds:sybase://dbhost:5000;databaseName=test +batch.jdbc.user=spring +batch.jdbc.password=spring +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.SybaseMaxValueIncrementer +batch.schema.script=classpath*:/org/springframework/batch/core/schema-sybase.sql +batch.drop.script=classpath*:/org/springframework/batch/core/schema-drop-sybase.sql +batch.jdbc.testWhileIdle=true +batch.jdbc.validationQuery= + + +# Non-platform dependent settings that you might like to change +batch.data.source.init=true +batch.table.prefix=BATCH_ + diff --git a/spring-batch-core/src/main/resources/beanRefContext.xml b/spring-batch-core/src/main/resources/beanRefContext.xml new file mode 100644 index 0000000000..92a295f11d --- /dev/null +++ b/spring-batch-core/src/main/resources/beanRefContext.xml @@ -0,0 +1,13 @@ + + + + + + + + + diff --git a/spring-batch-core/src/main/resources/jsrBaseContext.xml b/spring-batch-core/src/main/resources/jsrBaseContext.xml new file mode 100644 index 0000000000..2abb27aad6 --- /dev/null +++ b/spring-batch-core/src/main/resources/jsrBaseContext.xml @@ -0,0 +1,68 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + classpath:batch-${ENVIRONMENT:hsql}.properties + + + + + + + + diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/configuration/xml/spring-batch-2.1.xsd b/spring-batch-core/src/main/resources/org/springframework/batch/core/configuration/xml/spring-batch-2.1.xsd index b2d4745f7f..2dd716caf7 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/configuration/xml/spring-batch-2.1.xsd +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/configuration/xml/spring-batch-2.1.xsd @@ -3,8 +3,8 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:tool="http://www.springframework.org/schema/tool" targetNamespace="http://www.springframework.org/schema/batch" elementFormDefault="qualified" attributeFormDefault="unqualified" - xsi:schemaLocation="http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-2.5.xsd-http://www.springframework.org/schema/tool%20http://www.springframework.org/schema/tool/spring-tool-2.5.xsd" + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-2.5.xsd+http://www.springframework.org/schema/tool%20https://www.springframework.org/schema/tool/spring-tool-2.5.xsd" version="2.1"> diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/configuration/xml/spring-batch-2.2.xsd b/spring-batch-core/src/main/resources/org/springframework/batch/core/configuration/xml/spring-batch-2.2.xsd index f021aa7b3e..bf349ebb67 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/configuration/xml/spring-batch-2.2.xsd +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/configuration/xml/spring-batch-2.2.xsd @@ -3,8 +3,8 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:tool="http://www.springframework.org/schema/tool" targetNamespace="http://www.springframework.org/schema/batch" elementFormDefault="qualified" attributeFormDefault="unqualified" - xsi:schemaLocation="http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd-http://www.springframework.org/schema/tool%20http://www.springframework.org/schema/tool/spring-tool-3.1.xsd" + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd+http://www.springframework.org/schema/tool%20https://www.springframework.org/schema/tool/spring-tool-3.1.xsd" version="2.2"> diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/configuration/xml/spring-batch-3.0.xsd b/spring-batch-core/src/main/resources/org/springframework/batch/core/configuration/xml/spring-batch-3.0.xsd new file mode 100644 index 0000000000..d9faa11d1b --- /dev/null +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/configuration/xml/spring-batch-3.0.xsd @@ -0,0 +1,1349 @@ + + + + + + + + + + + + + + Defines a job composed of a set of steps and + transitions between steps. The job will be exposed in + the enclosing + bean factory as a component of type Job + that can be launched using a + JobLauncher. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Defines a stage in job processing backed by a + Step. The id attribute must be specified since this + step definition + will be referred to from other elements + to form a Job flow. + + + + + + + + + + + + + + + + + Defines a flow composed of a set of steps and + transitions between steps. + + + + + + + + + + + + + + + + + + A reference to a JobExecutionListener (or a POJO + if using before-job-method / after-job-method or + source level + annotations). + + + + + + + + + + + + + + + A bean definition for a step listener (or POJO if + using *-method attributes or source level + annotations) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Defines a stage in job processing backed by a + Step. The id attribute must be specified. The + step + requires either + a chunk definition, + a tasklet reference, or a reference to a + (possibly abstract) parent step. + + + + + + + + + + + + + + + + Declares job should split here into two or more + subflows. + + + + + + + + A subflow within a job, having the same + format as a job, but without a separate identity. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Declares job should include an externalized flow + here. + + + + + + + + + + + + + + + + + + + + + + Declares job should query a decider to determine + where execution should go next. + + + + + + + + + The decider is a reference to a + JobExecutionDecider that can produce a status to base + the next + transition on. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + The tasklet is a reference to another bean + definition that implements + the Tasklet interface. + + + + + + + + + + If the tasklet is specified as a bean definition, then a method can be specified and a POJO + will + be adapted to the Tasklet interface. The method suggested should have the same arguments + as Tasklet.execute (or a subset), and have a compatible return type (boolean, void or RepeatStatus). + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + An exception class name. + + + + + + + + + + + + + + + + + Classify an exception as "included" in the set. Exceptions of this type or a subclass are + included. + + + + + + + + + + + + + + + + Classify an exception as "excluded" from the + set. Exceptions of this type or a subclass are + excluded + + + + + + + + + + + + + + + A reference to a listener, a POJO with a + listener-annotated method, or a POJO with + a method + referenced by a + *-method attribute. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Defines a transition from this step to the + next + one depending on the value of the exit + status. + + + + + + A pattern to match against the exit status + code. Use * and ? as wildcard characters. When a + step finishes + the most + specific match will be chosen to select the next step. + Hint: + always include a default + transition with on="*". + + + + + + + The name of the step to go to next. Must + resolve to one of the other steps in this job. + + + + + + + + + Declares job should be stop at this point and + provides pointer where execution should continue + when + the job is + restarted. + + + + + + A pattern to match against the exit status + code. Use * and ? as wildcard characters. + When a step + finishes + the most specific match will be chosen to + select the next step. + + + + + + The name of the step to start on when the + stopped job is restarted. + Must resolve to one of the + other steps + in this job. + + + + + + The exit code value to end on, defaults to + STOPPED. + + + + + + + + Declares job should end at this point, without + the possibility of restart. + BatchStatus will be + COMPLETED. + ExitStatus is configurable. + + + + + + A pattern to match against the exit status + code. Use * and ? as wildcard characters. + When a step + finishes + the most specific match will be chosen to + select the next step. + + + + + + The exit code value to end on, defaults to + COMPLETED. + + + + + + + + Declares job should fail at this point. + BatchStatus will be FAILED. ExitStatus is configurable. + + + + + + A pattern to match against the exit status + code. Use * and ? as wildcard characters. + When a step + finishes + the most specific match will be chosen to + select the next step. + + + + + + The exit code value to end on, defaults to + FAILED. + + + + + + + + + + + + + + + + + + + + + + + + + The name of the parent bean from which the + configuration should inherit. + + + + + + + + + + + + + Is this bean "abstract", that is, not meant to be + instantiated itself + but rather just serving as + parent for concrete + child bean definitions? + The default is "false". Specify "true" to + tell the bean factory to not + try + to instantiate that particular bean + in any case. + + Note: This attribute will not be inherited by child + bean definitions. + Hence, it needs to be specified per abstract bean + definition. + + + + + + + + + + Should this list be merged with the corresponding + list provided + by the parent? If not, it will + overwrite the parent + list. + + + + + + + + + + This attribute indicates the method from the + class that should + be used to dynamically create a + proxy. + + + + + + + + + + + + + diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/jsr/configuration/xml/batchXML_1_0.xsd b/spring-batch-core/src/main/resources/org/springframework/batch/core/jsr/configuration/xml/batchXML_1_0.xsd new file mode 100644 index 0000000000..fdbd192c10 --- /dev/null +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/jsr/configuration/xml/batchXML_1_0.xsd @@ -0,0 +1,34 @@ + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/jsr/configuration/xml/jobXML_1_0.xsd b/spring-batch-core/src/main/resources/org/springframework/batch/core/jsr/configuration/xml/jobXML_1_0.xsd new file mode 100755 index 0000000000..a0504bad04 --- /dev/null +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/jsr/configuration/xml/jobXML_1_0.xsd @@ -0,0 +1,435 @@ + + + + + + + Job Specification Language (JSL) specifies a job, + its steps, and directs their execution. + JSL also can be referred to as "Job XML". + + + + + + + This is a helper type. Though it is not otherwise + called out by this name + in the specification, it captures the fact + that the xs:string value refers + to a batch artifact, across numerous + other JSL type definitions. + + + + + + + + + The type of a job definition, whether concrete or + abstract. This is the type of the root element of any JSL document. + + + + + + + The job-level properties, which are accessible + via the JobContext.getProperties() API in a batch artifact. + + + + + + + Note that "listeners" sequence order in XML does + not imply order of execution by + the batch runtime, per the + specification. + + + + + + + + + + + + + + + + + + + The definition of an job, whether concrete or + abstract. This is the + type of the root element of any JSL document. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + This grouping provides allows for the reuse of the + 'end', 'fail', 'next', 'stop' element sequences which + may appear at the end of a 'step', 'flow', 'split' or 'decision'. + The term 'TransitionElements' does not formally appear in the spec, it is + a schema convenience. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Note that "listeners" sequence order in XML does + not imply order of execution by + the batch runtime, per the + specification. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Specifies the checkpoint policy that governs + commit behavior for this chunk. + Valid values are: "item" or + "custom". The "item" policy means the + chunk is checkpointed after a + specified number of items are + processed. The "custom" policy means + the chunk is checkpointed + according to a checkpoint algorithm + implementation. Specifying + "custom" requires that the + checkpoint-algorithm element is also + specified. It is an optional + attribute. The default policy is + "item". However, we chose not to define + a schema-specified default for this attribute. + + + + + + + Specifies the number of items to process per chunk + when using the item + checkpoint policy. It must be valid XML integer. + It is an optional + attribute. The default is 10. The item-count + attribute is ignored + for "custom" checkpoint policy. However, to + make it easier for implementations to support JSL inheritance + we + abstain from defining a schema-specified default for this + attribute. + + + + + + + Specifies the amount of time in seconds before + taking a checkpoint for the + item checkpoint policy. It must be valid + XML integer. It is an + optional attribute. The default is 0, which + means no limit. However, to + make it easier for implementations to + support JSL inheritance + we abstain from defining a schema-specified + default for this attribute. + When a value greater than zero is + specified, a checkpoint is taken when + time-limit is reached or + item-count items have been processed, + whichever comes first. The + time-limit attribute is ignored for + "custom" checkpoint policy. + + + + + + + Specifies the number of exceptions a step will + skip if any configured + skippable exceptions are thrown by chunk + processing. It must be a + valid XML integer value. It is an optional + attribute. The default + is no limit. + + + + + + + Specifies the number of times a step will retry if + any configured retryable + exceptions are thrown by chunk processing. + It must be a valid XML + integer value. It is an optional attribute. + The default is no + limit. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-db2.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-db2.sql similarity index 94% rename from spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-db2.sql rename to spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-db2.sql index c3f08c8865..e52da193b6 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-db2.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-db2.sql @@ -21,7 +21,7 @@ CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( INSERT INTO BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID , TYPE_CD, KEY_NAME, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING ) SELECT - JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 1 + JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 'Y' FROM BATCH_JOB_PARAMS JP,BATCH_JOB_EXECUTION JE WHERE diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-derby.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-derby.sql similarity index 94% rename from spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-derby.sql rename to spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-derby.sql index c3f08c8865..e52da193b6 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-derby.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-derby.sql @@ -21,7 +21,7 @@ CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( INSERT INTO BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID , TYPE_CD, KEY_NAME, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING ) SELECT - JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 1 + JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 'Y' FROM BATCH_JOB_PARAMS JP,BATCH_JOB_EXECUTION JE WHERE diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-h2.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-h2.sql similarity index 94% rename from spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-h2.sql rename to spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-h2.sql index c3f08c8865..e52da193b6 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-h2.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-h2.sql @@ -21,7 +21,7 @@ CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( INSERT INTO BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID , TYPE_CD, KEY_NAME, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING ) SELECT - JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 1 + JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 'Y' FROM BATCH_JOB_PARAMS JP,BATCH_JOB_EXECUTION JE WHERE diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-hsqldb.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-hsqldb.sql similarity index 94% rename from spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-hsqldb.sql rename to spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-hsqldb.sql index c3f08c8865..e52da193b6 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-hsqldb.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-hsqldb.sql @@ -21,7 +21,7 @@ CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( INSERT INTO BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID , TYPE_CD, KEY_NAME, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING ) SELECT - JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 1 + JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 'Y' FROM BATCH_JOB_PARAMS JP,BATCH_JOB_EXECUTION JE WHERE diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-insert-only.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-insert-only.sql similarity index 89% rename from spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-insert-only.sql rename to spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-insert-only.sql index 10114d9743..2d9fe8bd1e 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-insert-only.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-insert-only.sql @@ -6,7 +6,7 @@ INSERT INTO BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID , TYPE_CD, KEY_NAME, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING ) SELECT - JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 1 + JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 'Y' FROM BATCH_JOB_PARAMS JP,BATCH_JOB_EXECUTION JE WHERE diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-mysql.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-mysql.sql similarity index 94% rename from spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-mysql.sql rename to spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-mysql.sql index bfe9c6b307..56eaa20d15 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-mysql.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-mysql.sql @@ -21,7 +21,7 @@ CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( INSERT INTO BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID , TYPE_CD, KEY_NAME, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING ) SELECT - JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 1 + JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 'Y' FROM BATCH_JOB_PARAMS JP,BATCH_JOB_EXECUTION JE WHERE diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-oracle10g.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-oracle10g.sql new file mode 100644 index 0000000000..dd3d074daf --- /dev/null +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-oracle10g.sql @@ -0,0 +1,28 @@ + +-- create the requisite table + +CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( + JOB_EXECUTION_ID NUMBER(19,0) NOT NULL , + TYPE_CD VARCHAR2(6) NOT NULL , + KEY_NAME VARCHAR2(100) NOT NULL , + STRING_VAL VARCHAR2(250) , + DATE_VAL TIMESTAMP DEFAULT NULL , + LONG_VAL NUMBER(19,0) , + DOUBLE_VAL NUMBER , + IDENTIFYING CHAR(1) NOT NULL , + constraint JOB_EXEC_PARAMS_FK foreign key (JOB_EXECUTION_ID) + references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) +) ; + +-- insert script that 'copies' existing batch_job_params to batch_job_execution_params +-- sets new params to identifying ones +-- verified on h2, + +INSERT INTO BATCH_JOB_EXECUTION_PARAMS + ( JOB_EXECUTION_ID , TYPE_CD, KEY_NAME, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING ) +SELECT + JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 'Y' +FROM + BATCH_JOB_PARAMS JP,BATCH_JOB_EXECUTION JE +WHERE + JP.JOB_INSTANCE_ID = JE.JOB_INSTANCE_ID; \ No newline at end of file diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-postgresql.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-postgresql.sql similarity index 94% rename from spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-postgresql.sql rename to spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-postgresql.sql index c3f08c8865..e52da193b6 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-postgresql.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-postgresql.sql @@ -21,7 +21,7 @@ CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( INSERT INTO BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID , TYPE_CD, KEY_NAME, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING ) SELECT - JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 1 + JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 'Y' FROM BATCH_JOB_PARAMS JP,BATCH_JOB_EXECUTION JE WHERE diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-sqlf.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-sqlf.sql similarity index 94% rename from spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-sqlf.sql rename to spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-sqlf.sql index c3f08c8865..e52da193b6 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-sqlf.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-sqlf.sql @@ -21,7 +21,7 @@ CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( INSERT INTO BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID , TYPE_CD, KEY_NAME, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING ) SELECT - JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 1 + JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 'Y' FROM BATCH_JOB_PARAMS JP,BATCH_JOB_EXECUTION JE WHERE diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-sqlserver.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-sqlserver.sql similarity index 94% rename from spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-sqlserver.sql rename to spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-sqlserver.sql index bacb5a53b7..4c24789ee8 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-sqlserver.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-sqlserver.sql @@ -21,7 +21,7 @@ CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( INSERT INTO BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID , TYPE_CD, KEY_NAME, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING ) SELECT - JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 1 + JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 'Y' FROM BATCH_JOB_PARAMS JP,BATCH_JOB_EXECUTION JE WHERE diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-sybase.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-sybase.sql similarity index 94% rename from spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-sybase.sql rename to spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-sybase.sql index 586712c58c..d8def25a63 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-sybase.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/2.2/migration-sybase.sql @@ -21,7 +21,7 @@ CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( INSERT INTO BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID , TYPE_CD, KEY_NAME, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING ) SELECT - JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 1 + JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 'Y' FROM BATCH_JOB_PARAMS JP,BATCH_JOB_EXECUTION JE WHERE diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/4.1/migration-oracle10g.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/4.1/migration-oracle10g.sql new file mode 100644 index 0000000000..79c931b1c9 --- /dev/null +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/4.1/migration-oracle10g.sql @@ -0,0 +1,19 @@ +ALTER TABLE BATCH_JOB_INSTANCE MODIFY JOB_NAME VARCHAR2(100 char); +ALTER TABLE BATCH_JOB_INSTANCE MODIFY JOB_KEY VARCHAR2(32 char); + +ALTER TABLE BATCH_JOB_EXECUTION MODIFY STATUS VARCHAR2(10 char); +ALTER TABLE BATCH_JOB_EXECUTION MODIFY EXIT_CODE VARCHAR2(2500 char); +ALTER TABLE BATCH_JOB_EXECUTION MODIFY EXIT_MESSAGE VARCHAR2(2500 char); +ALTER TABLE BATCH_JOB_EXECUTION MODIFY JOB_CONFIGURATION_LOCATION VARCHAR2(2500 char); + +ALTER TABLE BATCH_JOB_EXECUTION_PARAMS MODIFY TYPE_CD VARCHAR2(6 char); +ALTER TABLE BATCH_JOB_EXECUTION_PARAMS MODIFY KEY_NAME VARCHAR2(100 char); +ALTER TABLE BATCH_JOB_EXECUTION_PARAMS MODIFY STRING_VAL VARCHAR2(250 char); + +ALTER TABLE BATCH_STEP_EXECUTION MODIFY STEP_NAME VARCHAR2(100 char); +ALTER TABLE BATCH_STEP_EXECUTION MODIFY STATUS VARCHAR2(10 char); +ALTER TABLE BATCH_STEP_EXECUTION MODIFY EXIT_CODE VARCHAR2(2500 char); +ALTER TABLE BATCH_STEP_EXECUTION MODIFY EXIT_MESSAGE VARCHAR2(2500 char); + +ALTER TABLE BATCH_STEP_EXECUTION_CONTEXT MODIFY SHORT_CONTEXT VARCHAR2(2500 char); +ALTER TABLE BATCH_JOB_EXECUTION_CONTEXT MODIFY SHORT_CONTEXT VARCHAR2(2500 char); diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-oracle10g.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-oracle10g.sql deleted file mode 100644 index c63a012404..0000000000 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/migration/migration-oracle10g.sql +++ /dev/null @@ -1,28 +0,0 @@ - --- create the requisite table - -CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( - JOB_EXECUTION_ID NUMBER(19,0) NOT NULL , - TYPE_CD VARCHAR2(6) NOT NULL , - KEY_NAME VARCHAR2(100) NOT NULL , - STRING_VAL VARCHAR2(250) , - DATE_VAL TIMESTAMP DEFAULT NULL , - LONG_VAL NUMBER(19,0) , - DOUBLE_VAL NUMBER , - IDENTIFYING CHAR(1) NOT NULL , - constraint JOB_EXEC_PARAMS_FK foreign key (JOB_EXECUTION_ID) - references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) -) ; - --- insert script that 'copies' existing batch_job_params to batch_job_execution_params --- sets new params to identifying ones --- verified on h2, - -INSERT INTO BATCH_JOB_EXECUTION_PARAMS - ( JOB_EXECUTION_ID , TYPE_CD, KEY_NAME, STRING_VAL, DATE_VAL, LONG_VAL, DOUBLE_VAL, IDENTIFYING ) -SELECT - JE.JOB_EXECUTION_ID , JP.TYPE_CD , JP.KEY_NAME , JP.STRING_VAL , JP.DATE_VAL , JP.LONG_VAL , JP.DOUBLE_VAL , 1 -FROM - BATCH_JOB_PARAMS JP,BATCH_JOB_EXECUTION JE -WHERE - JP.JOB_INSTANCE_ID = JE.JOB_INSTANCE_ID; \ No newline at end of file diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-db2.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-db2.sql index 264b52ad20..e37118ad95 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-db2.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-db2.sql @@ -16,9 +16,10 @@ CREATE TABLE BATCH_JOB_EXECUTION ( START_TIME TIMESTAMP DEFAULT NULL , END_TIME TIMESTAMP DEFAULT NULL , STATUS VARCHAR(10) , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, + JOB_CONFIGURATION_LOCATION VARCHAR(2500) NULL, constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) ) ; @@ -52,7 +53,7 @@ CREATE TABLE BATCH_STEP_EXECUTION ( WRITE_SKIP_COUNT BIGINT , PROCESS_SKIP_COUNT BIGINT , ROLLBACK_COUNT BIGINT , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, constraint JOB_EXEC_STEP_FK foreign key (JOB_EXECUTION_ID) diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-derby.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-derby.sql index 8794edb696..09a8e53efe 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-derby.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-derby.sql @@ -16,9 +16,10 @@ CREATE TABLE BATCH_JOB_EXECUTION ( START_TIME TIMESTAMP DEFAULT NULL , END_TIME TIMESTAMP DEFAULT NULL , STATUS VARCHAR(10) , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, + JOB_CONFIGURATION_LOCATION VARCHAR(2500) DEFAULT NULL, constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) ) ; @@ -52,7 +53,7 @@ CREATE TABLE BATCH_STEP_EXECUTION ( WRITE_SKIP_COUNT BIGINT , PROCESS_SKIP_COUNT BIGINT , ROLLBACK_COUNT BIGINT , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, constraint JOB_EXEC_STEP_FK foreign key (JOB_EXECUTION_ID) diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-drop-sqlite.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-drop-sqlite.sql new file mode 100644 index 0000000000..1b4e013909 --- /dev/null +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-drop-sqlite.sql @@ -0,0 +1,12 @@ +-- Autogenerated: do not edit this file + DROP TABLE IF EXISTS BATCH_STEP_EXECUTION_CONTEXT ; +DROP TABLE IF EXISTS BATCH_JOB_EXECUTION_CONTEXT ; +DROP TABLE IF EXISTS BATCH_JOB_EXECUTION_PARAMS ; +DROP TABLE IF EXISTS BATCH_STEP_EXECUTION ; +DROP TABLE IF EXISTS BATCH_JOB_EXECUTION ; +DROP TABLE IF EXISTS BATCH_JOB_PARAMS ; +DROP TABLE IF EXISTS BATCH_JOB_INSTANCE ; + +DROP TABLE IF EXISTS BATCH_STEP_EXECUTION_SEQ ; +DROP TABLE IF EXISTS BATCH_JOB_EXECUTION_SEQ ; +DROP TABLE IF EXISTS BATCH_JOB_SEQ ; diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-h2.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-h2.sql index a8bf524279..fb19c65549 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-h2.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-h2.sql @@ -16,9 +16,10 @@ CREATE TABLE BATCH_JOB_EXECUTION ( START_TIME TIMESTAMP DEFAULT NULL , END_TIME TIMESTAMP DEFAULT NULL , STATUS VARCHAR(10) , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, + JOB_CONFIGURATION_LOCATION VARCHAR(2500) NULL, constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) ) ; @@ -52,7 +53,7 @@ CREATE TABLE BATCH_STEP_EXECUTION ( WRITE_SKIP_COUNT BIGINT , PROCESS_SKIP_COUNT BIGINT , ROLLBACK_COUNT BIGINT , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, constraint JOB_EXEC_STEP_FK foreign key (JOB_EXECUTION_ID) diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-hsqldb.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-hsqldb.sql index 5ac3d3544d..4de04851fe 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-hsqldb.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-hsqldb.sql @@ -16,9 +16,10 @@ CREATE TABLE BATCH_JOB_EXECUTION ( START_TIME TIMESTAMP DEFAULT NULL , END_TIME TIMESTAMP DEFAULT NULL , STATUS VARCHAR(10) , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, + JOB_CONFIGURATION_LOCATION VARCHAR(2500) NULL, constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) ) ; @@ -52,7 +53,7 @@ CREATE TABLE BATCH_STEP_EXECUTION ( WRITE_SKIP_COUNT BIGINT , PROCESS_SKIP_COUNT BIGINT , ROLLBACK_COUNT BIGINT , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, constraint JOB_EXEC_STEP_FK foreign key (JOB_EXECUTION_ID) diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-mysql.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-mysql.sql index 9d7e9009b0..5bd10960ec 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-mysql.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-mysql.sql @@ -16,9 +16,10 @@ CREATE TABLE BATCH_JOB_EXECUTION ( START_TIME DATETIME DEFAULT NULL , END_TIME DATETIME DEFAULT NULL , STATUS VARCHAR(10) , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED DATETIME, + JOB_CONFIGURATION_LOCATION VARCHAR(2500) NULL, constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) ) ENGINE=InnoDB; @@ -52,7 +53,7 @@ CREATE TABLE BATCH_STEP_EXECUTION ( WRITE_SKIP_COUNT BIGINT , PROCESS_SKIP_COUNT BIGINT , ROLLBACK_COUNT BIGINT , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED DATETIME, constraint JOB_EXEC_STEP_FK foreign key (JOB_EXECUTION_ID) @@ -75,9 +76,26 @@ CREATE TABLE BATCH_JOB_EXECUTION_CONTEXT ( references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) ) ENGINE=InnoDB; -CREATE TABLE BATCH_STEP_EXECUTION_SEQ (ID BIGINT NOT NULL) ENGINE=MYISAM; -INSERT INTO BATCH_STEP_EXECUTION_SEQ values(0); -CREATE TABLE BATCH_JOB_EXECUTION_SEQ (ID BIGINT NOT NULL) ENGINE=MYISAM; -INSERT INTO BATCH_JOB_EXECUTION_SEQ values(0); -CREATE TABLE BATCH_JOB_SEQ (ID BIGINT NOT NULL) ENGINE=MYISAM; -INSERT INTO BATCH_JOB_SEQ values(0); +CREATE TABLE BATCH_STEP_EXECUTION_SEQ ( + ID BIGINT NOT NULL, + UNIQUE_KEY CHAR(1) NOT NULL, + constraint UNIQUE_KEY_UN unique (UNIQUE_KEY) +) ENGINE=InnoDB; + +INSERT INTO BATCH_STEP_EXECUTION_SEQ (ID, UNIQUE_KEY) select * from (select 0 as ID, '0' as UNIQUE_KEY) as tmp where not exists(select * from BATCH_STEP_EXECUTION_SEQ); + +CREATE TABLE BATCH_JOB_EXECUTION_SEQ ( + ID BIGINT NOT NULL, + UNIQUE_KEY CHAR(1) NOT NULL, + constraint UNIQUE_KEY_UN unique (UNIQUE_KEY) +) ENGINE=InnoDB; + +INSERT INTO BATCH_JOB_EXECUTION_SEQ (ID, UNIQUE_KEY) select * from (select 0 as ID, '0' as UNIQUE_KEY) as tmp where not exists(select * from BATCH_JOB_EXECUTION_SEQ); + +CREATE TABLE BATCH_JOB_SEQ ( + ID BIGINT NOT NULL, + UNIQUE_KEY CHAR(1) NOT NULL, + constraint UNIQUE_KEY_UN unique (UNIQUE_KEY) +) ENGINE=InnoDB; + +INSERT INTO BATCH_JOB_SEQ (ID, UNIQUE_KEY) select * from (select 0 as ID, '0' as UNIQUE_KEY) as tmp where not exists(select * from BATCH_JOB_SEQ); diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-oracle10g.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-oracle10g.sql index 73403b90b3..b445e35797 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-oracle10g.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-oracle10g.sql @@ -3,8 +3,8 @@ CREATE TABLE BATCH_JOB_INSTANCE ( JOB_INSTANCE_ID NUMBER(19,0) NOT NULL PRIMARY KEY , VERSION NUMBER(19,0) , - JOB_NAME VARCHAR2(100) NOT NULL, - JOB_KEY VARCHAR2(32) NOT NULL, + JOB_NAME VARCHAR2(100 char) NOT NULL, + JOB_KEY VARCHAR2(32 char) NOT NULL, constraint JOB_INST_UN unique (JOB_NAME, JOB_KEY) ) ; @@ -15,19 +15,20 @@ CREATE TABLE BATCH_JOB_EXECUTION ( CREATE_TIME TIMESTAMP NOT NULL, START_TIME TIMESTAMP DEFAULT NULL , END_TIME TIMESTAMP DEFAULT NULL , - STATUS VARCHAR2(10) , - EXIT_CODE VARCHAR2(100) , - EXIT_MESSAGE VARCHAR2(2500) , + STATUS VARCHAR2(10 char) , + EXIT_CODE VARCHAR2(2500 char) , + EXIT_MESSAGE VARCHAR2(2500 char) , LAST_UPDATED TIMESTAMP, + JOB_CONFIGURATION_LOCATION VARCHAR(2500 char) NULL, constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) ) ; CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID NUMBER(19,0) NOT NULL , - TYPE_CD VARCHAR2(6) NOT NULL , - KEY_NAME VARCHAR2(100) NOT NULL , - STRING_VAL VARCHAR2(250) , + TYPE_CD VARCHAR2(6 char) NOT NULL , + KEY_NAME VARCHAR2(100 char) NOT NULL , + STRING_VAL VARCHAR2(250 char) , DATE_VAL TIMESTAMP DEFAULT NULL , LONG_VAL NUMBER(19,0) , DOUBLE_VAL NUMBER , @@ -39,11 +40,11 @@ CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( CREATE TABLE BATCH_STEP_EXECUTION ( STEP_EXECUTION_ID NUMBER(19,0) NOT NULL PRIMARY KEY , VERSION NUMBER(19,0) NOT NULL, - STEP_NAME VARCHAR2(100) NOT NULL, + STEP_NAME VARCHAR2(100 char) NOT NULL, JOB_EXECUTION_ID NUMBER(19,0) NOT NULL, START_TIME TIMESTAMP NOT NULL , END_TIME TIMESTAMP DEFAULT NULL , - STATUS VARCHAR2(10) , + STATUS VARCHAR2(10 char) , COMMIT_COUNT NUMBER(19,0) , READ_COUNT NUMBER(19,0) , FILTER_COUNT NUMBER(19,0) , @@ -52,8 +53,8 @@ CREATE TABLE BATCH_STEP_EXECUTION ( WRITE_SKIP_COUNT NUMBER(19,0) , PROCESS_SKIP_COUNT NUMBER(19,0) , ROLLBACK_COUNT NUMBER(19,0) , - EXIT_CODE VARCHAR2(100) , - EXIT_MESSAGE VARCHAR2(2500) , + EXIT_CODE VARCHAR2(2500 char) , + EXIT_MESSAGE VARCHAR2(2500 char) , LAST_UPDATED TIMESTAMP, constraint JOB_EXEC_STEP_FK foreign key (JOB_EXECUTION_ID) references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) @@ -61,7 +62,7 @@ CREATE TABLE BATCH_STEP_EXECUTION ( CREATE TABLE BATCH_STEP_EXECUTION_CONTEXT ( STEP_EXECUTION_ID NUMBER(19,0) NOT NULL PRIMARY KEY, - SHORT_CONTEXT VARCHAR2(2500) NOT NULL, + SHORT_CONTEXT VARCHAR2(2500 char) NOT NULL, SERIALIZED_CONTEXT CLOB , constraint STEP_EXEC_CTX_FK foreign key (STEP_EXECUTION_ID) references BATCH_STEP_EXECUTION(STEP_EXECUTION_ID) @@ -69,7 +70,7 @@ CREATE TABLE BATCH_STEP_EXECUTION_CONTEXT ( CREATE TABLE BATCH_JOB_EXECUTION_CONTEXT ( JOB_EXECUTION_ID NUMBER(19,0) NOT NULL PRIMARY KEY, - SHORT_CONTEXT VARCHAR2(2500) NOT NULL, + SHORT_CONTEXT VARCHAR2(2500 char) NOT NULL, SERIALIZED_CONTEXT CLOB , constraint JOB_EXEC_CTX_FK foreign key (JOB_EXECUTION_ID) references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-postgresql.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-postgresql.sql index d0435b3f83..fe3299a076 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-postgresql.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-postgresql.sql @@ -16,9 +16,10 @@ CREATE TABLE BATCH_JOB_EXECUTION ( START_TIME TIMESTAMP DEFAULT NULL , END_TIME TIMESTAMP DEFAULT NULL , STATUS VARCHAR(10) , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, + JOB_CONFIGURATION_LOCATION VARCHAR(2500) NULL, constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) ) ; @@ -52,7 +53,7 @@ CREATE TABLE BATCH_STEP_EXECUTION ( WRITE_SKIP_COUNT BIGINT , PROCESS_SKIP_COUNT BIGINT , ROLLBACK_COUNT BIGINT , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, constraint JOB_EXEC_STEP_FK foreign key (JOB_EXECUTION_ID) diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sqlf.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sqlf.sql index 8794edb696..7910641065 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sqlf.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sqlf.sql @@ -16,9 +16,10 @@ CREATE TABLE BATCH_JOB_EXECUTION ( START_TIME TIMESTAMP DEFAULT NULL , END_TIME TIMESTAMP DEFAULT NULL , STATUS VARCHAR(10) , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, + JOB_CONFIGURATION_LOCATION VARCHAR(2500) NULL, constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) ) ; @@ -52,7 +53,7 @@ CREATE TABLE BATCH_STEP_EXECUTION ( WRITE_SKIP_COUNT BIGINT , PROCESS_SKIP_COUNT BIGINT , ROLLBACK_COUNT BIGINT , - EXIT_CODE VARCHAR(100) , + EXIT_CODE VARCHAR(2500) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, constraint JOB_EXEC_STEP_FK foreign key (JOB_EXECUTION_ID) diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sqlite.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sqlite.sql new file mode 100644 index 0000000000..5df5e404a0 --- /dev/null +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sqlite.sql @@ -0,0 +1,87 @@ +-- Autogenerated: do not edit this file + +CREATE TABLE BATCH_JOB_INSTANCE ( + JOB_INSTANCE_ID INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, + VERSION INTEGER , + JOB_NAME VARCHAR(100) NOT NULL, + JOB_KEY VARCHAR(32) NOT NULL, + constraint JOB_INST_UN unique (JOB_NAME, JOB_KEY) +) ; + +CREATE TABLE BATCH_JOB_EXECUTION ( + JOB_EXECUTION_ID INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, + VERSION INTEGER , + JOB_INSTANCE_ID INTEGER NOT NULL, + CREATE_TIME TIMESTAMP NOT NULL, + START_TIME TIMESTAMP DEFAULT NULL , + END_TIME TIMESTAMP DEFAULT NULL , + STATUS VARCHAR(10) , + EXIT_CODE VARCHAR(100) , + EXIT_MESSAGE VARCHAR(2500) , + LAST_UPDATED TIMESTAMP, + JOB_CONFIGURATION_LOCATION VARCHAR(2500), + constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) + references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) +) ; + +CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( + JOB_EXECUTION_ID INTEGER NOT NULL , + TYPE_CD VARCHAR(6) NOT NULL , + KEY_NAME VARCHAR(100) NOT NULL , + STRING_VAL VARCHAR(250) , + DATE_VAL TIMESTAMP DEFAULT NULL , + LONG_VAL INTEGER , + DOUBLE_VAL DOUBLE PRECISION , + IDENTIFYING CHAR(1) NOT NULL , + constraint JOB_EXEC_PARAMS_FK foreign key (JOB_EXECUTION_ID) + references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) +) ; + +CREATE TABLE BATCH_STEP_EXECUTION ( + STEP_EXECUTION_ID INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, + VERSION INTEGER NOT NULL, + STEP_NAME VARCHAR(100) NOT NULL, + JOB_EXECUTION_ID INTEGER NOT NULL, + START_TIME TIMESTAMP NOT NULL , + END_TIME TIMESTAMP DEFAULT NULL , + STATUS VARCHAR(10) , + COMMIT_COUNT INTEGER , + READ_COUNT INTEGER , + FILTER_COUNT INTEGER , + WRITE_COUNT INTEGER , + READ_SKIP_COUNT INTEGER , + WRITE_SKIP_COUNT INTEGER , + PROCESS_SKIP_COUNT INTEGER , + ROLLBACK_COUNT INTEGER , + EXIT_CODE VARCHAR(100) , + EXIT_MESSAGE VARCHAR(2500) , + LAST_UPDATED TIMESTAMP, + constraint JOB_EXEC_STEP_FK foreign key (JOB_EXECUTION_ID) + references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) +) ; + +CREATE TABLE BATCH_STEP_EXECUTION_CONTEXT ( + STEP_EXECUTION_ID INTEGER NOT NULL PRIMARY KEY, + SHORT_CONTEXT VARCHAR(2500) NOT NULL, + SERIALIZED_CONTEXT CLOB , + constraint STEP_EXEC_CTX_FK foreign key (STEP_EXECUTION_ID) + references BATCH_STEP_EXECUTION(STEP_EXECUTION_ID) +) ; + +CREATE TABLE BATCH_JOB_EXECUTION_CONTEXT ( + JOB_EXECUTION_ID INTEGER NOT NULL PRIMARY KEY, + SHORT_CONTEXT VARCHAR(2500) NOT NULL, + SERIALIZED_CONTEXT CLOB , + constraint JOB_EXEC_CTX_FK foreign key (JOB_EXECUTION_ID) + references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) +) ; + +CREATE TABLE BATCH_STEP_EXECUTION_SEQ ( + ID INTEGER PRIMARY KEY AUTOINCREMENT +); +CREATE TABLE BATCH_JOB_EXECUTION_SEQ ( + ID INTEGER PRIMARY KEY AUTOINCREMENT +); +CREATE TABLE BATCH_JOB_SEQ ( + ID INTEGER PRIMARY KEY AUTOINCREMENT +); diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sqlserver.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sqlserver.sql index 560a304bb3..9f93513f72 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sqlserver.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sqlserver.sql @@ -2,7 +2,7 @@ CREATE TABLE BATCH_JOB_INSTANCE ( JOB_INSTANCE_ID BIGINT NOT NULL PRIMARY KEY , - VERSION BIGINT , + VERSION BIGINT NULL, JOB_NAME VARCHAR(100) NOT NULL, JOB_KEY VARCHAR(32) NOT NULL, constraint JOB_INST_UN unique (JOB_NAME, JOB_KEY) @@ -10,15 +10,16 @@ CREATE TABLE BATCH_JOB_INSTANCE ( CREATE TABLE BATCH_JOB_EXECUTION ( JOB_EXECUTION_ID BIGINT NOT NULL PRIMARY KEY , - VERSION BIGINT , + VERSION BIGINT NULL, JOB_INSTANCE_ID BIGINT NOT NULL, CREATE_TIME DATETIME NOT NULL, START_TIME DATETIME DEFAULT NULL , END_TIME DATETIME DEFAULT NULL , - STATUS VARCHAR(10) , - EXIT_CODE VARCHAR(100) , - EXIT_MESSAGE VARCHAR(2500) , - LAST_UPDATED DATETIME, + STATUS VARCHAR(10) NULL, + EXIT_CODE VARCHAR(2500) NULL, + EXIT_MESSAGE VARCHAR(2500) NULL, + LAST_UPDATED DATETIME NULL, + JOB_CONFIGURATION_LOCATION VARCHAR(2500) NULL, constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) ) ; @@ -27,10 +28,10 @@ CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID BIGINT NOT NULL , TYPE_CD VARCHAR(6) NOT NULL , KEY_NAME VARCHAR(100) NOT NULL , - STRING_VAL VARCHAR(250) , + STRING_VAL VARCHAR(250) NULL, DATE_VAL DATETIME DEFAULT NULL , - LONG_VAL BIGINT , - DOUBLE_VAL DOUBLE PRECISION , + LONG_VAL BIGINT NULL, + DOUBLE_VAL DOUBLE PRECISION NULL, IDENTIFYING CHAR(1) NOT NULL , constraint JOB_EXEC_PARAMS_FK foreign key (JOB_EXECUTION_ID) references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) @@ -43,18 +44,18 @@ CREATE TABLE BATCH_STEP_EXECUTION ( JOB_EXECUTION_ID BIGINT NOT NULL, START_TIME DATETIME NOT NULL , END_TIME DATETIME DEFAULT NULL , - STATUS VARCHAR(10) , - COMMIT_COUNT BIGINT , - READ_COUNT BIGINT , - FILTER_COUNT BIGINT , - WRITE_COUNT BIGINT , - READ_SKIP_COUNT BIGINT , - WRITE_SKIP_COUNT BIGINT , - PROCESS_SKIP_COUNT BIGINT , - ROLLBACK_COUNT BIGINT , - EXIT_CODE VARCHAR(100) , - EXIT_MESSAGE VARCHAR(2500) , - LAST_UPDATED DATETIME, + STATUS VARCHAR(10) NULL, + COMMIT_COUNT BIGINT NULL, + READ_COUNT BIGINT NULL, + FILTER_COUNT BIGINT NULL, + WRITE_COUNT BIGINT NULL, + READ_SKIP_COUNT BIGINT NULL, + WRITE_SKIP_COUNT BIGINT NULL, + PROCESS_SKIP_COUNT BIGINT NULL, + ROLLBACK_COUNT BIGINT NULL, + EXIT_CODE VARCHAR(2500) NULL, + EXIT_MESSAGE VARCHAR(2500) NULL, + LAST_UPDATED DATETIME NULL, constraint JOB_EXEC_STEP_FK foreign key (JOB_EXECUTION_ID) references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) ) ; @@ -62,7 +63,7 @@ CREATE TABLE BATCH_STEP_EXECUTION ( CREATE TABLE BATCH_STEP_EXECUTION_CONTEXT ( STEP_EXECUTION_ID BIGINT NOT NULL PRIMARY KEY, SHORT_CONTEXT VARCHAR(2500) NOT NULL, - SERIALIZED_CONTEXT TEXT , + SERIALIZED_CONTEXT TEXT NULL, constraint STEP_EXEC_CTX_FK foreign key (STEP_EXECUTION_ID) references BATCH_STEP_EXECUTION(STEP_EXECUTION_ID) ) ; @@ -70,7 +71,7 @@ CREATE TABLE BATCH_STEP_EXECUTION_CONTEXT ( CREATE TABLE BATCH_JOB_EXECUTION_CONTEXT ( JOB_EXECUTION_ID BIGINT NOT NULL PRIMARY KEY, SHORT_CONTEXT VARCHAR(2500) NOT NULL, - SERIALIZED_CONTEXT TEXT , + SERIALIZED_CONTEXT TEXT NULL, constraint JOB_EXEC_CTX_FK foreign key (JOB_EXECUTION_ID) references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) ) ; diff --git a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sybase.sql b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sybase.sql index 6f122e5230..aeea56c270 100644 --- a/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sybase.sql +++ b/spring-batch-core/src/main/resources/org/springframework/batch/core/schema-sybase.sql @@ -16,9 +16,10 @@ CREATE TABLE BATCH_JOB_EXECUTION ( START_TIME DATETIME DEFAULT NULL NULL, END_TIME DATETIME DEFAULT NULL NULL, STATUS VARCHAR(10) NULL, - EXIT_CODE VARCHAR(100) NULL, + EXIT_CODE VARCHAR(2500) NULL, EXIT_MESSAGE VARCHAR(2500) NULL, LAST_UPDATED DATETIME, + JOB_CONFIGURATION_LOCATION VARCHAR(2500) NULL, constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) ) ; @@ -52,7 +53,7 @@ CREATE TABLE BATCH_STEP_EXECUTION ( WRITE_SKIP_COUNT BIGINT NULL, PROCESS_SKIP_COUNT BIGINT NULL, ROLLBACK_COUNT BIGINT NULL, - EXIT_CODE VARCHAR(100) NULL, + EXIT_CODE VARCHAR(2500) NULL, EXIT_MESSAGE VARCHAR(2500) NULL, LAST_UPDATED DATETIME, constraint JOB_EXEC_STEP_FK foreign key (JOB_EXECUTION_ID) diff --git a/spring-batch-core/src/main/sql/destroy.sql.vpp b/spring-batch-core/src/main/sql/destroy.sql.vpp index 1c95ce2a87..b834009856 100644 --- a/spring-batch-core/src/main/sql/destroy.sql.vpp +++ b/spring-batch-core/src/main/sql/destroy.sql.vpp @@ -1,8 +1,8 @@ DROP TABLE $!{IFEXISTSBEFORE} BATCH_STEP_EXECUTION_CONTEXT $!{IFEXISTS}; DROP TABLE $!{IFEXISTSBEFORE} BATCH_JOB_EXECUTION_CONTEXT $!{IFEXISTS}; +DROP TABLE $!{IFEXISTSBEFORE} BATCH_JOB_EXECUTION_PARAMS $!{IFEXISTS}; DROP TABLE $!{IFEXISTSBEFORE} BATCH_STEP_EXECUTION $!{IFEXISTS}; DROP TABLE $!{IFEXISTSBEFORE} BATCH_JOB_EXECUTION $!{IFEXISTS}; -DROP TABLE $!{IFEXISTSBEFORE} BATCH_JOB_PARAMS $!{IFEXISTS}; DROP TABLE $!{IFEXISTSBEFORE} BATCH_JOB_INSTANCE $!{IFEXISTS}; DROP ${SEQUENCE} $!{IFEXISTSBEFORE} BATCH_STEP_EXECUTION_SEQ $!{IFEXISTS}; diff --git a/spring-batch-core/src/main/sql/mysql.properties b/spring-batch-core/src/main/sql/mysql.properties index 470ca6bc1d..3d83bc702e 100644 --- a/spring-batch-core/src/main/sql/mysql.properties +++ b/spring-batch-core/src/main/sql/mysql.properties @@ -1,8 +1,8 @@ -platform=oracle10g +platform=mysql # SQL language oddities BIGINT = BIGINT -IDENTITY = -GENERATED = +IDENTITY = +GENERATED = VOODOO = ENGINE=InnoDB IFEXISTSBEFORE = IF EXISTS DOUBLE = DOUBLE PRECISION @@ -10,5 +10,6 @@ BLOB = BLOB CLOB = TEXT TIMESTAMP = DATETIME VARCHAR = VARCHAR +CHAR = CHAR # for generating drop statements... SEQUENCE = TABLE diff --git a/spring-batch-core/src/main/sql/mysql.vpp b/spring-batch-core/src/main/sql/mysql.vpp index fd65fad6e3..aca4243940 100644 --- a/spring-batch-core/src/main/sql/mysql.vpp +++ b/spring-batch-core/src/main/sql/mysql.vpp @@ -1,4 +1,6 @@ -#macro (sequence $name $value)CREATE TABLE ${name} (ID BIGINT NOT NULL) ENGINE=MYISAM; -INSERT INTO ${name} values(0); +#macro (sequence $name $value)CREATE TABLE ${name} (ID BIGINT NOT NULL, + UNIQUE_KEY CHAR(1) NOT NULL, + constraint ${name}_UN unique (UNIQUE_KEY)) ENGINE=InnoDB; +INSERT INTO ${name} (ID, UNIQUE_KEY) select * from (select 0 as ID, '0' as UNIQUE_KEY) as tmp where not exists(select * from ${name}); #end #macro (notnull $name $type)MODIFY COLUMN ${name} ${type} NOT NULL#end diff --git a/spring-batch-core/src/main/sql/schema.sql.vpp b/spring-batch-core/src/main/sql/schema.sql.vpp index fd7a0bd923..3dcc730fab 100644 --- a/spring-batch-core/src/main/sql/schema.sql.vpp +++ b/spring-batch-core/src/main/sql/schema.sql.vpp @@ -1,33 +1,34 @@ -- Autogenerated: do not edit this file CREATE TABLE BATCH_JOB_INSTANCE ( - JOB_INSTANCE_ID ${BIGINT} $!{IDENTITY} NOT NULL PRIMARY KEY $!{GENERATED}, - VERSION ${BIGINT} $!{NULL}, - JOB_NAME ${VARCHAR}(100) NOT NULL, + JOB_INSTANCE_ID ${BIGINT} $!{IDENTITY} NOT NULL PRIMARY KEY $!{GENERATED}, + VERSION ${BIGINT} $!{NULL}, + JOB_NAME ${VARCHAR}(100) NOT NULL, JOB_KEY ${VARCHAR}(32) NOT NULL, constraint JOB_INST_UN unique (JOB_NAME, JOB_KEY) ) $!{VOODOO}; CREATE TABLE BATCH_JOB_EXECUTION ( JOB_EXECUTION_ID ${BIGINT} $!{IDENTITY} NOT NULL PRIMARY KEY $!{GENERATED}, - VERSION ${BIGINT} $!{NULL}, + VERSION ${BIGINT} $!{NULL}, JOB_INSTANCE_ID ${BIGINT} NOT NULL, CREATE_TIME ${TIMESTAMP} NOT NULL, - START_TIME ${TIMESTAMP} DEFAULT NULL $!{NULL}, + START_TIME ${TIMESTAMP} DEFAULT NULL $!{NULL}, END_TIME ${TIMESTAMP} DEFAULT NULL $!{NULL}, STATUS ${VARCHAR}(10) $!{NULL}, - EXIT_CODE ${VARCHAR}(100) $!{NULL}, + EXIT_CODE ${VARCHAR}(2500) $!{NULL}, EXIT_MESSAGE ${VARCHAR}(2500) $!{NULL}, LAST_UPDATED ${TIMESTAMP}, + JOB_CONFIGURATION_LOCATION ${VARCHAR}(2500) $!{NULL}, constraint JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) ) $!{VOODOO}; - + CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( JOB_EXECUTION_ID ${BIGINT} NOT NULL , TYPE_CD ${VARCHAR}(6) NOT NULL , - KEY_NAME ${VARCHAR}(100) NOT NULL , - STRING_VAL ${VARCHAR}(250) $!{NULL}, + KEY_NAME ${VARCHAR}(100) NOT NULL , + STRING_VAL ${VARCHAR}(250) $!{NULL}, DATE_VAL ${TIMESTAMP} DEFAULT NULL $!{NULL}, LONG_VAL ${BIGINT} $!{NULL}, DOUBLE_VAL ${DOUBLE} $!{NULL}, @@ -35,24 +36,24 @@ CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( constraint JOB_EXEC_PARAMS_FK foreign key (JOB_EXECUTION_ID) references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) ) $!{VOODOO}; - + CREATE TABLE BATCH_STEP_EXECUTION ( STEP_EXECUTION_ID ${BIGINT} $!{IDENTITY} NOT NULL PRIMARY KEY $!{GENERATED}, - VERSION ${BIGINT} NOT NULL, + VERSION ${BIGINT} NOT NULL, STEP_NAME ${VARCHAR}(100) NOT NULL, JOB_EXECUTION_ID ${BIGINT} NOT NULL, - START_TIME ${TIMESTAMP} NOT NULL , - END_TIME ${TIMESTAMP} DEFAULT NULL $!{NULL}, + START_TIME ${TIMESTAMP} NOT NULL , + END_TIME ${TIMESTAMP} DEFAULT NULL $!{NULL}, STATUS ${VARCHAR}(10) $!{NULL}, - COMMIT_COUNT ${BIGINT} $!{NULL}, + COMMIT_COUNT ${BIGINT} $!{NULL}, READ_COUNT ${BIGINT} $!{NULL}, FILTER_COUNT ${BIGINT} $!{NULL}, WRITE_COUNT ${BIGINT} $!{NULL}, READ_SKIP_COUNT ${BIGINT} $!{NULL}, WRITE_SKIP_COUNT ${BIGINT} $!{NULL}, PROCESS_SKIP_COUNT ${BIGINT} $!{NULL}, - ROLLBACK_COUNT ${BIGINT} $!{NULL}, - EXIT_CODE ${VARCHAR}(100) $!{NULL}, + ROLLBACK_COUNT ${BIGINT} $!{NULL}, + EXIT_CODE ${VARCHAR}(2500) $!{NULL}, EXIT_MESSAGE ${VARCHAR}(2500) $!{NULL}, LAST_UPDATED ${TIMESTAMP}, constraint JOB_EXEC_STEP_FK foreign key (JOB_EXECUTION_ID) @@ -62,7 +63,7 @@ CREATE TABLE BATCH_STEP_EXECUTION ( CREATE TABLE BATCH_STEP_EXECUTION_CONTEXT ( STEP_EXECUTION_ID ${BIGINT} NOT NULL PRIMARY KEY, SHORT_CONTEXT ${VARCHAR}(2500) NOT NULL, - SERIALIZED_CONTEXT ${CLOB} $!{NULL}, + SERIALIZED_CONTEXT ${CLOB} $!{NULL}, constraint STEP_EXEC_CTX_FK foreign key (STEP_EXECUTION_ID) references BATCH_STEP_EXECUTION(STEP_EXECUTION_ID) ) $!{VOODOO}; @@ -70,7 +71,7 @@ CREATE TABLE BATCH_STEP_EXECUTION_CONTEXT ( CREATE TABLE BATCH_JOB_EXECUTION_CONTEXT ( JOB_EXECUTION_ID ${BIGINT} NOT NULL PRIMARY KEY, SHORT_CONTEXT ${VARCHAR}(2500) NOT NULL, - SERIALIZED_CONTEXT ${CLOB} $!{NULL}, + SERIALIZED_CONTEXT ${CLOB} $!{NULL}, constraint JOB_EXEC_CTX_FK foreign key (JOB_EXECUTION_ID) references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) ) $!{VOODOO}; diff --git a/spring-batch-core/src/main/sql/sqlite.properties b/spring-batch-core/src/main/sql/sqlite.properties new file mode 100644 index 0000000000..807467dcaf --- /dev/null +++ b/spring-batch-core/src/main/sql/sqlite.properties @@ -0,0 +1,14 @@ +platform=sqlite +# SQL language oddities +BIGINT = INTEGER +IDENTITY = +GENERATED = AUTOINCREMENT +IFEXISTSBEFORE = IF EXISTS +DOUBLE = DOUBLE PRECISION +BLOB = BLOB +CLOB = CLOB +TIMESTAMP = TIMESTAMP +CHAR = CHAR +VARCHAR = VARCHAR +# for generating drop statements... +SEQUENCE = TABLE \ No newline at end of file diff --git a/spring-batch-core/src/main/sql/sqlite.vpp b/spring-batch-core/src/main/sql/sqlite.vpp new file mode 100644 index 0000000000..a3da1792e2 --- /dev/null +++ b/spring-batch-core/src/main/sql/sqlite.vpp @@ -0,0 +1,5 @@ +#macro (sequence $name $value)CREATE TABLE ${name} ( + ID INTEGER PRIMARY KEY AUTOINCREMENT +); +#end +#macro (notnull $name $type)ALTER COLUMN ${name} ${type} NOT NULL#end diff --git a/spring-batch-core/src/site/apt/changelog.apt b/spring-batch-core/src/site/apt/changelog.apt deleted file mode 100644 index 18947ba75f..0000000000 --- a/spring-batch-core/src/site/apt/changelog.apt +++ /dev/null @@ -1,8 +0,0 @@ -Changelog: Spring Batch Core - -* 1.0-M2 - -** 2007/07/12 - - * No-one uses this file: we should just switch to auto-generated changelogs? - diff --git a/spring-batch-core/src/site/apt/executable.apt b/spring-batch-core/src/site/apt/executable.apt deleted file mode 100644 index ec70e99369..0000000000 --- a/spring-batch-core/src/site/apt/executable.apt +++ /dev/null @@ -1,38 +0,0 @@ - - Tried to create an executable jar with this: - -+--- - - - - org.apache.maven.plugins - maven-jar-plugin - - - - org.springframework.batch.container.bootstrap.BatchCommandLineLauncher - true - - - - - - -+--- - - But the resulting MANIFEST.MF is rubbish. Look at the classpath - (where did that come from)? - -+--- -Manifest-Version: 1.0 -Archiver-Version: Plexus Archiver -Created-By: Apache Maven -Built-By: dsyer -Build-Jdk: 1.5.0_09 -Main-Class: org.springframework.batch.container.bootstrap.BatchCommand - LineLauncher -Class-Path: spring-2.1-m2.jar commons-logging-1.1.jar log4j-1.2.12.jar - dom4j-1.6.1.jar commons-lang-2.1.jar spring-batch-infrastructure-1.0 - -m2-SNAPSHOT.jar antlr-2.7.6.jar commons-collections-2.1.1.jar hibern - ate-3.2.3.ga.jar spring-mock-2.1-m2.jar ehcache-1.2.3.jar -+--- diff --git a/spring-batch-core/src/site/apt/glossary.apt b/spring-batch-core/src/site/apt/glossary.apt deleted file mode 100644 index 1b2e1aac5b..0000000000 --- a/spring-batch-core/src/site/apt/glossary.apt +++ /dev/null @@ -1,46 +0,0 @@ - ------ - Glossary - ------ - Wayne Lund - ------ - May 2007 - - - [[1]]<>: An accumulation of business transactions over time. - - [[2]]<>: Term used to designate batch as an application style in its own right similar to online, Web or SOA. It has standard elements of input, validation, transformation of information to business model, business processing and output. In addition, it requires monitoring at a macro level. - - [[3]]<>: The handling of a batch of many business transactions that have accumulated over a period of time (e.g. an hour, day, week, month, or year). It is the application of a process, or set of processes, to many data entities or objects in a repetitive and predictable fashion with either no manual element, or a separate manual element for error processing. - - [[4]]<>: The time frame within which a batch job must complete. This can be constrained by other systems coming online, other dependent jobs needing to execute or other factors specific to the batch environment. - - [[5]]<>: It is the main batch task or Unit of Work controller. It initializes the module, and controls the transaction environment based on commit interval setting, etc. - - [[6]]<>: The main application program created by application developer to process the business logic for each LUW. - - [[7]]<>: Job Types describe application of jobs for particular type of processing. Common areas are interface processing (typically flat files), forms processing (either for online pdf generation or print formats), report processing. s - - [[8]]<>: A driving query identifies the set of work for a job to do; the job then breaks that work into individual units of work. For instance, identify all financial transactions that have a status of "pending transmission" and send them to our partner system. The driving query returns a set of record IDs to process; each record ID then becomes a unit of work. A driving query may involve a join (if the criteria for selection falls across two or more tables) or it may work with a single table. - - [[9]]<>: A batch job iterates through a driving query (or another input source such as a file) to perform the set of work that the job must accomplish. Each iteration of work performed is a unit of work. - - [[10]]<>: A set of LUWs constitute a commit interval. - - [[11]]<>: Splitting a job into multiple threads where each thread is responsible for a subset of the overall data to be processed. The threads of execution may be within the same JVM or they may span JVMs in a clustered environment that supports workload balancing. - - [[12]]<>: A table that holds temporary data while it is being processed. - - [[13]]<>: - a job that can be executed again and will assume the same identity as when run initially. In othewords, it is has the same job instance id. - - [[14]]Rerunnable - a job that is restartable and manages it's own state in terms of previous run's record processing. Note>>: Rerunnable is tied to the driving query. If the query can be formed so that it will limit the processed rows when the job is restarted than re-runnable = true. Often times a condition is added to the where statement to limit the rows returned by the driving query with something like "and processedFlag != true". - ---------------------------------------------------------------------- -Note: If its false the architecture assumes responsibility for tracking which rows have been processed. There is a default strategy for tracking the last record processed by partition. Most batch jobs only have one partition. The option is only valid for a restartable job. The reason being is that we have to persist the restart data which is only available on a restartable job. - -In DSL it is the following: - StartOver ::= restartable = false. Restartable ::= true | false - If (Restartable) - re-runnable ::= true | false - -We don't persist restart information for a non-restartable job. As you can see, it doesn't make sense. Rerunnable has always confused the best of us. ----------------------------------------------------------------------------- diff --git a/spring-batch-core/src/site/apt/index.apt b/spring-batch-core/src/site/apt/index.apt deleted file mode 100644 index 626f7c58fb..0000000000 --- a/spring-batch-core/src/site/apt/index.apt +++ /dev/null @@ -1,77 +0,0 @@ - ------ - Spring Batch Core - ------ - Dave Syer - ------ - August 2007 - -Overview of the Spring Batch Core - - The Spring Batch Core Domain consists of an API for launching, - monitoring and managing batch jobs. - -[images/core-domain-overview.png] The Spring Batch Core Domain with -dependencies to infrastructure indicated schematically. - - The figure above shows the central parts of the core domain and its - main touch points with the batch application develepor (<<>> - and <<>>). To launch a job there is a <<>> - interface that can be used to simplify the launching for dumb - clients like JMX or a command line. - - A <<>> is composed of a list of <<>>s, each of which is - executed in turn by the <<>>. The <<>> is a central - strategy in the Spring Batch Core. Implementations of <<>> - are responsible for sharing the work out, but in ways that the - configuration doesn't need to be aware of. For instance, the same - or very similar <<>> configuration might be used in a simple - in-process sequential executor, or in a multi-threaded - implementation, or one that delegates to remote calls to a - distributed system. - -[images/core-domain-extended.png] The Spring Batch Core Domain -extended to include the datababase entities and identifier strategy. - - A <<>> can be re-used to create multiple job - instances and this is reflected in the figure above showing an - extended picture of the core domain. When a <<>> - is launched it first checks to see if a job with - the same <<>> was already executed. We expect one of - the following outcomes, depending on the <<>>: - - * If the job was not previously launched then it can be created - and executed. A new <<>> is created and stored in a - repository (usually a database). A new <<>> is also - created to track the progress of this particular execution. - - * If the job was previously launched and failed the <<>> is - responsible for indicating whether it believes it is restartable - (is a restart legal and expected). There is a flag for this - purpose on the <<>>. If the there was a previous failure - - maybe the operator has fixed some bad input and wants to run it - again - then we might want to restart the previous job. - - If the job was previously launched with teh same - <<>> and completed successfully, then it is an - error to restart it. An ad-hoc request needs to be distinguished - from previous runs by adding a unique job parameter. In either - case a new <<>> is created and stored to monitor - this execution of the <<>>. - -Overview of the Spring Batch Launch Environment - - The diagram below provides an overview of the high level components, technical services, and basic operations required by a batch architecture. This architecture framework is a blueprint that has been proven through decades of implementations on the last several generations of platforms (COBOL/Mainframe, C++/Unix, and now Java/anywhere). Simple Batch provides a physical implementation of the layers, components and technical services commonly found in robust, maintainable systems used to address the creation of simple to complex batch applications, with the infrastructure and extensions to address very complex processing needs. The materials below will walk through the details of the diagram. - -[images/spring-batch-reference-model.png] Simple Batch Launch Environment high level flow and interaction of the architecture. - -* Tiers - - The application style is organized into four logical tiers, which include Run, Job, Application, and Data tiers. The primary goal for organizing an application according to the tiers is to embed what is known as "separation of concerns" within the system. Effective separation of concerns results in reducing the impact of change to the system. - - * <> The Run Tier is concerned with the scheduling and launching of the application. A vendor product is typically used in this tier to allow time-based and interdependent scheduling of batch jobs as well as providing parallel processing capabilities. - - * <> The Job Tier is responsible for the overall execution of a batch job. It sequentially executes batch steps, ensuring that all steps are in the correct state and all appropriate policies are enforced. - - * <> The Application Tier contains components required to execute the program. It contains specific modules that address the required batch functionality and enforces policies around a module execution (e.g., commit intervals, capture of statistics, etc.) - - * <> The Data Tier provides the integration with the physical data sources that might include databases, files, or queues. <>: In some cases the Job tier can be completely missing and in other cases one job script can start several batch job instances. diff --git a/spring-batch-core/src/site/apt/introduction.apt b/spring-batch-core/src/site/apt/introduction.apt deleted file mode 100644 index 44b7a3764a..0000000000 --- a/spring-batch-core/src/site/apt/introduction.apt +++ /dev/null @@ -1,201 +0,0 @@ - ------ - Batch Processing Strategy - ------ - Scott Wintermute - ------ - May 2007 - -Batch Processing Strategy - - To help design and implement batch systems, basic batch application building blocks and patterns should be provided to the designers and programmers in form of sample structure charts and code shells. When starting to design a batch job, the business logic should be decomposed into a series of steps which can be implemented using the following standard building blocks: - - * Conversion Applications: For each type of file supplied by or generated to an external system, a conversion application will need to be created to convert the transaction records supplied into a standard format required for processing. This type of batch application can partly or entirely consist of translation utility modules (see Basic Batch Services). - - * Validation Applications: Validation applications ensure that all input/output records are correct and consistent. Validation is typically based on file headers and trailers, checksums and validation algorithms as well as record level cross-checks. - - * Extract Applications: An application that reads a set of records from a database or input file, selects records based on predefined rules, and writes the records to an output file. - - * Extract/Update Applications: An application that reads records from a database or an input file, and makes changes to a database or an output file driven by the data found in each input record. - - * Processing and Updating Applications: An application that performs processing on input transactions from an extract or a validation application. The processing will usually involve reading a database to obtain data required for processing, potentially updating the database and creating records for output processing. - - * Output/Format Applications: Applications reading an input file, restructures data from this record according to a standard format, and produces an output file for printing or transmission to another program or system. - - <> - - Additionally a basic application shell should be provided for business logic that cannot be built using the previously mentioned building blocks. - - In addition to the main building blocks, each application may use one or more of standard utility steps, such as: - - * Sort - A Program that reads an input file and produces an output file where records have been re-sequenced according to a sort key field in the records. Sorts are usually performed by standard system utilities. - - * Split - A program that reads a single input file, and writes each record to one of several output files based on a field value. Splits can be tailored or performed by parameter-driven standard system utilities. - - * Merge - A program that reads records from multiple input files and produces one output file with combined data from the input files. Merges can be tailored or performed by parameter-driven standard system utilities. - - Batch applications can additionally be categorized by their input source: - - * Database-driven applications are driven by rows or values retrieved from the database. - - * File-driven applications are driven by records or values retrieved from a file - - The foundation of any batch system is the processing strategy. Factors affecting the selection of the strategy include estimated batch system volume, concurrency with on-line or with another batch systems, available batch windows etc. Also with more enterprises wanting to be up and running 24x7, leaving no obvious batch windows. - - Typical processing options for batch are: - - * Normal processing in a batch window during off-line - - * Concurrent batch / on-line processing - - * Parallel processing of many different batch runs or jobs at the same time - - * Streaming i.e. processing of many instances of the same job at the same time - - * A combination of these - - The order in the list above reflects the implementation complexity, processing in a batch window being the easiest and streaming the most complex to implement. - - Some or all of these options may be supported by a commercial scheduler. - - In the following section these processing options will be discussed in more detail. It is important to notice that the commit and locking strategy adopted by batch processes will be dependent on the type of processing performed and as a rule of thumb, the on-line locking should use the same principles. Therefore a batch architecture cannot be simply an afterthought when designing an overall architecture. - - The locking strategy can use only normal database locks, or an additional custom locking service can be implemented in the architecture. The locking service would track database locking (for example by storing the necessary information in a dedicated db-table) and give or deny permissions to the application programs requesting a db operation. Retry logic could also be implemented by this architecture to avoid aborting a batch job in case of a lock situation. - - <<1. Normal processing in a batch window>> - For simple batch processes running in a separate batch window, where the data being updated is not required by on-line users or other batch processes, concurrency is not an issue and a single commit can be done at the end of the batch run. - - In most cases a more robust approach is more appropriate. A thing to keep in mind is that batch systems have a tendency to grow as time goes by, both in terms of complexity and the data volumes they will handle. If no locking strategy is in place and the system still relies on a single commit point, modifying the batch programs can be painful. Therefore, even with the simplest batch systems, consider the need for commit logic depicted in the [Restart/Recovery section|Restart & Recovery] as well as the information concerning the more complex cases below. - - <<2. Concurrent batch / on-line processing>> - Batch applications processing data that can simultaneously be updated by on-line users, should not lock any data (either in the database or in files) which could be required by on-line users for more than a few seconds. Also updates should be committed to the database at the end of every few transaction. This minimizes the portion of data that is unavailable to other processes and the elapsed time the data is unavailable. - - Another option to minimize physical locking is to have a logical row-level locking implemented using either an Optimistic Locking Pattern or a Pessimistic Locking Pattern. - - * Optimistic locking assumes a low likelihood of record contention. It typically means inserting a timestamp column in each database table used concurrently by both batch and on-line processing. When an application fetches a row for processing, it also fetches the timestamp. As the application then tries to update the processed row, the update uses the original timestamp in the WHERE clause. If the timestamp matches, the data and the timestamp will be updated successfully. If the timestamp does not match, this indicates that another application has updated the same row between the fetch and the update attempt and therefore the update cannot be performed. - - * Pessimistic locking is any locking strategy that assumes there is a high likelihood of record contention and therefore either a physical or logical lock needs to be obtained at retrieval time. One type of pessimistic logical locking uses a dedicated lock-column in the database table. When an application retrieves the row for update, it sets a flag in the lock column. With the flag in place, other applications attempting to retrieve the same row will logically fail. When the application that set the flag updates the row, it also clears the flag, enabling the row to be retrieved by other applications. Please note, that the integrity of data must be maintained also between the initial fetch and the setting of the flag, for example by using db locks (e.g.,SELECT FOR UPDATE). Note also that this method suffers from the same downside as physical locking except that it is somewhat easier to manage building a time-out mechanism that will get the lock released if the user goes to lunch while the record is locked. - - These patterns are not necessarily suitable for batch processing, but they might be used for concurrent batch and on-line processing for example in cases where the database doesn't support row-level locking. As a general rule, optimistic locking is more suitable for on-line applications, while pessimistic locking is more suitable for batch applications. Whenever logical locking is used, the same scheme must be used for all applications accessing data entities protected by logical locks. - - Note that both of these solutions only address locking a single record. Often we may need to lock a logically related group of records. With physical locks, you have to manage these very carefully in order to avoid potential deadlocks. With logical locks, it is usually best to build a logical lock manager that understands the logical record groups you want to protect and can ensure that locks are coherent and non-deadlocking. This logical lock manager usually uses its own tables for lock management, contention reporting, time-out mechanism, etc. - - <<3. Parallel Processing>> - Parallel processing allows multiple batch runs / jobs to run in parallel to minimize the total elapsed batch processing time. This is not a problem as long as the jobs are not sharing the same files, db-tables or index spaces. If they do, this service should be implemented using partitioned data. Another option is to build an architecture module for maintaining interdependencies using a control table. A control table should contain a row for each shared resource and whether it is in use by an application or not. The batch architecture (Control Program Tasklet) or the application in a parallel job would then retrieve information from that table to determine if it can get access to the resource it needs or not. - - If the data access is not a problem, parallel processing can be implemented in a mainframe environment using parallel job classes, in order to ensure adequate CPU time for all the processes. In an environment other than the mainframe, a similar solution can be put in place with for example threads. The solution has to be robust enough to ensure time slices for all the running processes. - - Other key issues in parallel processing include load balancing and the availability of general system resources such as files, database buffer pools etc. Also note that the control table itself can easily become a critical resource. - - <<4. Partitioning>> - Using partitioning allows multiple versions of large batch applications to run in concurrent. The purpose of this is to reduce the elapsed time required to process long batch jobs. Processes which can be successfully partitioned are those where the input file can be split and/or the main database tables partitioned to allow the application to run against different sets of data. - - In addition, processes which are partitioned must be designed to only process their assigned data set. A partitioning architecture has to be closely tied to the database design and the database partitioning strategy. Please note, that the database partitioning doesn't necessarily mean physical partitioning of the database, although in most cases this is advisable. The following picture illustrates the partitioning approach:!app_style_batch_processing.png|align=center! - - The architecture should be flexible enough to allow dynamic configuration of the number of partitions. Both automatic and user controlled configuration should be considered. Automatic configuration may be based on parameters such as the input file size and/or the number of input records. - - <<4.1 Streaming Approaches>> - The following lists some of the possible streaming approaches. Selecting a streaming approach has to be done on a case-by-case basis. - - <1. Fixed and Even Break-Up of Record Set> - - This involves breaking the input record set into an even number of portions (e.g. 10, where each portion will have exactly 1/10th of the entire record set). Each portion is then processed by one instance of the batch/extract application. - - In order to use this approach, preprocessing will be required to split the recordset up. The result of this split will be a lower and upper bound placement number which can be used as input to the batch/extract application in order to restrict its processing to its portion alone. - - Preprocessing could be a large overhead as it has to calculate and determine the bounds of each portion of the record set. - - <2. Breakup by a Key Column> - - This involves breaking up the input record set by a key column such as a location code, and assigning data from each key to a batch instance. In order to achieve this, column values can either be - - <3. Assigned to a batch instance via a streaming table (see below for details).> - - <4. Assigned to a batch instance by a portion of the value (e.g. values 0000-0999, 1000 - 1999, etc.)> - - Under option 1, addition of new values will mean a manual reconfiguration of the batch/extract to ensure that the new value is added to a particular instance. - - Under option 2, this will ensure that all values are covered via an instance of the batch job. However, the number of values processed by one instance is dependent on the distribution of column values (i.e. there may be a large number of locations in the 0000-0999 range, and few in the 1000-1999 range). Under this option, the data range should be designed with streaming in mind. - - Under both options, the optimal even distribution of records to batch instances cannot be realized. There is no dynamic configuration of the number of batch instances used. - - <5. Breakup by Views> - - This approach is basically breakup by a key column, but on the database level. It involves breaking up the recordset into views. These views will be used by each instance of the batch application during its processing. The breakup will be done by grouping the data. - - With this option, each instance of a batch application will have to be configured to hit a particular view (instead of the master table). Also, with the addition of new data values, this new group of data will have to be included into a view. There is no dynamic configuration capability, as a change in the number of instances will result in a change to the views. - - <6. Addition of a Processing Indicator> - - This involves the addition of a new column to the input table, which acts as an indicator. As a preprocessing step, all indicators would be marked to non-processed. During the record fetch stage of the batch application, records are read on the condition that that record is marked non-processed, and once they are read (with lock), they are marked processing. When that record is completed, the indicator is updated to either complete or error. Many instances of a batch application can be started without an change, as the additional column ensures that a record is only processed once. - - With this option, I/O on the table increased dynamically. In the case of a updating batch application, this impact is reduced, as a write will have to occur anyway. - - <7. Extract Table to a Flat File> - - This involves the extraction of the table into a file. This file can then be split into multiple segments and used as input to the batch instances. - - With this option, the additional overhead of extracting the table into a file, and splitting it, may cancel out the effect of multi-streaming. Dynamic configuration can be achieved via changing the file splitting script. - - <8. Use of a Hashing Column> - - This scheme involves the addition of a hash column (key/index) to the database tables used to retrieve the driver record. This hash column will have an indicator to determine which instance of the batch application will process this particular row. For example, if there are three batch instances to be started, then an indicator of 'A' will mark that row for processing by instance 1, an indicator of 'B' will mark that row for processing by instance 2, etc. - - The procedure used to retrieve the records would then have an additional WHERE clause to select all rows marked by a particular indicator. The inserts in this table would involve the addition of the marker field, which would be defaulted to one of the instances (e.g. 'A'). - - A simple batch application would be used to update the indicators such as to redistribute the load between the different instances. When a sufficiently large number of new rows have been added, this batch can be run (anytime, except in the batch window) to redistribute the new rows to other instances. - - Additional instances of the batch application only require the running of the batch application as above to redistribute the indicators to cater for a new number of instances. - - - 4.2 Database and Application design Principles - - An architecture that supports multi-streamed applications which run against partitioned database tables using the key column approach, should include a central streaming repository for storing streaming parameters. This provides flexibility and ensures maintainability. The repository will generally consist of a single table known as the streaming table. - - Information stored in the streaming table will be static and in general should be maintained by the DBA. The table should consist of one row of information for each stream of a multi-streamed application. The table should have a similar layout to the following table: - - {center} - || Streaming Table || - | Program ID Code - Stream Number (Logical ID of the stream) - Low Value of the db key column for this stream - High Value of the db key column for this stream | - {center} - - On program start-up the program id and stream number should be passed to the application from the architecture (Control Processing Tasklet). These variables are used to read the streaming table, to determine what range of data the application is to process (if a key column approach is used). In addition the stream number must be used throughout the processing to: - - * Add to the output files/database updates in order for the merge process to work properly - - * Report normal processing to the batch log and any errors that occur during execution to the architecture error handler - - 4.3 Minimizing Deadlocks - When applications run in parallel or streamed, contention in database resources and deadlocks may occur. It is critical that the database design team eliminates potential contention situations as far as possible as part of the database design. - - Also ensure that the database index tables are designed with deadlock prevention and performance in mind. - - Deadlocks or hot spots often occur in administration or architecture tables such as log tables, control tables, lock tables etc.. The implications of these should be taken into account as well. A realistic stress test is crucial for identifying the possible bottlenecks in the architecture. - - To minimize the impact of conflicts on data, the architecture should provide services such as wait-and-retry intervals when attaching to a database or when encountering a deadlock. This means a built-in mechanism to react to certain database return codes and instead of issuing an immediate error handling, waiting a predetermined amount of time and retrying the database operation. - - 4.4 Parameter Passing and Validation - - The streaming architecture should be relatively transparent to application developers. The architecture should perform all tasks associated with running the application in a streamed mode i.e. - - * Retrieve streaming parameters before application start-up - - * Validate streaming parameters before application start-up - - * Pass parameters to application at start-up - - The validation should include checks to ensure that: - - * the application has sufficient streams to cover the whole data range - - * there are no gaps between streams - - If the database is partitioned, some additional validation may be necessary to ensure that a single stream does not span database partitions. - - Also the architecture should take into consideration the consolidation of streams. Key questions include: - - * Must all the streams be finished before going into the next job step? - - * What happens if one of the streams aborts? diff --git a/spring-batch-core/src/site/apt/outline.apt b/spring-batch-core/src/site/apt/outline.apt deleted file mode 100644 index d500ed3344..0000000000 --- a/spring-batch-core/src/site/apt/outline.apt +++ /dev/null @@ -1,124 +0,0 @@ - ------------------------------------------ - The Spring Batch - Reference Documentation - ---------------------------------------- - Wayne Lund, Waseem Malik, Lucas Ward, Scott Wintermute, - Kerry O'Brien, Tomi Vanek - ------------------------------------------- - May 2007 - -Preface - -*1. {{{./introduction.html}Spring Container Batch Processing}} - - *1.1. Overview - - *1.2 Usage Scenarios - -*2. {{{./overview.html}Architecture Overview}} - - **2.1. Introduction to Architecture Layers - - **2.2. Batch Applications - - **2.3 Container Application layer - - **2.4 Container Support Layer - - **2.5 The Container Core Layer - - **2.6 Using the Spring-batch infrastructure - - *2.6.1 Infrastructure Provided I/O Support - - *2.6.2 Infrastucture Provided Base Services - - *2.7. Batch Execution Container Configurations - - *2.7.1. Single VM Simple Batch Execution Container - One Job, One Step, One Partition - - *2.7.2. Single VM Multi-threaded Batch Execution Container Configuration - One Job, One Step, Multiple Partitions - - *2.7.3 Batch Execution Container Hosted in J2EE Container - managed environment - - -*3. Core Batch Services - - *3.1. Launching Batch Jobs - - *3.2. Mapping Batch Error Codes to Launch Client Error Codes - - *3.2.1. Returning error codes to Enterprise Schedulers with command line interfaces - - *3.2.2. Web Request - returning error codes to "On-Demand Batch Requests" - - *3.3. Job Services - - *3.3.1. Step Configuration - - *3.3.2. Job Status Service - - *3.3.3. Job Status - - *3.3.4. Job Statistics - - *3.4. Step Services - - *3.4.1. Step Execution - - *3.4.2. Restart Services - - *3.4.3. Skip Services - - *3.4.5. Step Statistics - - *3.5. Data Providers - Operations, Templates and Convenience Callbacks - - *3.5.1. Wrapping input sources - - *3.5.2. Validation of input - - *3.5.3. Delimited File Data Providers - - *3.5.4. Fixed Position File Data Providers - - *3.5.5. XML File Data Providers - - *3.5.6. SQL Input Source Data Provider - - *3.6. Transaction Management - - *3.6.1. Transaction Synchronization with non-transactional resources - - *3.7. Partitioning Batch Jobs - - *3.7.1. Partitioning Strategies - - *3.7.2. Partitioning Job Steps - - *3.7.3. Partition Status - - *3.7.4. Partition Statistics - - *3.7.5. Handling Exceptions within partitions - -*4. [Batch in a J2EE Container] - -*5. [Testing Batch Jobs] - - *5.1. Unit Testing & Mock Objects Provided by the Framework - - *5.2. Integration Testing - - *5.3. Performance Testing Batch Jobs - -*6. {{{samples.html}Practical Examples for Spring Batch}} - - *6.1. Sample Applications|Spring Reference-Application Job-Map - - *6.2. Running the Sample Batch Applications - -*7. [INCUB:Batch XML Schema] - -*8. {{{./glossary.html}Glossary}} - - diff --git a/spring-batch-core/src/site/apt/overview.apt b/spring-batch-core/src/site/apt/overview.apt deleted file mode 100644 index 2edcc8b038..0000000000 --- a/spring-batch-core/src/site/apt/overview.apt +++ /dev/null @@ -1,145 +0,0 @@ - ------ - Architecture Overview - ------ - Wayne Lund - ------ - May 2007 - -2. Architecture Overview - - -*2.1 Introduction - - This chapter covers the overall spring batch architecture. The Spring Container Archtiecture is made up of five logical layers; 1) the Batch Application, 2) the Batch Application Layer, 3) the batch core layer, and 4) the batch infrastucture layer. - -*----------*----------------*------------+ -|Provided By | Layer | Description -*----------*----------------*------------* -| Application Developer | Batch Application | This is where the application writes their batch jobs and modules. | -*----------*----------------*------------* -| Spring Batch Execution Container | Container Application Layer | Allows for extending and overwriting of the batch support layer for custom requirements. Facilities implemented in this layer could migrate down to Batch Support Layer. This is also the layer to add the project specific jars required by job types (e.g. reporting jars like Crystal, Brio, etc, form generation jars like Central Pro or Adobe, etc). | -*----------*----------------*------------* -| Spring Batch Execution Container | Container Support Layer | Provides default implementations of batch core services including I/O, Restart, Partitioning, Statistics, and configurations | -*----------*----------------*------------* -| Spring Batch Execution Container | Container Core Layer | Enables configuration, Common Services & Interfaces, management | -*----------*----------------*------------* -| Spring Batch Infrastructure | Batch-Infrastructure | Provides IO support, Batch style transactions, advanced exception handling, batch-template, batch-retry | -*----------*----------------*------------* - - [Figure 2.0] - Batch Architecture Layers - - The batch architecture is modeled after a container architecture, meaning that there are managed resources essential to high performance batch architectures that are configured through a spring context. The following sections will provide a quick review of each layer and their role in the batch architecture. - -*2.2 Batch Applications - -*2.3 Container Application Layer - -*2.4 Container Support Layer - - The batch support layer provides default implementations for all interfaces, interceptors, advice and other core batch services. Figure 2.3.1 illustrates the following logical packages. !Batch Support.png! -Although physically they break out into many more than depicted, logically you can think of the groupings in the following manner: -* I/O Support packages -* Restart Support -* Lifecycle Support packages -* DAO support layer - - **2.4.1 I/O Support Packages - - The I/O related packages are currently the richest packages in the batch architecture. They are modeled after Spring Patterns of Operations and Templates. For example, you'll see FlatFileInputOperations accompanied with a FlatFileInputTemplate. The FlatFileInputTemplate is wired up with a File Descriptor, which contains a Record Descriptor along with various other properties. With the File and Record Descriptors the InputTemplate supports a callback method that allows for the mapping of a record into an object. This support applies to fixed length records, delimited records and XML records. To further simplify this a DefaultFlatFileDataProvider is supplied an input template, which contains the field and record descriptions, along with a line mapper that knows how to map the line to an object. The next() operation on a record simply needs to readAndMap(lineMapper) a record. This pattern is used over again for XML and SQL input for simple mapping of input records to objects. - - In addition to declarative descriptions of the records that can be re-used by multiple batch jobs, the I/O facilities also support configurable validation strategies. The two currently supported are Apache Commons Validator and Spring's VALang. - - **2.4.2 Restart Support - - The Restart Support provides implementations for a few common restart strategies that will be discussed further in the respective section. The following are provided out-of-the-box: - * IDList Restart Strategy - a strategy that supports a batch application where the application does not have a "process" flag and needs the batch architecture to track which records have been processed. This is not the ideal scenario. - * Last Processed Restart Strategy - when the record can be identified through a where and order by only the last record(s) processed needs to be saved for restart. - * No Restart Strategy - some batch jobs simply can't support restart. When they are re-run they are considered to be a new instance of a batch job. - * Sql Restart Strategy - \[need some additional javadoc for this strategy\]. - - **2.4.3 Lifecycle Support - -*2.5. The Container Core Layer - - The Batch Core interfaces and services are illiustrated in a simplied view of a package diagram. There are roughly seven logical packages: - * Core Spring Extensions - * Core Batch Advice - * Core Batch Configuration - * Core Batch Repository - * Core Batch Tasklet - \\ !Batch Core.png! - [Figure 2.5] Batch Core Layer - - In the actual physical packaging there are a few more packages but the above illistration serves as an overview of the logical services that the batch container provides. The following sections will provide an introduction into each set of core batch facilities. - - **2.5.1 Core Spring Extensions - - The Core Spring extensions provide the scaffolding for a batch container. This includes facilities for managing the batch architecture in terms of launching, suspending and stopping batch jobs. There is house keeping that goes on, especially in concurrent batch jobs, related to ensuring that batch jobs quiese properly. The lifecycle management provides services for the proper initialization and subsequent shutdown of batch resources and services. The batch architecture is flexible in terms of how batch jobs may be launched. For example, batch jobs can be started via JMX facilities, scripts from the command line that launch a Java VM. It can also support launching batch jobs through web services or http. There are no restrictions. Finally, there are standard batch error codes. These error codes can be exposed to external utilities, like Schedulers, to ensure that batch jobs expose the status of jobs to an operational environment. This is especially important in the batch context where the modus operandi is headless, meaning unattended operation. - - **2.5.2 Core Batch Advice - - Core Batch Advice is an inventory of the type of advice that batch architectures will inject during the runtime of a batch application. These are defined as a set of extensible interfaces, with a number of default implementations in the support layer that provide some of the most common types of advice. Partition Advice is helpful with large datasets that need to be "chunked" up and run concurrently for better through put. Resource Advice is helpful for registering interest in transactional information so that file locations can be kept in sync with information processed within a transaction. In addition, the resource is associated with the correct step context and its associated configuration properties. Skip advice is applied for records that the module is unable to process. Restart Advice is helpful for Restartable jobs where for advising the job on how to restart. There is considerable variability on how restart can occur. For example, a job may be marking records as "processed" and the restart advice will advise the process with query that restarts the job at the last successfully processed record. Finally, Statistics are vital in operational environments to report on records processed, records skipped and total number of records read. In addition, certain batch jobs lend themselves to custom reporting to expose additional business level information like the number of trades processed or cases opened, etc. - - **2.5.3 Core Batch Configuration - - Batch configuration is considerably different from online web applications or SOA based applications. The Core Batch Configuration provides a place for configuring runtime properties related to the batch application style. This includes the ability to add Commit Policy. In a batch style application it is often advantageous to keep the commit interval as high as possible when processing Logical Units of Work. Whereas in an online web application with declarative transaction the transaction scope would be at the entrance to a business service, a batch transaction scope may include many logical units of work before a transaction commit is executed. A Start Policy allows a configuration to tell the batch job whether it is Restartable, and if so, what type of restart to initiate. Some jobs are not restartable and care should be taken to ensure that information is not applied multiple times when the business rules do not allow for it. Exception policies deal with what to do when exceptions occur. This impacts logging policies and exception handling. The architecture defines a common set of exceptions that projects can apply handlers to like processing errors, validation errors, parsing errors, missing configuration parameters, etc. - - **2.5.4 Container Repository - - This is an internal package for storing the state of a batch job and any associated partition and step status. - - **2.5.5 Core Batch Tasklet - - The core batch module is where control is handed off to the application. There are a number of patterns that have been observed in processing batch data. Spring Core Batch Tasklet implements the most common patterns and provides and extension point for additional Tasklet processing implementations. The basic idea of module provides the facilities for reading and processing data. The simplest implementation of Tasklet, the ReadProcessTasklet, handles both the input and output of data within one class. An alternative implementation, the DataProviderProcessTasklet, provides functionality for 'split processing'. This type of processing is characterized by separating the reading and processing of batch data into two seperate classes: DataProvider and TaskletProcessor. The DataProvider class provides a solid means for reusablility and enforces good architecture practices. Because an object \*must\* be returned by the DataProider to continue processing, (Returning null indicates processing should end) a developer is forced to read in all relevant data, place it into domain or value objects, and return the object. The TaskletProcessor will then use this object within the business logic and final output. - -2.6 Container's Use of batch infrastructure - - **2.6.1 Infrastructure Provided I/O - - The I/O core interfaces and implementations provide facilities for simplifying the extraction of data from I/O sources like files and database tables. The key concepts are FieldDescriptors and FieldSets along with appropriate CallBack Handlers. These are modeled after common spring operations and templates like JdbcTemplate. Through the use of LineMappers a developer needs only to describe a record format and write the appropriate callback method that maps the parsed record into an object of their choice. These can either be true POJO objects or Value Objects (structures) that are subsequently available for the module to processs. The interface for Field Descriptors also allows for a level of validation through the use of Spring's VALang or Apache's Common Validator. - - **2.6.2 Core Batch Interceptors & Interceptor Services - - * Batch Operations & Batch Template - - Interceptors and the associated services are the key to how advise is applied in the batch architecture. The interceptors are Point Cuts in the batch lifecycle that allow the injection of advise. The shared lifecycle behavior abstracted through the BatchLifeCycleInterceptor defineds three methods; init, onError and finalize. All subclasses of LifeCycleInterceptor define default behavior for these three methods. The JobLifecycleInterceptor further exposes the methods beforeJob(), beforeStep(), afterJob(), and afterStep() allowing hooks into the lifecycle for specific advise. The Batch Architecture provides default implementations for all lifecycle point cuts, or interception points. The Tasklet Interceptor, in addition to the standard lifecycle methods, implements logic around beforeLuw(), afterLuw(), commitIntervalStarted() and commitIntervalCompleted(). Having well defined lifecycle interception points allows for the easy insertion of custom advice into the batch runtime environment. - -2.7 Batch Esecution Container Configurations - - In addition to core facilities for configuring or wiring together jobs and steps with required resources, policies, and interceptors, spring batch allows considerable flexibility in how scalability is achieved. More options for scalability will be available in the future. The important key for scalability in Java is the recognition that there is a limit to what one JVM may scale up to in terms of number of threads, managed resources, memory configuration, etc. The spring batch architecture allows for the configuration of simple batch jobs where one VM and one process is sufficient to do perform the work within a batch window all the way through many threads distributed within a cluster of JEE servers. The figure below illistrates the scalability spectrum. - - This is not to be understood as the only way to scale batch jobs as there are many factors. For example, other federated java architectures hold potential like Teracotta or Gigaspaces although there is no current implementation for these distributed models in the current batch architecture. - !scalability-model.png! - [Figure 2.3.1] - Scalability Model - -*2.7.1. Single VM Simple Batch Execution Container - One Job, One Step, One Partition - - The simplest configuration is one job with with step and hence, one implied partition. Implied means that there is nothing for the developer to consider because the default number of partitions is one. There is typically one input source and one output source in this simple configuration. See the Simple Tasklet Job for an example of what this configuration looks like. A simple configuration still typically configures a datasource context, the batch configuration for describing the Job, Step, along with the associated configured policies, field descriptors, and line mappers. !SimpleTradeConfiguration.jpg! - [Figure 2.3.1] Simple Container Configuration - - The details of this configuration will be covered thoroughly in subsequent sections of the document but for now it should be understood that Job, the Step, the input template, the file descriptor with its associated line mapper, and the output (e.g. the TradeWriter). - -*2.3.2. Single VM Multi-threaded Batch Execution Container Configuration - One Job, One Step, Multiple Partitions - - In a Single JVM using partitioning a multi-threaded execution is supported. \[This is still work in progress\] - -*2.3.3. Batch Execution Container Hosted in J2EE Container - managed environment - - The J2EE container model has fallen under fire over the past few years for many valid reasons. There are some things that the J2EE container do very well though that projects should consider when planning for scalability with batch architectures. Commercial and open source containers like WebSphere, BEA and JBOSS typically: - - * manage datasources effectively along with attendent services like prepared statement caching. - - * manage transactions effectively including many configurable properties for long lived transactions. - - * manage thread pools more effectively. - - * supply robust implementations of JTA, a requirement when batch jobs output to multiple XA resources like JMS and JDBC. - - * manage distribution effectively including domains, clusters and cells - - * provide robust JMX management for configuring, managing and administering distributed applications. - - * workload management facilities (clusters) provided by J2EE vendors - - Projects are encouraged to deploy batch applications with the simplest configuration possible, but when federated JVMs are a requirement to process volumes of data within a batch window, batch-in-container provides an effective way of distributing the processing. Spring Batch supports this through a simple change in configuration. \[Work in progress on the exact implementation - being released as part of M2\]. - diff --git a/spring-batch-core/src/site/resources/images/core-domain-extended.png b/spring-batch-core/src/site/resources/images/core-domain-extended.png deleted file mode 100644 index 4ac1358c43..0000000000 Binary files a/spring-batch-core/src/site/resources/images/core-domain-extended.png and /dev/null differ diff --git a/spring-batch-core/src/site/resources/images/core-domain-overview.png b/spring-batch-core/src/site/resources/images/core-domain-overview.png deleted file mode 100644 index ced0330ab5..0000000000 Binary files a/spring-batch-core/src/site/resources/images/core-domain-overview.png and /dev/null differ diff --git a/spring-batch-core/src/site/resources/images/spring-batch-reference-model.png b/spring-batch-core/src/site/resources/images/spring-batch-reference-model.png deleted file mode 100644 index 39d963a757..0000000000 Binary files a/spring-batch-core/src/site/resources/images/spring-batch-reference-model.png and /dev/null differ diff --git a/spring-batch-core/src/site/site.xml b/spring-batch-core/src/site/site.xml deleted file mode 100644 index b0d91e7f6d..0000000000 --- a/spring-batch-core/src/site/site.xml +++ /dev/null @@ -1,19 +0,0 @@ - - - - Spring Batch: ${project.name} - - - - - - - - - - - - - - - diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/AbstractExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/AbstractExceptionTests.java index a6dce689ca..945bf3cac4 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/AbstractExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/AbstractExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/AbstractExceptionWithCauseTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/AbstractExceptionWithCauseTests.java index 10ae0e21c4..8b8e50cea7 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/AbstractExceptionWithCauseTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/AbstractExceptionWithCauseTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/BatchStatusTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/BatchStatusTests.java index c4ac077fbc..1b0efc5fc8 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/BatchStatusTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/BatchStatusTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,7 +30,7 @@ /** * @author Dave Syer - * + * */ public class BatchStatusTests { @@ -118,4 +118,15 @@ public void testSerialization() throws Exception { BatchStatus status = (BatchStatus) in.readObject(); assertEquals(BatchStatus.COMPLETED, status); } + + @Test + public void testJsrConversion() { + assertEquals(javax.batch.runtime.BatchStatus.ABANDONED, BatchStatus.ABANDONED.getBatchStatus()); + assertEquals(javax.batch.runtime.BatchStatus.COMPLETED, BatchStatus.COMPLETED.getBatchStatus()); + assertEquals(javax.batch.runtime.BatchStatus.STARTED, BatchStatus.STARTED.getBatchStatus()); + assertEquals(javax.batch.runtime.BatchStatus.STARTING, BatchStatus.STARTING.getBatchStatus()); + assertEquals(javax.batch.runtime.BatchStatus.STOPPED, BatchStatus.STOPPED.getBatchStatus()); + assertEquals(javax.batch.runtime.BatchStatus.STOPPING, BatchStatus.STOPPING.getBatchStatus()); + assertEquals(javax.batch.runtime.BatchStatus.FAILED, BatchStatus.FAILED.getBatchStatus()); + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/DefaultJobKeyGeneratorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/DefaultJobKeyGeneratorTests.java index cdca6355b8..c76415335f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/DefaultJobKeyGeneratorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/DefaultJobKeyGeneratorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,13 +29,18 @@ public void setUp() throws Exception { jobKeyGenerator = new DefaultJobKeyGenerator(); } + @Test(expected = IllegalArgumentException.class) + public void testNullParameters() { + jobKeyGenerator.generateKey(null); + } + @Test public void testMixedParameters() { JobParameters jobParameters1 = new JobParametersBuilder().addString( "foo", "bar").addString("bar", "foo").toJobParameters(); JobParameters jobParameters2 = new JobParametersBuilder().addString( "foo", "bar", true).addString("bar", "foo", true) - .addString("ignoreMe", "irrelivant", false).toJobParameters(); + .addString("ignoreMe", "irrelevant", false).toJobParameters(); String key1 = jobKeyGenerator.generateKey(jobParameters1); String key2 = jobKeyGenerator.generateKey(jobParameters2); assertEquals(key1, key2); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/EntityTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/EntityTests.java index e5dc60ec59..79ba98752c 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/EntityTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/EntityTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/ExitStatusTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/ExitStatusTests.java index 30e9ddc300..3bba8cf066 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/ExitStatusTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/ExitStatusTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,11 +20,12 @@ import static org.junit.Assert.assertTrue; import org.junit.Test; -import org.springframework.batch.support.SerializationUtils; +import org.springframework.util.SerializationUtils; /** * @author Dave Syer - * + * @author Mahmoud Ben Hassine + * */ public class ExitStatusTests { @@ -54,7 +55,7 @@ public void testExitStatusConstantsFinished() { /** * Test equality of exit statuses. - * + * * @throws Exception */ @Test @@ -75,7 +76,7 @@ public void testEquals() { /** * Test equality of exit statuses. - * + * * @throws Exception */ @Test @@ -85,7 +86,7 @@ public void testEqualsWithNull() throws Exception { /** * Test equality of exit statuses. - * + * * @throws Exception */ @Test @@ -223,7 +224,7 @@ public void testAddExitCodeWithDescription() throws Exception { } @Test - public void testUnkownIsRunning() throws Exception { + public void testUnknownIsRunning() throws Exception { assertTrue(ExitStatus.UNKNOWN.isRunning()); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/IgnoredTestSuite.java b/spring-batch-core/src/test/java/org/springframework/batch/core/IgnoredTestSuite.java index b6116a0f9b..96251fef38 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/IgnoredTestSuite.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/IgnoredTestSuite.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/JobExecutionExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/JobExecutionExceptionTests.java index 7596c3b6f2..4e6d747aac 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/JobExecutionExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/JobExecutionExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/JobExecutionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/JobExecutionTests.java index 76cc1c64b0..a81ccbe783 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/JobExecutionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/JobExecutionTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,16 +26,17 @@ import java.util.List; import org.junit.Test; -import org.springframework.batch.support.SerializationUtils; +import org.springframework.util.SerializationUtils; /** * @author Dave Syer + * @author Dimitrios Liapis * */ public class JobExecutionTests { private JobExecution execution = new JobExecution(new JobInstance(new Long(11), "foo"), - new Long(12), new JobParameters()); + new Long(12), new JobParameters(), null); @Test public void testJobExecution() { @@ -53,12 +54,19 @@ public void testGetEndTime() { assertEquals(100L, execution.getEndTime().getTime()); } + @Test + public void testGetJobConfigurationName() { + execution = new JobExecution(new JobInstance(null, "foo"), null, "/META-INF/batch-jobs/someJob.xml"); + assertEquals("/META-INF/batch-jobs/someJob.xml", execution.getJobConfigurationName()); + } + /** * Test method for * {@link org.springframework.batch.core.JobExecution#getEndTime()}. */ @Test public void testIsRunning() { + execution.setStartTime(new Date()); assertTrue(execution.isRunning()); execution.setEndTime(new Date(100L)); assertFalse(execution.isRunning()); @@ -70,6 +78,7 @@ public void testIsRunning() { */ @Test public void testIsRunningWithStoppedExecution() { + execution.setStartTime(new Date()); assertTrue(execution.isRunning()); execution.stop(); assertTrue(execution.isRunning()); @@ -126,7 +135,7 @@ public void testDowngradeStatus() { @Test public void testGetJobId() { assertEquals(11, execution.getJobId().longValue()); - execution = new JobExecution(new JobInstance(new Long(23), "testJob"), null, new JobParameters()); + execution = new JobExecution(new JobInstance(new Long(23), "testJob"), null, new JobParameters(), null); assertEquals(23, execution.getJobId().longValue()); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/JobInstanceTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/JobInstanceTests.java index c3a1ac5947..fb9be52eba 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/JobInstanceTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/JobInstanceTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,15 +15,17 @@ */ package org.springframework.batch.core; -import junit.framework.TestCase; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; -import org.springframework.batch.support.SerializationUtils; +import org.junit.Test; +import org.springframework.util.SerializationUtils; /** * @author dsyer * */ -public class JobInstanceTests extends TestCase { +public class JobInstanceTests { private JobInstance instance = new JobInstance(new Long(11), "job"); @@ -31,15 +33,18 @@ public class JobInstanceTests extends TestCase { * Test method for * {@link org.springframework.batch.core.JobInstance#getJobName()}. */ + @Test public void testGetName() { instance = new JobInstance(new Long(1), "foo"); assertEquals("foo", instance.getJobName()); } + @Test public void testGetJob() { assertEquals("job", instance.getJobName()); } + @Test public void testCreateWithNulls() { try { new JobInstance(null, null); @@ -52,6 +57,7 @@ public void testCreateWithNulls() { assertEquals("testJob", instance.getJobName()); } + @Test public void testSerialization() { instance = new JobInstance(new Long(1), "jobName"); @@ -59,4 +65,9 @@ public void testSerialization() { assertEquals(instance, SerializationUtils.deserialize(serialized)); } + + @Test + public void testGetInstanceId() { + assertEquals(11, instance.getInstanceId()); + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/JobInterruptedExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/JobInterruptedExceptionTests.java index 12b540731c..0cdb1c5944 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/JobInterruptedExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/JobInterruptedExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/JobParameterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/JobParameterTests.java index 1baca91687..a3bab89508 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/JobParameterTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/JobParameterTests.java @@ -1,5 +1,17 @@ -/** +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ package org.springframework.batch.core; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/JobParametersBuilderTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/JobParametersBuilderTests.java index aca692377f..0b66037dbb 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/JobParametersBuilderTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/JobParametersBuilderTests.java @@ -1,75 +1,157 @@ +/* + * Copyright 2008-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; - +import java.util.ArrayList; import java.util.Date; import java.util.Iterator; +import java.util.List; import java.util.Map; +import java.util.Properties; +import org.junit.Before; +import org.junit.Rule; import org.junit.Test; +import org.junit.rules.ExpectedException; + +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.job.SimpleJob; +import org.springframework.batch.core.launch.support.RunIdIncrementer; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNull; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; /** * @author Lucas Ward * @author Michael Minella + * @author Glenn Renfro + * @author Mahmoud Ben Hassine * */ public class JobParametersBuilderTests { - JobParametersBuilder parametersBuilder = new JobParametersBuilder(); + private JobParametersBuilder parametersBuilder; + + private SimpleJob job; + + private JobExplorer jobExplorer; + + private List jobInstanceList; + + private List jobExecutionList; + + private Date date = new Date(System.currentTimeMillis()); - Date date = new Date(System.currentTimeMillis()); + @Rule + public ExpectedException expectedException = ExpectedException.none(); + + @Before + public void initialize() { + this.job = new SimpleJob("simpleJob"); + this.jobExplorer = mock(JobExplorer.class); + this.jobInstanceList = new ArrayList<>(1); + this.jobExecutionList = new ArrayList<>(1); + this.parametersBuilder = new JobParametersBuilder(this.jobExplorer); + } + + @Test + public void testAddingExistingJobParameters() { + JobParameters params1 = new JobParametersBuilder() + .addString("foo", "bar") + .addString("bar", "baz") + .toJobParameters(); + + JobParameters params2 = new JobParametersBuilder() + .addString("foo", "baz") + .toJobParameters(); + + JobParameters finalParams = new JobParametersBuilder() + .addString("baz", "quix") + .addJobParameters(params1) + .addJobParameters(params2) + .toJobParameters(); + + assertEquals(finalParams.getString("foo"), "baz"); + assertEquals(finalParams.getString("bar"), "baz"); + assertEquals(finalParams.getString("baz"), "quix"); + } @Test public void testNonIdentifyingParameters() { - parametersBuilder.addDate("SCHEDULE_DATE", date, false); - parametersBuilder.addLong("LONG", new Long(1), false); - parametersBuilder.addString("STRING", "string value", false); - JobParameters parameters = parametersBuilder.toJobParameters(); + this.parametersBuilder.addDate("SCHEDULE_DATE", date, false); + this.parametersBuilder.addLong("LONG", new Long(1), false); + this.parametersBuilder.addString("STRING", "string value", false); + this.parametersBuilder.addDouble("DOUBLE", new Double(1), false); + + JobParameters parameters = this.parametersBuilder.toJobParameters(); assertEquals(date, parameters.getDate("SCHEDULE_DATE")); assertEquals(1L, parameters.getLong("LONG").longValue()); assertEquals("string value", parameters.getString("STRING")); + assertEquals(1, parameters.getDouble("DOUBLE").doubleValue(), 1e-15); assertFalse(parameters.getParameters().get("SCHEDULE_DATE").isIdentifying()); assertFalse(parameters.getParameters().get("LONG").isIdentifying()); assertFalse(parameters.getParameters().get("STRING").isIdentifying()); + assertFalse(parameters.getParameters().get("DOUBLE").isIdentifying()); } @Test - public void testToJobRuntimeParamters(){ - parametersBuilder.addDate("SCHEDULE_DATE", date); - parametersBuilder.addLong("LONG", new Long(1)); - parametersBuilder.addString("STRING", "string value"); - JobParameters parameters = parametersBuilder.toJobParameters(); + public void testToJobRuntimeParameters(){ + this.parametersBuilder.addDate("SCHEDULE_DATE", date); + this.parametersBuilder.addLong("LONG", new Long(1)); + this.parametersBuilder.addString("STRING", "string value"); + this.parametersBuilder.addDouble("DOUBLE", new Double(1)); + JobParameters parameters = this.parametersBuilder.toJobParameters(); assertEquals(date, parameters.getDate("SCHEDULE_DATE")); assertEquals(1L, parameters.getLong("LONG").longValue()); + assertEquals(1, parameters.getDouble("DOUBLE").doubleValue(), 1e-15); assertEquals("string value", parameters.getString("STRING")); } @Test - public void testNullRuntimeParamters(){ - parametersBuilder.addDate("SCHEDULE_DATE", null); - parametersBuilder.addLong("LONG", null); - parametersBuilder.addString("STRING", null); - JobParameters parameters = parametersBuilder.toJobParameters(); - assertEquals(null, parameters.getDate("SCHEDULE_DATE")); - assertEquals(0L, parameters.getLong("LONG").longValue()); - assertEquals(null, parameters.getString("STRING")); + public void testNullRuntimeParameters(){ + this.parametersBuilder.addDate("SCHEDULE_DATE", null); + this.parametersBuilder.addLong("LONG", null); + this.parametersBuilder.addString("STRING", null); + this.parametersBuilder.addDouble("DOUBLE", null); + + JobParameters parameters = this.parametersBuilder.toJobParameters(); + assertNull(parameters.getDate("SCHEDULE_DATE")); + assertNull(parameters.getLong("LONG")); + assertNull(parameters.getString("STRING")); + assertNull(parameters.getLong("DOUBLE")); } @Test public void testCopy(){ - parametersBuilder.addString("STRING", "string value"); - parametersBuilder = new JobParametersBuilder(parametersBuilder.toJobParameters()); - Iterator parameters = parametersBuilder.toJobParameters().getParameters().keySet().iterator(); + this.parametersBuilder.addString("STRING", "string value"); + this.parametersBuilder = new JobParametersBuilder(this.parametersBuilder.toJobParameters()); + Iterator parameters = this.parametersBuilder.toJobParameters().getParameters().keySet().iterator(); assertEquals("STRING", parameters.next()); } @Test public void testOrderedTypes(){ - parametersBuilder.addDate("SCHEDULE_DATE", date); - parametersBuilder.addLong("LONG", new Long(1)); - parametersBuilder.addString("STRING", "string value"); - Iterator parameters = parametersBuilder.toJobParameters().getParameters().keySet().iterator(); + this.parametersBuilder.addDate("SCHEDULE_DATE", date); + this.parametersBuilder.addLong("LONG", new Long(1)); + this.parametersBuilder.addString("STRING", "string value"); + Iterator parameters = this.parametersBuilder.toJobParameters().getParameters().keySet().iterator(); assertEquals("SCHEDULE_DATE", parameters.next()); assertEquals("LONG", parameters.next()); assertEquals("STRING", parameters.next()); @@ -77,10 +159,10 @@ public void testOrderedTypes(){ @Test public void testOrderedStrings(){ - parametersBuilder.addString("foo", "value foo"); - parametersBuilder.addString("bar", "value bar"); - parametersBuilder.addString("spam", "value spam"); - Iterator parameters = parametersBuilder.toJobParameters().getParameters().keySet().iterator(); + this.parametersBuilder.addString("foo", "value foo"); + this.parametersBuilder.addString("bar", "value bar"); + this.parametersBuilder.addString("spam", "value spam"); + Iterator parameters = this.parametersBuilder.toJobParameters().getParameters().keySet().iterator(); assertEquals("foo", parameters.next()); assertEquals("bar", parameters.next()); assertEquals("spam", parameters.next()); @@ -89,9 +171,111 @@ public void testOrderedStrings(){ @Test public void testAddJobParameter(){ JobParameter jobParameter = new JobParameter("bar"); - parametersBuilder.addParameter("foo", jobParameter); - Map parameters = parametersBuilder.toJobParameters().getParameters(); + this.parametersBuilder.addParameter("foo", jobParameter); + Map parameters = this.parametersBuilder.toJobParameters().getParameters(); assertEquals(1, parameters.size()); assertEquals("bar", parameters.get("foo").getValue()); } + + @Test + public void testProperties() { + Properties props = new Properties(); + props.setProperty("SCHEDULE_DATE", "A DATE"); + props.setProperty("LONG", "1"); + props.setProperty("STRING", "string value"); + this.parametersBuilder = new JobParametersBuilder(props); + JobParameters parameters = this.parametersBuilder.toJobParameters(); + assertEquals("A DATE", parameters.getString("SCHEDULE_DATE")); + assertEquals("1", parameters.getString("LONG")); + assertEquals("string value", parameters.getString("STRING")); + assertFalse(parameters.getParameters().get("SCHEDULE_DATE").isIdentifying()); + assertFalse(parameters.getParameters().get("LONG").isIdentifying()); + assertFalse(parameters.getParameters().get("STRING").isIdentifying()); + } + + + @Test + public void testGetNextJobParametersFirstRun(){ + job.setJobParametersIncrementer(new RunIdIncrementer()); + initializeForNextJobParameters(); + this.parametersBuilder.getNextJobParameters(this.job); + defaultNextJobParametersVerify(this.parametersBuilder.toJobParameters(), 4); + } + + @Test + public void testGetNextJobParametersNoIncrementer(){ + expectedException.expect(IllegalArgumentException.class); + expectedException.expectMessage("No job parameters incrementer found for job=simpleJob"); + initializeForNextJobParameters(); + this.parametersBuilder.getNextJobParameters(this.job); + } + + @Test + public void testGetNextJobParameters(){ + this.job.setJobParametersIncrementer(new RunIdIncrementer()); + this.jobInstanceList.add(new JobInstance(1L, "simpleJobInstance")); + this.jobExecutionList.add(getJobExecution(this.jobInstanceList.get(0), null)); + when(this.jobExplorer.getJobInstances("simpleJob",0,1)).thenReturn(this.jobInstanceList); + when(this.jobExplorer.getJobExecutions(any())).thenReturn(this.jobExecutionList); + initializeForNextJobParameters(); + this.parametersBuilder.getNextJobParameters(this.job); + defaultNextJobParametersVerify(this.parametersBuilder.toJobParameters(), 4); + } + + @Test + public void testGetNextJobParametersRestartable(){ + this.job.setRestartable(true); + this.job.setJobParametersIncrementer(new RunIdIncrementer()); + this.jobInstanceList.add(new JobInstance(1L, "simpleJobInstance")); + this.jobExecutionList.add(getJobExecution(this.jobInstanceList.get(0), BatchStatus.FAILED)); + when(this.jobExplorer.getJobInstances("simpleJob",0,1)).thenReturn(this.jobInstanceList); + when(this.jobExplorer.getJobExecutions(any())).thenReturn(this.jobExecutionList); + initializeForNextJobParameters(); + this.parametersBuilder.addLong("NON_IDENTIFYING_LONG", new Long(1), false); + this.parametersBuilder.getNextJobParameters(this.job); + baseJobParametersVerify(this.parametersBuilder.toJobParameters(), 5); + } + + @Test + public void testGetNextJobParametersNoPreviousExecution(){ + this.job.setJobParametersIncrementer(new RunIdIncrementer()); + this.jobInstanceList.add(new JobInstance(1L, "simpleJobInstance")); + when(this.jobExplorer.getJobInstances("simpleJob",0,1)).thenReturn(this.jobInstanceList); + when(this.jobExplorer.getJobExecutions(any())).thenReturn(this.jobExecutionList); + initializeForNextJobParameters(); + this.parametersBuilder.getNextJobParameters(this.job); + baseJobParametersVerify(this.parametersBuilder.toJobParameters(), 4); + } + + @Test(expected = IllegalStateException.class) + public void testMissingJobExplorer() { + this.parametersBuilder = new JobParametersBuilder(); + this.parametersBuilder.getNextJobParameters(this.job); + } + + private void initializeForNextJobParameters() { + this.parametersBuilder.addDate("SCHEDULE_DATE", date); + this.parametersBuilder.addLong("LONG", new Long(1)); + this.parametersBuilder.addString("STRING", "string value"); + } + + private void defaultNextJobParametersVerify(JobParameters parameters, int paramCount) { + baseJobParametersVerify(parameters, paramCount); + assertEquals("1", parameters.getString("run.id")); + } + private void baseJobParametersVerify(JobParameters parameters, int paramCount) { + assertEquals(date, parameters.getDate("SCHEDULE_DATE")); + assertEquals(1L, parameters.getLong("LONG").longValue()); + assertEquals("string value", parameters.getString("STRING")); + assertEquals(paramCount, parameters.getParameters().size()); + } + + private JobExecution getJobExecution(JobInstance jobInstance, BatchStatus batchStatus) { + JobExecution jobExecution = new JobExecution(jobInstance, 1L, null, "TestConfig"); + if(batchStatus != null) { + jobExecution.setStatus(batchStatus); + } + return jobExecution; + + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/JobParametersTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/JobParametersTests.java index e7a8c62d79..68fa481fc4 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/JobParametersTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/JobParametersTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core; import static org.junit.Assert.assertEquals; @@ -13,12 +28,13 @@ import org.junit.Before; import org.junit.Test; -import org.springframework.batch.support.SerializationUtils; +import org.springframework.util.SerializationUtils; /** * @author Lucas Ward * @author Dave Syer * @author Michael Minella + * @author Mahmoud Ben Hassine * */ public class JobParametersTests { @@ -36,7 +52,7 @@ public void setUp() throws Exception { private JobParameters getNewParameters() { - Map parameterMap = new HashMap(); + Map parameterMap = new HashMap<>(); parameterMap.put("string.key1", new JobParameter("value1", true)); parameterMap.put("string.key2", new JobParameter("value2", true)); parameterMap.put("long.key1", new JobParameter(1L, true)); @@ -59,7 +75,7 @@ public void testGetString() { @Test public void testGetNullString() { parameters = new JobParameters(Collections.singletonMap("string.key1", new JobParameter((String) null, true))); - assertEquals(null, parameters.getDate("string.key1")); + assertNull(parameters.getDate("string.key1")); } @Test @@ -83,23 +99,23 @@ public void testGetDate() { @Test public void testGetNullDate() { parameters = new JobParameters(Collections.singletonMap("date.key1", new JobParameter((Date)null, true))); - assertEquals(null, parameters.getDate("date.key1")); + assertNull(parameters.getDate("date.key1")); } @Test public void testGetEmptyLong() { parameters = new JobParameters(Collections.singletonMap("long1", new JobParameter((Long)null, true))); - assertEquals(0L, parameters.getLong("long1").longValue()); + assertNull(parameters.getLong("long1")); } @Test public void testGetMissingLong() { - assertEquals(0L, parameters.getLong("missing.long1").longValue()); + assertNull(parameters.getLong("missing.long1")); } @Test public void testGetMissingDouble() { - assertEquals(0.0, parameters.getDouble("missing.double1"), 0.0001); + assertNull(parameters.getDouble("missing.double1")); } @Test @@ -142,14 +158,14 @@ public void testEqualsNull() { public void testToStringOrder() { Map props = parameters.getParameters(); - StringBuffer stringBuilder = new StringBuffer(); + StringBuilder stringBuilder = new StringBuilder(); for (Entry entry : props.entrySet()) { - stringBuilder.append(entry.toString() + ";"); + stringBuilder.append(entry.toString()).append(";"); } String string1 = stringBuilder.toString(); - Map parameterMap = new HashMap(); + Map parameterMap = new HashMap<>(); parameterMap.put("string.key2", new JobParameter("value2", true)); parameterMap.put("string.key1", new JobParameter("value1", true)); parameterMap.put("long.key2", new JobParameter(2L, true)); @@ -162,9 +178,9 @@ public void testToStringOrder() { JobParameters testProps = new JobParameters(parameterMap); props = testProps.getParameters(); - stringBuilder = new StringBuffer(); + stringBuilder = new StringBuilder(); for (Entry entry : props.entrySet()) { - stringBuilder.append(entry.toString() + ";"); + stringBuilder.append(entry.toString()).append(";"); } String string2 = stringBuilder.toString(); @@ -194,8 +210,8 @@ public void testSerialization() { } @Test - public void testLongReturns0WhenKeyDoesntExit(){ - assertEquals(0L,new JobParameters().getLong("keythatdoesntexist").longValue()); + public void testLongReturnsNullWhenKeyDoesntExit(){ + assertNull(new JobParameters().getLong("keythatdoesntexist")); } @Test @@ -204,8 +220,8 @@ public void testStringReturnsNullWhenKeyDoesntExit(){ } @Test - public void testDoubleReturns0WhenKeyDoesntExit(){ - assertEquals(0.0,new JobParameters().getLong("keythatdoesntexist"), 0.0001); + public void testDoubleReturnsNullWhenKeyDoesntExit(){ + assertNull(new JobParameters().getDouble("keythatdoesntexist")); } @Test diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/PooledEmbeddedDataSource.java b/spring-batch-core/src/test/java/org/springframework/batch/core/PooledEmbeddedDataSource.java new file mode 100644 index 0000000000..0da5b44137 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/PooledEmbeddedDataSource.java @@ -0,0 +1,120 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core; + +import java.io.PrintWriter; +import java.sql.Connection; +import java.sql.SQLException; +import java.util.logging.Logger; + +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabase; + +/** + * As of Spring 3.2, when a context is closed, the shutdown method is + * called on any beans that are registered. With an embedded database + * that uses a connection pool, this can leave the connection pool open + * with stale connections. This wraps an {@link EmbeddedDatabase} and + * ignores calls to {@link EmbeddedDatabase#shutdown()}. + * + * @author Phil Webb + * @since 3.0 + */ +public class PooledEmbeddedDataSource implements EmbeddedDatabase { + + private final EmbeddedDatabase dataSource; + + /** + * @param dataSource The database to be wrapped + */ + public PooledEmbeddedDataSource(EmbeddedDatabase dataSource) { + this.dataSource = dataSource; + } + + /* (non-Javadoc) + * @see javax.sql.DataSource#getConnection() + */ + @Override + public Connection getConnection() throws SQLException { + return this.dataSource.getConnection(); + } + + /* (non-Javadoc) + * @see javax.sql.DataSource#getConnection(java.lang.String, java.lang.String) + */ + @Override + public Connection getConnection(String username, String password) throws SQLException { + return this.dataSource.getConnection(username, password); + } + + /* (non-Javadoc) + * @see javax.sql.CommonDataSource#getLogWriter() + */ + @Override + public PrintWriter getLogWriter() throws SQLException { + return this.dataSource.getLogWriter(); + } + + /* (non-Javadoc) + * @see javax.sql.CommonDataSource#setLogWriter(java.io.PrintWriter) + */ + @Override + public void setLogWriter(PrintWriter out) throws SQLException { + this.dataSource.setLogWriter(out); + } + + /* (non-Javadoc) + * @see javax.sql.CommonDataSource#getLoginTimeout() + */ + @Override + public int getLoginTimeout() throws SQLException { + return this.dataSource.getLoginTimeout(); + } + + /* (non-Javadoc) + * @see javax.sql.CommonDataSource#setLoginTimeout(int) + */ + @Override + public void setLoginTimeout(int seconds) throws SQLException { + this.dataSource.setLoginTimeout(seconds); + } + + /* (non-Javadoc) + * @see java.sql.Wrapper#unwrap(java.lang.Class) + */ + @Override + public T unwrap(Class iface) throws SQLException { + return this.dataSource.unwrap(iface); + } + + /* (non-Javadoc) + * @see java.sql.Wrapper#isWrapperFor(java.lang.Class) + */ + @Override + public boolean isWrapperFor(Class iface) throws SQLException { + return this.dataSource.isWrapperFor(iface); + } + + public Logger getParentLogger() { + return Logger.getLogger(Logger.GLOBAL_LOGGER_NAME); + } + + /* (non-Javadoc) + * @see org.springframework.jdbc.datasource.embedded.EmbeddedDatabase#shutdown() + */ + @Override + public void shutdown() { + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/SpringBeanJobTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/SpringBeanJobTests.java index 5f9e242bba..923b41771e 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/SpringBeanJobTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/SpringBeanJobTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,8 +16,10 @@ package org.springframework.batch.core; -import junit.framework.TestCase; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import org.junit.Test; import org.springframework.batch.core.job.JobSupport; import org.springframework.beans.factory.config.ConstructorArgumentValues; import org.springframework.beans.factory.support.ChildBeanDefinition; @@ -25,34 +27,42 @@ import org.springframework.context.support.GenericApplicationContext; import org.springframework.context.support.StaticApplicationContext; -public class SpringBeanJobTests extends TestCase { +public class SpringBeanJobTests { + @Test public void testBeanName() throws Exception { StaticApplicationContext context = new StaticApplicationContext(); JobSupport configuration = new JobSupport(); context.getAutowireCapableBeanFactory().initializeBean(configuration, "bean"); + context.refresh(); assertNotNull(configuration.getName()); configuration.setBeanName("foo"); context.getAutowireCapableBeanFactory().initializeBean(configuration, "bean"); assertEquals("bean", configuration.getName()); + context.close(); } + @Test public void testBeanNameWithBeanDefinition() throws Exception { GenericApplicationContext context = new GenericApplicationContext(); ConstructorArgumentValues args = new ConstructorArgumentValues(); args.addGenericArgumentValue("foo"); context.registerBeanDefinition("bean", new RootBeanDefinition( JobSupport.class, args, null)); + + context.refresh(); JobSupport configuration = (JobSupport) context .getBean("bean"); assertNotNull(configuration.getName()); assertEquals("foo", configuration.getName()); configuration.setBeanName("bar"); assertEquals("foo", configuration.getName()); + context.close(); } + @Test public void testBeanNameWithParentBeanDefinition() throws Exception { GenericApplicationContext context = new GenericApplicationContext(); ConstructorArgumentValues args = new ConstructorArgumentValues(); @@ -60,6 +70,7 @@ public void testBeanNameWithParentBeanDefinition() throws Exception { context.registerBeanDefinition("parent", new RootBeanDefinition( JobSupport.class, args, null)); context.registerBeanDefinition("bean", new ChildBeanDefinition("parent")); + context.refresh(); JobSupport configuration = (JobSupport) context .getBean("bean"); assertNotNull(configuration.getName()); @@ -68,5 +79,6 @@ public void testBeanNameWithParentBeanDefinition() throws Exception { assertEquals("bar", configuration.getName()); configuration.setName("foo"); assertEquals("foo", configuration.getName()); + context.close(); } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/StepContributionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/StepContributionTests.java index 29a3ffdf09..19b8f0ed51 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/StepContributionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/StepContributionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/StepExecutionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/StepExecutionTests.java index 043e60860f..d83869f8d4 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/StepExecutionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/StepExecutionTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,7 +30,7 @@ import org.junit.Test; import org.springframework.batch.core.step.StepSupport; import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.support.SerializationUtils; +import org.springframework.util.SerializationUtils; /** * @author Dave Syer @@ -256,7 +256,7 @@ public void testHashCodeWithNullIds() throws Exception { @Test public void testHashCodeViaHashSet() throws Exception { - Set set = new HashSet(); + Set set = new HashSet<>(); set.add(execution); assertTrue(set.contains(execution)); execution.setExecutionContext(foobarEc); @@ -305,7 +305,7 @@ private StepExecution newStepExecution(Step step, Long jobExecutionId) { private StepExecution newStepExecution(Step step, Long jobExecutionId, long stepExecutionId) { JobInstance job = new JobInstance(3L, "testJob"); - StepExecution execution = new StepExecution(step.getName(), new JobExecution(job, jobExecutionId, new JobParameters()), stepExecutionId); + StepExecution execution = new StepExecution(step.getName(), new JobExecution(job, jobExecutionId, new JobParameters(), null), stepExecutionId); return execution; } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/DuplicateJobExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/DuplicateJobExceptionTests.java index 52a7a80a7e..fe134af6e7 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/DuplicateJobExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/DuplicateJobExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/DataSourceConfiguration.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/DataSourceConfiguration.java index 76aa3d8058..e307afdf1f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/DataSourceConfiguration.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/DataSourceConfiguration.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,9 +15,6 @@ */ package org.springframework.batch.core.configuration.annotation; -import javax.annotation.PostConstruct; -import javax.sql.DataSource; - import org.springframework.batch.core.Step; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; @@ -29,15 +26,18 @@ import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator; import org.springframework.util.ClassUtils; +import javax.annotation.PostConstruct; +import javax.sql.DataSource; + @Configuration public class DataSourceConfiguration { - + @Autowired private Environment environment; - + @Autowired private ResourceLoader resourceLoader; - + @PostConstruct protected void initialize() { ResourceDatabasePopulator populator = new ResourceDatabasePopulator(); @@ -45,7 +45,7 @@ protected void initialize() { populator.setContinueOnError(true); DatabasePopulatorUtils.execute(populator, dataSource()); } - + @Bean public DataSource dataSource() { return new EmbeddedDatabaseFactory().getDatabase(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/JobBuilderConfigurationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/JobBuilderConfigurationTests.java index 993d1345d6..30d8497a34 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/JobBuilderConfigurationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/JobBuilderConfigurationTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2011 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -39,10 +39,12 @@ import org.springframework.context.annotation.AnnotationConfigApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.lang.Nullable; /** * @author Dave Syer - * + * @author Mahmoud Ben Hassine + * */ public class JobBuilderConfigurationTests { @@ -131,6 +133,7 @@ protected Step step2() throws Exception { @Bean protected Tasklet tasklet() { return new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext context) throws Exception { if (fail) { @@ -176,7 +179,7 @@ public static class TestConfigurer extends DefaultBatchConfigurer { private SimpleBatchConfiguration jobs; @Bean - public Job testConfigererJob() throws Exception { + public Job testConfigurerJob() throws Exception { SimpleJobBuilder builder = jobs.jobBuilders().get("configurer").start(step1()); return builder.build(); } @@ -216,6 +219,7 @@ public Job beansConfigurerJob() throws Exception { protected Step step1() throws Exception { return steps.get("step1").tasklet(new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { return null; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/JobLoaderConfigurationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/JobLoaderConfigurationTests.java index 34d0e0ac41..8a1ff3b743 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/JobLoaderConfigurationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/JobLoaderConfigurationTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ import javax.annotation.PostConstruct; import org.junit.Test; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; @@ -30,6 +31,7 @@ import org.springframework.batch.core.configuration.support.ApplicationContextFactory; import org.springframework.batch.core.configuration.support.AutomaticJobRegistrar; import org.springframework.batch.core.configuration.support.GenericApplicationContextFactory; +import org.springframework.batch.core.explore.JobExplorer; import org.springframework.batch.core.job.builder.SimpleJobBuilder; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.scope.context.ChunkContext; @@ -39,6 +41,8 @@ import org.springframework.context.annotation.AnnotationConfigApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.support.ApplicationObjectSupport; +import org.springframework.lang.Nullable; /** * @author Dave Syer @@ -72,6 +76,8 @@ private void testJob(String jobName, BatchStatus status, int stepExecutionCount, .toJobParameters()); assertEquals(status, execution.getStatus()); assertEquals(stepExecutionCount, execution.getStepExecutions().size()); + JobExplorer jobExplorer = context.getBean(JobExplorer.class); + assertEquals(1, jobExplorer.getJobInstanceCount(jobName)); context.close(); } @@ -111,6 +117,11 @@ public void initialize() { @Configuration public static class TestConfiguration { + @Bean + public ApplicationObjectSupport fakeApplicationObjectSupport() { + return new ApplicationObjectSupport() {}; + } + @Autowired private JobBuilderFactory jobs; @@ -136,6 +147,7 @@ protected Step step2() throws Exception { @Bean protected Tasklet tasklet() { return new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext context) throws Exception { return RepeatStatus.FINISHED; @@ -162,6 +174,7 @@ public Job vanillaJob() throws Exception { @Bean protected Step step3() throws Exception { return steps.get("step3").tasklet(new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext context) throws Exception { return RepeatStatus.FINISHED; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTests.java new file mode 100644 index 0000000000..dc3ec0498e --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTests.java @@ -0,0 +1,301 @@ +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.core.configuration.annotation; + +import static org.junit.Assert.assertEquals; + +import java.util.concurrent.Callable; + +import org.junit.After; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.scope.context.JobSynchronizationManager; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.beans.factory.BeanCreationException; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.ImportResource; +import org.springframework.context.annotation.Scope; +import org.springframework.context.annotation.ScopedProxyMode; +import org.springframework.context.support.ClassPathXmlApplicationContext; +import org.springframework.lang.Nullable; + +/** + * @author Dave Syer + * @author Michael Minella + * @author Mahmoud Ben Hassine + * + */ +public class JobScopeConfigurationTests { + + private ConfigurableApplicationContext context; + + private JobExecution jobExecution; + + @Rule + public ExpectedException expected = ExpectedException.none(); + + @Test + public void testXmlJobScopeWithProxyTargetClass() throws Exception { + context = new ClassPathXmlApplicationContext( + "org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsProxyTargetClass-context.xml"); + JobSynchronizationManager.register(jobExecution); + SimpleHolder value = context.getBean(SimpleHolder.class); + assertEquals("JOB", value.call()); + } + + @Test + public void testXmlJobScopeWithInterface() throws Exception { + context = new ClassPathXmlApplicationContext( + "org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsInterface-context.xml"); + JobSynchronizationManager.register(jobExecution); + @SuppressWarnings("unchecked") + Callable value = context.getBean(Callable.class); + assertEquals("JOB", value.call()); + } + + @Test + public void testXmlJobScopeWithInheritance() throws Exception { + context = new ClassPathXmlApplicationContext( + "org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsInheritance-context.xml"); + JobSynchronizationManager.register(jobExecution); + SimpleHolder value = (SimpleHolder) context.getBean("child"); + assertEquals("JOB", value.call()); + } + + @Test + public void testJobScopeWithProxyTargetClass() throws Exception { + init(JobScopeConfigurationRequiringProxyTargetClass.class); + SimpleHolder value = context.getBean(SimpleHolder.class); + assertEquals("JOB", value.call()); + } + + @Test + public void testStepScopeXmlImportUsingNamespace() throws Exception { + init(JobScopeConfigurationXmlImportUsingNamespace.class); + + SimpleHolder value = (SimpleHolder) context.getBean("xmlValue"); + assertEquals("JOB", value.call()); + value = (SimpleHolder) context.getBean("javaValue"); + assertEquals("JOB", value.call()); + } + + @Test + public void testJobScopeWithProxyTargetClassInjected() throws Exception { + init(JobScopeConfigurationInjectingProxy.class); + SimpleHolder value = context.getBean(Wrapper.class).getValue(); + assertEquals("JOB", value.call()); + } + + @Test + public void testIntentionallyBlowUpOnMissingContextWithProxyTargetClass() throws Exception { + init(JobScopeConfigurationRequiringProxyTargetClass.class); + JobSynchronizationManager.release(); + expected.expect(BeanCreationException.class); + expected.expectMessage("job scope"); + SimpleHolder value = context.getBean(SimpleHolder.class); + assertEquals("JOB", value.call()); + } + + @Test + public void testIntentionallyBlowupWithForcedInterface() throws Exception { + init(JobScopeConfigurationForcingInterfaceProxy.class); + JobSynchronizationManager.release(); + expected.expect(BeanCreationException.class); + expected.expectMessage("job scope"); + SimpleHolder value = context.getBean(SimpleHolder.class); + assertEquals("JOB", value.call()); + } + + @Test + public void testJobScopeWithDefaults() throws Exception { + init(JobScopeConfigurationWithDefaults.class); + @SuppressWarnings("unchecked") + Callable value = context.getBean(Callable.class); + assertEquals("JOB", value.call()); + } + + @Test + public void testIntentionallyBlowUpOnMissingContextWithInterface() throws Exception { + init(JobScopeConfigurationWithDefaults.class); + JobSynchronizationManager.release(); + expected.expect(BeanCreationException.class); + expected.expectMessage("job scope"); + @SuppressWarnings("unchecked") + Callable value = context.getBean(Callable.class); + assertEquals("JOB", value.call()); + } + + public void init(Class... config) throws Exception { + Class[] configs = new Class[config.length + 1]; + System.arraycopy(config, 0, configs, 1, config.length); + configs[0] = DataSourceConfiguration.class; + AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(); + context.register(configs); + context.refresh(); + this.context = context; + JobSynchronizationManager.register(jobExecution); + } + + @Before + public void setup() { + JobSynchronizationManager.release(); + jobExecution = new JobExecution(new JobInstance(5l, "JOB"), null, null); + } + + @After + public void close() { + JobSynchronizationManager.release(); + if (context != null) { + context.close(); + } + } + + public static class SimpleCallable implements Callable { + private final String value; + + private SimpleCallable(String value) { + this.value = value; + } + + @Override + public String call() throws Exception { + return value; + } + } + + public static class SimpleHolder { + private final String value; + + protected SimpleHolder() { + value = ""; + } + + public SimpleHolder(String value) { + this.value = value; + } + + public String call() throws Exception { + return value; + } + } + + public static class Wrapper { + + private SimpleHolder value; + + public Wrapper(SimpleHolder value) { + this.value = value; + } + + public SimpleHolder getValue() { + return value; + } + + } + + public static class TaskletSupport implements Tasklet { + + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + return RepeatStatus.FINISHED; + } + } + + @Configuration + @ImportResource("org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsXmlImportUsingNamespace-context.xml") + @EnableBatchProcessing + public static class JobScopeConfigurationXmlImportUsingNamespace { + + @Bean + @JobScope + protected SimpleHolder javaValue(@Value("#{jobName}") final String value) { + return new SimpleHolder(value); + } + + } + + @Configuration + @EnableBatchProcessing + public static class JobScopeConfigurationInjectingProxy { + + @Bean + public Wrapper wrapper(SimpleHolder value) { + return new Wrapper(value); + } + + @Bean + @Scope(value="job", proxyMode = ScopedProxyMode.TARGET_CLASS) + protected SimpleHolder value(@Value("#{jobName}") + final String value) { + return new SimpleHolder(value); + } + + } + + @Configuration + @EnableBatchProcessing + public static class JobScopeConfigurationRequiringProxyTargetClass { + + @Bean + @Scope(value="job", proxyMode = ScopedProxyMode.TARGET_CLASS) + protected SimpleHolder value(@Value("#{jobName}") + final String value) { + return new SimpleHolder(value); + } + + } + + @Configuration + @EnableBatchProcessing + public static class JobScopeConfigurationWithDefaults { + + @Bean + @JobScope + protected Callable value(@Value("#{jobName}") + final String value) { + return new SimpleCallable(value); + } + + } + + @Configuration + @EnableBatchProcessing + public static class JobScopeConfigurationForcingInterfaceProxy { + + @Bean + @Scope(value="job", proxyMode = ScopedProxyMode.INTERFACES) + protected SimpleHolder value(@Value("#{jobName}") + final String value) { + return new SimpleHolder(value); + } + + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/MapJobRepositoryConfigurationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/MapJobRepositoryConfigurationTests.java new file mode 100644 index 0000000000..501be92032 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/MapJobRepositoryConfigurationTests.java @@ -0,0 +1,163 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.configuration.annotation; + +import static org.junit.Assert.assertEquals; + +import javax.sql.DataSource; + +import org.junit.Test; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.PooledEmbeddedDataSource; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.beans.factory.UnsatisfiedDependencyException; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Primary; +import org.springframework.context.support.GenericApplicationContext; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder; +import org.springframework.lang.Nullable; +import org.springframework.stereotype.Component; + +public class MapJobRepositoryConfigurationTests { + + JobLauncher jobLauncher; + JobRepository jobRepository; + Job job; + JobExplorer jobExplorer; + + @Test + public void testRoseyScenario() throws Exception { + testConfigurationClass(MapRepositoryBatchConfiguration.class); + } + + @Test + public void testOneDataSource() throws Exception { + testConfigurationClass(HsqlBatchConfiguration.class); + } + + @Test(expected = UnsatisfiedDependencyException.class) + public void testMultipleDataSources_whenNoneOfThemIsPrimary() throws Exception { + testConfigurationClass(InvalidBatchConfiguration.class); + } + + @Test + public void testMultipleDataSources_whenNoneOfThemIsPrimaryButOneOfThemIsNamed_dataSource_() throws Exception { + testConfigurationClass(ValidBatchConfigurationWithoutPrimaryDataSource.class); + } + + @Test + public void testMultipleDataSources_whenOneOfThemIsPrimary() throws Exception { + testConfigurationClass(ValidBatchConfigurationWithPrimaryDataSource.class); + } + + private void testConfigurationClass(Class clazz) throws Exception { + GenericApplicationContext context = new AnnotationConfigApplicationContext(clazz); + this.jobLauncher = context.getBean(JobLauncher.class); + this.jobRepository = context.getBean(JobRepository.class); + this.job = context.getBean(Job.class); + this.jobExplorer = context.getBean(JobExplorer.class); + + JobExecution jobExecution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + JobExecution repositoryJobExecution = jobRepository.getLastJobExecution(job.getName(), new JobParameters()); + assertEquals(jobExecution.getId(), repositoryJobExecution.getId()); + assertEquals("job", jobExplorer.getJobNames().iterator().next()); + context.close(); + } + + public static class InvalidBatchConfiguration extends HsqlBatchConfiguration { + + @Bean + DataSource dataSource2() { + return new PooledEmbeddedDataSource(new EmbeddedDatabaseBuilder().setName("badDatabase").build()); + } + } + + public static class ValidBatchConfigurationWithPrimaryDataSource extends HsqlBatchConfiguration { + + @Primary + @Bean + DataSource dataSource2() { + return new PooledEmbeddedDataSource(new EmbeddedDatabaseBuilder(). + setName("dataSource2"). + addScript("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql"). + addScript("classpath:org/springframework/batch/core/schema-hsqldb.sql"). + build()); + } + } + + public static class ValidBatchConfigurationWithoutPrimaryDataSource extends HsqlBatchConfiguration { + + @Bean + DataSource dataSource() { // will be autowired by name + return new PooledEmbeddedDataSource(new EmbeddedDatabaseBuilder(). + setName("dataSource"). + addScript("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql"). + addScript("classpath:org/springframework/batch/core/schema-hsqldb.sql"). + build()); + } + } + + public static class HsqlBatchConfiguration extends MapRepositoryBatchConfiguration { + + @Bean + DataSource dataSource1() { + return new PooledEmbeddedDataSource(new EmbeddedDatabaseBuilder(). + setName("dataSource1"). + addScript("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql"). + addScript("classpath:org/springframework/batch/core/schema-hsqldb.sql"). + build()); + } + } + + @Component + @EnableBatchProcessing + public static class MapRepositoryBatchConfiguration { + @Autowired + JobBuilderFactory jobFactory; + + @Autowired + StepBuilderFactory stepFactory; + + @Bean + Step step1 () { + return stepFactory.get("step1").tasklet(new Tasklet() { + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + return RepeatStatus.FINISHED; + } + }).build(); + } + + @Bean + Job job() { + return jobFactory.get("job").start(step1()).build(); + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTests.java index 95e244dbaa..2af2dc03b9 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,29 +16,37 @@ package org.springframework.batch.core.configuration.annotation; -import static org.junit.Assert.assertEquals; - -import java.util.concurrent.Callable; - import org.junit.After; import org.junit.Before; import org.junit.Rule; import org.junit.Test; import org.junit.rules.ExpectedException; +import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.repeat.RepeatStatus; import org.springframework.beans.factory.BeanCreationException; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.ConfigurableApplicationContext; import org.springframework.context.annotation.AnnotationConfigApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.ImportResource; import org.springframework.context.annotation.Scope; import org.springframework.context.annotation.ScopedProxyMode; import org.springframework.context.support.ClassPathXmlApplicationContext; +import org.springframework.lang.Nullable; + +import java.util.concurrent.Callable; + +import static org.junit.Assert.assertEquals; /** * @author Dave Syer + * @author Michael Minella + * @author Mahmoud Ben Hassine * */ public class StepScopeConfigurationTests { @@ -70,9 +78,9 @@ public void testXmlStepScopeWithInterface() throws Exception { } @Test - public void testXmlStepScopeWithInheritence() throws Exception { + public void testXmlStepScopeWithInheritance() throws Exception { context = new ClassPathXmlApplicationContext( - "org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInheritence-context.xml"); + "org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInheritance-context.xml"); StepSynchronizationManager.register(stepExecution); SimpleHolder value = (SimpleHolder) context.getBean("child"); assertEquals("STEP", value.call()); @@ -85,6 +93,16 @@ public void testStepScopeWithProxyTargetClass() throws Exception { assertEquals("STEP", value.call()); } + @Test + public void testStepScopeXmlImportUsingNamespace() throws Exception { + init(StepScopeConfigurationXmlImportUsingNamespace.class); + + SimpleHolder value = (SimpleHolder) context.getBean("xmlValue"); + assertEquals("STEP", value.call()); + value = (SimpleHolder) context.getBean("javaValue"); + assertEquals("STEP", value.call()); + } + @Test public void testStepScopeWithProxyTargetClassInjected() throws Exception { init(StepScopeConfigurationInjectingProxy.class); @@ -199,6 +217,20 @@ public SimpleHolder getValue() { } + @Configuration + @ImportResource("org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsXmlImportUsingNamespace-context.xml") + @EnableBatchProcessing + public static class StepScopeConfigurationXmlImportUsingNamespace { + + @Bean + @StepScope + protected SimpleHolder javaValue(@Value("#{stepExecution.stepName}") + final String value) { + return new SimpleHolder(value); + } + + } + @Configuration @EnableBatchProcessing public static class StepScopeConfigurationInjectingProxy { @@ -256,4 +288,12 @@ protected SimpleHolder value(@Value("#{stepExecution.stepName}") } + public static class TaskletSupport implements Tasklet { + + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + return RepeatStatus.FINISHED; + } + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/TransactionManagerConfigurationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/TransactionManagerConfigurationTests.java new file mode 100644 index 0000000000..32685fddee --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/TransactionManagerConfigurationTests.java @@ -0,0 +1,69 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.core.configuration.annotation; + +import javax.sql.DataSource; + +import org.junit.runner.RunWith; +import org.mockito.Mock; +import org.mockito.junit.MockitoJUnitRunner; + +import org.springframework.aop.Advisor; +import org.springframework.aop.TargetSource; +import org.springframework.aop.framework.Advised; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.transaction.interceptor.TransactionInterceptor; + +/** + * @author Mahmoud Ben Hassine + */ +@RunWith(MockitoJUnitRunner.class) +public abstract class TransactionManagerConfigurationTests { + + @Mock + protected static PlatformTransactionManager transactionManager; + + @Mock + protected static PlatformTransactionManager transactionManager2; + + /* + * The transaction manager set on JobRepositoryFactoryBean in DefaultBatchConfigurer.createJobRepository + * ends up in the TransactionInterceptor advise applied to the (proxied) JobRepository. + * This method extracts the advise from the proxy and returns the transaction manager. + */ + PlatformTransactionManager getTransactionManagerSetOnJobRepository(JobRepository jobRepository) throws Exception { + TargetSource targetSource = ((Advised) jobRepository).getTargetSource(); // proxy created in SimpleBatchConfiguration.createLazyProxy + Advised target = (Advised) targetSource.getTarget(); // initial proxy created in AbstractJobRepositoryFactoryBean.initializeProxy + Advisor[] advisors = target.getAdvisors(); + for (Advisor advisor : advisors) { + if (advisor.getAdvice() instanceof TransactionInterceptor) { + TransactionInterceptor transactionInterceptor = (TransactionInterceptor) advisor.getAdvice(); + return (PlatformTransactionManager) transactionInterceptor.getTransactionManager(); + } + } + return null; + } + + static DataSource createDataSource() { + return new EmbeddedDatabaseBuilder() + .addScript("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql") + .addScript("classpath:org/springframework/batch/core/schema-hsqldb.sql") + .build(); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/TransactionManagerConfigurationWithBatchConfigurerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/TransactionManagerConfigurationWithBatchConfigurerTests.java new file mode 100644 index 0000000000..825c125e0f --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/TransactionManagerConfigurationWithBatchConfigurerTests.java @@ -0,0 +1,146 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.core.configuration.annotation; + +import javax.sql.DataSource; + +import org.junit.Assert; +import org.junit.Test; + +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.support.transaction.ResourcelessTransactionManager; +import org.springframework.context.ApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.jdbc.datasource.DataSourceTransactionManager; +import org.springframework.test.util.AopTestUtils; +import org.springframework.transaction.PlatformTransactionManager; + +/** + * @author Mahmoud Ben Hassine + */ +public class TransactionManagerConfigurationWithBatchConfigurerTests extends TransactionManagerConfigurationTests { + + @Test + public void testConfigurationWithNoDataSourceAndNoTransactionManager() throws Exception { + ApplicationContext applicationContext = new AnnotationConfigApplicationContext(BatchConfigurationWithNoDataSourceAndNoTransactionManager.class); + BatchConfigurer batchConfigurer = applicationContext.getBean(BatchConfigurer.class); + + PlatformTransactionManager platformTransactionManager = batchConfigurer.getTransactionManager(); + Assert.assertTrue(platformTransactionManager instanceof ResourcelessTransactionManager); + Assert.assertSame(getTransactionManagerSetOnJobRepository(applicationContext.getBean(JobRepository.class)), platformTransactionManager); + } + + @Test + public void testConfigurationWithNoDataSourceAndTransactionManager() throws Exception { + ApplicationContext applicationContext = new AnnotationConfigApplicationContext(BatchConfigurationWithNoDataSourceAndTransactionManager.class); + BatchConfigurer batchConfigurer = applicationContext.getBean(BatchConfigurer.class); + + PlatformTransactionManager platformTransactionManager = batchConfigurer.getTransactionManager(); + Assert.assertSame(transactionManager, platformTransactionManager); + Assert.assertSame(getTransactionManagerSetOnJobRepository(applicationContext.getBean(JobRepository.class)), transactionManager); + } + + @Test + public void testConfigurationWithDataSourceAndNoTransactionManager() throws Exception { + ApplicationContext applicationContext = new AnnotationConfigApplicationContext(BatchConfigurationWithDataSourceAndNoTransactionManager.class); + BatchConfigurer batchConfigurer = applicationContext.getBean(BatchConfigurer.class); + + PlatformTransactionManager platformTransactionManager = batchConfigurer.getTransactionManager(); + Assert.assertTrue(platformTransactionManager instanceof DataSourceTransactionManager); + DataSourceTransactionManager dataSourceTransactionManager = AopTestUtils.getTargetObject(platformTransactionManager); + Assert.assertEquals(applicationContext.getBean(DataSource.class), dataSourceTransactionManager.getDataSource()); + Assert.assertSame(getTransactionManagerSetOnJobRepository(applicationContext.getBean(JobRepository.class)), platformTransactionManager); + } + + @Test + public void testConfigurationWithDataSourceAndTransactionManager() throws Exception { + ApplicationContext applicationContext = new AnnotationConfigApplicationContext(BatchConfigurationWithDataSourceAndTransactionManager.class); + BatchConfigurer batchConfigurer = applicationContext.getBean(BatchConfigurer.class); + + PlatformTransactionManager platformTransactionManager = batchConfigurer.getTransactionManager(); + Assert.assertSame(transactionManager, platformTransactionManager); + Assert.assertSame(getTransactionManagerSetOnJobRepository(applicationContext.getBean(JobRepository.class)), transactionManager); + } + + @EnableBatchProcessing + public static class BatchConfigurationWithNoDataSourceAndNoTransactionManager { + @Bean + public BatchConfigurer batchConfigurer() { + return new DefaultBatchConfigurer(); + } + } + + @EnableBatchProcessing + public static class BatchConfigurationWithNoDataSourceAndTransactionManager { + + @Bean + public PlatformTransactionManager transactionManager() { + return transactionManager; + } + + @Bean + public BatchConfigurer batchConfigurer() { + return new DefaultBatchConfigurer() { + @Override + public PlatformTransactionManager getTransactionManager() { + return transactionManager(); + } + }; + } + + } + + @EnableBatchProcessing + public static class BatchConfigurationWithDataSourceAndNoTransactionManager { + @Bean + public DataSource dataSource() { + return createDataSource(); + } + + @Bean + public BatchConfigurer batchConfigurer(DataSource dataSource) { + return new DefaultBatchConfigurer(dataSource); + } + } + + @EnableBatchProcessing + public static class BatchConfigurationWithDataSourceAndTransactionManager { + + @Bean + public DataSource dataSource() { + return createDataSource(); + } + + @Bean + public PlatformTransactionManager transactionManager() { + return transactionManager; + } + + @Bean + public BatchConfigurer batchConfigurer(DataSource dataSource) { + return new DefaultBatchConfigurer(dataSource) { + @Override + public PlatformTransactionManager getTransactionManager() { + return transactionManager(); + } + }; + } + + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/TransactionManagerConfigurationWithoutBatchConfigurerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/TransactionManagerConfigurationWithoutBatchConfigurerTests.java new file mode 100644 index 0000000000..a2fb3860b6 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/annotation/TransactionManagerConfigurationWithoutBatchConfigurerTests.java @@ -0,0 +1,148 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.core.configuration.annotation; + +import javax.sql.DataSource; + +import org.junit.Assert; +import org.junit.Test; + +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.support.transaction.ResourcelessTransactionManager; +import org.springframework.context.ApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Primary; +import org.springframework.jdbc.datasource.DataSourceTransactionManager; +import org.springframework.test.util.AopTestUtils; +import org.springframework.transaction.PlatformTransactionManager; + +/** + * @author Mahmoud Ben Hassine + */ +public class TransactionManagerConfigurationWithoutBatchConfigurerTests extends TransactionManagerConfigurationTests { + + @Test + public void testConfigurationWithNoDataSourceAndNoTransactionManager() throws Exception { + ApplicationContext applicationContext = new AnnotationConfigApplicationContext(BatchConfigurationWithNoDataSourceAndNoTransactionManager.class); + Assert.assertTrue(applicationContext.containsBean("transactionManager")); + PlatformTransactionManager platformTransactionManager = applicationContext.getBean(PlatformTransactionManager.class); + Object targetObject = AopTestUtils.getTargetObject(platformTransactionManager); + Assert.assertTrue(targetObject instanceof ResourcelessTransactionManager); + Assert.assertSame(getTransactionManagerSetOnJobRepository(applicationContext.getBean(JobRepository.class)), targetObject); + } + + @Test + public void testConfigurationWithNoDataSourceAndTransactionManager() throws Exception { + ApplicationContext applicationContext = new AnnotationConfigApplicationContext(BatchConfigurationWithNoDataSourceAndTransactionManager.class); + PlatformTransactionManager platformTransactionManager = applicationContext.getBean(PlatformTransactionManager.class); + Assert.assertSame(transactionManager, platformTransactionManager); + // In this case, the supplied transaction manager won't be used by batch and a ResourcelessTransactionManager will be used instead. + // The user has to provide a custom BatchConfigurer. + Assert.assertTrue(getTransactionManagerSetOnJobRepository(applicationContext.getBean(JobRepository.class)) instanceof ResourcelessTransactionManager); + } + + @Test + public void testConfigurationWithDataSourceAndNoTransactionManager() throws Exception { + ApplicationContext applicationContext = new AnnotationConfigApplicationContext(BatchConfigurationWithDataSourceAndNoTransactionManager.class); + Assert.assertTrue(applicationContext.containsBean("transactionManager")); + PlatformTransactionManager platformTransactionManager = applicationContext.getBean(PlatformTransactionManager.class); + Object targetObject = AopTestUtils.getTargetObject(platformTransactionManager); + Assert.assertTrue(targetObject instanceof DataSourceTransactionManager); + DataSourceTransactionManager dataSourceTransactionManager = (DataSourceTransactionManager) targetObject; + Assert.assertEquals(applicationContext.getBean(DataSource.class), dataSourceTransactionManager.getDataSource()); + Assert.assertSame(getTransactionManagerSetOnJobRepository(applicationContext.getBean(JobRepository.class)), dataSourceTransactionManager); + } + + @Test + public void testConfigurationWithDataSourceAndOneTransactionManager() throws Exception { + ApplicationContext applicationContext = new AnnotationConfigApplicationContext(BatchConfigurationWithDataSourceAndOneTransactionManager.class); + PlatformTransactionManager platformTransactionManager = applicationContext.getBean(PlatformTransactionManager.class); + Assert.assertSame(transactionManager, platformTransactionManager); + // In this case, the supplied transaction manager won't be used by batch and a DataSourceTransactionManager will be used instead. + // The user has to provide a custom BatchConfigurer. + Assert.assertTrue(getTransactionManagerSetOnJobRepository(applicationContext.getBean(JobRepository.class)) instanceof DataSourceTransactionManager); + } + + @Test + public void testConfigurationWithDataSourceAndMultipleTransactionManagers() throws Exception { + ApplicationContext applicationContext = new AnnotationConfigApplicationContext(BatchConfigurationWithDataSourceAndMultipleTransactionManagers.class); + PlatformTransactionManager platformTransactionManager = applicationContext.getBean(PlatformTransactionManager.class); + Assert.assertSame(transactionManager2, platformTransactionManager); + // In this case, the supplied primary transaction manager won't be used by batch and a DataSourceTransactionManager will be used instead. + // The user has to provide a custom BatchConfigurer. + Assert.assertTrue(getTransactionManagerSetOnJobRepository(applicationContext.getBean(JobRepository.class)) instanceof DataSourceTransactionManager); + + } + + @EnableBatchProcessing + public static class BatchConfigurationWithNoDataSourceAndNoTransactionManager { + + } + + @EnableBatchProcessing + public static class BatchConfigurationWithNoDataSourceAndTransactionManager { + + @Bean + public PlatformTransactionManager transactionManager() { + return transactionManager; + } + } + + @EnableBatchProcessing + public static class BatchConfigurationWithDataSourceAndNoTransactionManager { + + @Bean + public DataSource dataSource() { + return createDataSource(); + } + } + + @EnableBatchProcessing + public static class BatchConfigurationWithDataSourceAndOneTransactionManager { + + @Bean + public DataSource dataSource() { + return createDataSource(); + } + + @Bean + public PlatformTransactionManager transactionManager() { + return transactionManager; + } + } + + @EnableBatchProcessing + public static class BatchConfigurationWithDataSourceAndMultipleTransactionManagers { + + @Bean + public DataSource dataSource() { + return createDataSource(); + } + + @Bean + public PlatformTransactionManager transactionManager() { + return transactionManager; + } + + @Primary + @Bean + public PlatformTransactionManager transactionManager2() { + return transactionManager2; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/ApplicationContextJobFactoryTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/ApplicationContextJobFactoryTests.java index 2b7b8960c9..784e8b3630 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/ApplicationContextJobFactoryTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/ApplicationContextJobFactoryTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.support; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarContextTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarContextTests.java index 0c9b3ea2c9..54c715cc95 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarContextTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarContextTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.support; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarTests.java index 69aaf61a56..65d7fe7be2 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.support; import static org.junit.Assert.assertEquals; @@ -10,11 +25,10 @@ import org.junit.Before; import org.junit.Test; +import org.mockito.Mockito; import org.springframework.batch.core.Job; import org.springframework.beans.factory.BeanCreationException; import org.springframework.context.ApplicationContext; -import org.springframework.context.event.ContextClosedEvent; -import org.springframework.context.event.ContextRefreshedEvent; import org.springframework.context.support.ClassPathXmlApplicationContext; import org.springframework.context.support.GenericApplicationContext; import org.springframework.core.Ordered; @@ -25,6 +39,7 @@ * * @author Dave Syer * @author Lucas Ward + * @author Mahmoud Ben Hassine * */ public class AutomaticJobRegistrarTests { @@ -40,6 +55,7 @@ public void setUp() { registrar.setJobLoader(jobLoader); } + @SuppressWarnings("cast") @Test public void testOrderedImplemented() throws Exception { @@ -50,6 +66,20 @@ public void testOrderedImplemented() throws Exception { } + @Test + public void testDefaultAutoStartup() throws Exception { + + assertTrue(registrar.isAutoStartup()); + + } + + @Test + public void testDefaultPhase() throws Exception { + + assertEquals(Integer.MIN_VALUE + 1000, registrar.getPhase()); + + } + @Test public void testLocateJob() throws Exception { @@ -57,6 +87,7 @@ public void testLocateJob() throws Exception { new ClassPathResource("org/springframework/batch/core/launch/support/job.xml"), new ClassPathResource("org/springframework/batch/core/launch/support/job2.xml") }; + @SuppressWarnings("resource") GenericApplicationContext applicationContext = new GenericApplicationContext(); applicationContext.refresh(); setUpApplicationContextFactories(jobPaths, applicationContext); @@ -79,6 +110,7 @@ public void testNoJobFound() throws Exception { Resource[] jobPaths = new Resource[] { new ClassPathResource( "org/springframework/batch/core/launch/support/test-environment.xml") }; + @SuppressWarnings("resource") GenericApplicationContext applicationContext = new GenericApplicationContext(); applicationContext.refresh(); setUpApplicationContextFactories(jobPaths, applicationContext); @@ -91,6 +123,7 @@ public void testDuplicateJobsInFile() throws Exception { Resource[] jobPaths = new Resource[] { new ClassPathResource( "org/springframework/batch/core/launch/support/2jobs.xml") }; + @SuppressWarnings("resource") GenericApplicationContext applicationContext = new GenericApplicationContext(); applicationContext.refresh(); setUpApplicationContextFactories(jobPaths, applicationContext); @@ -104,6 +137,7 @@ public void testChildContextOverridesBeanPostProcessor() throws Exception { Resource[] jobPaths = new Resource[] { new ClassPathResource( "org/springframework/batch/core/launch/support/2jobs.xml") }; + @SuppressWarnings("resource") ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext( "/org/springframework/batch/core/launch/support/test-environment-with-registry-and-auto-register.xml"); registrar.setApplicationContext(applicationContext); @@ -157,38 +191,25 @@ public void testStartStopRunning() throws Exception { } @Test - public void testInitCalledOnContextRefreshed() throws Exception { - - Resource[] jobPaths = new Resource[] { new ClassPathResource( - "org/springframework/batch/core/launch/support/2jobs.xml") }; - registrar.setApplicationContext(new ClassPathXmlApplicationContext( - "/org/springframework/batch/core/launch/support/test-environment-with-registry-and-auto-register.xml")); - GenericApplicationContext applicationContext = new GenericApplicationContext(); - applicationContext.refresh(); - setUpApplicationContextFactories(jobPaths, applicationContext); - registrar.setApplicationContext(applicationContext); - registrar.onApplicationEvent(new ContextRefreshedEvent(applicationContext)); - assertEquals(2, registry.getJobNames().size()); - } - - @Test - public void testClearCalledOnContextClosed() throws Exception { + public void testStartStopRunningWithCallback() throws Exception { + Runnable callback = Mockito.mock(Runnable.class); Resource[] jobPaths = new Resource[] { new ClassPathResource( "org/springframework/batch/core/launch/support/2jobs.xml") }; - GenericApplicationContext applicationContext = new GenericApplicationContext(); - applicationContext.refresh(); - setUpApplicationContextFactories(jobPaths, applicationContext); - registrar.setApplicationContext(applicationContext); + setUpApplicationContextFactories(jobPaths, null); + registrar.start(); + assertTrue(registrar.isRunning()); registrar.start(); assertEquals(2, registry.getJobNames().size()); - registrar.onApplicationEvent(new ContextClosedEvent(applicationContext)); + registrar.stop(callback); + assertFalse(registrar.isRunning()); assertEquals(0, registry.getJobNames().size()); + Mockito.verify(callback, Mockito.times(1)).run(); } private void setUpApplicationContextFactories(Resource[] jobPaths, ApplicationContext parent) { - Collection applicationContextFactories = new ArrayList(); + Collection applicationContextFactories = new ArrayList<>(); for (Resource resource : jobPaths) { GenericApplicationContextFactory factory = new GenericApplicationContextFactory(resource); factory.setApplicationContext(parent); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/DefaultJobLoaderTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/DefaultJobLoaderTests.java index 3d1538f52b..04b6126e76 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/DefaultJobLoaderTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/DefaultJobLoaderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2010 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ import java.util.Collection; import java.util.Collections; +import java.util.Map; import org.junit.Test; import org.springframework.batch.core.Job; @@ -28,7 +29,6 @@ import org.springframework.batch.core.JobParametersValidator; import org.springframework.batch.core.Step; import org.springframework.batch.core.configuration.DuplicateJobException; -import org.springframework.batch.core.configuration.JobFactory; import org.springframework.batch.core.configuration.JobRegistry; import org.springframework.batch.core.configuration.StepRegistry; import org.springframework.batch.core.launch.NoSuchJobException; @@ -36,10 +36,13 @@ import org.springframework.batch.core.step.StepLocator; import org.springframework.core.io.ByteArrayResource; import org.springframework.core.io.ClassPathResource; +import org.springframework.lang.Nullable; +import org.springframework.test.util.ReflectionTestUtils; /** * @author Dave Syer * @author Stephane Nicoll + * @author Mahmoud Ben Hassine */ public class DefaultJobLoaderTests { @@ -58,6 +61,18 @@ public class DefaultJobLoaderTests { private DefaultJobLoader jobLoader = new DefaultJobLoader(jobRegistry, stepRegistry); + @Test + public void testClear() throws Exception { + GenericApplicationContextFactory factory = new GenericApplicationContextFactory(new ByteArrayResource( + JOB_XML.getBytes())); + jobLoader.load(factory); + assertEquals(1, ((Map) ReflectionTestUtils.getField(jobLoader, "contexts")).size()); + assertEquals(1, ((Map) ReflectionTestUtils.getField(jobLoader, "contextToJobNames")).size()); + jobLoader.clear(); + assertEquals(0, ((Map) ReflectionTestUtils.getField(jobLoader, "contexts")).size()); + assertEquals(0, ((Map) ReflectionTestUtils.getField(jobLoader, "contextToJobNames")).size()); + } + @Test public void testLoadWithExplicitName() throws Exception { GenericApplicationContextFactory factory = new GenericApplicationContextFactory(new ByteArrayResource( @@ -208,13 +223,13 @@ protected void assertStepDoNotExist(String jobName, String... stepNames) { private static final String BASIC_JOB_XML = String .format( "", + + "xsi:schemaLocation='http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-2.5.xsd'>", DefaultJobLoaderTests.class.getName()); private static final String JOB_XML = String .format( "", + + "xsi:schemaLocation='http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-2.5.xsd'>", DefaultJobLoaderTests.class.getName()); public static class BasicStubJob implements Job { @@ -223,7 +238,8 @@ public static class BasicStubJob implements Job { public void execute(JobExecution execution) { } - @Override + @Nullable + @Override public JobParametersIncrementer getJobParametersIncrementer() { return null; } @@ -251,31 +267,10 @@ public Collection getStepNames() { return Collections.emptyList(); } - @Override + @Override public Step getStep(String stepName) throws NoSuchStepException { throw new NoSuchStepException("Step [" + stepName + "] does not exist"); } } - private static class JobRegistryMock implements JobRegistry { - @Override - public void register(JobFactory jobFactory) throws DuplicateJobException { - // dummy - } - - @Override - public void unregister(String jobName) { - // dummy - } - - @Override - public Collection getJobNames() { - return Collections.emptyList(); - } - - @Override - public Job getJob(String name) throws NoSuchJobException { - throw new NoSuchJobException("Mock implementation does not hold any job."); - } - } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/GenericApplicationContextFactoryTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/GenericApplicationContextFactoryTests.java index 922c575a63..954c4427ee 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/GenericApplicationContextFactoryTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/GenericApplicationContextFactoryTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,16 +20,30 @@ import static org.junit.Assert.assertTrue; import org.junit.Test; + import org.springframework.batch.core.Job; +import org.springframework.batch.core.job.JobSupport; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor; import org.springframework.beans.factory.config.BeanFactoryPostProcessor; +import org.springframework.beans.factory.config.BeanPostProcessor; +import org.springframework.beans.factory.support.AbstractBeanFactory; +import org.springframework.context.ApplicationContext; +import org.springframework.context.ApplicationContextAware; import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.EnvironmentAware; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; import org.springframework.context.support.ClassPathXmlApplicationContext; +import org.springframework.core.env.Environment; import org.springframework.core.io.ClassPathResource; import org.springframework.core.io.Resource; import org.springframework.util.ClassUtils; /** * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public class GenericApplicationContextFactoryTests { @@ -38,6 +52,7 @@ public class GenericApplicationContextFactoryTests { public void testCreateJob() { GenericApplicationContextFactory factory = new GenericApplicationContextFactory( new ClassPathResource(ClassUtils.addResourcePathToPackagePath(getClass(), "trivial-context.xml"))); + @SuppressWarnings("resource") ConfigurableApplicationContext context = factory.createApplicationContext(); assertNotNull(context); assertTrue("Wrong id: " + context, context.getId().contains("trivial-context.xml")); @@ -50,6 +65,7 @@ public void testGetJobName() { assertEquals("test-job", factory.createApplicationContext().getBeanNamesForType(Job.class)[0]); } + @SuppressWarnings("resource") @Test public void testParentConfigurationInherited() { GenericApplicationContextFactory factory = new GenericApplicationContextFactory( @@ -62,6 +78,7 @@ public void testParentConfigurationInherited() { assertEquals(4, context.getBean("foo", Foo.class).values[1], 0.01); } + @SuppressWarnings("resource") @Test public void testBeanFactoryPostProcessorOrderRespected() { GenericApplicationContextFactory factory = new GenericApplicationContextFactory( @@ -77,15 +94,18 @@ public void testBeanFactoryPostProcessorOrderRespected() { public void testBeanFactoryProfileRespected() { GenericApplicationContextFactory factory = new GenericApplicationContextFactory( new ClassPathResource(ClassUtils.addResourcePathToPackagePath(getClass(), "profiles.xml"))); + @SuppressWarnings("resource") ClassPathXmlApplicationContext parentContext = new ClassPathXmlApplicationContext(ClassUtils.addResourcePathToPackagePath( getClass(), "parent-context.xml")); parentContext.getEnvironment().setActiveProfiles("preferred"); factory.setApplicationContext(parentContext); + @SuppressWarnings("resource") ConfigurableApplicationContext context = factory.createApplicationContext(); assertEquals("test-job", context.getBeanNamesForType(Job.class)[0]); assertEquals("spam", context.getBean("test-job", Job.class).getName()); } + @SuppressWarnings("resource") @Test public void testBeanFactoryPostProcessorsNotCopied() { GenericApplicationContextFactory factory = new GenericApplicationContextFactory( @@ -101,6 +121,7 @@ public void testBeanFactoryPostProcessorsNotCopied() { assertEquals(4, context.getBean("foo", Foo.class).values[1], 0.01); } + @SuppressWarnings("resource") @Test public void testBeanFactoryConfigurationNotCopied() { GenericApplicationContextFactory factory = new GenericApplicationContextFactory(new ClassPathResource(ClassUtils.addResourcePathToPackagePath(getClass(), @@ -125,6 +146,83 @@ public void testEquals() throws Exception { assertEquals(other, factory); assertEquals(other.hashCode(), factory.hashCode()); } + + @Test + public void testEqualsMultipleConfigs() throws Exception { + Resource resource1 = new ClassPathResource(ClassUtils.addResourcePathToPackagePath(getClass(), + "abstract-context.xml")); + Resource resource2 = new ClassPathResource(ClassUtils.addResourcePathToPackagePath(getClass(), + "child-context-with-abstract-job.xml")); + GenericApplicationContextFactory factory = new GenericApplicationContextFactory(resource1, resource2); + GenericApplicationContextFactory other = new GenericApplicationContextFactory(resource1, resource2); + assertEquals(other, factory); + assertEquals(other.hashCode(), factory.hashCode()); + } + + @Test + public void testParentConfigurationInheritedMultipleConfigs() { + Resource resource1 = new ClassPathResource(ClassUtils.addResourcePathToPackagePath(getClass(), + "abstract-context.xml")); + Resource resource2 = new ClassPathResource(ClassUtils.addResourcePathToPackagePath(getClass(), + "child-context-with-abstract-job.xml")); + GenericApplicationContextFactory factory = new GenericApplicationContextFactory(resource1, resource2); + ConfigurableApplicationContext context = factory.createApplicationContext(); + assertEquals("concrete-job", context.getBeanNamesForType(Job.class)[0]); + assertEquals("bar", context.getBean("concrete-job", Job.class).getName()); + assertEquals(4, context.getBean("foo", Foo.class).values[1], 0.01); + assertNotNull(context.getBean("concrete-job", JobSupport.class).getStep("step31")); + assertNotNull(context.getBean("concrete-job", JobSupport.class).getStep("step32")); + boolean autowiredFound = false; + for (BeanPostProcessor postProcessor : ((AbstractBeanFactory) context.getBeanFactory()).getBeanPostProcessors()) { + if (postProcessor instanceof AutowiredAnnotationBeanPostProcessor) { + autowiredFound = true; + } + } + assertTrue(autowiredFound); + } + + @Test(expected = IllegalArgumentException.class) + public void testDifferentResourceTypes() throws Exception { + Resource resource1 = new ClassPathResource(ClassUtils.addResourcePathToPackagePath(getClass(), + "abstract-context.xml")); + GenericApplicationContextFactory factory = new GenericApplicationContextFactory(resource1, Configuration1.class); + factory.createApplicationContext(); + } + + @Test + public void testPackageScanning() throws Exception { + GenericApplicationContextFactory factory = new GenericApplicationContextFactory("org.springframework.batch.core.configuration.support"); + ConfigurableApplicationContext context = factory.createApplicationContext(); + + assertEquals(context.getBean("bean1"), "bean1"); + assertEquals(context.getBean("bean2"), "bean2"); + assertEquals(context.getBean("bean3"), "bean3"); + assertEquals(context.getBean("bean4"), "bean4"); + } + + @Test + public void testMultipleConfigurationClasses() throws Exception { + GenericApplicationContextFactory factory = new GenericApplicationContextFactory(Configuration1.class, Configuration2.class); + ConfigurableApplicationContext context = factory.createApplicationContext(); + + assertEquals(context.getBean("bean1"), "bean1"); + assertEquals(context.getBean("bean2"), "bean2"); + assertEquals(context.getBean("bean3"), "bean3"); + assertEquals(context.getBean("bean4"), "bean4"); + } + + @Test + public void testParentChildLifecycleEvents() throws InterruptedException { + AnnotationConfigApplicationContext parent = new AnnotationConfigApplicationContext(ParentContext.class); + GenericApplicationContextFactory child = new GenericApplicationContextFactory(ChildContextConfiguration.class); + child.setApplicationContext(parent); + ApplicationContext context = child.createApplicationContext(); + ChildBean bean = context.getBean(ChildBean.class); + assertEquals(1, bean.counter1); + assertEquals(1, bean.counter2); + } + + public static class Foo { private double[] values; @@ -134,4 +232,64 @@ public void setValues(double[] values) { } } + @Configuration + public static class Configuration1 { + @Bean + public String bean1() { + return "bean1"; + } + + @Bean + public String bean2() { + return "bean2"; + } + } + + @Configuration + public static class Configuration2 { + @Bean + public String bean3() { + return "bean3"; + } + + @Bean + public String bean4() { + return "bean4"; + } + } + + @Configuration + public static class ParentContext implements ApplicationContextAware { + @Override + public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { + } + } + + + @Configuration + public static class ChildContextConfiguration { + @Bean + public ChildBean childBean() { + return new ChildBean(); + } + } + + public static class ChildBean implements ApplicationContextAware, EnvironmentAware { + + private int counter1 = 0; + + private int counter2 = 0; + + @Override + public void setEnvironment(Environment environment) { + counter2++; + } + + @Override + public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { + counter1++; + } + } + + } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/GroupAwareJobTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/GroupAwareJobTests.java index 682a96938a..e68f183b35 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/GroupAwareJobTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/GroupAwareJobTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/JobFactoryRegistrationListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/JobFactoryRegistrationListenerTests.java index 1f65904862..55cbabf293 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/JobFactoryRegistrationListenerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/JobFactoryRegistrationListenerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/JobRegistryBeanPostProcessorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/JobRegistryBeanPostProcessorTests.java index 9e9d9d9344..4af662ea94 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/JobRegistryBeanPostProcessorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/JobRegistryBeanPostProcessorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -110,6 +110,7 @@ public void testUnregisterOnDestroy() throws Exception { } @Test + @SuppressWarnings("resource") public void testExecutionWithApplicationContext() throws Exception { ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext("test-context.xml", getClass()); MapJobRegistry registry = (MapJobRegistry) context.getBean("registry"); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/JobRegistryIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/JobRegistryIntegrationTests.java index a41cc93e59..b1285f448d 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/JobRegistryIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/JobRegistryIntegrationTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/MapJobRegistryTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/MapJobRegistryTests.java index 8e99211b35..4921c32dbd 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/MapJobRegistryTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/MapJobRegistryTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/MapStepRegistryTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/MapStepRegistryTests.java index fefa1a3dab..16c9e8dbdf 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/MapStepRegistryTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/MapStepRegistryTests.java @@ -1,5 +1,26 @@ +/* + * Copyright 2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.support; +import static org.junit.Assert.fail; + +import java.util.Arrays; +import java.util.Collection; +import java.util.HashSet; + import org.junit.Assert; import org.junit.Test; import org.springframework.batch.core.Step; @@ -9,12 +30,6 @@ import org.springframework.batch.core.step.NoSuchStepException; import org.springframework.batch.core.step.tasklet.TaskletStep; -import java.util.Arrays; -import java.util.Collection; -import java.util.HashSet; - -import static junit.framework.Assert.fail; - /** * @author Sebastien Gerard */ @@ -34,7 +49,7 @@ public void registerStepNullJobName() throws DuplicateJobException { final StepRegistry stepRegistry = createRegistry(); try { - stepRegistry.register(null, new HashSet()); + stepRegistry.register(null, new HashSet<>()); Assert.fail(EXCEPTION_NOT_THROWN_MSG); } catch (IllegalArgumentException e) { } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/OsgiBundleXmlApplicationContextFactoryTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/OsgiBundleXmlApplicationContextFactoryTests.java deleted file mode 100644 index 418b6fe2da..0000000000 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/OsgiBundleXmlApplicationContextFactoryTests.java +++ /dev/null @@ -1,66 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.core.configuration.support; - -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.junit.Assert.assertEquals; - -import org.junit.Test; -import org.osgi.framework.Bundle; -import org.osgi.framework.BundleContext; -import org.springframework.util.ClassUtils; - -/** - * @author Dave Syer - * @author Will Schipp - * - */ -public class OsgiBundleXmlApplicationContextFactoryTests { - - private OsgiBundleXmlApplicationContextFactory factory = new OsgiBundleXmlApplicationContextFactory(); - - /** - * Test method for {@link org.springframework.batch.core.configuration.support.OsgiBundleXmlApplicationContextFactory#setDisplayName(java.lang.String)}. - */ - @Test - public void testSetDisplayName() { - factory.setDisplayName("foo"); - factory.setPath("classpath:"+ClassUtils.addResourcePathToPackagePath(getClass(), "trivial-context.xml")); - BundleContext bundleContext = mock(BundleContext.class); - Bundle bundle = mock(Bundle.class); - when(bundleContext.getBundle()).thenReturn(bundle); - factory.setBundleContext(bundleContext); - // factory.createApplicationContext(); - } - - @Test - public void testEquals() throws Exception { - factory.setPath("child-context.xml"); - OsgiBundleXmlApplicationContextFactory other = new OsgiBundleXmlApplicationContextFactory(); - other.setPath("child-context.xml"); - assertEquals(other, factory); - assertEquals(other.hashCode(), factory.hashCode()); - } - - /** - * Test method for {@link org.springframework.batch.core.configuration.support.OsgiBundleXmlApplicationContextFactory#setApplicationContext(org.springframework.context.ApplicationContext)}. - */ - @Test - public void testSetApplicationContext() { - } - -} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/ReferenceJobFactoryTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/ReferenceJobFactoryTests.java index 9739ec7537..88bfeefaf1 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/ReferenceJobFactoryTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/support/ReferenceJobFactoryTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AbstractJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AbstractJobParserTests.java index 42a1aeea66..ac600c4999 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AbstractJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AbstractJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,14 +15,13 @@ */ package org.springframework.batch.core.configuration.xml; -import static org.junit.Assert.fail; - import java.util.ArrayList; import org.junit.Before; + import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; -import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; @@ -31,6 +30,8 @@ import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; import org.springframework.beans.factory.annotation.Autowired; +import static org.junit.Assert.fail; + /** * @author Dan Garrette * @since 2.0 @@ -42,12 +43,12 @@ public abstract class AbstractJobParserTests { @Autowired private JobRepository jobRepository; - + @Autowired private MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean; @Autowired - protected ArrayList stepNamesList = new ArrayList(); + protected ArrayList stepNamesList = new ArrayList<>(); @Before public void setUp() { @@ -59,15 +60,10 @@ public void setUp() { * @return JobExecution */ protected JobExecution createJobExecution() throws JobInstanceAlreadyCompleteException, JobRestartException, - JobExecutionAlreadyRunningException { - return jobRepository.createJobExecution(job.getName(), new JobParameters()); + JobExecutionAlreadyRunningException { + return jobRepository.createJobExecution(job.getName(), new JobParametersBuilder().addLong("key1", 1L).toJobParameters()); } - /** - * @param jobExecution - * @param stepName - * @return the StepExecution corresponding to the specified step - */ protected StepExecution getStepExecution(JobExecution jobExecution, String stepName) { for (StepExecution stepExecution : jobExecution.getStepExecutions()) { if (stepExecution.getStepName().equals(stepName)) { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AbstractTestComponent.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AbstractTestComponent.java index f8c82cc8b4..d4f0bf307b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AbstractTestComponent.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AbstractTestComponent.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; public abstract class AbstractTestComponent { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeTests.java new file mode 100644 index 0000000000..c8b334491c --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeTests.java @@ -0,0 +1,54 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.configuration.xml; + +import static org.junit.Assert.assertTrue; + +import java.util.Map; + +import org.junit.Test; +import org.springframework.batch.core.scope.JobScope; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.support.ClassPathXmlApplicationContext; + + +/** + * @author Thomas Risberg + * @author Jimmy Praet + */ +public class AutoRegisteringJobScopeTests { + + @Test + @SuppressWarnings("resource") + public void testJobElement() throws Exception { + ConfigurableApplicationContext ctx = + new ClassPathXmlApplicationContext( + "org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeForJobElementTests-context.xml"); + Map beans = ctx.getBeansOfType(JobScope.class); + assertTrue("JobScope not defined properly", beans.size() == 1); + } + + @Test + @SuppressWarnings("resource") + public void testStepElement() throws Exception { + ConfigurableApplicationContext ctx = + new ClassPathXmlApplicationContext( + "org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeForStepElementTests-context.xml"); + Map beans = ctx.getBeansOfType(JobScope.class); + assertTrue("JobScope not defined properly", beans.size() == 1); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeTests.java index 216eefce0b..8fcf60c254 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,15 +15,15 @@ */ package org.springframework.batch.core.configuration.xml; -import static org.junit.Assert.assertTrue; - -import java.util.Map; - import org.junit.Test; import org.springframework.batch.core.scope.StepScope; import org.springframework.context.ConfigurableApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; +import java.util.Map; + +import static org.junit.Assert.assertTrue; + /** * @author Thomas Risberg @@ -31,19 +31,19 @@ public class AutoRegisteringStepScopeTests { @Test + @SuppressWarnings("resource") public void testJobElement() throws Exception { ConfigurableApplicationContext ctx = new ClassPathXmlApplicationContext("org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForJobElementTests-context.xml"); - @SuppressWarnings("unchecked") Map beans = ctx.getBeansOfType(StepScope.class); assertTrue("StepScope not defined properly", beans.size() == 1); } @Test + @SuppressWarnings("resource") public void testStepElement() throws Exception { ConfigurableApplicationContext ctx = new ClassPathXmlApplicationContext("org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForStepElementTests-context.xml"); - @SuppressWarnings("unchecked") Map beans = ctx.getBeansOfType(StepScope.class); assertTrue("StepScope not defined properly", beans.size() == 1); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/BeanDefinitionOverrideTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/BeanDefinitionOverrideTests.java new file mode 100644 index 0000000000..fc2292279d --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/BeanDefinitionOverrideTests.java @@ -0,0 +1,41 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.configuration.xml; + +import org.junit.Test; +import org.springframework.context.support.ClassPathXmlApplicationContext; + +/** + *

      + * Test cases for BATCH-1863. + *

      + */ +public class BeanDefinitionOverrideTests { + @Test + public void testAllowBeanOverride() { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setConfigLocation("org/springframework/batch/core/configuration/xml/BeanDefinitionOverrideTests-context.xml"); + applicationContext.refresh(); + } + + @Test + public void testAllowBeanOverrideFalse() { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setAllowBeanDefinitionOverriding(false); + applicationContext.setConfigLocation("org/springframework/batch/core/configuration/xml/BeanDefinitionOverrideTests-context.xml"); + applicationContext.refresh(); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/BranchStepJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/BranchStepJobParserTests.java index 3af7f26867..0cc093956b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/BranchStepJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/BranchStepJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -54,7 +54,7 @@ public void testBranchStep() throws Exception { job.execute(jobExecution); assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); assertEquals(2, jobExecution.getStepExecutions().size()); - List names = new ArrayList(); + List names = new ArrayList<>(); for (StepExecution stepExecution : jobExecution.getStepExecutions()) { names.add(stepExecution.getStepName()); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/ChunkElementParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/ChunkElementParserTests.java index e71b349e48..9386ad7d83 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/ChunkElementParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/ChunkElementParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,18 +15,13 @@ */ package org.springframework.batch.core.configuration.xml; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - import java.io.IOException; import java.util.Arrays; import java.util.Collection; import java.util.Map; import org.junit.Test; + import org.springframework.batch.core.Step; import org.springframework.batch.core.step.item.SimpleChunkProcessor; import org.springframework.batch.core.step.skip.SkipPolicy; @@ -50,6 +45,12 @@ import org.springframework.test.util.ReflectionTestUtils; import org.springframework.util.StringUtils; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + /** * @author Dan Garrette * @author Dave Syer @@ -58,6 +59,7 @@ public class ChunkElementParserTests { @Test + @SuppressWarnings("resource") public void testSimpleAttributes() throws Exception { ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ChunkElementSimpleAttributeParserTests-context.xml"); @@ -69,27 +71,30 @@ public void testSimpleAttributes() throws Exception { } @Test + @SuppressWarnings("resource") public void testCommitIntervalLateBinding() throws Exception { ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ChunkElementLateBindingParserTests-context.xml"); - Step step = (Step) context.getBean("s1", Step.class); + Step step = context.getBean("s1", Step.class); assertNotNull("Step not parsed", step); } @Test + @SuppressWarnings("resource") public void testSkipAndRetryAttributes() throws Exception { ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ChunkElementSkipAndRetryAttributeParserTests-context.xml"); - Step step = (Step) context.getBean("s1", Step.class); + Step step = context.getBean("s1", Step.class); assertNotNull("Step not parsed", step); } @Test + @SuppressWarnings("resource") public void testIllegalSkipAndRetryAttributes() throws Exception { try { ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ChunkElementIllegalSkipAndRetryAttributeParserTests-context.xml"); - Step step = (Step) context.getBean("s1", Step.class); + Step step = context.getBean("s1", Step.class); assertNotNull("Step not parsed", step); fail("Expected BeanCreationException"); } catch (BeanCreationException e) { @@ -99,6 +104,7 @@ public void testIllegalSkipAndRetryAttributes() throws Exception { @Test public void testRetryPolicyAttribute() throws Exception { + @SuppressWarnings("resource") ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ChunkElementRetryPolicyParserTests-context.xml"); Map, Boolean> retryable = getNestedExceptionMap("s1", context, @@ -111,6 +117,7 @@ public void testRetryPolicyAttribute() throws Exception { @Test public void testRetryPolicyElement() throws Exception { + @SuppressWarnings("resource") ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ChunkElementRetryPolicyParserTests-context.xml"); SimpleRetryPolicy policy = (SimpleRetryPolicy) getPolicy("s2", context, @@ -120,6 +127,7 @@ public void testRetryPolicyElement() throws Exception { @Test public void testSkipPolicyAttribute() throws Exception { + @SuppressWarnings("resource") ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ChunkElementSkipPolicyParserTests-context.xml"); SkipPolicy policy = getSkipPolicy("s1", context); @@ -129,6 +137,7 @@ public void testSkipPolicyAttribute() throws Exception { @Test public void testSkipPolicyElement() throws Exception { + @SuppressWarnings("resource") ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ChunkElementSkipPolicyParserTests-context.xml"); SkipPolicy policy = getSkipPolicy("s2", context); @@ -137,6 +146,7 @@ public void testSkipPolicyElement() throws Exception { } @Test + @SuppressWarnings("resource") public void testProcessorTransactionalAttributes() throws Exception { ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ChunkElementTransactionalAttributeParserTests-context.xml"); @@ -150,6 +160,7 @@ public void testProcessorTransactionalAttributes() throws Exception { } @Test + @SuppressWarnings("resource") public void testProcessorTransactionalNotAllowedOnSimpleProcessor() throws Exception { ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ChunkElementIllegalAttributeParserTests-context.xml"); @@ -328,7 +339,7 @@ private Object getNestedPathInStep(String stepName, ApplicationContext ctx, Stri /** * @param object the target object * @param path the path to the required field - * @return + * @return The field */ private Object getNestedPath(Object object, String path) { while (StringUtils.hasText(path)) { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DecisionJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DecisionJobParserTests.java index 329b901e86..70bab1020b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DecisionJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DecisionJobParserTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,6 +30,7 @@ import org.springframework.batch.core.repository.JobRepository; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.lang.Nullable; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @@ -59,7 +60,7 @@ public void testDecisionState() throws Exception { public static class TestDecider implements JobExecutionDecider { @Override - public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) { + public FlowExecutionStatus decide(JobExecution jobExecution, @Nullable StepExecution stepExecution) { return new FlowExecutionStatus("FOO"); } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DefaultFailureJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DefaultFailureJobParserTests.java index 822e0b7397..9a9770ac04 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DefaultFailureJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DefaultFailureJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DefaultSuccessJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DefaultSuccessJobParserTests.java index 4aafa2fbce..74e77cfdf0 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DefaultSuccessJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DefaultSuccessJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DefaultUnknownJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DefaultUnknownJobParserTests.java index 7c8d37d5b9..04f6b8a406 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DefaultUnknownJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DefaultUnknownJobParserTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,6 +24,7 @@ import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.listener.StepExecutionListenerSupport; +import org.springframework.lang.Nullable; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @@ -57,6 +58,7 @@ public void testDefaultUnknown() throws Exception { } public static class UnknownListener extends StepExecutionListenerSupport { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { stepExecution.setStatus(BatchStatus.UNKNOWN); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyAnnotationJobExecutionListener.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyAnnotationJobExecutionListener.java index 1acef2c0af..6c994a72ee 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyAnnotationJobExecutionListener.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyAnnotationJobExecutionListener.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.core.annotation.BeforeJob; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyAnnotationStepExecutionListener.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyAnnotationStepExecutionListener.java index a9574d9d93..c250fd3bd6 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyAnnotationStepExecutionListener.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyAnnotationStepExecutionListener.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.core.annotation.BeforeStep; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyCompletionPolicy.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyCompletionPolicy.java index e27f05073a..34035195a9 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyCompletionPolicy.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyCompletionPolicy.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.repeat.CompletionPolicy; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemHandlerAdapter.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemHandlerAdapter.java index 5c9ee2edf1..a0c3887f4b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemHandlerAdapter.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemHandlerAdapter.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; /** diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemProcessor.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemProcessor.java index c8fd3e1a81..b36c6de459 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemProcessor.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemProcessor.java @@ -1,6 +1,22 @@ +/* + * Copyright 2009-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.item.ItemProcessor; +import org.springframework.lang.Nullable; /** * @author Dave Syer @@ -8,6 +24,7 @@ */ public class DummyItemProcessor implements ItemProcessor { + @Nullable @Override public Object process(Object item) throws Exception { return item; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemReader.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemReader.java index 8b6b580efb..2db54248c1 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemReader.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemReader.java @@ -1,8 +1,24 @@ +/* + * Copyright 2009-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ParseException; import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.lang.Nullable; /** * @author Dan Garrette @@ -10,6 +26,7 @@ */ public class DummyItemReader implements ItemReader { + @Nullable @Override public Object read() throws Exception, UnexpectedInputException, ParseException { return null; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemWriter.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemWriter.java index 7374d52ffd..53b10ea64b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemWriter.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyItemWriter.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import java.util.List; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyJobRepository.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyJobRepository.java index 8a0f04aaa3..fd2ebb050e 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyJobRepository.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyJobRepository.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,6 +26,7 @@ import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.JobRestartException; import org.springframework.beans.factory.BeanNameAware; +import org.springframework.lang.Nullable; /** * @author Dan Garrette @@ -55,11 +56,13 @@ public JobExecution createJobExecution(String jobName, JobParameters jobParamete return null; } + @Nullable @Override public JobExecution getLastJobExecution(String jobName, JobParameters jobParameters) { return null; } + @Nullable @Override public StepExecution getLastStepExecution(JobInstance jobInstance, String stepName) { return null; @@ -95,4 +98,15 @@ public void updateExecutionContext(JobExecution jobExecution) { public void addAll(Collection stepExecutions) { } + @Override + public JobInstance createJobInstance(String jobName, + JobParameters jobParameters) { + return null; + } + + @Override + public JobExecution createJobExecution(JobInstance jobInstance, + JobParameters jobParameters, String jobConfigurationLocation) { + return null; + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyPlatformTransactionManager.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyPlatformTransactionManager.java index aeec5b5c24..7abe4530c8 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyPlatformTransactionManager.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyPlatformTransactionManager.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyPojoStepExecutionListener.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyPojoStepExecutionListener.java index f40ade7ae1..0caf866e74 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyPojoStepExecutionListener.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyPojoStepExecutionListener.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyRetryListener.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyRetryListener.java index e4220e5470..80dc4c04dd 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyRetryListener.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyRetryListener.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.retry.RetryCallback; @@ -11,16 +26,16 @@ public class DummyRetryListener implements RetryListener { @Override - public boolean open(RetryContext context, RetryCallback callback) { + public boolean open(RetryContext context, RetryCallback callback) { return false; } @Override - public void close(RetryContext context, RetryCallback callback, Throwable throwable) { + public void close(RetryContext context, RetryCallback callback, Throwable throwable) { } @Override - public void onError(RetryContext context, RetryCallback callback, Throwable throwable) { + public void onError(RetryContext context, RetryCallback callback, Throwable throwable) { } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyStep.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyStep.java index 95333c05c0..4b2c36b681 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyStep.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyStep.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyTasklet.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyTasklet.java index 232888dca4..aefe4480b6 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyTasklet.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DummyTasklet.java @@ -1,9 +1,25 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; /** * @author Dan Garrette @@ -11,6 +27,7 @@ */ public class DummyTasklet implements Tasklet { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { return null; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DuplicateTransitionJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DuplicateTransitionJobParserTests.java index 0f52093f1e..159b2838df 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DuplicateTransitionJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/DuplicateTransitionJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -28,12 +28,14 @@ public class DuplicateTransitionJobParserTests { @Test(expected = BeanDefinitionStoreException.class) + @SuppressWarnings("resource") public void testNextAttributeWithNestedElement() throws Exception { new ClassPathXmlApplicationContext(ClassUtils.addResourcePathToPackagePath(getClass(), "NextAttributeMultipleFinalJobParserTests-context.xml")); } @Test(expected = BeanDefinitionStoreException.class) + @SuppressWarnings("resource") public void testDuplicateTransition() throws Exception { new ClassPathXmlApplicationContext(ClassUtils.addResourcePathToPackagePath(getClass(), "DuplicateTransitionJobParserTests-context.xml")); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/EndTransitionDefaultStatusJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/EndTransitionDefaultStatusJobParserTests.java index 6eab69b15a..2cee9723f0 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/EndTransitionDefaultStatusJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/EndTransitionDefaultStatusJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/EndTransitionJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/EndTransitionJobParserTests.java index f46a2c90d0..ad03e74624 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/EndTransitionJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/EndTransitionJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FailTransitionDefaultStatusJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FailTransitionDefaultStatusJobParserTests.java index 0971c05171..48fa900714 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FailTransitionDefaultStatusJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FailTransitionDefaultStatusJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FailTransitionJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FailTransitionJobParserTests.java index e74b7bdab3..d6a80fb600 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FailTransitionJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FailTransitionJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FailingTasklet.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FailingTasklet.java index dd3fbad06b..cfd4140395 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FailingTasklet.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FailingTasklet.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,17 +18,19 @@ import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; /** * This tasklet will call * {@link NameStoringTasklet#execute(StepContribution, ChunkContext)} and then - * throw an exeception. + * throw an exception. * * @author Dan Garrette * @since 2.0 */ public class FailingTasklet extends NameStoringTasklet { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { super.execute(contribution, chunkContext); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FlowJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FlowJobParserTests.java index eca377b953..ad3d8ec22b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FlowJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FlowJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -118,7 +118,7 @@ public void testFlowInSplit() throws Exception { } private List getStepNames(JobExecution jobExecution) { - List list = new ArrayList(); + List list = new ArrayList<>(); for (StepExecution stepExecution : jobExecution.getStepExecutions()) { list.add(stepExecution.getStepName()); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FlowStepParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FlowStepParserTests.java index d0f598895f..9bc05c7e73 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FlowStepParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/FlowStepParserTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,6 +35,7 @@ import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.lang.Nullable; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @@ -126,7 +127,7 @@ public void testRestartedFlow() throws Exception { } private List getStepNames(JobExecution jobExecution) { - List list = new ArrayList(); + List list = new ArrayList<>(); for (StepExecution stepExecution : jobExecution.getStepExecutions()) { list.add(stepExecution.getStepName()); } @@ -138,7 +139,7 @@ public static class Decider implements JobExecutionDecider { int count = 0; @Override - public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) { + public FlowExecutionStatus decide(JobExecution jobExecution, @Nullable StepExecution stepExecution) { if (count++ < 2) { return new FlowExecutionStatus("OK"); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/InlineItemHandlerParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/InlineItemHandlerParserTests.java index a5a5c6a7e0..b24ee8a29a 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/InlineItemHandlerParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/InlineItemHandlerParserTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import static org.junit.Assert.assertEquals; @@ -88,11 +103,10 @@ public void testInlineHandlersWithStepScope() throws Exception { "org/springframework/batch/core/configuration/xml/InlineItemHandlerWithStepScopeParserTests-context.xml"); StepSynchronizationManager.register(new StepExecution("step", new JobExecution(123L))); - @SuppressWarnings({ "unchecked", "rawtypes" }) + @SuppressWarnings({ "rawtypes" }) Map readers = context.getBeansOfType(ItemReader.class); // Should be 2 each (proxy and target) for the two readers in the steps defined assertEquals(4, readers.size()); - // System.err.println(readers); } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/InterruptibleTasklet.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/InterruptibleTasklet.java index 444d7f865e..aac3cf5704 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/InterruptibleTasklet.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/InterruptibleTasklet.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,6 +18,7 @@ import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; /** * This tasklet will call @@ -31,6 +32,7 @@ public class InterruptibleTasklet extends NameStoringTasklet { private volatile boolean started = false; + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { if (!started) { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobExecutionListenerMethodAttributeParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobExecutionListenerMethodAttributeParserTests.java index 9918bed9b8..ad5f597205 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobExecutionListenerMethodAttributeParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobExecutionListenerMethodAttributeParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,7 +15,7 @@ */ package org.springframework.batch.core.configuration.xml; -import static junit.framework.Assert.assertTrue; +import static org.junit.Assert.assertTrue; import org.junit.Test; import org.junit.runner.RunWith; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobExecutionListenerParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobExecutionListenerParserTests.java index 76b1cc250b..23971c56d6 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobExecutionListenerParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobExecutionListenerParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,7 +15,7 @@ */ package org.springframework.batch.core.configuration.xml; -import static junit.framework.Assert.assertTrue; +import static org.junit.Assert.assertTrue; import org.junit.Test; import org.junit.runner.RunWith; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserExceptionTests.java index 76761778fe..e7ce1ac0ef 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserExceptionTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import static org.junit.Assert.assertTrue; @@ -58,7 +73,7 @@ public void testWrongSchemaInRoot() { } catch (BeanDefinitionParsingException e) { String message = e.getMessage(); - assertTrue("Wrong message: "+message, message.matches("(?s).*You cannot use spring-batch-2.0.xsd.*")); + assertTrue("Wrong message: "+message, message.startsWith("Configuration problem: You are using a version of the spring-batch XSD")); } catch (BeanDefinitionStoreException e) { // Probably the internet is not available and the schema validation failed. fail("Wrong exception when schema didn't match: " + e.getMessage()); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserJobFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserJobFactoryBeanTests.java index 03b780d696..404c5a6fc9 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserJobFactoryBeanTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserJobFactoryBeanTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import static org.junit.Assert.*; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserParentAttributeTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserParentAttributeTests.java index e5bb8a9608..938c05eb8a 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserParentAttributeTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserParentAttributeTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -150,7 +150,7 @@ private List getListeners(Job job) throws Exception { Object composite = ReflectionTestUtils.getField(compositeListener, "listeners"); List list = (List) ReflectionTestUtils.getField(composite, "list"); - List listeners = new ArrayList(); + List listeners = new ArrayList<>(); for (Object listener : list) { while (listener instanceof Advised) { listener = ((Advised) listener).getTargetSource().getTarget(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserValidatorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserValidatorTests.java index f0bd525245..73f936b4f9 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserValidatorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobParserValidatorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRegistryJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRegistryJobParserTests.java index 5be6ae14e0..1a6af802d9 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRegistryJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRegistryJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRepositoryDefaultParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRepositoryDefaultParserTests.java index 79ef3ecfe1..d844e75655 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRepositoryDefaultParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRepositoryDefaultParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRepositoryParserReferenceTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRepositoryParserReferenceTests.java index 20eadf2f5d..87ecfbed2f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRepositoryParserReferenceTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRepositoryParserReferenceTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRepositoryParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRepositoryParserTests.java index 3091585c84..d4dc67ec44 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRepositoryParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobRepositoryParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobStepParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobStepParserTests.java index 443b25dd69..108e58c128 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobStepParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/JobStepParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -87,7 +87,7 @@ public void testFlowExternalStep() throws Exception { } private List getStepNames(JobExecution jobExecution) { - List list = new ArrayList(); + List list = new ArrayList<>(); for (StepExecution stepExecution : jobExecution.getStepExecutions()) { list.add(stepExecution.getStepName()); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NameStoringTasklet.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NameStoringTasklet.java index 63861a7a3f..bdb00fb619 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NameStoringTasklet.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NameStoringTasklet.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,6 +23,7 @@ import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; /** * This class will store the step name when it is executed. @@ -40,6 +41,7 @@ public void beforeStep(StepExecution stepExecution) { stepName = stepExecution.getStepName(); } + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { if (stepNamesList != null) { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NamespacePrefixedJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NamespacePrefixedJobParserTests.java index bdf5221390..f92e0d3b3f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NamespacePrefixedJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NamespacePrefixedJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NextAttributeJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NextAttributeJobParserTests.java index 3da2b9d1d2..1ce3a3b241 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NextAttributeJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NextAttributeJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NextAttributeUnknownJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NextAttributeUnknownJobParserTests.java index f2932fa28b..52cce65960 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NextAttributeUnknownJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NextAttributeUnknownJobParserTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,6 +24,7 @@ import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.listener.StepExecutionListenerSupport; +import org.springframework.lang.Nullable; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @@ -57,6 +58,7 @@ public void testDefaultUnknown() throws Exception { } public static class UnknownListener extends StepExecutionListenerSupport { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { stepExecution.setStatus(BatchStatus.UNKNOWN); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NoopTasklet.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NoopTasklet.java index 55151ed0fb..2d6682c9f4 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NoopTasklet.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/NoopTasklet.java @@ -1,13 +1,30 @@ +/* + * Copyright 2010-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; public class NoopTasklet extends NameStoringTasklet { - @Override + @Nullable + @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { super.execute(contribution, chunkContext); contribution.setExitStatus(ExitStatus.NOOP); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/OneStepJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/OneStepJobParserTests.java index 7941f24e87..cc8a396a50 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/OneStepJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/OneStepJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/ParentStepFactoryBeanParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/ParentStepFactoryBeanParserTests.java index 648628a7bb..32ab2c9f67 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/ParentStepFactoryBeanParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/ParentStepFactoryBeanParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,6 +32,7 @@ public class ParentStepFactoryBeanParserTests { @Test + @SuppressWarnings("resource") public void testSimpleAttributes() throws Exception { ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ParentStepFactoryBeanParserTests-context.xml"); @@ -43,6 +44,7 @@ public void testSimpleAttributes() throws Exception { } @Test + @SuppressWarnings("resource") public void testSkippableAttributes() throws Exception { ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ParentSkippableStepFactoryBeanParserTests-context.xml"); @@ -54,6 +56,7 @@ public void testSkippableAttributes() throws Exception { } @Test + @SuppressWarnings("resource") public void testRetryableAttributes() throws Exception { ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ParentRetryableStepFactoryBeanParserTests-context.xml"); @@ -66,6 +69,7 @@ public void testRetryableAttributes() throws Exception { // BATCH-1396 @Test + @SuppressWarnings("resource") public void testRetryableLateBindingAttributes() throws Exception { ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ParentRetryableLateBindingStepFactoryBeanParserTests-context.xml"); @@ -78,6 +82,7 @@ public void testRetryableLateBindingAttributes() throws Exception { // BATCH-1396 @Test + @SuppressWarnings("resource") public void testSkippableLateBindingAttributes() throws Exception { ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/ParentSkippableLateBindingStepFactoryBeanParserTests-context.xml"); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepParserTests.java index 04a5da4ef0..bba6bdfed5 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepParserTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -53,6 +53,7 @@ /** * @author Dave Syer * @author Josh Long + * @author Mahmoud Ben Hassine */ @ContextConfiguration @RunWith(SpringJUnit4ClassRunner.class) @@ -90,7 +91,7 @@ public class PartitionStepParserTests implements ApplicationContextAware { private ApplicationContext applicationContext; - private List savedStepNames = new ArrayList(); + private List savedStepNames = new ArrayList<>(); @Override public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { @@ -143,8 +144,8 @@ public void testHandlerRefStep() throws Exception { /** * BATCH-1509 we now support the ability define steps inline for partitioned - * steps. this demonstates that the execution proceeds as expected and that - * the partitionhandler has a reference to the inline step definition + * steps. this demonstrates that the execution proceeds as expected and that + * the partition handler has a reference to the inline step definition */ @Test public void testNestedPartitionStepStepReference() throws Throwable { @@ -178,8 +179,8 @@ public void testNestedPartitionStepStepReference() throws Throwable { /** * BATCH-1509 we now support the ability define steps inline for partitioned - * steps. this demonstates that the execution proceeds as expected and that - * the partitionhandler has a reference to the inline step definition + * steps. this demonstrates that the execution proceeds as expected and that + * the partition handler has a reference to the inline step definition */ @Test public void testNestedPartitionStep() throws Throwable { @@ -205,7 +206,7 @@ public void testNestedPartitionStep() throws Throwable { } } assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); - // Step names not saved by this one (it geosn't have that tasklet) + // Step names not saved by this one (it doesn't have that tasklet) assertEquals("[]", savedStepNames.toString()); List stepNames = getStepNames(jobExecution); assertEquals(7, stepNames.size()); @@ -224,7 +225,7 @@ public void testCustomHandlerRefStep() throws Exception { } private List getStepNames(JobExecution jobExecution) { - List list = new ArrayList(); + List list = new ArrayList<>(); for (StepExecution stepExecution : jobExecution.getStepExecutions()) { list.add(stepExecution.getStepName()); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepWithFlowParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepWithFlowParserTests.java index 4252c3e145..33aa602350 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepWithFlowParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepWithFlowParserTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,6 +36,7 @@ import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.lang.Nullable; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @@ -61,7 +62,7 @@ public class PartitionStepWithFlowParserTests { @Autowired private MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean; - private List savedStepNames = new ArrayList(); + private List savedStepNames = new ArrayList<>(); @Before public void setUp() { @@ -84,7 +85,7 @@ public void testRepeatedFlowStep() throws Exception { } private List getStepNames(JobExecution jobExecution) { - List list = new ArrayList(); + List list = new ArrayList<>(); for (StepExecution stepExecution : jobExecution.getStepExecutions()) { list.add(stepExecution.getStepName()); } @@ -96,7 +97,7 @@ public static class Decider implements JobExecutionDecider { int count = 0; @Override - public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) { + public FlowExecutionStatus decide(JobExecution jobExecution, @Nullable StepExecution stepExecution) { if (count++<2) { return new FlowExecutionStatus("OK"); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepWithLateBindingParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepWithLateBindingParserTests.java index 88c1bd0774..efdd794199 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepWithLateBindingParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepWithLateBindingParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -59,7 +59,7 @@ public class PartitionStepWithLateBindingParserTests { @Autowired private MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean; - private List savedStepNames = new ArrayList(); + private List savedStepNames = new ArrayList<>(); @Before public void setUp() { @@ -82,7 +82,7 @@ public void testExplicitHandlerStep() throws Exception { } private List getStepNames(JobExecution jobExecution) { - List list = new ArrayList(); + List list = new ArrayList<>(); for (StepExecution stepExecution : jobExecution.getStepExecutions()) { list.add(stepExecution.getStepName()); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepWithNonDefaultTransactionManagerParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepWithNonDefaultTransactionManagerParserTests.java index 96e4ea7090..37c5b4355a 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepWithNonDefaultTransactionManagerParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/PartitionStepWithNonDefaultTransactionManagerParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/RepositoryJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/RepositoryJobParserTests.java index ebbdf264e2..561ff07dec 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/RepositoryJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/RepositoryJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailFirstJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailFirstJobParserTests.java index 5413bddc25..06c3465578 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailFirstJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailFirstJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailSecondJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailSecondJobParserTests.java index b1ade7c37f..d684bf2a33 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailSecondJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailSecondJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitInterruptedJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitInterruptedJobParserTests.java index c185e8ee40..8a144b2472 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitInterruptedJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitInterruptedJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitJobParserTests.java index 1f0d942be3..a348234978 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -57,7 +57,7 @@ public void testSplitJob() throws Exception { job.execute(jobExecution); assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); assertEquals(4, jobExecution.getStepExecutions().size()); - ArrayList names = new ArrayList(((StepLocator)job).getStepNames()); + ArrayList names = new ArrayList<>(((StepLocator) job).getStepNames()); Collections.sort(names); assertEquals("[s1, s2, s3, s4]", names.toString()); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitNestedJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitNestedJobParserTests.java index 13a30132e7..d27603e377 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitNestedJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/SplitNestedJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepListenerInStepParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepListenerInStepParserTests.java index 0b8ca6f4b3..c25948d85e 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepListenerInStepParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepListenerInStepParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -90,7 +90,7 @@ private List getListeners(Step step) throws Exception { Object composite = ReflectionTestUtils.getField(compositeListener, "list"); List proxiedListeners = (List) ReflectionTestUtils.getField( composite, "list"); - List r = new ArrayList(); + List r = new ArrayList<>(); for (Object listener : proxiedListeners) { while (listener instanceof Advised) { listener = ((Advised) listener).getTargetSource().getTarget(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepListenerMethodAttributeParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepListenerMethodAttributeParserTests.java index ce78943edb..e027cec58c 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepListenerMethodAttributeParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepListenerMethodAttributeParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -59,7 +59,7 @@ private List getListeners(Step step) throws Exception { Object composite = ReflectionTestUtils.getField(compositeListener, "list"); List proxiedListeners = (List) ReflectionTestUtils.getField( composite, "list"); - List r = new ArrayList(); + List r = new ArrayList<>(); for (Object listener : proxiedListeners) { while (listener instanceof Advised) { listener = ((Advised) listener).getTargetSource().getTarget(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepListenerParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepListenerParserTests.java index e996582df5..7fa7fcbb05 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepListenerParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepListenerParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -129,7 +129,7 @@ private List getListeners(Step step) throws Exception { Object composite = ReflectionTestUtils.getField(compositeListener, "list"); List proxiedListeners = (List) ReflectionTestUtils.getField( composite, "list"); - List r = new ArrayList(); + List r = new ArrayList<>(); for (Object listener : proxiedListeners) { while (listener instanceof Advised) { listener = ((Advised) listener).getTargetSource().getTarget(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepNameTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepNameTests.java index d9a8e53641..52281c0f65 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepNameTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepNameTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -41,7 +41,7 @@ @RunWith(Parameterized.class) public class StepNameTests { - private Map stepLocators = new HashMap(); + private Map stepLocators = new HashMap<>(); private ApplicationContext context; @@ -55,7 +55,6 @@ public StepNameTests(Resource resource) throws Exception { catch (BeanCreationException e) { return; } - @SuppressWarnings("unchecked") Map stepLocators = context.getBeansOfType(StepLocator.class); this.stepLocators = stepLocators; } @@ -78,7 +77,7 @@ public void testStepNames() throws Exception { @Parameters public static List data() throws Exception { - List list = new ArrayList(); + List list = new ArrayList<>(); ResourceArrayPropertyEditor editor = new ResourceArrayPropertyEditor(); editor.setAsText("classpath*:" + ClassUtils.addResourcePathToPackagePath(StepNameTests.class, "*.xml")); Resource[] resources = (Resource[]) editor.getValue(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepParserStepFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepParserStepFactoryBeanTests.java index 5678cb4f71..c7e047d770 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepParserStepFactoryBeanTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepParserStepFactoryBeanTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,13 +16,11 @@ package org.springframework.batch.core.configuration.xml; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; - import java.util.HashMap; import java.util.Map; import org.junit.Test; + import org.springframework.aop.framework.Advised; import org.springframework.aop.framework.ProxyFactory; import org.springframework.batch.core.StepListener; @@ -50,6 +48,10 @@ import org.springframework.transaction.annotation.Isolation; import org.springframework.transaction.annotation.Propagation; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + /** * @author Dan Garrette * @since 2.0 @@ -58,13 +60,13 @@ public class StepParserStepFactoryBeanTests { @Test(expected = StepBuilderException.class) public void testNothingSet() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.getObject(); } @Test public void testOnlyTaskletSet() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.setName("step"); fb.setTransactionManager(new ResourcelessTransactionManager()); fb.setJobRepository(new JobRepositorySupport()); @@ -77,7 +79,7 @@ public void testOnlyTaskletSet() throws Exception { @Test public void testOnlyTaskletTaskExecutor() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.setName("step"); fb.setTransactionManager(new ResourcelessTransactionManager()); fb.setJobRepository(new JobRepositorySupport()); @@ -91,7 +93,7 @@ public void testOnlyTaskletTaskExecutor() throws Exception { @Test(expected = StepBuilderException.class) public void testSkipLimitSet() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.setName("step"); fb.setSkipLimit(5); fb.getObject(); @@ -99,7 +101,7 @@ public void testSkipLimitSet() throws Exception { @Test public void testTaskletStepAll() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.setBeanName("step1"); fb.setAllowStartIfComplete(true); fb.setJobRepository(new JobRepositorySupport()); @@ -118,7 +120,7 @@ public void testTaskletStepAll() throws Exception { @Test public void testTaskletStepMissingIsolation() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.setBeanName("step1"); fb.setJobRepository(new JobRepositorySupport()); fb.setTasklet(new DummyTasklet()); @@ -132,7 +134,7 @@ public void testTaskletStepMissingIsolation() throws Exception { @Test(expected = IllegalStateException.class) public void testSimpleStepAll() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.setBeanName("step1"); fb.setAllowStartIfComplete(true); fb.setJobRepository(new JobRepositorySupport()); @@ -147,7 +149,7 @@ public void testSimpleStepAll() throws Exception { fb.setTaskExecutor(new SyncTaskExecutor()); fb.setItemReader(new DummyItemReader()); fb.setItemWriter(new DummyItemWriter()); - fb.setStreams(new ItemStream[] { new FlatFileItemReader() }); + fb.setStreams(new ItemStream[] {new FlatFileItemReader<>() }); fb.setHasChunkElement(true); Object step = fb.getObject(); @@ -158,7 +160,7 @@ public void testSimpleStepAll() throws Exception { @Test(expected = IllegalArgumentException.class) public void testFaultTolerantStepAll() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.setBeanName("step1"); fb.setAllowStartIfComplete(true); fb.setJobRepository(new JobRepositorySupport()); @@ -173,14 +175,14 @@ public void testFaultTolerantStepAll() throws Exception { fb.setTaskExecutor(new SyncTaskExecutor()); fb.setItemReader(new DummyItemReader()); fb.setItemWriter(new DummyItemWriter()); - fb.setStreams(new ItemStream[] { new FlatFileItemReader() }); + fb.setStreams(new ItemStream[] {new FlatFileItemReader<>() }); fb.setCacheCapacity(5); fb.setIsReaderTransactionalQueue(true); fb.setRetryLimit(5); fb.setSkipLimit(100); fb.setRetryListeners(new RetryListenerSupport()); - fb.setSkippableExceptionClasses(new HashMap, Boolean>()); - fb.setRetryableExceptionClasses(new HashMap, Boolean>()); + fb.setSkippableExceptionClasses(new HashMap<>()); + fb.setRetryableExceptionClasses(new HashMap<>()); fb.setHasChunkElement(true); Object step = fb.getObject(); @@ -191,7 +193,7 @@ public void testFaultTolerantStepAll() throws Exception { @Test public void testSimpleStep() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.setHasChunkElement(true); fb.setBeanName("step1"); fb.setAllowStartIfComplete(true); @@ -205,9 +207,9 @@ public void testSimpleStep() throws Exception { fb.setChunkCompletionPolicy(new DummyCompletionPolicy()); fb.setTaskExecutor(new SyncTaskExecutor()); fb.setItemReader(new DummyItemReader()); - fb.setItemProcessor(new PassThroughItemProcessor()); + fb.setItemProcessor(new PassThroughItemProcessor<>()); fb.setItemWriter(new DummyItemWriter()); - fb.setStreams(new ItemStream[] { new FlatFileItemReader() }); + fb.setStreams(new ItemStream[] {new FlatFileItemReader<>() }); Object step = fb.getObject(); assertTrue(step instanceof TaskletStep); @@ -217,7 +219,7 @@ public void testSimpleStep() throws Exception { @Test public void testFaultTolerantStep() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.setHasChunkElement(true); fb.setBeanName("step1"); fb.setAllowStartIfComplete(true); @@ -228,9 +230,9 @@ public void testFaultTolerantStep() throws Exception { fb.setChunkCompletionPolicy(new DummyCompletionPolicy()); fb.setTaskExecutor(new SyncTaskExecutor()); fb.setItemReader(new DummyItemReader()); - fb.setItemProcessor(new PassThroughItemProcessor()); + fb.setItemProcessor(new PassThroughItemProcessor<>()); fb.setItemWriter(new DummyItemWriter()); - fb.setStreams(new ItemStream[] { new FlatFileItemReader() }); + fb.setStreams(new ItemStream[] {new FlatFileItemReader<>() }); fb.setCacheCapacity(5); fb.setIsReaderTransactionalQueue(true); fb.setRetryLimit(5); @@ -248,11 +250,16 @@ public void testFaultTolerantStep() throws Exception { assertEquals(new Integer(10), throttleLimit); Object tasklet = ReflectionTestUtils.getField(step, "tasklet"); assertTrue(tasklet instanceof ChunkOrientedTasklet); + assertFalse((Boolean) ReflectionTestUtils.getField(tasklet, "buffering")); + Object chunkProvider = ReflectionTestUtils.getField(tasklet, "chunkProvider"); + Object repeatOperations = ReflectionTestUtils.getField(chunkProvider, "repeatOperations"); + Object completionPolicy = ReflectionTestUtils.getField(repeatOperations, "completionPolicy"); + assertTrue(completionPolicy instanceof DummyCompletionPolicy); } @Test public void testPartitionStep() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.setBeanName("step1"); fb.setAllowStartIfComplete(true); fb.setJobRepository(new JobRepositorySupport()); @@ -272,7 +279,7 @@ public void testPartitionStep() throws Exception { @Test public void testPartitionStepWithProxyHandler() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.setBeanName("step1"); fb.setAllowStartIfComplete(true); fb.setJobRepository(new JobRepositorySupport()); @@ -295,7 +302,7 @@ public void testPartitionStepWithProxyHandler() throws Exception { @Test public void testFlowStep() throws Exception { - StepParserStepFactoryBean fb = new StepParserStepFactoryBean(); + StepParserStepFactoryBean fb = new StepParserStepFactoryBean<>(); fb.setBeanName("step1"); fb.setAllowStartIfComplete(true); fb.setJobRepository(new JobRepositorySupport()); @@ -311,8 +318,9 @@ public void testFlowStep() throws Exception { assertTrue(handler instanceof SimpleFlow); } + @SuppressWarnings("unchecked") private Map, Boolean> getExceptionMap(Class... args) { - Map, Boolean> map = new HashMap, Boolean>(); + Map, Boolean> map = new HashMap<>(); for (Class arg : args) { map.put(arg, true); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepParserTests.java index 5c1c1d829d..c322587302 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,13 +25,13 @@ import java.util.List; import java.util.Map; import java.util.Map.Entry; +import java.util.Set; import org.junit.BeforeClass; import org.junit.Test; import org.springframework.aop.framework.Advised; import org.springframework.batch.core.Step; import org.springframework.batch.core.StepExecutionListener; -import org.springframework.batch.core.StepListener; import org.springframework.batch.core.job.AbstractJob; import org.springframework.batch.core.listener.CompositeStepExecutionListener; import org.springframework.batch.core.listener.StepExecutionListenerSupport; @@ -77,10 +77,11 @@ public static void loadAppCtx() { } @Test + @SuppressWarnings("resource") public void testTaskletStepAttributes() throws Exception { ConfigurableApplicationContext ctx = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/StepParserTaskletAttributesTests-context.xml"); - @SuppressWarnings({ "unchecked", "rawtypes" }) + @SuppressWarnings({ "rawtypes" }) Map beans = ctx.getBeansOfType(StepParserStepFactoryBean.class); String factoryName = (String) beans.keySet().toArray()[0]; @SuppressWarnings("unchecked") @@ -92,10 +93,10 @@ public void testTaskletStepAttributes() throws Exception { } @Test + @SuppressWarnings("resource") public void testStepParserBeanName() throws Exception { ConfigurableApplicationContext ctx = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/StepParserBeanNameTests-context.xml"); - @SuppressWarnings("unchecked") Map beans = ctx.getBeansOfType(Step.class); assertTrue("'s1' bean not found", beans.containsKey("s1")); Step s1 = (Step) ctx.getBean("s1"); @@ -103,16 +104,17 @@ public void testStepParserBeanName() throws Exception { } @Test(expected = BeanDefinitionParsingException.class) + @SuppressWarnings("resource") public void testStepParserCommitIntervalCompletionPolicy() throws Exception { new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/StepParserCommitIntervalCompletionPolicyTests-context.xml"); } @Test + @SuppressWarnings("resource") public void testStepParserCommitInterval() throws Exception { ConfigurableApplicationContext ctx = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/StepParserCommitIntervalTests-context.xml"); - @SuppressWarnings("unchecked") Map beans = ctx.getBeansOfType(Step.class); assertTrue("'s1' bean not found", beans.containsKey("s1")); Step s1 = (Step) ctx.getBean("s1"); @@ -122,10 +124,10 @@ public void testStepParserCommitInterval() throws Exception { } @Test + @SuppressWarnings("resource") public void testStepParserCompletionPolicy() throws Exception { ConfigurableApplicationContext ctx = new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/StepParserCompletionPolicyTests-context.xml"); - @SuppressWarnings("unchecked") Map beans = ctx.getBeansOfType(Step.class); assertTrue("'s1' bean not found", beans.containsKey("s1")); Step s1 = (Step) ctx.getBean("s1"); @@ -141,18 +143,21 @@ private CompletionPolicy getCompletionPolicy(Step s1) throws NoSuchFieldExceptio } @Test(expected = BeanDefinitionParsingException.class) + @SuppressWarnings("resource") public void testStepParserNoCommitIntervalOrCompletionPolicy() throws Exception { new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/StepParserNoCommitIntervalOrCompletionPolicyTests-context.xml"); } @Test(expected = BeanDefinitionParsingException.class) + @SuppressWarnings("resource") public void testTaskletStepWithBadStepListener() throws Exception { new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/StepParserBadStepListenerTests-context.xml"); } @Test(expected = BeanDefinitionParsingException.class) + @SuppressWarnings("resource") public void testTaskletStepWithBadRetryListener() throws Exception { new ClassPathXmlApplicationContext( "org/springframework/batch/core/configuration/xml/StepParserBadRetryListenerTests-context.xml"); @@ -220,7 +225,7 @@ private List getListeners(String stepName, ApplicationCon Object composite = ReflectionTestUtils.getField(compositeListener, "list"); List list = (List) ReflectionTestUtils .getField(composite, "list"); - List unwrappedList = new ArrayList(); + List unwrappedList = new ArrayList<>(); for (StepExecutionListener listener : list) { while (listener instanceof Advised) { listener = (StepExecutionListener) ((Advised) listener).getTargetSource().getTarget(); @@ -422,13 +427,13 @@ public void testTaskletElementOverridesParentBeanClass() { public void testStepWithListsMerge() throws Exception { ApplicationContext ctx = stepParserParentAttributeTestsCtx; - Map, Boolean> skippable = new HashMap, Boolean>(); + Map, Boolean> skippable = new HashMap<>(); skippable.put(SkippableRuntimeException.class, true); skippable.put(SkippableException.class, true); skippable.put(FatalRuntimeException.class, false); skippable.put(FatalSkippableException.class, false); skippable.put(ForceRollbackForWriteSkipException.class, true); - Map, Boolean> retryable = new HashMap, Boolean>(); + Map, Boolean> retryable = new HashMap<>(); retryable.put(DeadlockLoserDataAccessException.class, true); retryable.put(FatalSkippableException.class, true); retryable.put(ForceRollbackForWriteSkipException.class, true); @@ -446,7 +451,7 @@ public void testStepWithListsMerge() throws Exception { Map, Boolean> retryableFound = getExceptionMap(fb, "retryableExceptionClasses"); ItemStream[] streamsFound = (ItemStream[]) ReflectionTestUtils.getField(fb, "streams"); RetryListener[] retryListenersFound = (RetryListener[]) ReflectionTestUtils.getField(fb, "retryListeners"); - StepListener[] stepListenersFound = (StepListener[]) ReflectionTestUtils.getField(fb, "listeners"); + Set stepListenersFound = (Set) ReflectionTestUtils.getField(fb, "stepExecutionListeners"); Collection> noRollbackFound = getExceptionList(fb, "noRollbackExceptionClasses"); assertSameMaps(skippable, skippableFound); @@ -462,11 +467,11 @@ public void testStepWithListsMerge() throws Exception { public void testStepWithListsNoMerge() throws Exception { ApplicationContext ctx = stepParserParentAttributeTestsCtx; - Map, Boolean> skippable = new HashMap, Boolean>(); + Map, Boolean> skippable = new HashMap<>(); skippable.put(SkippableException.class, true); skippable.put(FatalSkippableException.class, false); skippable.put(ForceRollbackForWriteSkipException.class, true); - Map, Boolean> retryable = new HashMap, Boolean>(); + Map, Boolean> retryable = new HashMap<>(); retryable.put(FatalSkippableException.class, true); retryable.put(ForceRollbackForWriteSkipException.class, true); List> streams = Arrays.asList(CompositeItemStream.class); @@ -480,7 +485,7 @@ public void testStepWithListsNoMerge() throws Exception { Map, Boolean> retryableFound = getExceptionMap(fb, "retryableExceptionClasses"); ItemStream[] streamsFound = (ItemStream[]) ReflectionTestUtils.getField(fb, "streams"); RetryListener[] retryListenersFound = (RetryListener[]) ReflectionTestUtils.getField(fb, "retryListeners"); - StepListener[] stepListenersFound = (StepListener[]) ReflectionTestUtils.getField(fb, "listeners"); + Set stepListenersFound = (Set) ReflectionTestUtils.getField(fb, "stepExecutionListeners"); Collection> noRollbackFound = getExceptionList(fb, "noRollbackExceptionClasses"); assertSameMaps(skippable, skippableFound); @@ -491,6 +496,7 @@ public void testStepWithListsNoMerge() throws Exception { assertSameCollections(noRollback, noRollbackFound); } + @SuppressWarnings("unchecked") @Test public void testStepWithListsOverrideWithEmpty() throws Exception { ApplicationContext ctx = stepParserParentAttributeTestsCtx; @@ -502,7 +508,7 @@ public void testStepWithListsOverrideWithEmpty() throws Exception { assertEquals(1, getExceptionMap(fb, "retryableExceptionClasses").size()); assertEquals(0, ((ItemStream[]) ReflectionTestUtils.getField(fb, "streams")).length); assertEquals(0, ((RetryListener[]) ReflectionTestUtils.getField(fb, "retryListeners")).length); - assertEquals(0, ((StepListener[]) ReflectionTestUtils.getField(fb, "listeners")).length); + assertEquals(0, ((Set) ReflectionTestUtils.getField(fb, "stepExecutionListeners")).size()); assertEquals(0, getExceptionList(fb, "noRollbackExceptionClasses").size()); } @@ -537,7 +543,7 @@ private Collection> toClassCollection(T[] in) throws Exce @SuppressWarnings("unchecked") private Collection> toClassCollection(Collection in) throws Exception { - Collection> out = new ArrayList>(); + Collection> out = new ArrayList<>(); for (T item : in) { while (item instanceof Advised) { item = (T) ((Advised) item).getTargetSource().getTarget(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithBasicProcessTaskJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithBasicProcessTaskJobParserTests.java index 580fa20d0c..9e5e9f5af0 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithBasicProcessTaskJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithBasicProcessTaskJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,13 +19,15 @@ import static org.junit.Assert.assertNotNull; import static org.junit.Assert.assertTrue; +import java.util.Set; + import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; -import org.springframework.batch.core.StepListener; +import org.springframework.batch.core.StepExecutionListener; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.item.ItemStream; import org.springframework.beans.factory.annotation.Autowired; @@ -42,36 +44,37 @@ @ContextConfiguration @RunWith(SpringJUnit4ClassRunner.class) public class StepWithBasicProcessTaskJobParserTests { - + @Autowired private Job job; @Autowired private JobRepository jobRepository; - + @Autowired private TestReader reader; - + @Autowired @Qualifier("listener") private TestListener listener; - + @Autowired private TestProcessor processor; - + @Autowired private TestWriter writer; - + @Autowired private StepParserStepFactoryBean factory; - + + @SuppressWarnings("unchecked") @Test public void testStepWithTask() throws Exception { assertNotNull(job); Object ci = ReflectionTestUtils.getField(factory, "commitInterval"); assertEquals("wrong chunk-size:", 10, ci); - Object listeners = ReflectionTestUtils.getField(factory, "listeners"); - assertEquals("wrong number of listeners:", 2, ((StepListener[])listeners).length); + Object listeners = ReflectionTestUtils.getField(factory, "stepExecutionListeners"); + assertEquals("wrong number of listeners:", 2, ((Set)listeners).size()); Object streams = ReflectionTestUtils.getField(factory, "streams"); assertEquals("wrong number of streams:", 1, ((ItemStream[])streams).length); JobExecution jobExecution = jobRepository.createJobExecution(job.getName(), new JobParameters()); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithFaultTolerantProcessTaskJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithFaultTolerantProcessTaskJobParserTests.java index 395404f316..6f0f6f27a3 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithFaultTolerantProcessTaskJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithFaultTolerantProcessTaskJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,13 +19,15 @@ import static org.junit.Assert.assertNotNull; import static org.junit.Assert.assertTrue; +import java.util.Set; + import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; -import org.springframework.batch.core.StepListener; +import org.springframework.batch.core.StepExecutionListener; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.item.ItemStream; import org.springframework.beans.factory.annotation.Autowired; @@ -40,7 +42,7 @@ /** * @author Thomas Risberg - * + * */ @ContextConfiguration @RunWith(SpringJUnit4ClassRunner.class) @@ -71,6 +73,7 @@ public class StepWithFaultTolerantProcessTaskJobParserTests { @Autowired private StepParserStepFactoryBean factory; + @SuppressWarnings("unchecked") @Test public void testStepWithTask() throws Exception { assertNotNull(job); @@ -91,8 +94,8 @@ public void testStepWithTask() throws Exception { assertEquals("wrong reader-transactional-queue:", true, txq); Object te = ReflectionTestUtils.getField(factory, "taskExecutor"); assertEquals("wrong task-executor:", ConcurrentTaskExecutor.class, te.getClass()); - Object listeners = ReflectionTestUtils.getField(factory, "listeners"); - assertEquals("wrong number of listeners:", 2, ((StepListener[]) listeners).length); + Object listeners = ReflectionTestUtils.getField(factory, "stepExecutionListeners"); + assertEquals("wrong number of listeners:", 2, ((Set) listeners).size()); Object retryListeners = ReflectionTestUtils.getField(factory, "retryListeners"); assertEquals("wrong number of retry-listeners:", 2, ((RetryListener[]) retryListeners).length); Object streams = ReflectionTestUtils.getField(factory, "streams"); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithPojoListenerJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithPojoListenerJobParserTests.java index be9b45a56e..cfcd8d93a3 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithPojoListenerJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithPojoListenerJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithSimpleTaskJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithSimpleTaskJobParserTests.java index 9071a486ff..58bab495c7 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithSimpleTaskJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StepWithSimpleTaskJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -78,7 +78,7 @@ public void testJob() throws Exception { private TestTasklet assertTasklet(Job job, String stepName, String taskletName) { System.err.println(((FlowJob) job).getStepNames()); Step step = ((FlowJob) job).getStep(stepName); - assertTrue("Wong type for step name="+stepName+": "+step, step instanceof TaskletStep); + assertTrue("Wrong type for step name="+stepName+": "+step, step instanceof TaskletStep); Object tasklet = ReflectionTestUtils.getField(step, "tasklet"); assertTrue(tasklet instanceof TestTasklet); TestTasklet testTasklet = (TestTasklet) tasklet; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopAndRestartFailedJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopAndRestartFailedJobParserTests.java index ebabe4c52c..4d355d5e4c 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopAndRestartFailedJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopAndRestartFailedJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopAndRestartJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopAndRestartJobParserTests.java index 1c5386f83e..bcb4ddb438 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopAndRestartJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopAndRestartJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopCustomStatusJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopCustomStatusJobParserTests.java index c7de1b0e69..9de10637ac 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopCustomStatusJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopCustomStatusJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopIncompleteJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopIncompleteJobParserTests.java index a68bba2c46..25585d52a6 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopIncompleteJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopIncompleteJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopJobParserTests.java index 95cc8f2395..a926b19c46 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopJobParserTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,6 +26,7 @@ import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.job.flow.FlowExecutionStatus; import org.springframework.batch.core.job.flow.JobExecutionDecider; +import org.springframework.lang.Nullable; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @@ -75,7 +76,7 @@ public void testStopState() throws Exception { public static class TestDecider implements JobExecutionDecider { @Override - public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) { + public FlowExecutionStatus decide(JobExecution jobExecution, @Nullable StepExecution stepExecution) { return new FlowExecutionStatus("FOO"); } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopRestartOnCompletedStepJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopRestartOnCompletedStepJobParserTests.java index c4c858eac7..ac7fb24186 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopRestartOnCompletedStepJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopRestartOnCompletedStepJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopRestartOnFailedStepJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopRestartOnFailedStepJobParserTests.java index 083e93fcd7..f05a2c67f8 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopRestartOnFailedStepJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/StopRestartOnFailedStepJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TaskletParserAdapterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TaskletParserAdapterTests.java index a3a9ac7a36..890bdcd2f8 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TaskletParserAdapterTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TaskletParserAdapterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TaskletParserBeanPropertiesTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TaskletParserBeanPropertiesTests.java index 224d6dc24d..36b3ec1082 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TaskletParserBeanPropertiesTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TaskletParserBeanPropertiesTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TaskletStepAllowStartIfCompleteTest.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TaskletStepAllowStartIfCompleteTest.java index 66cbcc8d83..bea5267789 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TaskletStepAllowStartIfCompleteTest.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TaskletStepAllowStartIfCompleteTest.java @@ -1,3 +1,18 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestCustomStatusListener.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestCustomStatusListener.java index 4248142182..e60356ccbc 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestCustomStatusListener.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestCustomStatusListener.java @@ -1,11 +1,28 @@ +/* + * Copyright 2009-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.StepExecutionListener; +import org.springframework.lang.Nullable; public class TestCustomStatusListener extends AbstractTestComponent implements StepExecutionListener { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { return new ExitStatus("FOO").and(stepExecution.getExitStatus()); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestIncrementer.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestIncrementer.java index 1fea100dfb..90250f84a2 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestIncrementer.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestIncrementer.java @@ -1,12 +1,28 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.JobParametersIncrementer; +import org.springframework.lang.Nullable; public class TestIncrementer implements JobParametersIncrementer{ @Override - public JobParameters getNext(JobParameters parameters) { + public JobParameters getNext(@Nullable JobParameters parameters) { return null; } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestJobListener.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestJobListener.java index 4b14b07255..8b87a9e81d 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestJobListener.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestJobListener.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.core.annotation.BeforeJob; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestListener.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestListener.java index ba24f7e670..4fa621acd0 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestListener.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestListener.java @@ -1,12 +1,29 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.StepExecutionListener; import org.springframework.batch.core.annotation.AfterRead; +import org.springframework.lang.Nullable; public class TestListener extends AbstractTestComponent implements StepExecutionListener { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { return null; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestPojoListener.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestPojoListener.java index 529672f7c7..3ae78dc127 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestPojoListener.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestPojoListener.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import java.util.List; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestProcessor.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestProcessor.java index 731ec465d9..71010c67e6 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestProcessor.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestProcessor.java @@ -1,9 +1,26 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.item.ItemProcessor; +import org.springframework.lang.Nullable; public class TestProcessor extends AbstractTestComponent implements ItemProcessor{ + @Nullable @Override public String process(String item) throws Exception { executed = true; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestReader.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestReader.java index 7aa95a896c..77c5047745 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestReader.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestReader.java @@ -1,13 +1,28 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import java.util.ArrayList; import java.util.Collections; import java.util.List; import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.item.ItemStreamException; import org.springframework.batch.item.ItemStreamReader; import org.springframework.batch.item.ParseException; import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.lang.Nullable; public class TestReader extends AbstractTestComponent implements ItemStreamReader { @@ -16,7 +31,7 @@ public class TestReader extends AbstractTestComponent implements ItemStreamReade List items = null; { - List l = new ArrayList(); + List l = new ArrayList<>(); l.add("Item *** 1 ***"); l.add("Item *** 2 ***"); this.items = Collections.synchronizedList(l); @@ -30,6 +45,7 @@ public void setOpened(boolean opened) { this.opened = opened; } + @Nullable @Override public String read() throws Exception, UnexpectedInputException, ParseException { executed = true; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestRetryListener.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestRetryListener.java index 7d6b9381a0..a07ddc6f6c 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestRetryListener.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestRetryListener.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.retry.RetryCallback; @@ -7,17 +22,17 @@ public class TestRetryListener extends AbstractTestComponent implements RetryListener { @Override - public void close(RetryContext context, RetryCallback callback, + public void close(RetryContext context, RetryCallback callback, Throwable throwable) { } @Override - public void onError(RetryContext context, RetryCallback callback, + public void onError(RetryContext context, RetryCallback callback, Throwable throwable) { } @Override - public boolean open(RetryContext context, RetryCallback callback) { + public boolean open(RetryContext context, RetryCallback callback) { executed = true; return true; } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestTasklet.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestTasklet.java index 3e526e515d..c5618b3ceb 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestTasklet.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestTasklet.java @@ -1,14 +1,31 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; public class TestTasklet extends AbstractTestComponent implements Tasklet { private String name; + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestWriter.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestWriter.java index b4e1ae90d4..6f86355165 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestWriter.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TestWriter.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.configuration.xml; import java.util.List; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TwoStepJobParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TwoStepJobParserTests.java index dd7876eb15..d8123956e4 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TwoStepJobParserTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/configuration/xml/TwoStepJobParserTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/converter/DefaultJobParametersConverterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/converter/DefaultJobParametersConverterTests.java index 680d1a69d1..7b6ef08721 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/converter/DefaultJobParametersConverterTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/converter/DefaultJobParametersConverterTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ import java.text.DateFormat; import java.text.DecimalFormat; +import java.text.DecimalFormatSymbols; import java.text.NumberFormat; import java.text.SimpleDateFormat; import java.util.Date; @@ -153,7 +154,7 @@ public void testGetParametersWithNumberFormat() throws Exception { String[] args = new String[] { "value(long)=1,000" }; - factory.setNumberFormat(new DecimalFormat("#,###")); + factory.setNumberFormat(new DecimalFormat("#,###", DecimalFormatSymbols.getInstance(Locale.ENGLISH))); JobParameters props = factory.getJobParameters(StringUtils.splitArrayElementsIntoProperties(args, "=")); assertNotNull(props); assertEquals(1000L, props.getLong("value").longValue()); @@ -178,7 +179,7 @@ public void testGetParametersWithBogusLong() throws Exception { public void testGetParametersWithDoubleValueDeclaredAsLong() throws Exception { String[] args = new String[] { "value(long)=1.03" }; - factory.setNumberFormat(new DecimalFormat("#.#")); + factory.setNumberFormat(new DecimalFormat("#.#", DecimalFormatSymbols.getInstance(Locale.ENGLISH))); try { factory.getJobParameters(StringUtils.splitArrayElementsIntoProperties(args, "=")); @@ -280,6 +281,22 @@ public void testRoundTrip() throws Exception { assertEquals("1.23", props.getProperty("double.key(double)")); } + @Test + public void testRoundTripWithIdentifyingAndNonIdentifying() throws Exception { + + String[] args = new String[] { "schedule.date(date)=2008/01/23", "+job.key=myKey", "-vendor.id(long)=33243243", + "double.key(double)=1.23" }; + + JobParameters parameters = factory.getJobParameters(StringUtils.splitArrayElementsIntoProperties(args, "=")); + + Properties props = factory.getProperties(parameters); + assertNotNull(props); + assertEquals("myKey", props.getProperty("job.key")); + assertEquals("33243243", props.getProperty("-vendor.id(long)")); + assertEquals("2008/01/23", props.getProperty("schedule.date(date)")); + assertEquals("1.23", props.getProperty("double.key(double)")); + } + @Test public void testRoundTripWithNumberFormat() throws Exception { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/converter/JobParametersConverterSupport.java b/spring-batch-core/src/test/java/org/springframework/batch/core/converter/JobParametersConverterSupport.java new file mode 100644 index 0000000000..ac06c9de19 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/converter/JobParametersConverterSupport.java @@ -0,0 +1,58 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.converter; + +import java.util.Map; +import java.util.Properties; + +import org.springframework.batch.core.JobParameter; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.lang.Nullable; + +public class JobParametersConverterSupport implements JobParametersConverter { + + @Override + public JobParameters getJobParameters(@Nullable Properties properties) { + JobParametersBuilder builder = new JobParametersBuilder(); + + if(properties != null) { + for (Map.Entry curParameter : properties.entrySet()) { + if(curParameter.getValue() != null) { + builder.addString(curParameter.getKey().toString(), curParameter.getValue().toString(), false); + } + } + } + + return builder.toJobParameters(); + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.converter.JobParametersConverter#getProperties(org.springframework.batch.core.JobParameters) + */ + @Override + public Properties getProperties(@Nullable JobParameters params) { + Properties properties = new Properties(); + + if(params != null) { + for(Map.Entry curParameter: params.getParameters().entrySet()) { + properties.setProperty(curParameter.getKey(), curParameter.getValue().getValue().toString()); + } + } + + return properties; + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/JobExplorerFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/JobExplorerFactoryBeanTests.java index 28a05bbac1..305ed2dc43 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/JobExplorerFactoryBeanTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/JobExplorerFactoryBeanTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,17 +15,20 @@ */ package org.springframework.batch.core.explore.support; -import static junit.framework.Assert.assertTrue; -import static junit.framework.Assert.fail; -import static org.mockito.Mockito.mock; +import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.mock; import javax.sql.DataSource; import org.junit.Before; import org.junit.Test; import org.springframework.batch.core.explore.JobExplorer; -import org.springframework.batch.core.explore.support.JobExplorerFactoryBean; +import org.springframework.jdbc.core.JdbcOperations; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.util.ReflectionTestUtils; /** * @author Dave Syer @@ -49,6 +52,24 @@ public void setUp() throws Exception { factory.setTablePrefix(tablePrefix); } + + + @Test + public void testDefaultJdbcOperations() throws Exception { + + factory.afterPropertiesSet(); + JdbcOperations jdbcOperations = (JdbcOperations) ReflectionTestUtils.getField(factory, "jdbcOperations"); + assertTrue(jdbcOperations instanceof JdbcTemplate); + } + + @Test + public void testCustomJdbcOperations() throws Exception { + + JdbcOperations customJdbcOperations = mock(JdbcOperations.class); + factory.setJdbcOperations(customJdbcOperations); + factory.afterPropertiesSet(); + assertEquals(customJdbcOperations, ReflectionTestUtils.getField(factory, "jdbcOperations")); + } @Test public void testMissingDataSource() throws Exception { @@ -70,7 +91,7 @@ public void testMissingDataSource() throws Exception { public void testCreateExplorer() throws Exception { factory.afterPropertiesSet(); - JobExplorer explorer = (JobExplorer) factory.getObject(); + JobExplorer explorer = factory.getObject(); assertNotNull(explorer); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/MapJobExplorerFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/MapJobExplorerFactoryBeanTests.java index 983bcd8dff..9ceb07593c 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/MapJobExplorerFactoryBeanTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/MapJobExplorerFactoryBeanTests.java @@ -1,14 +1,31 @@ +/* + * Copyright 2010-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.explore.support; import static org.junit.Assert.assertEquals; import org.junit.Test; +import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.explore.JobExplorer; -import org.springframework.batch.core.explore.support.MapJobExplorerFactoryBean; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; +import java.util.Date; + /** * Tests for {@link MapJobExplorerFactoryBean}. */ @@ -22,12 +39,17 @@ public class MapJobExplorerFactoryBeanTests { public void testCreateExplorer() throws Exception { MapJobRepositoryFactoryBean repositoryFactory = new MapJobRepositoryFactoryBean(); - ((JobRepository)repositoryFactory.getObject()).createJobExecution("foo", new JobParameters()); - + JobRepository jobRepository = repositoryFactory.getObject(); + JobExecution jobExecution = jobRepository.createJobExecution("foo", new JobParameters()); + + //simulating a running job execution + jobExecution.setStartTime(new Date()); + jobRepository.update(jobExecution); + MapJobExplorerFactoryBean tested = new MapJobExplorerFactoryBean(repositoryFactory); tested.afterPropertiesSet(); - JobExplorer explorer = (JobExplorer) tested.getObject(); + JobExplorer explorer = tested.getObject(); assertEquals(1, explorer.findRunningJobExecutions("foo").size()); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/MapJobExplorerIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/MapJobExplorerIntegrationTests.java index 1f0987506c..3fcc0ddef0 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/MapJobExplorerIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/MapJobExplorerIntegrationTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,10 +15,6 @@ */ package org.springframework.batch.core.explore.support; -import static org.junit.Assert.assertEquals; - -import java.util.Set; - import org.junit.Test; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParametersBuilder; @@ -33,6 +29,11 @@ import org.springframework.batch.core.step.tasklet.TaskletStep; import org.springframework.batch.repeat.RepeatStatus; import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.lang.Nullable; + +import java.util.Set; + +import static org.junit.Assert.assertEquals; /** * @author Dave Syer @@ -48,7 +49,7 @@ public void testRunningJobExecution() throws Exception { SimpleJobLauncher jobLauncher = new SimpleJobLauncher(); MapJobRepositoryFactoryBean repositoryFactory = new MapJobRepositoryFactoryBean(); repositoryFactory.afterPropertiesSet(); - JobRepository jobRepository = (JobRepository) repositoryFactory.getObject(); + JobRepository jobRepository = repositoryFactory.getObject(); jobLauncher.setJobRepository(jobRepository); jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor()); jobLauncher.afterPropertiesSet(); @@ -56,6 +57,7 @@ public void testRunningJobExecution() throws Exception { SimpleJob job = new SimpleJob("job"); TaskletStep step = new TaskletStep("step"); step.setTasklet(new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { while (block) { @@ -74,7 +76,7 @@ public RepeatStatus execute(StepContribution contribution, ChunkContext chunkCon jobLauncher.run(job, new JobParametersBuilder().addString("test", getClass().getName()).toJobParameters()); Thread.sleep(500L); - JobExplorer explorer = (JobExplorer) new MapJobExplorerFactoryBean(repositoryFactory).getObject(); + JobExplorer explorer = new MapJobExplorerFactoryBean(repositoryFactory).getObject(); Set executions = explorer.findRunningJobExecutions("job"); assertEquals(1, executions.size()); assertEquals(1, executions.iterator().next().getStepExecutions().size()); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/SimpleJobExplorerIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/SimpleJobExplorerIntegrationTests.java new file mode 100644 index 0000000000..04a856347f --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/SimpleJobExplorerIntegrationTests.java @@ -0,0 +1,156 @@ +/* + * Copyright 2013-2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.explore.support; + +import java.util.ArrayList; +import java.util.List; + +import org.apache.commons.dbcp2.BasicDataSource; +import org.junit.Test; +import org.junit.runner.RunWith; +import test.jdbc.datasource.DataSourceInitializer; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobInterruptedException; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.UnexpectedJobExecutionException; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.core.configuration.xml.DummyStep; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.job.flow.FlowExecutionStatus; +import org.springframework.batch.core.job.flow.FlowStep; +import org.springframework.batch.core.job.flow.support.SimpleFlow; +import org.springframework.batch.core.job.flow.support.StateTransition; +import org.springframework.batch.core.job.flow.support.state.EndState; +import org.springframework.batch.core.job.flow.support.state.StepState; +import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; +import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.core.io.ClassPathResource; +import org.springframework.core.io.Resource; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +import static org.junit.Assert.assertEquals; + +/** + * Integration test for the BATCH-2034 issue. + * The {@link FlowStep} execution should not fail in the remote partitioning use case because the {@link SimpleJobExplorer} + * doesn't retrieve the {@link JobInstance} from the {@link JobRepository}. + * To illustrate the issue the test simulates the behavior of the {@code StepExecutionRequestHandler} + * from the spring-batch-integration project. + * + * @author Sergey Shcherbakov + */ +@ContextConfiguration(classes={SimpleJobExplorerIntegrationTests.Config.class}) +@RunWith(SpringJUnit4ClassRunner.class) +public class SimpleJobExplorerIntegrationTests { + + @Configuration + @EnableBatchProcessing + static class Config { + + @Autowired + private StepBuilderFactory steps; + + @Bean + public JobExplorer jobExplorer() throws Exception { + return jobExplorerFactoryBean().getObject(); + } + + @Bean + public JobExplorerFactoryBean jobExplorerFactoryBean() { + JobExplorerFactoryBean jobExplorerFactoryBean = new JobExplorerFactoryBean(); + jobExplorerFactoryBean.setDataSource(dataSource()); + return jobExplorerFactoryBean; + } + + @Bean + public Step flowStep() throws Exception { + return steps.get("flowStep").flow(simpleFlow()).build(); + } + + @Bean + public Step dummyStep() { + return new DummyStep(); + } + + @Bean + public SimpleFlow simpleFlow() { + SimpleFlow simpleFlow = new SimpleFlow("simpleFlow"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(dummyStep()), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + simpleFlow.setStateTransitions(transitions); + return simpleFlow; + } + + @Bean + public BasicDataSource dataSource() { + BasicDataSource dataSource = new BasicDataSource(); + dataSource.setDriverClassName("org.hsqldb.jdbcDriver"); + dataSource.setUrl("jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true;hsqldb.tx=mvcc"); + dataSource.setUsername("sa"); + dataSource.setPassword(""); + return dataSource; + } + + @Bean + public DataSourceInitializer dataSourceInitializer() { + DataSourceInitializer dataSourceInitializer = new DataSourceInitializer(); + dataSourceInitializer.setDataSource(dataSource()); + dataSourceInitializer.setInitScripts(new Resource[] { + new ClassPathResource("org/springframework/batch/core/schema-drop-hsqldb.sql"), + new ClassPathResource("org/springframework/batch/core/schema-hsqldb.sql") + }); + return dataSourceInitializer; + } + } + + @Autowired + private JobRepository jobRepository; + + @Autowired + private JobExplorer jobExplorer; + + @Autowired + private FlowStep flowStep; + + @Test + public void testGetStepExecution() throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException, JobInterruptedException, UnexpectedJobExecutionException { + + // Prepare the jobRepository for the test + JobExecution jobExecution = jobRepository.createJobExecution("myJob", new JobParameters()); + StepExecution stepExecution = jobExecution.createStepExecution("flowStep"); + jobRepository.add(stepExecution); + + // Executed on the remote end in remote partitioning use case + StepExecution jobExplorerStepExecution = jobExplorer.getStepExecution(jobExecution.getId(), stepExecution.getId()); + flowStep.execute(jobExplorerStepExecution); + + assertEquals(BatchStatus.COMPLETED, jobExplorerStepExecution.getStatus()); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/SimpleJobExplorerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/SimpleJobExplorerTests.java index 0c9c1740ab..290243acc7 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/SimpleJobExplorerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/explore/support/SimpleJobExplorerTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,9 +16,11 @@ package org.springframework.batch.core.explore.support; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; -import static org.junit.Assert.assertNull; import java.util.Collections; @@ -28,6 +30,7 @@ import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.launch.NoSuchJobException; import org.springframework.batch.core.repository.dao.ExecutionContextDao; import org.springframework.batch.core.repository.dao.JobExecutionDao; import org.springframework.batch.core.repository.dao.JobInstanceDao; @@ -38,7 +41,8 @@ * * @author Dave Syer * @author Will Schipp - * + * @author Michael Minella + * @author Mahmoud Ben Hassine * */ public class SimpleJobExplorerTests { @@ -55,7 +59,7 @@ public class SimpleJobExplorerTests { private ExecutionContextDao ecDao; - private JobExecution jobExecution = new JobExecution(jobInstance, 1234L, new JobParameters()); + private JobExecution jobExecution = new JobExecution(jobInstance, 1234L, new JobParameters(), null); @Before public void setUp() throws Exception { @@ -79,6 +83,13 @@ public void testGetJobExecution() throws Exception { jobExplorer.getJobExecution(123L); } + @Test + public void testGetLastJobExecution() { + when(jobExecutionDao.getLastJobExecution(jobInstance)).thenReturn(jobExecution); + JobExecution lastJobExecution = jobExplorer.getLastJobExecution(jobInstance); + assertEquals(jobExecution, lastJobExecution); + } + @Test public void testMissingGetJobExecution() throws Exception { when(jobExecutionDao.getJobExecution(123L)).thenReturn(null); @@ -88,18 +99,24 @@ public void testMissingGetJobExecution() throws Exception { @Test public void testGetStepExecution() throws Exception { when(jobExecutionDao.getJobExecution(jobExecution.getId())).thenReturn(jobExecution); + when(jobInstanceDao.getJobInstance(jobExecution)).thenReturn(jobInstance); StepExecution stepExecution = jobExecution.createStepExecution("foo"); when(stepExecutionDao.getStepExecution(jobExecution, 123L)) - .thenReturn(stepExecution); + .thenReturn(stepExecution); when(ecDao.getExecutionContext(stepExecution)).thenReturn(null); - jobExplorer.getStepExecution(jobExecution.getId(), 123L); + stepExecution = jobExplorer.getStepExecution(jobExecution.getId(), 123L); + + assertEquals(jobInstance, + stepExecution.getJobExecution().getJobInstance()); + + verify(jobInstanceDao).getJobInstance(jobExecution); } @Test public void testGetStepExecutionMissing() throws Exception { when(jobExecutionDao.getJobExecution(jobExecution.getId())).thenReturn(jobExecution); when(stepExecutionDao.getStepExecution(jobExecution, 123L)) - .thenReturn(null); + .thenReturn(null); assertNull(jobExplorer.getStepExecution(jobExecution.getId(), 123L)); } @@ -147,10 +164,30 @@ public void testGetLastJobInstances() throws Exception { jobExplorer.getJobInstances("foo", 0, 1); } + @Test + public void testGetLastJobInstance() { + when(jobInstanceDao.getLastJobInstance("foo")).thenReturn(jobInstance); + JobInstance lastJobInstance = jobExplorer.getLastJobInstance("foo"); + assertEquals(jobInstance, lastJobInstance); + } + @Test public void testGetJobNames() throws Exception { jobInstanceDao.getJobNames(); jobExplorer.getJobNames(); } + @Test + public void testGetJobInstanceCount() throws Exception { + when(jobInstanceDao.getJobInstanceCount("myJob")).thenReturn(4); + + assertEquals(4, jobExplorer.getJobInstanceCount("myJob")); + } + + @Test(expected=NoSuchJobException.class) + public void testGetJobInstanceCountException() throws Exception { + when(jobInstanceDao.getJobInstanceCount("throwException")).thenThrow(new NoSuchJobException("expected")); + + jobExplorer.getJobInstanceCount("throwException"); + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/CompositeJobParametersValidatorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/CompositeJobParametersValidatorTests.java index 455b84ee38..5e4b812a80 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/CompositeJobParametersValidatorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/CompositeJobParametersValidatorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -44,7 +44,7 @@ public void testValidatorsCanNotBeNull() throws Exception{ @Test(expected=IllegalArgumentException.class) public void testValidatorsCanNotBeEmpty() throws Exception{ - compositeJobParametersValidator.setValidators(new ArrayList()); + compositeJobParametersValidator.setValidators(new ArrayList<>()); compositeJobParametersValidator.afterPropertiesSet(); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/DefaultJobParametersValidatorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/DefaultJobParametersValidatorTests.java index 68848a182e..0a29a5ac68 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/DefaultJobParametersValidatorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/DefaultJobParametersValidatorTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.job; import org.junit.Test; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/ExtendedAbstractJobTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/ExtendedAbstractJobTests.java index f046a2cdb4..574f0e3c49 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/ExtendedAbstractJobTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/ExtendedAbstractJobTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,16 +15,6 @@ */ package org.springframework.batch.core.job; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - -import java.util.Collection; -import java.util.Collections; -import java.util.Date; - import org.junit.Before; import org.junit.Test; import org.springframework.batch.core.BatchStatus; @@ -38,9 +28,21 @@ import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; import org.springframework.batch.core.step.StepSupport; +import org.springframework.lang.Nullable; + +import java.util.Collection; +import java.util.Collections; +import java.util.Date; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; /** * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public class ExtendedAbstractJobTests { @@ -51,7 +53,7 @@ public class ExtendedAbstractJobTests { @Before public void setUp() throws Exception { MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(); - jobRepository = (JobRepository) factory.getObject(); + jobRepository = factory.getObject(); job = new StubJob("job", jobRepository); } @@ -130,7 +132,7 @@ public void testValidatorWithNotNullParameters() throws Exception { public void testSetValidator() throws Exception { job.setJobParametersValidator(new DefaultJobParametersValidator() { @Override - public void validate(JobParameters parameters) throws JobParametersInvalidException { + public void validate(@Nullable JobParameters parameters) throws JobParametersInvalidException { throw new JobParametersInvalidException("FOO"); } }); @@ -166,7 +168,7 @@ public void execute(StepExecution stepExecution) throws JobInterruptedException MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(); factory.afterPropertiesSet(); - JobRepository repository = (JobRepository) factory.getObject(); + JobRepository repository = factory.getObject(); job.setJobRepository(repository); job.setRestartable(true); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/JobSupport.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/JobSupport.java index f7143557f5..8a0b14a712 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/JobSupport.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/JobSupport.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,6 +30,7 @@ import org.springframework.batch.core.step.NoSuchStepException; import org.springframework.batch.core.step.StepLocator; import org.springframework.beans.factory.BeanNameAware; +import org.springframework.lang.Nullable; import org.springframework.util.ClassUtils; /** @@ -40,10 +41,11 @@ * * @author Lucas Ward * @author Dave Syer + * @author Mahmoud Ben Hassine */ public class JobSupport implements BeanNameAware, Job, StepLocator { - private Map steps = new HashMap(); + private Map steps = new HashMap<>(); private String name; @@ -175,6 +177,7 @@ public String toString() { * * @see org.springframework.batch.core.Job#getJobParametersIncrementer() */ + @Nullable @Override public JobParametersIncrementer getJobParametersIncrementer() { return null; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/SimpleJobFailureTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/SimpleJobFailureTests.java index e9b686379a..2a1043aa94 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/SimpleJobFailureTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/SimpleJobFailureTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.job; import static org.junit.Assert.assertEquals; @@ -32,7 +47,7 @@ public class SimpleJobFailureTests { @Before public void init() throws Exception { - JobRepository jobRepository = new MapJobRepositoryFactoryBean().getJobRepository(); + JobRepository jobRepository = new MapJobRepositoryFactoryBean().getObject(); job.setJobRepository(jobRepository); execution = jobRepository.createJobExecution("job", new JobParameters()); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/SimpleJobTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/SimpleJobTests.java index 0ba40976f8..f8ecb0c44a 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/SimpleJobTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/SimpleJobTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,7 +17,6 @@ package org.springframework.batch.core.job; import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertNotNull; @@ -66,6 +65,7 @@ * * @author Lucas Ward * @author Will Schipp + * @author Mahmoud Ben Hassine */ public class SimpleJobTests { @@ -79,7 +79,7 @@ public class SimpleJobTests { private ExecutionContextDao ecDao; - private List list = new ArrayList(); + private List list = new ArrayList<>(); private JobInstance jobInstance; @@ -123,7 +123,7 @@ public void run() { } }); - List steps = new ArrayList(); + List steps = new ArrayList<>(); steps.add(step1); steps.add(step2); job.setName("testJob"); @@ -195,7 +195,7 @@ public boolean isAllowStartIfComplete() { return false; } }; - List steps = new ArrayList(); + List steps = new ArrayList<>(); steps.add(testStep); job.setSteps(steps); job.execute(jobExecution); @@ -356,7 +356,7 @@ public void testStepAlreadyComplete() throws Exception { @Test public void testStepAlreadyCompleteInSameExecution() throws Exception { - List steps = new ArrayList(); + List steps = new ArrayList<>(); steps.add(step1); steps.add(step2); // Two steps with the same name should both be executed, since @@ -373,7 +373,7 @@ public void testStepAlreadyCompleteInSameExecution() throws Exception { @Test public void testNoSteps() throws Exception { - job.setSteps(new ArrayList()); + job.setSteps(new ArrayList<>()); job.execute(jobExecution); ExitStatus exitStatus = jobExecution.getExitStatus(); @@ -529,7 +529,7 @@ public void testGetStepNotExists() { */ private void checkRepository(BatchStatus status, ExitStatus exitStatus) { assertEquals(jobInstance, jobInstanceDao.getJobInstance(job.getName(), jobParameters)); - // because map dao stores in memory, it can be checked directly + // because map DAO stores in memory, it can be checked directly JobExecution jobExecution = jobExecutionDao.findJobExecutions(jobInstance).get(0); assertEquals(jobInstance.getId(), jobExecution.getJobId()); assertEquals(status, jobExecution.getStatus()); @@ -579,7 +579,7 @@ public void setCallback(Runnable runnable) { /* * (non-Javadoc) * - * @seeorg.springframework.batch.core.step.StepSupport#execute(org. + * @see org.springframework.batch.core.step.StepSupport#execute(org. * springframework.batch.core.StepExecution) */ @Override diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/SimpleStepHandlerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/SimpleStepHandlerTests.java index 2764c69a94..ee5ec3e3e5 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/SimpleStepHandlerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/SimpleStepHandlerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -45,7 +45,7 @@ public class SimpleStepHandlerTests { @Before public void setUp() throws Exception { MapJobRepositoryFactoryBean jobRepositoryFactoryBean = new MapJobRepositoryFactoryBean(); - jobRepository = jobRepositoryFactoryBean.getJobRepository(); + jobRepository = jobRepositoryFactoryBean.getObject(); jobExecution = jobRepository.createJobExecution("job", new JobParameters()); stepHandler = new SimpleStepHandler(jobRepository); stepHandler.afterPropertiesSet(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/builder/FlowBuilderTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/builder/FlowBuilderTests.java index 067c094851..68b2eadbf1 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/builder/FlowBuilderTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/builder/FlowBuilderTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,8 +36,8 @@ public class FlowBuilderTests { @Test public void test() throws Exception { - FlowBuilder builder = new FlowBuilder("flow"); - JobRepository jobRepository = new MapJobRepositoryFactoryBean().getJobRepository(); + FlowBuilder builder = new FlowBuilder<>("flow"); + JobRepository jobRepository = new MapJobRepositoryFactoryBean().getObject(); JobExecution execution = jobRepository.createJobExecution("foo", new JobParameters()); builder.start(new StepSupport("step") { @Override diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/builder/FlowJobBuilderTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/builder/FlowJobBuilderTests.java index daead9e305..3ed08c5c28 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/builder/FlowJobBuilderTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/builder/FlowJobBuilderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,6 +34,7 @@ import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; import org.springframework.batch.core.step.StepSupport; import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.lang.Nullable; /** * @author Dave Syer @@ -87,7 +88,7 @@ public void execute(StepExecution stepExecution) throws JobInterruptedException, @Before public void init() throws Exception { - jobRepository = new MapJobRepositoryFactoryBean().getJobRepository(); + jobRepository = new MapJobRepositoryFactoryBean().getObject(); execution = jobRepository.createJobExecution("flow", new JobParameters()); } @@ -139,12 +140,36 @@ public void testBuildSplit() throws Exception { assertEquals(2, execution.getStepExecutions().size()); } + @Test + public void testBuildSplitUsingStartAndAdd_BATCH_2346() throws Exception { + Flow subflow1 = new FlowBuilder("subflow1").from(step2).end(); + Flow subflow2 = new FlowBuilder("subflow2").from(step3).end(); + Flow splitflow = new FlowBuilder("splitflow").start(subflow1).split(new SimpleAsyncTaskExecutor()) + .add(subflow2).build(); + + FlowJobBuilder builder = new JobBuilder("flow").repository(jobRepository).start(splitflow).end(); + builder.preventRestart().build().execute(execution); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(2, execution.getStepExecutions().size()); + } + + @Test + public void testBuildSplit_BATCH_2282() throws Exception { + Flow flow1 = new FlowBuilder("subflow1").from(step1).end(); + Flow flow2 = new FlowBuilder("subflow2").from(step2).end(); + Flow splitFlow = new FlowBuilder("splitflow").split(new SimpleAsyncTaskExecutor()).add(flow1, flow2).build(); + FlowJobBuilder builder = new JobBuilder("flow").repository(jobRepository).start(splitFlow).end(); + builder.preventRestart().build().execute(execution); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(2, execution.getStepExecutions().size()); + } + @Test public void testBuildDecision() throws Exception { JobExecutionDecider decider = new JobExecutionDecider() { private int count = 0; @Override - public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) { + public FlowExecutionStatus decide(JobExecution jobExecution, @Nullable StepExecution stepExecution) { count++; return count<2 ? new FlowExecutionStatus("ONGOING") : FlowExecutionStatus.COMPLETED; } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowExecutionExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowExecutionExceptionTests.java index 12d8914598..04415a3e3f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowExecutionExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowExecutionExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowExecutionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowExecutionTests.java index be007997f9..112caba58e 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowExecutionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowExecutionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowJobFailureTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowJobFailureTests.java index 5f93182fb8..9bd7e31088 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowJobFailureTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowJobFailureTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.job.flow; import static org.junit.Assert.assertEquals; @@ -37,7 +52,7 @@ public class FlowJobFailureTests { @Before public void init() throws Exception { - JobRepository jobRepository = new MapJobRepositoryFactoryBean().getJobRepository(); + JobRepository jobRepository = new MapJobRepositoryFactoryBean().getObject(); job.setJobRepository(jobRepository); execution = jobRepository.createJobExecution("job", new JobParameters()); } @@ -45,7 +60,7 @@ public void init() throws Exception { @Test public void testStepFailure() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); StepState step = new StepState(new StepSupport("step")); transitions.add(StateTransition.createStateTransition(step, ExitStatus.FAILED.getExitCode(), "end0")); transitions.add(StateTransition.createStateTransition(step, ExitStatus.COMPLETED.getExitCode(), "end1")); @@ -61,7 +76,7 @@ public void testStepFailure() throws Exception { @Test public void testStepStatusUnknown() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); StepState step = new StepState(new StepSupport("step") { @Override public void execute(StepExecution stepExecution) throws JobInterruptedException, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowJobTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowJobTests.java index e9a09856a2..d253fc54a2 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowJobTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowJobTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,16 +15,6 @@ */ package org.springframework.batch.core.job.flow; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.fail; - -import java.util.ArrayList; -import java.util.Arrays; -import java.util.Collections; -import java.util.List; - import org.junit.Before; import org.junit.Test; import org.springframework.batch.core.BatchStatus; @@ -35,6 +25,7 @@ import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.Step; import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.job.flow.support.DefaultStateTransitionComparator; import org.springframework.batch.core.job.flow.support.SimpleFlow; import org.springframework.batch.core.job.flow.support.StateTransition; import org.springframework.batch.core.job.flow.support.state.DecisionState; @@ -42,13 +33,29 @@ import org.springframework.batch.core.job.flow.support.state.FlowState; import org.springframework.batch.core.job.flow.support.state.SplitState; import org.springframework.batch.core.job.flow.support.state.StepState; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.jsr.partition.JsrPartitionHandler; +import org.springframework.batch.core.jsr.step.PartitionStep; +import org.springframework.batch.core.jsr.partition.JsrStepExecutionSplitter; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.dao.JobExecutionDao; import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; import org.springframework.batch.core.step.StepSupport; +import org.springframework.lang.Nullable; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collections; +import java.util.List; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; /** * @author Dave Syer + * @author Michael Minella * */ public class FlowJobTests { @@ -68,7 +75,7 @@ public void setUp() throws Exception { MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(); factory.afterPropertiesSet(); jobExecutionDao = factory.getJobExecutionDao(); - jobRepository = (JobRepository) factory.getObject(); + jobRepository = factory.getObject(); job.setJobRepository(jobRepository); jobExecution = jobRepository.createJobExecution("job", new JobParameters()); } @@ -76,7 +83,7 @@ public void setUp() throws Exception { @Test public void testGetSteps() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); @@ -90,7 +97,7 @@ public void testGetSteps() throws Exception { @Test public void testTwoSteps() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); StepState step2 = new StepState(new StubStep("step2")); transitions.add(StateTransition.createStateTransition(step2, ExitStatus.FAILED.getExitCode(), "end0")); @@ -109,7 +116,7 @@ public void testTwoSteps() throws Exception { @Test public void testFailedStep() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StateSupport("step1", FlowExecutionStatus.FAILED), "step2")); StepState step2 = new StepState(new StubStep("step2")); @@ -130,7 +137,7 @@ public void testFailedStep() throws Exception { @Test public void testFailedStepRestarted() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); State step2State = new StateSupport("step2") { @Override @@ -167,7 +174,7 @@ public FlowExecutionStatus handle(FlowExecutor executor) throws Exception { @Test public void testStoppingStep() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); State state2 = new StateSupport("step2", FlowExecutionStatus.FAILED); transitions.add(StateTransition.createStateTransition(state2, ExitStatus.FAILED.getExitCode(), "end0")); @@ -188,7 +195,7 @@ public void testStoppingStep() throws Exception { @Test public void testInterrupted() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1") { @Override public void execute(StepExecution stepExecution) throws JobInterruptedException { @@ -211,7 +218,7 @@ public void execute(StepExecution stepExecution) throws JobInterruptedException @Test public void testUnknownStatusStopsJob() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1") { @Override public void execute(StepExecution stepExecution) throws JobInterruptedException { @@ -239,7 +246,7 @@ public void testInterruptedSplit() throws Exception { SimpleFlow flow1 = new SimpleFlow("flow1"); SimpleFlow flow2 = new SimpleFlow("flow2"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1") { @Override public void execute(StepExecution stepExecution) throws JobInterruptedException { @@ -254,12 +261,12 @@ public void execute(StepExecution stepExecution) throws JobInterruptedException } }), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); - flow1.setStateTransitions(new ArrayList(transitions)); + flow1.setStateTransitions(new ArrayList<>(transitions)); flow1.afterPropertiesSet(); - flow2.setStateTransitions(new ArrayList(transitions)); + flow2.setStateTransitions(new ArrayList<>(transitions)); flow2.afterPropertiesSet(); - transitions = new ArrayList(); + transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new SplitState(Arrays. asList(flow1, flow2), "split"), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); @@ -282,7 +289,7 @@ public void execute(StepExecution stepExecution) throws JobInterruptedException @Test public void testInterruptedException() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1") { @Override public void execute(StepExecution stepExecution) throws JobInterruptedException { @@ -307,7 +314,7 @@ public void testInterruptedSplitException() throws Exception { SimpleFlow flow1 = new SimpleFlow("flow1"); SimpleFlow flow2 = new SimpleFlow("flow2"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1") { @Override public void execute(StepExecution stepExecution) throws JobInterruptedException { @@ -315,12 +322,12 @@ public void execute(StepExecution stepExecution) throws JobInterruptedException } }), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); - flow1.setStateTransitions(new ArrayList(transitions)); + flow1.setStateTransitions(new ArrayList<>(transitions)); flow1.afterPropertiesSet(); - flow2.setStateTransitions(new ArrayList(transitions)); + flow2.setStateTransitions(new ArrayList<>(transitions)); flow2.afterPropertiesSet(); - transitions = new ArrayList(); + transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new SplitState(Arrays. asList(flow1, flow2), "split"), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); @@ -339,7 +346,7 @@ public void execute(StepExecution stepExecution) throws JobInterruptedException @Test public void testEndStateStopped() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "end")); transitions.add(StateTransition .createStateTransition(new EndState(FlowExecutionStatus.STOPPED, "end"), "step2")); @@ -358,7 +365,7 @@ public void testEndStateStopped() throws Exception { public void testEndStateFailed() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "end")); transitions .add(StateTransition.createStateTransition(new EndState(FlowExecutionStatus.FAILED, "end"), "step2")); @@ -379,7 +386,7 @@ public void testEndStateFailed() throws Exception { @Test public void testEndStateStoppedWithRestart() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "end")); transitions.add(StateTransition .createStateTransition(new EndState(FlowExecutionStatus.STOPPED, "end"), "step2")); @@ -407,7 +414,7 @@ public void testEndStateStoppedWithRestart() throws Exception { @Test public void testBranching() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); StepState step1 = new StepState(new StubStep("step1")); transitions.add(StateTransition.createStateTransition(step1, "step2")); transitions.add(StateTransition.createStateTransition(step1, "COMPLETED", "step3")); @@ -422,6 +429,7 @@ public void testBranching() throws Exception { transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end2"))); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end3"))); flow.setStateTransitions(transitions); + flow.setStateTransitionComparator(new DefaultStateTransitionComparator()); job.setFlow(flow); job.afterPropertiesSet(); job.doExecute(jobExecution); @@ -433,7 +441,7 @@ public void testBranching() throws Exception { @Test public void testBasicFlow() throws Throwable { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step")), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); flow.setStateTransitions(transitions); @@ -451,13 +459,13 @@ public void testDecisionFlow() throws Throwable { SimpleFlow flow = new SimpleFlow("job"); JobExecutionDecider decider = new JobExecutionDecider() { @Override - public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) { + public FlowExecutionStatus decide(JobExecution jobExecution, @Nullable StepExecution stepExecution) { assertNotNull(stepExecution); return new FlowExecutionStatus("SWITCH"); } }; - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "decision")); DecisionState decision = new DecisionState(decider, "decision"); transitions.add(StateTransition.createStateTransition(decision, "step2")); @@ -473,6 +481,7 @@ public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepE transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end2"))); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end3"))); flow.setStateTransitions(transitions); + flow.setStateTransitionComparator(new DefaultStateTransitionComparator()); job.setFlow(flow); job.doExecute(jobExecution); @@ -492,13 +501,13 @@ public void testDecisionFlowWithExceptionInDecider() throws Throwable { SimpleFlow flow = new SimpleFlow("job"); JobExecutionDecider decider = new JobExecutionDecider() { @Override - public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) { + public FlowExecutionStatus decide(JobExecution jobExecution, @Nullable StepExecution stepExecution) { assertNotNull(stepExecution); throw new RuntimeException("Foo"); } }; - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "decision")); DecisionState decision = new DecisionState(decider, "decision"); transitions.add(StateTransition.createStateTransition(decision, "step2")); @@ -533,7 +542,7 @@ public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepE @Test public void testGetStepExists() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); @@ -547,10 +556,42 @@ public void testGetStepExists() throws Exception { assertEquals("step2", step.getName()); } + @Test + public void testGetPartitionedStep() throws Exception { + SimpleFlow flow = new SimpleFlow("job"); + List transitions = new ArrayList<>(); + PartitionStep step = new PartitionStep(); + step.setName("step1"); + JsrPartitionHandler partitionHandler = new JsrPartitionHandler(); + partitionHandler.setPropertyContext(new BatchPropertyContext()); + partitionHandler.setPartitions(3); + partitionHandler.setJobRepository(jobRepository); + partitionHandler.setStep(new StubStep("subStep")); + partitionHandler.afterPropertiesSet(); + step.setPartitionHandler(partitionHandler); + step.setStepExecutionSplitter(new JsrStepExecutionSplitter(jobRepository, false, "step1", true)); + step.setJobRepository(jobRepository); + step.afterPropertiesSet(); + transitions.add(StateTransition.createStateTransition(new StepState("job.step", step), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + job.setFlow(flow); + job.afterPropertiesSet(); + + job.execute(jobRepository.createJobExecution("partitionJob", new JobParameters())); + + assertEquals(3, step.getStepNames().size()); + Step subStep = job.getStep("step1:partition0"); + assertNotNull(subStep); + assertEquals("subStep", subStep.getName()); + assertNull(job.getStep("step that does not exist")); + } + @Test public void testGetStepExistsWithPrefix() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState("job.step", new StubStep("step")), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); flow.setStateTransitions(transitions); @@ -567,7 +608,7 @@ public void testGetStepExistsWithPrefix() throws Exception { @Test public void testGetStepNamesWithPrefix() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState("job.step", new StubStep("step")), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); flow.setStateTransitions(transitions); @@ -582,7 +623,7 @@ public void testGetStepNamesWithPrefix() throws Exception { @Test public void testGetStepNotExists() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); @@ -598,7 +639,7 @@ public void testGetStepNotExists() throws Exception { @Test public void testGetStepNotStepState() throws Exception { SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); @@ -614,14 +655,14 @@ public void testGetStepNotStepState() throws Exception { @Test public void testGetStepNestedFlow() throws Exception { SimpleFlow nested = new SimpleFlow("nested"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end1")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end1"))); nested.setStateTransitions(transitions); nested.afterPropertiesSet(); SimpleFlow flow = new SimpleFlow("job"); - transitions = new ArrayList(); + transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "nested")); transitions.add(StateTransition.createStateTransition(new FlowState(nested, "nested"), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); @@ -630,7 +671,7 @@ public void testGetStepNestedFlow() throws Exception { job.setFlow(flow); job.afterPropertiesSet(); - List names = new ArrayList(job.getStepNames()); + List names = new ArrayList<>(job.getStepNames()); Collections.sort(names); assertEquals("[step1, step2]", names.toString()); } @@ -641,18 +682,18 @@ public void testGetStepSplitFlow() throws Exception { SimpleFlow flow1 = new SimpleFlow("flow1"); SimpleFlow flow2 = new SimpleFlow("flow2"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); - flow1.setStateTransitions(new ArrayList(transitions)); + flow1.setStateTransitions(new ArrayList<>(transitions)); flow1.afterPropertiesSet(); - transitions = new ArrayList(); + transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end1")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end1"))); - flow2.setStateTransitions(new ArrayList(transitions)); + flow2.setStateTransitions(new ArrayList<>(transitions)); flow2.afterPropertiesSet(); - transitions = new ArrayList(); + transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new SplitState(Arrays. asList(flow1, flow2), "split"), "end2")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end2"))); @@ -661,7 +702,7 @@ public void testGetStepSplitFlow() throws Exception { job.setFlow(flow); job.afterPropertiesSet(); - List names = new ArrayList(job.getStepNames()); + List names = new ArrayList<>(job.getStepNames()); Collections.sort(names); assertEquals("[step1, step2]", names.toString()); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowStepTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowStepTests.java index a4c084114f..b953df51ee 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowStepTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/FlowStepTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -49,7 +49,7 @@ public class FlowStepTests { @Before public void setUp() throws Exception { - jobRepository = new MapJobRepositoryFactoryBean().getJobRepository(); + jobRepository = new MapJobRepositoryFactoryBean().getObject(); jobExecution = jobRepository.createJobExecution("job", new JobParameters()); } @@ -73,7 +73,7 @@ public void testDoExecute() throws Exception { step.setJobRepository(jobRepository); SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); StepState step2 = new StepState(new StubStep("step2")); transitions.add(StateTransition.createStateTransition(step2, ExitStatus.FAILED.getExitCode(), "end0")); @@ -105,7 +105,7 @@ public void testDoExecuteAndFail() throws Exception { step.setJobRepository(jobRepository); SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); StepState step2 = new StepState(new StubStep("step2", true)); transitions.add(StateTransition.createStateTransition(step2, ExitStatus.FAILED.getExitCode(), "end0")); @@ -141,7 +141,7 @@ public void testExecuteWithParentContext() throws Exception { step.setJobRepository(jobRepository); SimpleFlow flow = new SimpleFlow("job"); - List transitions = new ArrayList(); + List transitions = new ArrayList<>(); transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "end0")); transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); flow.setStateTransitions(transitions); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/StateSupport.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/StateSupport.java index 317cf0d901..2b1217657c 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/StateSupport.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/StateSupport.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,7 +26,7 @@ */ public class StateSupport extends AbstractState { - private FlowExecutionStatus status; + protected FlowExecutionStatus status; public StateSupport(String name) { this(name, FlowExecutionStatus.COMPLETED); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/DefaultStateTransitionComparatorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/DefaultStateTransitionComparatorTests.java new file mode 100644 index 0000000000..6a762f12c9 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/DefaultStateTransitionComparatorTests.java @@ -0,0 +1,82 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.job.flow.support; + +import static org.junit.Assert.assertEquals; + +import java.util.Comparator; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.job.flow.State; +import org.springframework.batch.core.job.flow.StateSupport; + +public class DefaultStateTransitionComparatorTests { + + private State state = new StateSupport("state1"); + private Comparator comparator; + + @Before + public void setUp() throws Exception { + comparator = new DefaultStateTransitionComparator(); + } + + @Test + public void testSimpleOrderingEqual() { + StateTransition transition = StateTransition.createStateTransition(state, "CONTIN???LE", "start"); + assertEquals(0, comparator.compare(transition, transition)); + } + + @Test + public void testSimpleOrderingMoreGeneral() { + StateTransition transition = StateTransition.createStateTransition(state, "CONTIN???LE", "start"); + StateTransition other = StateTransition.createStateTransition(state, "CONTINUABLE", "start"); + assertEquals(1, comparator.compare(transition, other)); + assertEquals(-1, comparator.compare(other, transition)); + } + + @Test + public void testSimpleOrderingMostGeneral() { + StateTransition transition = StateTransition.createStateTransition(state, "*", "start"); + StateTransition other = StateTransition.createStateTransition(state, "CONTINUABLE", "start"); + assertEquals(1, comparator.compare(transition, other)); + assertEquals(-1, comparator.compare(other, transition)); + } + + @Test + public void testSubstringAndWildcard() { + StateTransition transition = StateTransition.createStateTransition(state, "CONTIN*", "start"); + StateTransition other = StateTransition.createStateTransition(state, "CONTINUABLE", "start"); + assertEquals(1, comparator.compare(transition, other)); + assertEquals(-1, comparator.compare(other, transition)); + } + + @Test + public void testSimpleOrderingMostToNextGeneral() { + StateTransition transition = StateTransition.createStateTransition(state, "*", "start"); + StateTransition other = StateTransition.createStateTransition(state, "C?", "start"); + assertEquals(1, comparator.compare(transition, other)); + assertEquals(-1, comparator.compare(other, transition)); + } + + @Test + public void testSimpleOrderingAdjacent() { + StateTransition transition = StateTransition.createStateTransition(state, "CON*", "start"); + StateTransition other = StateTransition.createStateTransition(state, "CON?", "start"); + assertEquals(1, comparator.compare(transition, other)); + assertEquals(-1, comparator.compare(other, transition)); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/JobFlowExecutorSupport.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/JobFlowExecutorSupport.java index c4be052e0e..a97ec37a35 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/JobFlowExecutorSupport.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/JobFlowExecutorSupport.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,6 +25,7 @@ import org.springframework.batch.core.job.flow.FlowExecutionStatus; import org.springframework.batch.core.job.flow.FlowExecutor; import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.lang.Nullable; /** * @author Dave Syer @@ -43,6 +44,7 @@ public JobExecution getJobExecution() { return null; } + @Nullable @Override public StepExecution getStepExecution() { return null; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/SimpleFlowTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/SimpleFlowTests.java index 91f812024c..293a504f5b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/SimpleFlowTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/SimpleFlowTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2013 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,6 +25,7 @@ import java.util.Collections; import java.util.List; +import org.junit.Before; import org.junit.Test; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.job.flow.FlowExecution; @@ -36,13 +37,19 @@ /** * @author Dave Syer - * + * @author Michael Minella + * */ public class SimpleFlowTests { - private SimpleFlow flow = new SimpleFlow("job"); + protected SimpleFlow flow; + + protected FlowExecutor executor = new JobFlowExecutorSupport(); - private FlowExecutor executor = new JobFlowExecutorSupport(); + @Before + public void setUp() { + flow = new SimpleFlow("job"); + } @Test(expected = IllegalArgumentException.class) public void testEmptySteps() throws Exception { @@ -115,7 +122,7 @@ public void testOneStepWithListenerCallsClose() throws Exception { flow.setStateTransitions(Collections.singletonList(StateTransition.createEndStateTransition(new StubState( "step1")))); flow.afterPropertiesSet(); - final List list = new ArrayList(); + final List list = new ArrayList<>(); executor = new JobFlowExecutorSupport() { @Override public void close(FlowExecution result) { @@ -177,7 +184,8 @@ public void testBranching() throws Exception { flow.setStateTransitions(collect(StateTransition.createStateTransition(new StubState("step1"), "step2"), StateTransition.createStateTransition(new StubState("step1"), ExitStatus.COMPLETED.getExitCode(), "step3"), StateTransition.createEndStateTransition(new StubState("step2")), StateTransition - .createEndStateTransition(new StubState("step3")))); + .createEndStateTransition(new StubState("step3")))); + flow.setStateTransitionComparator(new DefaultStateTransitionComparator()); flow.afterPropertiesSet(); FlowExecution execution = flow.start(executor); assertEquals(FlowExecutionStatus.COMPLETED, execution.getStatus()); @@ -203,30 +211,21 @@ public void testGetStateDoesNotExist() throws Exception { assertNull(state); } - private List collect(StateTransition s1, StateTransition s2) { - List list = new ArrayList(); - list.add(s1); - list.add(s2); - return list; - } + protected List collect(StateTransition... states) { + List list = new ArrayList<>(); - private List collect(StateTransition s1, StateTransition s2, StateTransition s3) { - List list = collect(s1, s2); - list.add(s3); - return list; - } + for (StateTransition stateTransition : states) { + list.add(stateTransition); + } - private List collect(StateTransition s1, StateTransition s2, StateTransition s3, StateTransition s4) { - List list = collect(s1, s2, s3); - list.add(s4); return list; } /** * @author Dave Syer - * + * */ - private static class StubState extends StateSupport { + protected static class StubState extends StateSupport { /** * @param string diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/StateTransitionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/StateTransitionTests.java index 28d1889b3c..e095a01997 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/StateTransitionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/StateTransitionTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2013 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,7 +15,6 @@ */ package org.springframework.batch.core.job.flow.support; -import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertNull; import static org.junit.Assert.assertTrue; @@ -25,12 +24,13 @@ /** * @author Dave Syer - * + * @author Michael Minella + * */ public class StateTransitionTests { State state = new StateSupport("state1"); - + @Test public void testIsEnd() { StateTransition transition = StateTransition.createEndStateTransition(state, ""); @@ -74,52 +74,6 @@ public void testMatchesPlaceholder() { assertTrue(transition.matches("CONTINUABLE")); } - @Test - public void testSimpleOrderingEqual() { - StateTransition transition = StateTransition.createStateTransition(state, "CONTIN???LE", "start"); - assertEquals(0, transition.compareTo(transition)); - } - - @Test - public void testSimpleOrderingMoreGeneral() { - StateTransition transition = StateTransition.createStateTransition(state, "CONTIN???LE", "start"); - StateTransition other = StateTransition.createStateTransition(state, "CONTINUABLE", "start"); - assertEquals(1, transition.compareTo(other)); - assertEquals(-1, other.compareTo(transition)); - } - - @Test - public void testSimpleOrderingMostGeneral() { - StateTransition transition = StateTransition.createStateTransition(state, "*", "start"); - StateTransition other = StateTransition.createStateTransition(state, "CONTINUABLE", "start"); - assertEquals(1, transition.compareTo(other)); - assertEquals(-1, other.compareTo(transition)); - } - - @Test - public void testSubstringAndWildcard() { - StateTransition transition = StateTransition.createStateTransition(state, "CONTIN*", "start"); - StateTransition other = StateTransition.createStateTransition(state, "CONTINUABLE", "start"); - assertEquals(1, transition.compareTo(other)); - assertEquals(-1, other.compareTo(transition)); - } - - @Test - public void testSimpleOrderingMostToNextGeneral() { - StateTransition transition = StateTransition.createStateTransition(state, "*", "start"); - StateTransition other = StateTransition.createStateTransition(state, "C?", "start"); - assertEquals(1, transition.compareTo(other)); - assertEquals(-1, other.compareTo(transition)); - } - - @Test - public void testSimpleOrderingAdjacent() { - StateTransition transition = StateTransition.createStateTransition(state, "CON*", "start"); - StateTransition other = StateTransition.createStateTransition(state, "CON?", "start"); - assertEquals(1, transition.compareTo(other)); - assertEquals(-1, other.compareTo(transition)); - } - @Test public void testToString() { StateTransition transition = StateTransition.createStateTransition(state, "CONTIN???LE", "start"); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/state/EndStateTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/state/EndStateTests.java index d34d6019e0..2eb6dd197e 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/state/EndStateTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/state/EndStateTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/state/SimpleFlowExecutionAggregatorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/state/SimpleFlowExecutionAggregatorTests.java index c70d937138..fd0677901f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/state/SimpleFlowExecutionAggregatorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/state/SimpleFlowExecutionAggregatorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/state/SplitStateTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/state/SplitStateTests.java index 4f26c60f61..a4ff7088ac 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/state/SplitStateTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/job/flow/support/state/SplitStateTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -43,7 +43,7 @@ public class SplitStateTests { @Test public void testBasicHandling() throws Exception { - Collection flows = new ArrayList(); + Collection flows = new ArrayList<>(); Flow flow1 = mock(Flow.class); Flow flow2 = mock(Flow.class); flows.add(flow1); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/AbstractJsrTestCase.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/AbstractJsrTestCase.java new file mode 100644 index 0000000000..571efc9288 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/AbstractJsrTestCase.java @@ -0,0 +1,122 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import java.util.Date; +import java.util.Properties; +import java.util.concurrent.TimeoutException; + +import javax.batch.operations.JobOperator; +import javax.batch.runtime.BatchRuntime; +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.JobExecution; +import javax.batch.runtime.Metric; +import javax.batch.runtime.StepExecution; + +/** + * @author mminella + */ +public abstract class AbstractJsrTestCase { + + protected static JobOperator operator; + + static { + operator = BatchRuntime.getJobOperator(); + } + + /** + * Executes a job and waits for it's status to be any of {@link javax.batch.runtime.BatchStatus#STOPPED}, + * {@link javax.batch.runtime.BatchStatus#COMPLETED}, or {@link javax.batch.runtime.BatchStatus#FAILED}. If the job does not + * reach one of those statuses within the given timeout, a {@link java.util.concurrent.TimeoutException} is + * thrown. + * + * @param jobName Name of the job to run + * @param properties Properties to pass the job + * @param timeout length of time to wait for a job to finish + * @return the {@link javax.batch.runtime.JobExecution} for the final state of the job + * @throws java.util.concurrent.TimeoutException if the timeout occurs + */ + public static JobExecution runJob(String jobName, Properties properties, long timeout) throws TimeoutException { + System.out.println("Operator = " + operator); + long executionId = operator.start(jobName, properties); + JobExecution execution = operator.getJobExecution(executionId); + + Date curDate = new Date(); + BatchStatus curBatchStatus = execution.getBatchStatus(); + + while(true) { + if(curBatchStatus == BatchStatus.STOPPED || curBatchStatus == BatchStatus.COMPLETED || curBatchStatus == BatchStatus.FAILED) { + break; + } + + if(new Date().getTime() - curDate.getTime() > timeout) { + throw new TimeoutException("Job processing did not complete in time"); + } + + execution = operator.getJobExecution(executionId); + curBatchStatus = execution.getBatchStatus(); + } + return execution; + } + + /** + * Restarts a job and waits for it's status to be any of {@link BatchStatus#STOPPED}, + * {@link BatchStatus#COMPLETED}, or {@link BatchStatus#FAILED}. If the job does not + * reach one of those statuses within the given timeout, a {@link java.util.concurrent.TimeoutException} is + * thrown. + * + * @param executionId The execution id to restart + * @param properties The Properties to pass to the new run + * @param timeout The length of time to wait for the job to run + * @return the {@link JobExecution} for the final state of the job + * @throws java.util.concurrent.TimeoutException if the timeout occurs + */ + public static JobExecution restartJob(long executionId, Properties properties, long timeout) throws TimeoutException { + long restartId = operator.restart(executionId, properties); + JobExecution execution = operator.getJobExecution(restartId); + + Date curDate = new Date(); + BatchStatus curBatchStatus = execution.getBatchStatus(); + + while(true) { + if(curBatchStatus == BatchStatus.STOPPED || curBatchStatus == BatchStatus.COMPLETED || curBatchStatus == BatchStatus.FAILED) { + break; + } + + if(new Date().getTime() - curDate.getTime() > timeout) { + throw new TimeoutException("Job processing did not complete in time"); + } + + execution = operator.getJobExecution(restartId); + curBatchStatus = execution.getBatchStatus(); + } + return execution; + } + + public static Metric getMetric(StepExecution stepExecution, Metric.MetricType type) { + Metric[] metrics = stepExecution.getMetrics(); + + for (Metric metric : metrics) { + if(metric.getType() == type) { + return metric; + } + } + + return null; + } + + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/ChunkListenerAdapterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/ChunkListenerAdapterTests.java new file mode 100644 index 0000000000..aa0aef9db9 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/ChunkListenerAdapterTests.java @@ -0,0 +1,98 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.mockito.Mockito.doThrow; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import javax.batch.api.chunk.listener.ChunkListener; +import javax.batch.operations.BatchRuntimeException; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.UncheckedTransactionException; + +public class ChunkListenerAdapterTests { + + private ChunkListenerAdapter adapter; + @Mock + private ChunkListener delegate; + @Mock + private ChunkContext context; + + @Before + public void setUp() { + MockitoAnnotations.initMocks(this); + adapter = new ChunkListenerAdapter(delegate); + } + + @Test(expected=IllegalArgumentException.class) + public void testNullDelegate() { + adapter = new ChunkListenerAdapter(null); + } + + @Test + public void testBeforeChunk() throws Exception { + adapter.beforeChunk(null); + + verify(delegate).beforeChunk(); + } + + @Test(expected=UncheckedTransactionException.class) + public void testBeforeChunkException() throws Exception { + doThrow(new Exception("This is expected")).when(delegate).beforeChunk(); + adapter.beforeChunk(null); + } + + @Test + public void testAfterChunk() throws Exception { + adapter.afterChunk(null); + + verify(delegate).afterChunk(); + } + + @Test(expected=UncheckedTransactionException.class) + public void testAfterChunkException() throws Exception { + doThrow(new Exception("This is expected")).when(delegate).afterChunk(); + adapter.afterChunk(null); + } + + @Test(expected=BatchRuntimeException.class) + public void testAfterChunkErrorNullContext() throws Exception { + adapter.afterChunkError(null); + } + + @Test(expected=UncheckedTransactionException.class) + public void testAfterChunkErrorException() throws Exception { + doThrow(new Exception("This is expected")).when(delegate).afterChunk(); + adapter.afterChunk(null); + } + + @Test + public void testAfterChunkError() throws Exception { + Exception exception = new Exception("This was expected"); + + when(context.getAttribute(org.springframework.batch.core.ChunkListener.ROLLBACK_EXCEPTION_KEY)).thenReturn(exception); + + adapter.afterChunkError(context); + + verify(delegate).onError(exception); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/ItemProcessListenerAdapterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/ItemProcessListenerAdapterTests.java new file mode 100644 index 0000000000..33ad9ba6f3 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/ItemProcessListenerAdapterTests.java @@ -0,0 +1,106 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.mockito.Mockito.doThrow; +import static org.mockito.Mockito.verify; + +import javax.batch.api.chunk.listener.ItemProcessListener; +import javax.batch.operations.BatchRuntimeException; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +public class ItemProcessListenerAdapterTests { + + private ItemProcessListenerAdapter adapter; + @Mock + private ItemProcessListener delegate; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + adapter = new ItemProcessListenerAdapter<>(delegate); + } + + @Test(expected=IllegalArgumentException.class) + public void testNullCreation() { + adapter = new ItemProcessListenerAdapter<>(null); + } + + @Test + public void testBeforeProcess() throws Exception { + String item = "This is my item"; + + adapter.beforeProcess(item); + + verify(delegate).beforeProcess(item); + } + + @Test(expected=BatchRuntimeException.class) + public void testBeforeProcessException() throws Exception { + Exception exception = new Exception("This should occur"); + String item = "This is the bad item"; + + doThrow(exception).when(delegate).beforeProcess(item); + + adapter.beforeProcess(item); + } + + @Test + public void testAfterProcess() throws Exception { + String item = "This is the input"; + String result = "This is the output"; + + adapter.afterProcess(item, result); + + verify(delegate).afterProcess(item, result); + } + + @Test(expected=BatchRuntimeException.class) + public void testAfterProcessException() throws Exception { + String item = "This is the input"; + String result = "This is the output"; + Exception exception = new Exception("This is expected"); + + doThrow(exception).when(delegate).afterProcess(item, result); + + adapter.afterProcess(item, result); + } + + @Test + public void testOnProcessError() throws Exception { + String item = "This is the input"; + Exception cause = new Exception("This was the cause"); + + adapter.onProcessError(item, cause); + + verify(delegate).onProcessError(item, cause); + } + + @Test(expected=BatchRuntimeException.class) + public void testOnProcessErrorException() throws Exception { + String item = "This is the input"; + Exception cause = new Exception("This was the cause"); + Exception exception = new Exception("This is expected"); + + doThrow(exception).when(delegate).onProcessError(item, cause); + + adapter.onProcessError(item, cause); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/ItemReadListenerAdapterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/ItemReadListenerAdapterTests.java new file mode 100644 index 0000000000..d5412a3a8b --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/ItemReadListenerAdapterTests.java @@ -0,0 +1,97 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.mockito.Mockito.doThrow; +import static org.mockito.Mockito.verify; + +import javax.batch.api.chunk.listener.ItemReadListener; +import javax.batch.operations.BatchRuntimeException; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +public class ItemReadListenerAdapterTests { + + private ItemReadListenerAdapter adapter; + @Mock + private ItemReadListener delegate; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + adapter = new ItemReadListenerAdapter<>(delegate); + } + + @Test(expected=IllegalArgumentException.class) + public void testNullDelegate() { + adapter = new ItemReadListenerAdapter<>(null); + } + + @Test + public void testBeforeRead() throws Exception { + adapter.beforeRead(); + + verify(delegate).beforeRead(); + } + + @Test(expected=BatchRuntimeException.class) + public void testBeforeReadException() throws Exception { + doThrow(new Exception("Should occur")).when(delegate).beforeRead(); + + adapter.beforeRead(); + } + + @Test + public void testAfterRead() throws Exception { + String item = "item"; + + adapter.afterRead(item); + + verify(delegate).afterRead(item); + } + + @Test(expected=BatchRuntimeException.class) + public void testAfterReadException() throws Exception { + String item = "item"; + Exception expected = new Exception("expected"); + + doThrow(expected).when(delegate).afterRead(item); + + adapter.afterRead(item); + } + + @Test + public void testOnReadError() throws Exception { + Exception cause = new Exception ("cause"); + + adapter.onReadError(cause); + + verify(delegate).onReadError(cause); + } + + @Test(expected=BatchRuntimeException.class) + public void testOnReadErrorException() throws Exception { + Exception cause = new Exception ("cause"); + Exception result = new Exception("result"); + + doThrow(result).when(delegate).onReadError(cause); + + adapter.onReadError(cause); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/ItemWriteListenerAdapterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/ItemWriteListenerAdapterTests.java new file mode 100644 index 0000000000..d1e7e85872 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/ItemWriteListenerAdapterTests.java @@ -0,0 +1,96 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.mockito.Mockito.doThrow; +import static org.mockito.Mockito.verify; + +import java.util.ArrayList; +import java.util.List; + +import javax.batch.api.chunk.listener.ItemWriteListener; +import javax.batch.operations.BatchRuntimeException; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +@SuppressWarnings({"rawtypes", "unchecked"}) +public class ItemWriteListenerAdapterTests { + + private ItemWriteListenerAdapter adapter; + @Mock + private ItemWriteListener delegate; + private List items = new ArrayList(); + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + adapter = new ItemWriteListenerAdapter<>(delegate); + } + + @Test(expected=IllegalArgumentException.class) + public void testCreateWithNull() { + adapter = new ItemWriteListenerAdapter<>(null); + } + + @Test + public void testBeforeWrite() throws Exception { + adapter.beforeWrite(items); + + verify(delegate).beforeWrite(items); + } + + @Test(expected=BatchRuntimeException.class) + public void testBeforeTestWriteException() throws Exception { + doThrow(new Exception("expected")).when(delegate).beforeWrite(items); + + adapter.beforeWrite(items); + } + + @Test + public void testAfterWrite() throws Exception { + adapter.afterWrite(items); + + verify(delegate).afterWrite(items); + } + + @Test(expected=BatchRuntimeException.class) + public void testAfterTestWriteException() throws Exception { + doThrow(new Exception("expected")).when(delegate).afterWrite(items); + + adapter.afterWrite(items); + } + + @Test + public void testOnWriteError() throws Exception { + Exception cause = new Exception("cause"); + + adapter.onWriteError(cause, items); + + verify(delegate).onWriteError(items, cause); + } + + @Test(expected=BatchRuntimeException.class) + public void testOnWriteErrorException() throws Exception { + Exception cause = new Exception("cause"); + + doThrow(new Exception("expected")).when(delegate).onWriteError(items, cause); + + adapter.onWriteError(cause, items); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JobListenerAdapterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JobListenerAdapterTests.java new file mode 100644 index 0000000000..e948ae748f --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JobListenerAdapterTests.java @@ -0,0 +1,73 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.mockito.Mockito.doThrow; +import static org.mockito.Mockito.verify; + +import javax.batch.api.listener.JobListener; +import javax.batch.operations.BatchRuntimeException; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +public class JobListenerAdapterTests { + + private JobListenerAdapter adapter; + @Mock + private JobListener delegate; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + adapter = new JobListenerAdapter(delegate); + } + + @Test(expected=IllegalArgumentException.class) + public void testCreateWithNull() { + adapter = new JobListenerAdapter(null); + } + + @Test + public void testBeforeJob() throws Exception { + adapter.beforeJob(null); + + verify(delegate).beforeJob(); + } + + @Test(expected=BatchRuntimeException.class) + public void testBeforeJobException() throws Exception { + doThrow(new Exception("expected")).when(delegate).beforeJob(); + + adapter.beforeJob(null); + } + + @Test + public void testAfterJob() throws Exception { + adapter.afterJob(null); + + verify(delegate).afterJob(); + } + + @Test(expected=BatchRuntimeException.class) + public void testAfterJobException() throws Exception { + doThrow(new Exception("expected")).when(delegate).afterJob(); + + adapter.afterJob(null); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrJobContextFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrJobContextFactoryBeanTests.java new file mode 100644 index 0000000000..fbcee4b199 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrJobContextFactoryBeanTests.java @@ -0,0 +1,126 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +import java.util.ArrayList; +import java.util.HashSet; +import java.util.List; +import java.util.Set; +import java.util.concurrent.Callable; +import java.util.concurrent.Future; + +import javax.batch.runtime.context.JobContext; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.beans.factory.FactoryBeanNotInitializedException; +import org.springframework.core.task.AsyncTaskExecutor; +import org.springframework.core.task.SimpleAsyncTaskExecutor; + +public class JsrJobContextFactoryBeanTests { + + private JsrJobContextFactoryBean factoryBean; + private BatchPropertyContext propertyContext; + + @Before + public void setUp() throws Exception { + StepSynchronizationManager.close(); + propertyContext = new BatchPropertyContext(); + factoryBean = new JsrJobContextFactoryBean(); + } + + @After + public void tearDown() throws Exception { + factoryBean.close(); + StepSynchronizationManager.close(); + } + + @Test + public void testInitialCreationSingleThread() throws Exception { + factoryBean.setJobExecution(new JobExecution(5L)); + factoryBean.setBatchPropertyContext(propertyContext); + + assertTrue(factoryBean.getObjectType().isAssignableFrom(JobContext.class)); + assertFalse(factoryBean.isSingleton()); + + JobContext jobContext1 = factoryBean.getObject(); + JobContext jobContext2 = factoryBean.getObject(); + + assertEquals(5L, jobContext1.getExecutionId()); + assertEquals(5L, jobContext2.getExecutionId()); + assertTrue(jobContext1 == jobContext2); + } + + @Test + public void testInitialCreationSingleThreadUsingStepScope() throws Exception { + factoryBean.setBatchPropertyContext(propertyContext); + + StepSynchronizationManager.register(new StepExecution("step1", new JobExecution(5L))); + + JobContext jobContext = factoryBean.getObject(); + + assertEquals(5L, jobContext.getExecutionId()); + StepSynchronizationManager.close(); + } + + @Test(expected=FactoryBeanNotInitializedException.class) + public void testNoJobExecutionProvided() throws Exception { + factoryBean.getObject(); + } + + @Test + public void testOneJobContextPerThread() throws Exception { + List> jobContexts = new ArrayList<>(); + + AsyncTaskExecutor executor = new SimpleAsyncTaskExecutor(); + + for(int i = 0; i < 4; i++) { + final long count = i; + jobContexts.add(executor.submit(new Callable() { + + @Override + public JobContext call() throws Exception { + try { + StepSynchronizationManager.register(new StepExecution("step" + count, new JobExecution(count))); + JobContext context = factoryBean.getObject(); + Thread.sleep(1000L); + return context; + } catch (Throwable ignore) { + return null; + }finally { + StepSynchronizationManager.release(); + } + } + })); + } + + Set contexts = new HashSet<>(); + for (Future future : jobContexts) { + contexts.add(future.get()); + } + + assertEquals(4, contexts.size()); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrJobContextTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrJobContextTests.java new file mode 100644 index 0000000000..9ec6cb1918 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrJobContextTests.java @@ -0,0 +1,127 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import java.util.Properties; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; + +public class JsrJobContextTests { + + private JsrJobContext context; + @Mock + private JobExecution execution; + @Mock + private JobInstance instance; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + + Properties properties = new Properties(); + properties.put("jobLevelProperty1", "jobLevelValue1"); + + context = new JsrJobContext(); + context.setProperties(properties); + context.setJobExecution(execution); + + when(execution.getJobInstance()).thenReturn(instance); + } + + @Test(expected=IllegalArgumentException.class) + public void testCreateWithNull() { + context = new JsrJobContext(); + context.setJobExecution(null); + } + + @Test + public void testGetJobName() { + when(instance.getJobName()).thenReturn("jobName"); + + assertEquals("jobName", context.getJobName()); + } + + @Test + public void testTransientUserData() { + context.setTransientUserData("This is my data"); + assertEquals("This is my data", context.getTransientUserData()); + } + + @Test + public void testGetInstanceId() { + when(instance.getId()).thenReturn(5L); + + assertEquals(5L, context.getInstanceId()); + } + + @Test + public void testGetExecutionId() { + when(execution.getId()).thenReturn(5L); + + assertEquals(5L, context.getExecutionId()); + } + + @Test + public void testJobParameters() { + JobParameters params = new JobParametersBuilder() + .addString("key1", "value1") + .toJobParameters(); + + when(execution.getJobParameters()).thenReturn(params); + + assertEquals("value1", execution.getJobParameters().getString("key1")); + } + + @Test + public void testJobProperties() { + assertEquals("jobLevelValue1", context.getProperties().get("jobLevelProperty1")); + } + + @Test + public void testGetBatchStatus() { + when(execution.getStatus()).thenReturn(BatchStatus.COMPLETED); + + assertEquals(javax.batch.runtime.BatchStatus.COMPLETED, context.getBatchStatus()); + } + + @Test + public void testExitStatus() { + context.setExitStatus("my exit status"); + verify(execution).setExitStatus(new ExitStatus("my exit status")); + + when(execution.getExitStatus()).thenReturn(new ExitStatus("exit")); + assertEquals("exit", context.getExitStatus()); + } + + @Test + public void testInitialNullExitStatus() { + when(execution.getExitStatus()).thenReturn(new ExitStatus("exit")); + assertEquals(null, context.getExitStatus()); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrJobExecutionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrJobExecutionTests.java new file mode 100644 index 0000000000..14948fa729 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrJobExecutionTests.java @@ -0,0 +1,78 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; + +import java.util.Date; +import java.util.Properties; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.converter.JobParametersConverterSupport; + +public class JsrJobExecutionTests { + + private JsrJobExecution adapter; + + @Before + public void setUp() throws Exception { + JobInstance instance = new JobInstance(2L, "job name"); + + JobParameters params = new JobParametersBuilder().addString("key1", "value1").toJobParameters(); + + org.springframework.batch.core.JobExecution execution = new org.springframework.batch.core.JobExecution(instance, params); + + execution.setId(5L); + execution.setCreateTime(new Date(0)); + execution.setEndTime(new Date(999999999L)); + execution.setExitStatus(new ExitStatus("exit status")); + execution.setLastUpdated(new Date(12345)); + execution.setStartTime(new Date(98765)); + execution.setStatus(BatchStatus.FAILED); + execution.setVersion(21); + + adapter = new JsrJobExecution(execution, new JobParametersConverterSupport()); + } + + @Test(expected=IllegalArgumentException.class) + public void testCreateWithNull() { + adapter = new JsrJobExecution(null, new JobParametersConverterSupport()); + } + + @Test + public void testGetBasicValues() { + assertEquals(javax.batch.runtime.BatchStatus.FAILED, adapter.getBatchStatus()); + assertEquals(new Date(0), adapter.getCreateTime()); + assertEquals(new Date(999999999L), adapter.getEndTime()); + assertEquals(5L, adapter.getExecutionId()); + assertEquals("exit status", adapter.getExitStatus()); + assertEquals("job name", adapter.getJobName()); + assertEquals(new Date(12345), adapter.getLastUpdatedTime()); + assertEquals(new Date(98765), adapter.getStartTime()); + + Properties props = adapter.getJobParameters(); + + assertEquals("value1", props.get("key1")); + assertNull(props.get(JsrJobParametersConverter.JOB_RUN_ID)); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrJobParametersConverterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrJobParametersConverterTests.java new file mode 100644 index 0000000000..0961106f8c --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrJobParametersConverterTests.java @@ -0,0 +1,126 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +import java.util.Map.Entry; +import java.util.Properties; +import java.util.Set; + +import javax.sql.DataSource; + +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.PooledEmbeddedDataSource; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder; + +public class JsrJobParametersConverterTests { + + private JsrJobParametersConverter converter; + private static DataSource dataSource; + + @BeforeClass + public static void setupDatabase() { + dataSource = new PooledEmbeddedDataSource(new EmbeddedDatabaseBuilder(). + addScript("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql"). + addScript("classpath:org/springframework/batch/core/schema-hsqldb.sql"). + build()); + } + + @Before + public void setUp() throws Exception { + converter = new JsrJobParametersConverter(dataSource); + converter.afterPropertiesSet(); + } + + @Test + public void testNullJobParameters() { + Properties props = converter.getProperties((JobParameters) null); + assertNotNull(props); + Set> properties = props.entrySet(); + assertEquals(1, properties.size()); + assertTrue(props.containsKey(JsrJobParametersConverter.JOB_RUN_ID)); + } + + @Test + public void testStringJobParameters() { + JobParameters parameters = new JobParametersBuilder().addString("key", "value", false).toJobParameters(); + Properties props = converter.getProperties(parameters); + assertNotNull(props); + Set> properties = props.entrySet(); + assertEquals(2, properties.size()); + assertTrue(props.containsKey(JsrJobParametersConverter.JOB_RUN_ID)); + assertEquals("value", props.getProperty("key")); + } + + @Test + public void testNonStringJobParameters() { + JobParameters parameters = new JobParametersBuilder().addLong("key", 5L, false).toJobParameters(); + Properties props = converter.getProperties(parameters); + assertNotNull(props); + Set> properties = props.entrySet(); + assertEquals(2, properties.size()); + assertTrue(props.containsKey(JsrJobParametersConverter.JOB_RUN_ID)); + assertEquals("5", props.getProperty("key")); + } + + @Test + public void testJobParametersWithRunId() { + JobParameters parameters = new JobParametersBuilder().addLong("key", 5L, false).addLong(JsrJobParametersConverter.JOB_RUN_ID, 2L).toJobParameters(); + Properties props = converter.getProperties(parameters); + assertNotNull(props); + Set> properties = props.entrySet(); + assertEquals(2, properties.size()); + assertEquals("2", props.getProperty(JsrJobParametersConverter.JOB_RUN_ID)); + assertEquals("5", props.getProperty("key")); + } + + @Test + public void testNullProperties() { + JobParameters parameters = converter.getJobParameters((Properties)null); + assertNotNull(parameters); + assertEquals(1, parameters.getParameters().size()); + assertTrue(parameters.getParameters().containsKey(JsrJobParametersConverter.JOB_RUN_ID)); + } + + @Test + public void testProperties() { + Properties properties = new Properties(); + properties.put("key", "value"); + JobParameters parameters = converter.getJobParameters(properties); + assertEquals(2, parameters.getParameters().size()); + assertEquals("value", parameters.getString("key")); + assertTrue(parameters.getParameters().containsKey(JsrJobParametersConverter.JOB_RUN_ID)); + } + + @Test + public void testPropertiesWithRunId() { + Properties properties = new Properties(); + properties.put("key", "value"); + properties.put(JsrJobParametersConverter.JOB_RUN_ID, "3"); + JobParameters parameters = converter.getJobParameters(properties); + assertEquals(2, parameters.getParameters().size()); + assertEquals("value", parameters.getString("key")); + assertEquals(Long.valueOf(3L), parameters.getLong(JsrJobParametersConverter.JOB_RUN_ID)); + assertTrue(parameters.getParameters().get(JsrJobParametersConverter.JOB_RUN_ID).isIdentifying()); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrStepContextFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrStepContextFactoryBeanTests.java new file mode 100644 index 0000000000..006e1a85c6 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrStepContextFactoryBeanTests.java @@ -0,0 +1,199 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.mockito.Mockito.when; + +import java.util.ArrayList; +import java.util.HashSet; +import java.util.List; +import java.util.Properties; +import java.util.Set; +import java.util.concurrent.Callable; +import java.util.concurrent.Future; + +import javax.batch.runtime.context.StepContext; + +import org.junit.After; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.beans.factory.FactoryBeanNotInitializedException; +import org.springframework.core.task.AsyncTaskExecutor; +import org.springframework.core.task.SimpleAsyncTaskExecutor; + +public class JsrStepContextFactoryBeanTests { + + private JsrStepContextFactoryBean factory; + @Mock + private BatchPropertyContext propertyContext; + + /** + * Added to clean up left overs from other tests. + * @throws Exception + */ + @BeforeClass + public static void setUpClass() throws Exception { + StepSynchronizationManager.close(); + } + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + factory = new JsrStepContextFactoryBean(); + factory.setBatchPropertyContext(propertyContext); + } + + @After + public void tearDown() throws Exception { + StepSynchronizationManager.close(); + } + + @Test(expected=FactoryBeanNotInitializedException.class) + public void testNoStepExecutionRegistered() throws Exception { + factory.getObject(); + } + + @Test + public void getObjectSingleThread() throws Exception { + StepSynchronizationManager.register(new StepExecution("step1", new JobExecution(5L), 3L)); + + StepContext context1 = factory.getObject(); + StepContext context2 = factory.getObject(); + + assertTrue(context1 == context2); + assertEquals(3L, context1.getStepExecutionId()); + + StepSynchronizationManager.close(); + + StepSynchronizationManager.register(new StepExecution("step2", new JobExecution(5L), 2L)); + + StepContext context3 = factory.getObject(); + StepContext context4 = factory.getObject(); + + assertTrue(context3 == context4); + assertTrue(context3 != context2); + assertEquals(2L, context3.getStepExecutionId()); + + StepSynchronizationManager.close(); + } + + @Test + public void getObjectSingleThreadWithProperties() throws Exception { + Properties props = new Properties(); + props.put("key1", "value1"); + + when(propertyContext.getStepProperties("step3")).thenReturn(props); + + StepSynchronizationManager.register(new StepExecution("step3", new JobExecution(5L), 3L)); + + StepContext context1 = factory.getObject(); + StepContext context2 = factory.getObject(); + + assertTrue(context1 == context2); + assertEquals(3L, context1.getStepExecutionId()); + assertEquals("value1", context1.getProperties().get("key1")); + + StepSynchronizationManager.close(); + } + + @Test + public void getObjectMultiThread() throws Exception { + List> stepContexts = new ArrayList<>(); + + AsyncTaskExecutor executor = new SimpleAsyncTaskExecutor(); + + for(int i = 0; i < 4; i++) { + final long count = i; + stepContexts.add(executor.submit(new Callable() { + + @Override + public StepContext call() throws Exception { + try { + StepSynchronizationManager.register(new StepExecution("step" + count, new JobExecution(count))); + StepContext context = factory.getObject(); + Thread.sleep(1000L); + return context; + } catch (Throwable ignore) { + return null; + }finally { + StepSynchronizationManager.close(); + } + } + })); + } + + Set contexts = new HashSet<>(); + for (Future future : stepContexts) { + contexts.add(future.get()); + } + + assertEquals(4, contexts.size()); + } + + @Test + public void getObjectMultiThreadWithProperties() throws Exception { + for(int i = 0; i < 4; i++) { + Properties props = new Properties(); + props.put("step" + i, "step" + i + "value"); + + when(propertyContext.getStepProperties("step" + i)).thenReturn(props); + } + + List> stepContexts = new ArrayList<>(); + + AsyncTaskExecutor executor = new SimpleAsyncTaskExecutor(); + + for(int i = 0; i < 4; i++) { + final long count = i; + stepContexts.add(executor.submit(new Callable() { + + @Override + public StepContext call() throws Exception { + try { + StepSynchronizationManager.register(new StepExecution("step" + count, new JobExecution(count))); + StepContext context = factory.getObject(); + Thread.sleep(1000L); + return context; + } catch (Throwable ignore) { + return null; + }finally { + StepSynchronizationManager.close(); + } + } + })); + } + + Set contexts = new HashSet<>(); + for (Future future : stepContexts) { + contexts.add(future.get()); + } + + assertEquals(4, contexts.size()); + + for (StepContext stepContext : contexts) { + assertEquals(stepContext.getStepName() + "value", stepContext.getProperties().get(stepContext.getStepName())); + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrStepContextTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrStepContextTests.java new file mode 100644 index 0000000000..86ae93b65e --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrStepContextTests.java @@ -0,0 +1,158 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +import java.util.Properties; + +import javax.batch.runtime.Metric; +import javax.batch.runtime.context.StepContext; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.util.ExecutionContextUserSupport; +import org.springframework.util.ClassUtils; + +public class JsrStepContextTests { + + private StepExecution stepExecution; + private StepContext stepContext; + private ExecutionContext executionContext; + private ExecutionContextUserSupport executionContextUserSupport = new ExecutionContextUserSupport(ClassUtils.getShortName(JsrStepContext.class)); + + @Before + public void setUp() throws Exception { + JobExecution jobExecution = new JobExecution(1L, new JobParametersBuilder().addString("key", "value").toJobParameters()); + + stepExecution = new StepExecution("testStep", jobExecution); + stepExecution.setId(5L); + stepExecution.setStatus(BatchStatus.STARTED); + stepExecution.setExitStatus(new ExitStatus("customExitStatus")); + stepExecution.setCommitCount(1); + stepExecution.setFilterCount(2); + stepExecution.setProcessSkipCount(3); + stepExecution.setReadCount(4); + stepExecution.setReadSkipCount(5); + stepExecution.setRollbackCount(6); + stepExecution.setWriteCount(7); + stepExecution.setWriteSkipCount(8); + executionContext = new ExecutionContext(); + stepExecution.setExecutionContext(executionContext); + + Properties properties = new Properties(); + properties.put("key", "value"); + + stepContext = new JsrStepContext(stepExecution, properties); + stepContext.setTransientUserData("This is my transient data"); + } + + @Test + public void testBasicProperties() { + assertEquals(javax.batch.runtime.BatchStatus.STARTED, stepContext.getBatchStatus()); + assertEquals(null, stepContext.getExitStatus()); + stepContext.setExitStatus("customExitStatus"); + assertEquals("customExitStatus", stepContext.getExitStatus()); + assertEquals(5L, stepContext.getStepExecutionId()); + assertEquals("testStep", stepContext.getStepName()); + assertEquals("This is my transient data", stepContext.getTransientUserData()); + + Properties params = stepContext.getProperties(); + assertEquals("value", params.get("key")); + + Metric[] metrics = stepContext.getMetrics(); + + for (Metric metric : metrics) { + switch (metric.getType()) { + case COMMIT_COUNT: + assertEquals(1, metric.getValue()); + break; + case FILTER_COUNT: + assertEquals(2, metric.getValue()); + break; + case PROCESS_SKIP_COUNT: + assertEquals(3, metric.getValue()); + break; + case READ_COUNT: + assertEquals(4, metric.getValue()); + break; + case READ_SKIP_COUNT: + assertEquals(5, metric.getValue()); + break; + case ROLLBACK_COUNT: + assertEquals(6, metric.getValue()); + break; + case WRITE_COUNT: + assertEquals(7, metric.getValue()); + break; + case WRITE_SKIP_COUNT: + assertEquals(8, metric.getValue()); + break; + default: + fail("Invalid metric type"); + } + } + } + + @Test + public void testSetExitStatus() { + stepContext.setExitStatus("new Exit Status"); + assertEquals("new Exit Status", stepExecution.getExitStatus().getExitCode()); + } + + @Test + public void testPersistentUserData() { + String data = "saved data"; + stepContext.setPersistentUserData(data); + assertEquals(data, stepContext.getPersistentUserData()); + assertEquals(data, executionContext.get(executionContextUserSupport.getKey("batch_jsr_persistentUserData"))); + } + + @Test + public void testGetExceptionEmpty() { + assertNull(stepContext.getException()); + } + + @Test + public void testGetExceptionException() { + stepExecution.addFailureException(new Exception("expected")); + assertEquals("expected", stepContext.getException().getMessage()); + } + + @Test + public void testGetExceptionThrowable() { + stepExecution.addFailureException(new Throwable("expected")); + assertTrue(stepContext.getException().getMessage().endsWith("expected")); + } + + @Test + public void testGetExceptionMultiple() { + stepExecution.addFailureException(new Exception("not me")); + stepExecution.addFailureException(new Exception("not me either")); + stepExecution.addFailureException(new Exception("me")); + + assertEquals("me", stepContext.getException().getMessage()); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrStepExecutionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrStepExecutionTests.java new file mode 100644 index 0000000000..6fd8b255e5 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/JsrStepExecutionTests.java @@ -0,0 +1,121 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; + +import java.util.Date; + +import javax.batch.runtime.Metric; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.item.util.ExecutionContextUserSupport; +import org.springframework.util.ClassUtils; + +public class JsrStepExecutionTests { + + private StepExecution stepExecution; + private javax.batch.runtime.StepExecution jsrStepExecution; + //The API that sets the persisted user data is on the JsrStepContext so the key within the ExecutionContext is JsrStepContext + private ExecutionContextUserSupport executionContextUserSupport = new ExecutionContextUserSupport(ClassUtils.getShortName(JsrStepContext.class)); + + @Before + public void setUp() throws Exception { + JobExecution jobExecution = new JobExecution(1L, new JobParametersBuilder().addString("key", "value").toJobParameters()); + + stepExecution = new StepExecution("testStep", jobExecution); + stepExecution.setId(5L); + stepExecution.setStatus(BatchStatus.STARTED); + stepExecution.setExitStatus(new ExitStatus("customExitStatus")); + stepExecution.setCommitCount(1); + stepExecution.setFilterCount(2); + stepExecution.setProcessSkipCount(3); + stepExecution.setReadCount(4); + stepExecution.setReadSkipCount(5); + stepExecution.setRollbackCount(6); + stepExecution.setWriteCount(7); + stepExecution.setWriteSkipCount(8); + stepExecution.setStartTime(new Date(0)); + stepExecution.setEndTime(new Date(10000000)); + stepExecution.getExecutionContext().put(executionContextUserSupport.getKey("batch_jsr_persistentUserData"), "persisted data"); + + jsrStepExecution = new JsrStepExecution(stepExecution); + } + + @Test(expected=IllegalArgumentException.class) + public void testWithNullStepExecution() { + new JsrStepExecution(null); + } + + @Test + public void testNullExitStatus() { + stepExecution.setExitStatus(null); + + assertNull(jsrStepExecution.getExitStatus()); + } + + @Test + public void testBaseValues() { + assertEquals(5L, jsrStepExecution.getStepExecutionId()); + assertEquals("testStep", jsrStepExecution.getStepName()); + assertEquals(javax.batch.runtime.BatchStatus.STARTED, jsrStepExecution.getBatchStatus()); + assertEquals(new Date(0), jsrStepExecution.getStartTime()); + assertEquals(new Date(10000000), jsrStepExecution.getEndTime()); + assertEquals("customExitStatus", jsrStepExecution.getExitStatus()); + assertEquals("persisted data", jsrStepExecution.getPersistentUserData()); + + Metric[] metrics = jsrStepExecution.getMetrics(); + + for (Metric metric : metrics) { + switch (metric.getType()) { + case COMMIT_COUNT: + assertEquals(1, metric.getValue()); + break; + case FILTER_COUNT: + assertEquals(2, metric.getValue()); + break; + case PROCESS_SKIP_COUNT: + assertEquals(3, metric.getValue()); + break; + case READ_COUNT: + assertEquals(4, metric.getValue()); + break; + case READ_SKIP_COUNT: + assertEquals(5, metric.getValue()); + break; + case ROLLBACK_COUNT: + assertEquals(6, metric.getValue()); + break; + case WRITE_COUNT: + assertEquals(7, metric.getValue()); + break; + case WRITE_SKIP_COUNT: + assertEquals(8, metric.getValue()); + break; + default: + fail("Invalid metric type"); + } + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/SimpleMetricTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/SimpleMetricTests.java new file mode 100644 index 0000000000..364b662eac --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/SimpleMetricTests.java @@ -0,0 +1,39 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.junit.Assert.assertEquals; + +import javax.batch.runtime.Metric; +import javax.batch.runtime.Metric.MetricType; + +import org.junit.Test; + +public class SimpleMetricTests { + + @Test(expected=IllegalArgumentException.class) + public void testNullType() { + new SimpleMetric(null, 0); + } + + @Test + public void test() { + Metric metric = new SimpleMetric(MetricType.FILTER_COUNT, 3); + + assertEquals(3, metric.getValue()); + assertEquals(MetricType.FILTER_COUNT, metric.getType()); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/StepListenerAdapterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/StepListenerAdapterTests.java new file mode 100644 index 0000000000..102dc31941 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/StepListenerAdapterTests.java @@ -0,0 +1,83 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.doThrow; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import javax.batch.api.listener.StepListener; +import javax.batch.operations.BatchRuntimeException; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.StepExecution; + +public class StepListenerAdapterTests { + + private StepListenerAdapter adapter; + @Mock + private StepListener delegate; + @Mock + private StepExecution execution; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + + adapter = new StepListenerAdapter(delegate); + } + + @Test(expected=IllegalArgumentException.class) + public void testCreateWithNull() { + adapter = new StepListenerAdapter(null); + } + + @Test + public void testBeforeStep() throws Exception { + adapter.beforeStep(null); + + verify(delegate).beforeStep(); + } + + @Test(expected=BatchRuntimeException.class) + public void testBeforeStepException() throws Exception { + doThrow(new Exception("expected")).when(delegate).beforeStep(); + + adapter.beforeStep(null); + } + + @Test + public void testAfterStep() throws Exception { + ExitStatus exitStatus = new ExitStatus("complete"); + when(execution.getExitStatus()).thenReturn(exitStatus); + + assertEquals(exitStatus, adapter.afterStep(execution)); + + verify(delegate).afterStep(); + } + + @Test(expected=BatchRuntimeException.class) + public void testAfterStepException() throws Exception { + doThrow(new Exception("expected")).when(delegate).afterStep(); + + adapter.afterStep(null); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/support/BatchPropertyContextTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/support/BatchPropertyContextTests.java new file mode 100644 index 0000000000..8c47bd41ce --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/support/BatchPropertyContextTests.java @@ -0,0 +1,210 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.support; + +import static org.junit.Assert.assertEquals; + +import java.util.HashMap; +import java.util.Map; +import java.util.Properties; + +import org.junit.Before; +import org.junit.Test; + +/** + *

      + * Test cases around {@link BatchPropertyContext}. + *

      + * + * @author Chris Schaefer + */ +public class BatchPropertyContextTests { + private Properties jobProperties = new Properties(); + private Map stepProperties = new HashMap<>(); + private Map artifactProperties = new HashMap<>(); + private Map> partitionProperties = new HashMap<>(); + private Map> stepArtifactProperties = new HashMap<>(); + + @SuppressWarnings("serial") + @Before + public void setUp() { + Properties step1Properties = new Properties(); + step1Properties.setProperty("step1PropertyName1", "step1PropertyValue1"); + step1Properties.setProperty("step1PropertyName2", "step1PropertyValue2"); + this.stepProperties.put("step1", step1Properties); + + Properties step2Properties = new Properties(); + step2Properties.setProperty("step2PropertyName1", "step2PropertyValue1"); + step2Properties.setProperty("step2PropertyName2", "step2PropertyValue2"); + this.stepProperties.put("step2", step2Properties); + + Properties jobProperties = new Properties(); + jobProperties.setProperty("jobProperty1", "jobProperty1value"); + jobProperties.setProperty("jobProperty2", "jobProperty2value"); + this.jobProperties.putAll(jobProperties); + + Properties artifactProperties = new Properties(); + artifactProperties.setProperty("deciderProperty1", "deciderProperty1value"); + artifactProperties.setProperty("deciderProperty2", "deciderProperty2value"); + this.artifactProperties.put("decider1", artifactProperties); + + final Properties stepArtifactProperties = new Properties(); + stepArtifactProperties.setProperty("readerProperty1", "readerProperty1value"); + stepArtifactProperties.setProperty("readerProperty2", "readerProperty2value"); + + this.stepArtifactProperties.put("step1", new HashMap() {{ + put("reader", stepArtifactProperties); + }}); + + final Properties partitionProperties = new Properties(); + partitionProperties.setProperty("writerProperty1", "writerProperty1valuePartition0"); + partitionProperties.setProperty("writerProperty2", "writerProperty2valuePartition0"); + + this.partitionProperties.put("step2:partition0", new HashMap() {{ + put("writer", partitionProperties); + }}); + + final Properties partitionStepProperties = new Properties(); + partitionStepProperties.setProperty("writerProperty1Step", "writerProperty1"); + partitionStepProperties.setProperty("writerProperty2Step", "writerProperty2"); + + this.partitionProperties.put("step2", new HashMap() {{ + put("writer", partitionStepProperties); + }}); + } + + @Test + public void testStepLevelProperties() { + BatchPropertyContext batchPropertyContext = new BatchPropertyContext(); + batchPropertyContext.setJobProperties(jobProperties); + batchPropertyContext.setStepProperties(stepProperties); + + Properties step1Properties = batchPropertyContext.getStepProperties("step1"); + assertEquals(2, step1Properties.size()); + assertEquals("step1PropertyValue1", step1Properties.getProperty("step1PropertyName1")); + assertEquals("step1PropertyValue2", step1Properties.getProperty("step1PropertyName2")); + + Properties step2Properties = batchPropertyContext.getStepProperties("step2"); + assertEquals(2, step2Properties.size()); + assertEquals("step2PropertyValue1", step2Properties.getProperty("step2PropertyName1")); + assertEquals("step2PropertyValue2", step2Properties.getProperty("step2PropertyName2")); + } + + @Test + public void testJobLevelProperties() { + BatchPropertyContext batchPropertyContext = new BatchPropertyContext(); + batchPropertyContext.setJobProperties(jobProperties); + + Properties jobProperties = batchPropertyContext.getJobProperties(); + assertEquals(2, jobProperties.size()); + assertEquals("jobProperty1value", jobProperties.getProperty("jobProperty1")); + assertEquals("jobProperty2value", jobProperties.getProperty("jobProperty2")); + } + + @Test + public void testAddPropertiesToExistingStep() { + BatchPropertyContext batchPropertyContext = new BatchPropertyContext(); + batchPropertyContext.setJobProperties(jobProperties); + batchPropertyContext.setStepProperties(stepProperties); + + Properties step1 = batchPropertyContext.getStepProperties("step1"); + assertEquals(2, step1.size()); + assertEquals("step1PropertyValue1", step1.getProperty("step1PropertyName1")); + assertEquals("step1PropertyValue2", step1.getProperty("step1PropertyName2")); + + Properties step1properties = new Properties(); + step1properties.setProperty("newStep1PropertyName", "newStep1PropertyValue"); + + batchPropertyContext.setStepProperties("step1", step1properties); + + Properties step1updated = batchPropertyContext.getStepProperties("step1"); + assertEquals(3, step1updated.size()); + assertEquals("step1PropertyValue1", step1updated.getProperty("step1PropertyName1")); + assertEquals("step1PropertyValue2", step1updated.getProperty("step1PropertyName2")); + assertEquals("newStep1PropertyValue", step1updated.getProperty("newStep1PropertyName")); + } + + @Test + public void testNonStepLevelArtifactProperties() { + BatchPropertyContext batchPropertyContext = new BatchPropertyContext(); + batchPropertyContext.setJobProperties(jobProperties); + batchPropertyContext.setArtifactProperties(artifactProperties); + batchPropertyContext.setStepProperties(stepProperties); + + Properties artifactProperties = batchPropertyContext.getArtifactProperties("decider1"); + assertEquals(2, artifactProperties.size()); + assertEquals("deciderProperty1value", artifactProperties.getProperty("deciderProperty1")); + assertEquals("deciderProperty2value", artifactProperties.getProperty("deciderProperty2")); + } + + @Test + public void testStepLevelArtifactProperties() { + BatchPropertyContext batchPropertyContext = new BatchPropertyContext(); + batchPropertyContext.setJobProperties(jobProperties); + batchPropertyContext.setArtifactProperties(artifactProperties); + batchPropertyContext.setStepProperties(stepProperties); + batchPropertyContext.setStepArtifactProperties(stepArtifactProperties); + + Properties artifactProperties = batchPropertyContext.getStepArtifactProperties("step1", "reader"); + assertEquals(4, artifactProperties.size()); + assertEquals("readerProperty1value", artifactProperties.getProperty("readerProperty1")); + assertEquals("readerProperty2value", artifactProperties.getProperty("readerProperty2")); + assertEquals("step1PropertyValue1", artifactProperties.getProperty("step1PropertyName1")); + assertEquals("step1PropertyValue2", artifactProperties.getProperty("step1PropertyName2")); + } + + @Test + public void testArtifactNonOverridingJobProperties() { + BatchPropertyContext batchPropertyContext = new BatchPropertyContext(); + batchPropertyContext.setJobProperties(jobProperties); + batchPropertyContext.setArtifactProperties(artifactProperties); + + Properties jobProperties = new Properties(); + jobProperties.setProperty("deciderProperty1", "decider1PropertyOverride"); + + batchPropertyContext.setJobProperties(jobProperties); + + Properties step1 = batchPropertyContext.getArtifactProperties("decider1"); + assertEquals(2, step1.size()); + assertEquals("deciderProperty1value", step1.getProperty("deciderProperty1")); + assertEquals("deciderProperty2value", step1.getProperty("deciderProperty2")); + + Properties job = batchPropertyContext.getJobProperties(); + assertEquals(3, job.size()); + assertEquals("decider1PropertyOverride", job.getProperty("deciderProperty1")); + assertEquals("jobProperty1value", job.getProperty("jobProperty1")); + assertEquals("jobProperty2value", job.getProperty("jobProperty2")); + } + + @Test + public void testPartitionProperties() { + BatchPropertyContext batchPropertyContext = new BatchPropertyContext(); + batchPropertyContext.setJobProperties(jobProperties); + batchPropertyContext.setArtifactProperties(artifactProperties); + batchPropertyContext.setStepProperties(stepProperties); + batchPropertyContext.setStepArtifactProperties(stepArtifactProperties); + batchPropertyContext.setStepArtifactProperties(partitionProperties); + + Properties artifactProperties = batchPropertyContext.getStepArtifactProperties("step2:partition0", "writer"); + assertEquals(6, artifactProperties.size()); + assertEquals("writerProperty1", artifactProperties.getProperty("writerProperty1Step")); + assertEquals("writerProperty2", artifactProperties.getProperty("writerProperty2Step")); + assertEquals("writerProperty1valuePartition0", artifactProperties.getProperty("writerProperty1")); + assertEquals("writerProperty2valuePartition0", artifactProperties.getProperty("writerProperty2")); + assertEquals("step2PropertyValue1", artifactProperties.getProperty("step2PropertyName1")); + assertEquals("step2PropertyValue2", artifactProperties.getProperty("step2PropertyName2")); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/BatchParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/BatchParserTests.java new file mode 100644 index 0000000000..407d0c6079 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/BatchParserTests.java @@ -0,0 +1,101 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.junit.Test; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.configuration.xml.DummyItemProcessor; +import org.springframework.batch.core.scope.StepScope; +import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor; +import org.springframework.beans.factory.support.GenericBeanDefinition; +import org.springframework.core.io.ClassPathResource; +import org.springframework.core.io.Resource; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +public class BatchParserTests { + + @Test + @SuppressWarnings("unchecked") + public void testRoseyScenario() throws Exception { + JsrXmlApplicationContext context = new JsrXmlApplicationContext(); + Resource batchXml = new ClassPathResource("/org/springframework/batch/core/jsr/configuration/xml/batch.xml"); + context.setValidating(false); + context.load(batchXml); + + GenericBeanDefinition stepScope = new GenericBeanDefinition(); + stepScope.setBeanClass(StepScope.class); + context.registerBeanDefinition("stepScope", stepScope); + + GenericBeanDefinition bd = new GenericBeanDefinition(); + bd.setBeanClass(AutowiredAnnotationBeanPostProcessor.class); + context.registerBeanDefinition("postProcessor", bd); + context.refresh(); + + ItemProcessor itemProcessor = context.getBean(ItemProcessor.class); + + assertNotNull(itemProcessor); + StepSynchronizationManager.register(new StepExecution("step1", new JobExecution(5l))); + assertEquals("Test", itemProcessor.process("Test")); + StepSynchronizationManager.close(); + + context.close(); + } + + @Test + @SuppressWarnings("unchecked") + public void testOverrideBeansFirst() throws Exception { + JsrXmlApplicationContext context = new JsrXmlApplicationContext(); + Resource overrideXml = new ClassPathResource("/org/springframework/batch/core/jsr/configuration/xml/override_batch.xml"); + Resource batchXml = new ClassPathResource("/org/springframework/batch/core/jsr/configuration/xml/batch.xml"); + + context.setValidating(false); + context.load(overrideXml, batchXml); + context.refresh(); + + ItemProcessor itemProcessor = context.getBean("itemProcessor", ItemProcessor.class); + + assertNotNull(itemProcessor); + StepSynchronizationManager.register(new StepExecution("step1", new JobExecution(5l))); + assertEquals("Test", itemProcessor.process("Test")); + StepSynchronizationManager.close(); + + context.close(); + } + + @Test + @SuppressWarnings({"resource", "rawtypes"}) + public void testOverrideBeansLast() { + JsrXmlApplicationContext context = new JsrXmlApplicationContext(); + Resource overrideXml = new ClassPathResource("/org/springframework/batch/core/jsr/configuration/xml/override_batch.xml"); + Resource batchXml = new ClassPathResource("/org/springframework/batch/core/jsr/configuration/xml/batch.xml"); + + context.setValidating(false); + context.load(batchXml, overrideXml); + context.refresh(); + + ItemProcessor processor = (ItemProcessor) context.getBean("itemProcessor"); + + assertNotNull(processor); + assertTrue(processor instanceof DummyItemProcessor); + context.close(); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ChunkListenerParsingTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ChunkListenerParsingTests.java new file mode 100644 index 0000000000..0aa0660d24 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ChunkListenerParsingTests.java @@ -0,0 +1,117 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import static org.junit.Assert.assertEquals; + +import java.util.List; + +import javax.batch.api.chunk.AbstractItemWriter; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ChunkListener; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class ChunkListenerParsingTests { + + @Autowired + public Job job; + + @Autowired + public JobLauncher jobLauncher; + + @Autowired + public SpringChunkListener springChunkListener; + + @Autowired + public JsrChunkListener jsrChunkListener; + + @Test + public void test() throws Exception { + JobExecution execution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.FAILED, execution.getStatus()); + assertEquals(3, execution.getStepExecutions().size()); + assertEquals(4, springChunkListener.beforeChunkCount); + assertEquals(3, springChunkListener.afterChunkCount); + assertEquals(4, jsrChunkListener.beforeChunkCount); + assertEquals(3, jsrChunkListener.afterChunkCount); + assertEquals(1, springChunkListener.afterChunkErrorCount); + assertEquals(1, jsrChunkListener.afterChunkErrorCount); + } + + public static class SpringChunkListener implements ChunkListener { + + protected int beforeChunkCount = 0; + protected int afterChunkCount = 0; + protected int afterChunkErrorCount = 0; + + @Override + public void beforeChunk(ChunkContext context) { + beforeChunkCount++; + } + + @Override + public void afterChunk(ChunkContext context) { + afterChunkCount++; + } + + @Override + public void afterChunkError(ChunkContext context) { + afterChunkErrorCount++; + } + } + + public static class JsrChunkListener implements javax.batch.api.chunk.listener.ChunkListener { + + protected int beforeChunkCount = 0; + protected int afterChunkCount = 0; + protected int afterChunkErrorCount = 0; + + @Override + public void beforeChunk() throws Exception { + beforeChunkCount++; + } + + @Override + public void onError(Exception ex) throws Exception { + afterChunkErrorCount++; + } + + @Override + public void afterChunk() throws Exception { + afterChunkCount++; + } + } + + public static class ErrorThrowingItemWriter extends AbstractItemWriter { + + @Override + public void writeItems(List items) throws Exception { + throw new Exception("This should cause the rollback"); + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/CountingItemProcessor.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/CountingItemProcessor.java new file mode 100644 index 0000000000..5444c3c55a --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/CountingItemProcessor.java @@ -0,0 +1,29 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import javax.batch.api.chunk.ItemProcessor; + + +public class CountingItemProcessor implements ItemProcessor { + protected int count = 0; + + @Override + public Object processItem(Object item) throws Exception { + count++; + return item; + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/CustomWiredJsrJobOperatorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/CustomWiredJsrJobOperatorTests.java new file mode 100644 index 0000000000..037c627c36 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/CustomWiredJsrJobOperatorTests.java @@ -0,0 +1,61 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.util.Date; +import java.util.Properties; +import java.util.concurrent.TimeoutException; + +import javax.batch.operations.JobOperator; +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.JobExecution; + +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +/** + * @author Michael Minella + */ +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration +public class CustomWiredJsrJobOperatorTests { + + @Autowired + JobOperator jobOperator; + + @Test + public void testRunningJobWithManuallyWiredJsrJobOperator() throws Exception { + Date startTime = new Date(); + long jobExecutionId = jobOperator.start("jsrJobOperatorTestJob", new Properties()); + + JobExecution jobExecution = jobOperator.getJobExecution(jobExecutionId); + + long timeout = startTime.getTime() + 10000; + + while(!jobExecution.getBatchStatus().equals(BatchStatus.COMPLETED)) { + Thread.sleep(500); + jobExecution = jobOperator.getJobExecution(jobExecutionId); + + if(new Date().getTime() > timeout) { + throw new TimeoutException("Job didn't finish within 10 seconds"); + } + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/DecisionStepFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/DecisionStepFactoryBeanTests.java new file mode 100644 index 0000000000..3626d92f66 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/DecisionStepFactoryBeanTests.java @@ -0,0 +1,85 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +import javax.batch.api.Decider; +import javax.batch.runtime.StepExecution; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.jsr.step.DecisionStep; + +public class DecisionStepFactoryBeanTests { + + private DecisionStepFactoryBean factoryBean; + + @Before + public void setUp() throws Exception { + factoryBean = new DecisionStepFactoryBean(); + } + + @Test + public void testGetObjectType() { + assertEquals(DecisionStep.class, factoryBean.getObjectType()); + } + + @Test + public void testIsSingleton() { + assertTrue(factoryBean.isSingleton()); + } + + @Test(expected=IllegalArgumentException.class) + public void testNullDeciderAndName() throws Exception { + factoryBean.afterPropertiesSet(); + } + + @Test(expected=IllegalArgumentException.class) + public void testNullDecider() throws Exception{ + factoryBean.setName("state1"); + factoryBean.afterPropertiesSet(); + } + + @Test(expected=IllegalArgumentException.class) + public void testNullName() throws Exception { + factoryBean.setDecider(new DeciderSupport()); + factoryBean.afterPropertiesSet(); + } + + @Test + public void testDeciderDeciderState() throws Exception { + factoryBean.setDecider(new DeciderSupport()); + factoryBean.setName("IL"); + + factoryBean.afterPropertiesSet(); + + Step step = factoryBean.getObject(); + + assertEquals("IL", step.getName()); + assertEquals(DecisionStep.class, step.getClass()); + } + + public static class DeciderSupport implements Decider { + + @Override + public String decide(StepExecution[] executions) throws Exception { + return null; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ExceptionHandlingParsingTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ExceptionHandlingParsingTests.java new file mode 100644 index 0000000000..454f9c6374 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ExceptionHandlingParsingTests.java @@ -0,0 +1,111 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.junit.Test; +import org.springframework.batch.core.jsr.AbstractJsrTestCase; + +import javax.batch.api.BatchProperty; +import javax.batch.api.chunk.ItemProcessor; +import javax.batch.operations.JobOperator; +import javax.batch.runtime.BatchRuntime; +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.JobExecution; +import javax.batch.runtime.Metric; +import javax.batch.runtime.StepExecution; +import javax.inject.Inject; +import java.util.List; +import java.util.Properties; + +import static org.junit.Assert.assertEquals; + +public class ExceptionHandlingParsingTests extends AbstractJsrTestCase { + + @Test + public void testSkippable() throws Exception { + JobOperator jobOperator = BatchRuntime.getJobOperator(); + + Properties jobParameters = new Properties(); + jobParameters.setProperty("run", "1"); + JobExecution execution1 = runJob("ExceptionHandlingParsingTests-context", jobParameters, 10000l); + + List stepExecutions = jobOperator.getStepExecutions(execution1.getExecutionId()); + assertEquals(BatchStatus.FAILED, execution1.getBatchStatus()); + assertEquals(1, stepExecutions.size()); + assertEquals(1, getMetric(stepExecutions.get(0), Metric.MetricType.PROCESS_SKIP_COUNT).getValue()); + + jobParameters = new Properties(); + jobParameters.setProperty("run", "2"); + JobExecution execution2 = restartJob(execution1.getExecutionId(), jobParameters, 10000l); + stepExecutions = jobOperator.getStepExecutions(execution2.getExecutionId()); + assertEquals(BatchStatus.FAILED, execution2.getBatchStatus()); + assertEquals(2, stepExecutions.size()); + + jobParameters = new Properties(); + jobParameters.setProperty("run", "3"); + JobExecution execution3 = restartJob(execution2.getExecutionId(), jobParameters, 10000l); + stepExecutions = jobOperator.getStepExecutions(execution3.getExecutionId()); + assertEquals(BatchStatus.COMPLETED, execution3.getBatchStatus()); + assertEquals(2, stepExecutions.size()); + + assertEquals(0, getMetric(stepExecutions.get(1), Metric.MetricType.ROLLBACK_COUNT).getValue()); + + jobParameters = new Properties(); + jobParameters.setProperty("run", "4"); + JobExecution execution4 = runJob("ExceptionHandlingParsingTests-context", jobParameters, 10000l); + stepExecutions = jobOperator.getStepExecutions(execution4.getExecutionId()); + assertEquals(BatchStatus.COMPLETED, execution4.getBatchStatus()); + assertEquals(3, stepExecutions.size()); + } + + public static class ProblemProcessor implements ItemProcessor { + + @Inject + @BatchProperty + private String runId = "0"; + + private boolean hasRetried = false; + + private void throwException(Object item) throws Exception { + int runId = Integer.parseInt(this.runId); + + if(runId == 1) { + if(item.equals("One")) { + throw new Exception("skip me"); + } else if(item.equals("Two")){ + throw new RuntimeException("But don't skip me"); + } + } else if(runId == 2) { + if(item.equals("Three") && !hasRetried) { + hasRetried = true; + throw new Exception("retry me"); + } else if(item.equals("Four")){ + throw new RuntimeException("But don't retry me"); + } + } else if(runId == 3) { + if(item.equals("Five")) { + throw new Exception("Don't rollback on my account"); + } + } + } + + @Override + public Object processItem(Object item) throws Exception { + throwException(item); + return item; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/FlowParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/FlowParserTests.java new file mode 100644 index 0000000000..fc0f54bb8e --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/FlowParserTests.java @@ -0,0 +1,124 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.junit.Test; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.jsr.AbstractJsrTestCase; + +import javax.batch.api.AbstractBatchlet; +import javax.batch.operations.JobOperator; +import javax.batch.runtime.BatchRuntime; +import javax.batch.runtime.JobExecution; +import javax.batch.runtime.StepExecution; +import javax.batch.runtime.context.StepContext; +import javax.inject.Inject; +import java.util.List; +import java.util.Properties; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +/** + *

      + * Unit tests around {@link FlowParser}. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class FlowParserTests extends AbstractJsrTestCase { + @Test + public void testDuplicateTransitionPatternsAllowed() throws Exception { + JobExecution stoppedExecution = runJob("FlowParserTests-context", new Properties(), 10000L); + assertEquals(ExitStatus.STOPPED.getExitCode(), stoppedExecution.getExitStatus()); + + JobExecution endedExecution = restartJob(stoppedExecution.getExecutionId(), new Properties(), 10000L); + assertEquals(ExitStatus.COMPLETED.getExitCode(), endedExecution.getExitStatus()); + } + + @Test + public void testWildcardAddedLastWhenUsedWithNextAttrAndNoTransitionElements() throws Exception { + JobExecution jobExecution = runJob("FlowParserTestsWildcardAndNextAttrJob", new Properties(), 1000L); + assertEquals(ExitStatus.FAILED.getExitCode(), jobExecution.getExitStatus()); + + JobOperator jobOperator = BatchRuntime.getJobOperator(); + List stepExecutions = jobOperator.getStepExecutions(jobExecution.getExecutionId()); + assertEquals(1, stepExecutions.size()); + StepExecution failedStep = stepExecutions.get(0); + assertTrue("step1".equals(failedStep.getStepName())); + } + + @Test + public void testStepGetsFailedTransitionWhenNextAttributePresent() throws Exception { + JobExecution jobExecution = runJob("FlowParserTestsStepGetsFailedTransitionWhenNextAttributePresent", new Properties(), 10000L); + assertEquals(ExitStatus.FAILED.getExitCode(), jobExecution.getExitStatus()); + + JobOperator jobOperator = BatchRuntime.getJobOperator(); + List stepExecutions = jobOperator.getStepExecutions(jobExecution.getExecutionId()); + assertEquals(1, stepExecutions.size()); + StepExecution failedStep = stepExecutions.get(0); + assertTrue("failedExitStatusStep".equals(failedStep.getStepName())); + assertTrue("FAILED".equals(failedStep.getExitStatus())); + } + + @Test + public void testStepNoOverrideWhenNextAndFailedTransitionElementExists() throws Exception { + JobExecution jobExecution = runJob("FlowParserTestsStepNoOverrideWhenNextAndFailedTransitionElementExists", new Properties(), 10000L); + assertEquals(ExitStatus.FAILED.getExitCode(), jobExecution.getExitStatus()); + + JobOperator jobOperator = BatchRuntime.getJobOperator(); + List stepExecutions = jobOperator.getStepExecutions(jobExecution.getExecutionId()); + assertEquals(1, stepExecutions.size()); + StepExecution failedStep = stepExecutions.get(0); + assertTrue("failedExitStatusStepDontOverride".equals(failedStep.getStepName())); + assertTrue("CUSTOM_FAIL".equals(failedStep.getExitStatus())); + } + + public static class TestBatchlet extends AbstractBatchlet { + private static int CNT; + + @Inject + private StepContext stepContext; + + @Override + public String process() throws Exception { + String exitCode = "DISTINCT"; + + if("step3".equals(stepContext.getStepName())) { + exitCode = CNT % 2 == 0 ? "DISTINCT" : "RESTART"; + CNT++; + } + + if("failedExitStatusStep".equals(stepContext.getStepName())) { + exitCode = "FAILED"; + } + + if("failedExitStatusStepDontOverride".equals(stepContext.getStepName())) { + exitCode = "CUSTOM_FAIL"; + } + + return exitCode; + } + } + + public static class FailingTestBatchlet extends AbstractBatchlet { + @Override + public String process() throws Exception { + throw new RuntimeException("blah"); + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ItemListenerParsingTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ItemListenerParsingTests.java new file mode 100644 index 0000000000..c27e95b1e7 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ItemListenerParsingTests.java @@ -0,0 +1,188 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import static org.junit.Assert.assertEquals; + +import java.util.List; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ItemProcessListener; +import org.springframework.batch.core.ItemReadListener; +import org.springframework.batch.core.ItemWriteListener; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.lang.Nullable; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class ItemListenerParsingTests { + + @Autowired + public Job job; + + @Autowired + public JobLauncher jobLauncher; + + @Autowired + public SpringItemListener springListener; + + @Autowired + public JsrItemListener jsrListener; + + @Test + public void test() throws Exception { + JobExecution execution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(3, execution.getStepExecutions().size()); + assertEquals(6, springListener.beforeReadCount); + assertEquals(4, springListener.afterReadCount); + assertEquals(4, springListener.beforeProcessCount); + assertEquals(4, springListener.afterProcessCount); + assertEquals(2, springListener.beforeWriteCount); + assertEquals(2, springListener.afterWriteCount); + assertEquals(6, jsrListener.beforeReadCount); + assertEquals(4, jsrListener.afterReadCount); + assertEquals(4, jsrListener.beforeProcessCount); + assertEquals(4, jsrListener.afterProcessCount); + assertEquals(2, jsrListener.beforeWriteCount); + assertEquals(2, jsrListener.afterWriteCount); + } + + public static class SpringItemListener implements ItemReadListener, ItemProcessListener, ItemWriteListener { + + protected int beforeReadCount = 0; + protected int afterReadCount = 0; + protected int onReadErrorCount = 0; + protected int beforeProcessCount = 0; + protected int afterProcessCount = 0; + protected int onProcessErrorCount = 0; + protected int beforeWriteCount = 0; + protected int afterWriteCount = 0; + protected int onWriteErrorCount = 0; + + @Override + public void beforeRead() { + beforeReadCount++; + } + + @Override + public void afterRead(Object item) { + afterReadCount++; + } + + @Override + public void onReadError(Exception ex) { + onReadErrorCount++; + } + + @Override + public void beforeWrite(List items) { + beforeWriteCount++; + } + + @Override + public void afterWrite(List items) { + afterWriteCount++; + } + + @Override + public void onWriteError(Exception exception, List items) { + onWriteErrorCount++; + } + + @Override + public void beforeProcess(Object item) { + beforeProcessCount++; + } + + @Override + public void afterProcess(Object item, @Nullable Object result) { + afterProcessCount++; + } + + @Override + public void onProcessError(Object item, Exception e) { + onProcessErrorCount++; + } + } + + public static class JsrItemListener implements javax.batch.api.chunk.listener.ItemReadListener, javax.batch.api.chunk.listener.ItemProcessListener, javax.batch.api.chunk.listener.ItemWriteListener { + + protected int beforeReadCount = 0; + protected int afterReadCount = 0; + protected int onReadErrorCount = 0; + protected int beforeProcessCount = 0; + protected int afterProcessCount = 0; + protected int onProcessErrorCount = 0; + protected int beforeWriteCount = 0; + protected int afterWriteCount = 0; + protected int onWriteErrorCount = 0; + + @Override + public void beforeWrite(List items) throws Exception { + beforeWriteCount++; + } + + @Override + public void afterWrite(List items) throws Exception { + afterWriteCount++; + } + + @Override + public void onWriteError(List items, Exception ex) + throws Exception { + onWriteErrorCount++; + } + + @Override + public void beforeProcess(Object item) throws Exception { + beforeProcessCount++; + } + + @Override + public void afterProcess(Object item, Object result) throws Exception { + afterProcessCount++; + } + + @Override + public void onProcessError(Object item, Exception ex) throws Exception { + onProcessErrorCount++; + } + + @Override + public void beforeRead() throws Exception { + beforeReadCount++; + } + + @Override + public void afterRead(Object item) throws Exception { + afterReadCount++; + } + + @Override + public void onReadError(Exception ex) throws Exception { + onReadErrorCount++; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ItemSkipParsingTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ItemSkipParsingTests.java new file mode 100644 index 0000000000..190db67a05 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ItemSkipParsingTests.java @@ -0,0 +1,176 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.junit.Test; +import org.springframework.batch.core.jsr.AbstractJsrTestCase; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemWriter; +import org.springframework.lang.Nullable; + +import javax.batch.api.chunk.listener.SkipProcessListener; +import javax.batch.api.chunk.listener.SkipReadListener; +import javax.batch.api.chunk.listener.SkipWriteListener; +import javax.batch.operations.JobOperator; +import javax.batch.runtime.BatchRuntime; +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.Metric; +import javax.batch.runtime.StepExecution; +import java.util.ArrayList; +import java.util.List; +import java.util.Properties; + +import static org.junit.Assert.assertEquals; + +public class ItemSkipParsingTests extends AbstractJsrTestCase { + + @Test + public void test() throws Exception { + javax.batch.runtime.JobExecution execution = runJob("ItemSkipParsingTests-context", new Properties(), 10000l); + JobOperator jobOperator = BatchRuntime.getJobOperator(); + + assertEquals(BatchStatus.FAILED, execution.getBatchStatus()); + List stepExecutions = jobOperator.getStepExecutions(execution.getExecutionId()); + assertEquals(1, getMetric(stepExecutions.get(0), Metric.MetricType.READ_SKIP_COUNT).getValue()); + assertEquals(1, TestSkipListener.readSkips); + assertEquals(0, TestSkipListener.processSkips); + assertEquals(0, TestSkipListener.writeSkips); + + // Process skip and fail + execution = restartJob(execution.getExecutionId(), new Properties(), 10000l); + + assertEquals(BatchStatus.FAILED, execution.getBatchStatus()); + stepExecutions = jobOperator.getStepExecutions(execution.getExecutionId()); + assertEquals(1, getMetric(stepExecutions.get(0), Metric.MetricType.PROCESS_SKIP_COUNT).getValue()); + assertEquals(0, TestSkipListener.readSkips); + assertEquals(1, TestSkipListener.processSkips); + assertEquals(0, TestSkipListener.writeSkips); + + // Write skip and fail + execution = restartJob(execution.getExecutionId(), new Properties(), 10000l); + + assertEquals(BatchStatus.FAILED, execution.getBatchStatus()); + stepExecutions = jobOperator.getStepExecutions(execution.getExecutionId()); + assertEquals(1, getMetric(stepExecutions.get(0), Metric.MetricType.WRITE_SKIP_COUNT).getValue()); + assertEquals(0, TestSkipListener.readSkips); + assertEquals(0, TestSkipListener.processSkips); + assertEquals(1, TestSkipListener.writeSkips); + + // Complete + execution = restartJob(execution.getExecutionId(), new Properties(), 10000l); + + assertEquals(BatchStatus.COMPLETED, execution.getBatchStatus()); + stepExecutions = jobOperator.getStepExecutions(execution.getExecutionId()); + assertEquals(0, getMetric(stepExecutions.get(0), Metric.MetricType.WRITE_SKIP_COUNT).getValue()); + assertEquals(0, TestSkipListener.readSkips); + assertEquals(0, TestSkipListener.processSkips); + assertEquals(0, TestSkipListener.writeSkips); + } + + public static class SkipErrorGeneratingReader implements ItemReader { + private static int count = 0; + + @Nullable + @Override + public String read() throws Exception { + count++; + + if(count == 1) { + throw new Exception("read skip me"); + } else if (count == 2) { + return "item" + count; + } else if(count == 3) { + throw new RuntimeException("read fail because of me"); + } else if(count < 15) { + return "item" + count; + } else { + return null; + } + } + } + + public static class SkipErrorGeneratingProcessor implements ItemProcessor { + private static int count = 0; + + @Nullable + @Override + public String process(String item) throws Exception { + count++; + + if(count == 4) { + throw new Exception("process skip me"); + } else if(count == 5) { + return item; + } else if(count == 6) { + throw new RuntimeException("process fail because of me"); + } else { + return item; + } + } + } + + public static class SkipErrorGeneratingWriter implements ItemWriter { + private static int count = 0; + protected List writtenItems = new ArrayList<>(); + private List skippedItems = new ArrayList<>(); + + @Override + public void write(List items) throws Exception { + if(items.size() > 0 && !skippedItems.contains(items.get(0))) { + count++; + } + + if(count == 7) { + skippedItems.addAll(items); + throw new Exception("write skip me"); + } else if(count == 9) { + skippedItems = new ArrayList<>(); + throw new RuntimeException("write fail because of me"); + } else { + writtenItems.addAll(items); + } + } + } + + public static class TestSkipListener implements SkipReadListener, SkipProcessListener, SkipWriteListener { + + protected static int readSkips = 0; + protected static int processSkips = 0; + protected static int writeSkips = 0; + + public TestSkipListener() { + readSkips = 0; + processSkips = 0; + writeSkips = 0; + } + + @Override + public void onSkipProcessItem(Object item, Exception ex) throws Exception { + processSkips++; + } + + @Override + public void onSkipReadItem(Exception ex) throws Exception { + readSkips++; + } + + @Override + public void onSkipWriteItem(List items, Exception ex) throws Exception { + writeSkips++; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JobListenerParsingTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JobListenerParsingTests.java new file mode 100644 index 0000000000..2c9bafad8d --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JobListenerParsingTests.java @@ -0,0 +1,96 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import javax.batch.api.listener.JobListener; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobExecutionListener; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class JobListenerParsingTests { + + @Autowired + public Job job; + + @Autowired + public JobLauncher jobLauncher; + + @Autowired + public SpringJobListener springListener; + + @Autowired + public JsrJobListener jsrListener; + + @Test + public void test() throws Exception { + assertNotNull(job); + assertEquals("job1", job.getName()); + + JobExecution execution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(2, execution.getStepExecutions().size()); + assertEquals(1, springListener.countAfterJob); + assertEquals(1, springListener.countBeforeJob); + assertEquals(1, jsrListener.countAfterJob); + assertEquals(1, jsrListener.countBeforeJob); + } + + public static class SpringJobListener implements JobExecutionListener { + + protected int countBeforeJob = 0; + protected int countAfterJob = 0; + + @Override + public void beforeJob(JobExecution jobExecution) { + countBeforeJob++; + } + + @Override + public void afterJob(JobExecution jobExecution) { + countAfterJob++; + } + } + + public static class JsrJobListener implements JobListener { + + protected int countBeforeJob = 0; + protected int countAfterJob = 0; + + @Override + public void afterJob() throws Exception { + countBeforeJob++; + } + + @Override + public void beforeJob() throws Exception { + countAfterJob++; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JobPropertySubstitutionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JobPropertySubstitutionTests.java new file mode 100644 index 0000000000..2a3ba0fde2 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JobPropertySubstitutionTests.java @@ -0,0 +1,145 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.io.Serializable; +import java.util.List; +import javax.batch.api.BatchProperty; +import javax.batch.api.chunk.ItemProcessor; +import javax.batch.api.chunk.ItemReader; +import javax.batch.api.chunk.ItemWriter; +import javax.inject.Inject; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import static org.junit.Assert.assertEquals; + +/** + *

      + * Test cases for JSR-352 job property substitution. + *

      + * + * TODO: enhance test cases with more complex substitutions + * + * @author Chris Schaefer + * @since 3.0 + */ +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class JobPropertySubstitutionTests { + @Autowired + private Job job; + + @Autowired + private JobLauncher jobLauncher; + + @Test + public void testPropertySubstitutionSimple() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, + new JobParametersBuilder() + .addString("testParam", "testParamValue") + .addString("file.name.junit", "myfile2") + .toJobParameters()); + assertEquals(ExitStatus.COMPLETED, jobExecution.getExitStatus()); + } + + public static final class TestItemReader implements ItemReader { + private int cnt; + + @Inject + @BatchProperty + String readerPropertyName1; + + @Override + public void open(Serializable serializable) throws Exception { + assertEquals(System.getProperty("file.separator"), readerPropertyName1); + } + + @Override + public void close() throws Exception { + } + + @Override + public Object readItem() throws Exception { + if (cnt == 0) { + cnt++; + return "blah"; + } + + return null; + } + + @Override + public Serializable checkpointInfo() throws Exception { + return null; + } + } + + public static final class TestItemWriter implements ItemWriter { + @Inject + @BatchProperty + String writerPropertyName1; + + @Override + public void open(Serializable serializable) throws Exception { + assertEquals("jobPropertyValue1", writerPropertyName1); + } + + @Override + public void close() throws Exception { + } + + @Override + public void writeItems(List objects) throws Exception { + System.out.println(objects); + } + + @Override + public Serializable checkpointInfo() throws Exception { + return null; + } + } + + public static final class TestItemProcessor implements ItemProcessor { + @Inject + @BatchProperty + String processorProperty1; + + @Inject + @BatchProperty + String processorProperty2; + + @Inject + @BatchProperty + String processorProperty3; + + @Override + public Object processItem(Object item) throws Exception { + assertEquals("testParamValue", processorProperty1); + assertEquals("myfile1.txt", processorProperty2); + assertEquals(System.getProperty("file.separator") + "myfile2.txt", processorProperty3); + + return item; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JobPropertyTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JobPropertyTests.java new file mode 100644 index 0000000000..0bc24eadb3 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JobPropertyTests.java @@ -0,0 +1,300 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.io.Serializable; +import java.util.List; +import java.util.Properties; +import javax.batch.api.BatchProperty; +import javax.batch.api.Batchlet; +import javax.batch.api.Decider; +import javax.batch.api.chunk.CheckpointAlgorithm; +import javax.batch.api.chunk.ItemProcessor; +import javax.batch.api.chunk.ItemReader; +import javax.batch.api.chunk.ItemWriter; +import javax.batch.api.listener.StepListener; +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.JobExecution; +import javax.batch.runtime.context.JobContext; +import javax.inject.Inject; + +import org.junit.Test; + +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.jsr.AbstractJsrTestCase; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; + +import static org.junit.Assert.assertEquals; + +/** + *

      + * Configuration test for parsing various <properties /> elements defined by JSR-352. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class JobPropertyTests extends AbstractJsrTestCase { + @Test + public void testJobPropertyConfiguration() throws Exception { + Properties jobParameters = new Properties(); + jobParameters.setProperty("allow.start.if.complete", "true"); + jobParameters.setProperty("deciderName", "stepDecider"); + jobParameters.setProperty("deciderNumber", "1"); + + JobExecution jobExecution = runJob("jsrJobPropertyTestsContext", jobParameters, 5000L); + assertEquals(BatchStatus.COMPLETED, jobExecution.getBatchStatus()); + } + + public static final class TestItemReader implements ItemReader { + private int cnt; + + @Inject @BatchProperty String readerPropertyName1; + @Inject @BatchProperty String readerPropertyName2; + @Inject @BatchProperty String readerPropertyName3; + @Inject @BatchProperty(name = "annotationNamedReaderPropertyName") String annotationNamedProperty; + @Inject @BatchProperty String notDefinedProperty; + @Inject @BatchProperty(name = "notDefinedAnnotationNamedProperty") String notDefinedAnnotationNamedProperty; + @Inject @BatchProperty String jobPropertyName1; + @Inject @BatchProperty String jobPropertyName2; + @Inject JobContext injectAnnotatedOnlyField; + @BatchProperty String batchAnnotatedOnlyField; + @Inject javax.batch.runtime.context.StepContext stepContext; + + @Override + public void open(Serializable serializable) throws Exception { + org.springframework.util.Assert.notNull(stepContext, "stepContext is not null"); + org.springframework.util.Assert.isNull(stepContext.getProperties().get("step2PropertyName1"), "step2PropertyName1 is not null"); + org.springframework.util.Assert.isNull(stepContext.getProperties().get("step2PropertyName2"), "step2PropertyName2 is not null"); + org.springframework.util.Assert.isTrue(stepContext.getProperties().get("step1PropertyName1").equals("step1PropertyValue1"), "The value of step2PropertyName1 does not equal step2PropertyName1"); + org.springframework.util.Assert.isTrue(stepContext.getProperties().get("step1PropertyName2").equals("step1PropertyValue2"), "The value of step2PropertyName2 does not equal step2PropertyName2"); + org.springframework.util.Assert.isNull(stepContext.getProperties().get("jobPropertyName1"), "jobPropertyName1 is not null"); + org.springframework.util.Assert.isNull(stepContext.getProperties().get("jobPropertyName2") == null, "jobPropertyName2 is not null"); + org.springframework.util.Assert.isTrue("readerPropertyValue1".equals(readerPropertyName1), "The value of readerPropertyValue1 does not equal readerPropertyValue1"); + org.springframework.util.Assert.isTrue("readerPropertyValue2".equals(readerPropertyName2), "The value of readerPropertyValue2 does not equal readerPropertyValue2"); + org.springframework.util.Assert.isTrue("annotationNamedReaderPropertyValue".equals(annotationNamedProperty), "The value of annotationNamedReaderPropertyValue does not equal annotationNamedReaderPropertyValue"); + org.springframework.util.Assert.isNull(notDefinedProperty, "notDefinedProperty is not null"); + org.springframework.util.Assert.isNull(notDefinedAnnotationNamedProperty, "notDefinedAnnotationNamedProperty is not null"); + org.springframework.util.Assert.isNull(batchAnnotatedOnlyField, "batchAnnotatedOnlyField is not null"); + org.springframework.util.Assert.notNull(injectAnnotatedOnlyField, "injectAnnotatedOnlyField is not null"); + org.springframework.util.Assert.isTrue("job1".equals(injectAnnotatedOnlyField.getJobName()), "injectAnnotatedOnlyField does not equal job1"); + org.springframework.util.Assert.isNull(readerPropertyName3, "readerPropertyName3 is not null"); + + Properties jobProperties = injectAnnotatedOnlyField.getProperties(); + org.springframework.util.Assert.isTrue(jobProperties.size() == 5, "jobProperties has the wrong number of values. Expected 5, got " + jobProperties.size()); + org.springframework.util.Assert.isTrue(jobProperties.get("jobPropertyName1").equals("jobPropertyValue1"), "The value of jobPropertyName1 does not equal jobPropertyName1"); + org.springframework.util.Assert.isTrue(jobProperties.get("jobPropertyName2").equals("jobPropertyValue2"), "The value of jobPropertyName2 does not equal jobPropertyName2"); + org.springframework.util.Assert.isTrue(jobProperties.get("step2name").equals("step2"), "The value of step2name does note equal step2"); + org.springframework.util.Assert.isTrue(jobProperties.get("filestem").equals("postings"), "The value of filestem does not equal postings"); + org.springframework.util.Assert.isTrue(jobProperties.get("x").equals("xVal"), "The value of x does not equal xVal"); + } + + @Override + public void close() throws Exception { + } + + @Override + public Object readItem() throws Exception { + if (cnt == 0) { + cnt++; + return "blah"; + } + + return null; + } + + @Override + public Serializable checkpointInfo() throws Exception { + return null; + } + } + + public static final class TestItemProcessor implements ItemProcessor { + @Inject @BatchProperty String processorPropertyName1; + @Inject @BatchProperty String processorPropertyName2; + @Inject @BatchProperty(name = "annotationNamedProcessorPropertyName") String annotationNamedProperty; + @Inject @BatchProperty String notDefinedProperty; + @Inject @BatchProperty(name = "notDefinedAnnotationNamedProperty") String notDefinedAnnotationNamedProperty; + + @Override + public Object processItem(Object o) throws Exception { + org.springframework.util.Assert.isTrue("processorPropertyValue1".equals(processorPropertyName1), "The value of processorPropertyValue1 does not equal processorPropertyValue1"); + org.springframework.util.Assert.isTrue("processorPropertyValue2".equals(processorPropertyName2), "The value of processorPropertyValue2 does not equal processorPropertyValue2"); + org.springframework.util.Assert.isTrue("annotationNamedProcessorPropertyValue".equals(annotationNamedProperty), "The value of annotationNamedProcessorPropertyValue does not equal annotationNamedProcessorPropertyValue"); + org.springframework.util.Assert.isNull(notDefinedProperty, "The notDefinedProperty is not null"); + org.springframework.util.Assert.isNull(notDefinedAnnotationNamedProperty, "The notDefinedNamedProperty is not null"); + + return o; + } + } + + public static final class TestItemWriter implements ItemWriter { + @Inject @BatchProperty String writerPropertyName1; + @Inject @BatchProperty String writerPropertyName2; + @Inject @BatchProperty(name = "annotationNamedWriterPropertyName") String annotationNamedProperty; + @Inject @BatchProperty String notDefinedProperty; + @Inject @BatchProperty(name = "notDefinedAnnotationNamedProperty") String notDefinedAnnotationNamedProperty; + + @Override + public void open(Serializable serializable) throws Exception { + org.springframework.util.Assert.isTrue("writerPropertyValue1".equals(writerPropertyName1), "The value of writerPropertyValue1 does not equal writerPropertyValue1"); + org.springframework.util.Assert.isTrue("writerPropertyValue2".equals(writerPropertyName2), "The value of writerPropertyValue2 does not equal writerPropertyValue2"); + org.springframework.util.Assert.isTrue("annotationNamedWriterPropertyValue".equals(annotationNamedProperty), "The value of annotationNamedWriterPropertyValue does not equal annotationNamedWriterPropertyValue"); + org.springframework.util.Assert.isNull(notDefinedProperty, "notDefinedProperty is not null"); + org.springframework.util.Assert.isNull(notDefinedAnnotationNamedProperty, "notDefinedAnnotationNamedProperty is not null"); + } + + @Override + public void close() throws Exception { + } + + @Override + public void writeItems(List objects) throws Exception { + System.out.println(objects); + } + + @Override + public Serializable checkpointInfo() throws Exception { + return null; + } + } + + public static final class TestCheckpointAlgorithm implements CheckpointAlgorithm { + @Inject @BatchProperty String algorithmPropertyName1; + @Inject @BatchProperty String algorithmPropertyName2; + @Inject @BatchProperty(name = "annotationNamedAlgorithmPropertyName") String annotationNamedProperty; + @Inject @BatchProperty String notDefinedProperty; + @Inject @BatchProperty(name = "notDefinedAnnotationNamedProperty") String notDefinedAnnotationNamedProperty; + + @Override + public int checkpointTimeout() throws Exception { + return 0; + } + + @Override + public void beginCheckpoint() throws Exception { + org.springframework.util.Assert.isTrue("algorithmPropertyValue1".equals(algorithmPropertyName1), "The value of algorithmPropertyValue1 does not equal algorithmPropertyValue1"); + org.springframework.util.Assert.isTrue("algorithmPropertyValue2".equals(algorithmPropertyName2), "The value of algorithmPropertyValue2 does not equal algorithmPropertyValue2"); + org.springframework.util.Assert.isTrue("annotationNamedAlgorithmPropertyValue".equals(annotationNamedProperty), "The annotationNamedAlgorithmPropertyValue does not equal annotationNamedAlgorithmPropertyValue"); + org.springframework.util.Assert.isNull(notDefinedProperty, "notDefinedProperty is not null"); + org.springframework.util.Assert.isNull(notDefinedAnnotationNamedProperty, "notDefinedAnnotationNamedProperty is not null"); + } + + @Override + public boolean isReadyToCheckpoint() throws Exception { + return true; + } + + @Override + public void endCheckpoint() throws Exception { + } + } + + public static class TestDecider implements Decider { + @Inject @BatchProperty String deciderPropertyName1; + @Inject @BatchProperty String deciderPropertyName2; + @Inject @BatchProperty(name = "annotationNamedDeciderPropertyName") String annotationNamedProperty; + @Inject @BatchProperty String notDefinedProperty; + @Inject @BatchProperty(name = "notDefinedAnnotationNamedProperty") String notDefinedAnnotationNamedProperty; + + @Override + public String decide(javax.batch.runtime.StepExecution[] executions) throws Exception { + org.springframework.util.Assert.isTrue("deciderPropertyValue1".equals(deciderPropertyName1), "The value of deciderPropertyValue1 does not equal deciderPropertyValue1"); + org.springframework.util.Assert.isTrue("deciderPropertyValue2".equals(deciderPropertyName2), "The value of deciderPropertyValue2 does not equal deciderPropertyValue2"); + org.springframework.util.Assert.isTrue("annotationNamedDeciderPropertyValue".equals(annotationNamedProperty), "The value of annotationNamedDeciderPropertyValue does not equal annotationNamedDeciderPropertyValue"); + org.springframework.util.Assert.isNull(notDefinedProperty, "notDefinedProperty is not null"); + org.springframework.util.Assert.isNull(notDefinedAnnotationNamedProperty, "notDefinedAnnotationNamedProperty is not null"); + + return "step2"; + } + } + + public static class TestStepListener implements StepListener { + @Inject @BatchProperty String stepListenerPropertyName1; + @Inject @BatchProperty String stepListenerPropertyName2; + @Inject @BatchProperty(name = "annotationNamedStepListenerPropertyName") String annotationNamedProperty; + @Inject @BatchProperty String notDefinedProperty; + @Inject @BatchProperty(name = "notDefinedAnnotationNamedProperty") String notDefinedAnnotationNamedProperty; + + @Override + public void beforeStep() throws Exception { + org.springframework.util.Assert.isTrue("stepListenerPropertyValue1".equals(stepListenerPropertyName1), "The value of stepListenerPropertyValue1 does not equal stepListenerPropertyValue1"); + org.springframework.util.Assert.isTrue("stepListenerPropertyValue2".equals(stepListenerPropertyName2), "The value of stepListenerPropertyValue2 does not equal stepListenerPropertyValue2"); + org.springframework.util.Assert.isTrue("annotationNamedStepListenerPropertyValue".equals(annotationNamedProperty), "The value of annotationNamedStepListenerPropertyValue does note equal annotationNamedStepListenerPropertyValue"); + org.springframework.util.Assert.isNull(notDefinedProperty, "notDefinedProperty is not null"); + org.springframework.util.Assert.isNull(notDefinedAnnotationNamedProperty, "notDefinedAnnotationNamedProperty is not null"); + } + + @Override + public void afterStep() throws Exception { + } + } + + public static class TestBatchlet implements Batchlet { + @Inject @BatchProperty String batchletPropertyName1; + @Inject @BatchProperty String batchletPropertyName2; + @Inject @BatchProperty(name = "annotationNamedBatchletPropertyName") String annotationNamedProperty; + @Inject @BatchProperty String notDefinedProperty; + @Inject @BatchProperty(name = "notDefinedAnnotationNamedProperty") String notDefinedAnnotationNamedProperty; + @Inject javax.batch.runtime.context.StepContext stepContext; + @Inject @BatchProperty(name = "infile.name") String infile; + @Inject @BatchProperty(name = "y") String y; + @Inject @BatchProperty(name = "x") String x; + + @Override + public String process() throws Exception { + org.springframework.util.Assert.notNull(stepContext, "StepContext is not null"); + org.springframework.util.Assert.isNull(stepContext.getProperties().get("step1PropertyName1"), "The value of step1PropertyName1 is not null"); + org.springframework.util.Assert.isNull(stepContext.getProperties().get("step1PropertyName2"), "The value of step1PropertyName2 is not null"); + org.springframework.util.Assert.isTrue(stepContext.getProperties().get("step2PropertyName1").equals("step2PropertyValue1"), "The value of step2PropertyName1 does not equal step2PropertyName1"); + org.springframework.util.Assert.isTrue(stepContext.getProperties().get("step2PropertyName2").equals("step2PropertyValue2"), "The value of step2PropertyName2 does not equal step2PropertyName2"); + org.springframework.util.Assert.isTrue(stepContext.getProperties().get("jobPropertyName1") == null, "jobPropertyName1 is not null"); + org.springframework.util.Assert.isTrue(stepContext.getProperties().get("jobPropertyName2") == null, "jobPropertyName2 is not null"); + + org.springframework.util.Assert.isTrue("batchletPropertyValue1".equals(batchletPropertyName1), "batchletPropertyValue1 does not equal batchletPropertyValue1"); + org.springframework.util.Assert.isTrue("batchletPropertyValue2".equals(batchletPropertyName2), "batchletPropertyValue2 does not equal batchletPropertyValue2"); + org.springframework.util.Assert.isTrue("annotationNamedBatchletPropertyValue".equals(annotationNamedProperty), "annotationNamedBatchletPropertyValue does not equal annotationNamedBatchletPropertyValue"); + org.springframework.util.Assert.isTrue("postings.txt".equals(infile), "infile does not equal postings.txt"); + org.springframework.util.Assert.isTrue("xVal".equals(y), "y does not equal xVal"); + org.springframework.util.Assert.isNull(notDefinedProperty, "notDefinedProperty is not null"); + org.springframework.util.Assert.isNull(notDefinedAnnotationNamedProperty, "notDefinedAnnotationNamedProperty is not null"); + org.springframework.util.Assert.isNull(x, "x is not null"); + + return null; + } + + @Override + public void stop() throws Exception { + } + } + + public static class TestTasklet implements Tasklet { + @Inject + @BatchProperty + private String p1; + + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + org.springframework.util.Assert.isTrue("p1val".equals(p1), "Expected p1val, got " + p1); + + return RepeatStatus.FINISHED; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrBeanDefinitionDocumentReaderTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrBeanDefinitionDocumentReaderTests.java new file mode 100644 index 0000000000..3d468033fc --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrBeanDefinitionDocumentReaderTests.java @@ -0,0 +1,279 @@ +/* + * Copyright 2013-2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.io.IOException; +import java.io.InputStream; +import java.util.Properties; +import javax.batch.api.Batchlet; +import javax.batch.runtime.JobExecution; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Test; +import org.w3c.dom.Document; +import org.xml.sax.ErrorHandler; +import org.xml.sax.InputSource; + +import org.springframework.batch.core.jsr.AbstractJsrTestCase; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.xml.DefaultDocumentLoader; +import org.springframework.beans.factory.xml.DelegatingEntityResolver; +import org.springframework.beans.factory.xml.DocumentLoader; +import org.springframework.core.io.ClassPathResource; +import org.springframework.util.StringUtils; +import org.springframework.util.xml.SimpleSaxErrorHandler; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +/** + *

      + * Test cases around {@link JsrBeanDefinitionDocumentReader}. + *

      + * + * @author Chris Schaefer + */ +public class JsrBeanDefinitionDocumentReaderTests extends AbstractJsrTestCase { + private static final String JOB_PARAMETERS_BEAN_DEFINITION_NAME = "jsr_jobParameters"; + + private Log logger = LogFactory.getLog(getClass()); + private DocumentLoader documentLoader = new DefaultDocumentLoader(); + private ErrorHandler errorHandler = new SimpleSaxErrorHandler(logger); + + @Test + @SuppressWarnings("resource") + public void testGetJobParameters() { + Properties jobParameters = new Properties(); + jobParameters.setProperty("jobParameter1", "jobParameter1Value"); + jobParameters.setProperty("jobParameter2", "jobParameter2Value"); + + JsrXmlApplicationContext applicationContext = new JsrXmlApplicationContext(jobParameters); + applicationContext.setValidating(false); + applicationContext.load(new ClassPathResource("jsrBaseContext.xml"), + new ClassPathResource("/META-INF/batch.xml"), + new ClassPathResource("/META-INF/batch-jobs/jsrPropertyPreparseTestJob.xml")); + applicationContext.refresh(); + + BeanDefinition beanDefinition = applicationContext.getBeanDefinition(JOB_PARAMETERS_BEAN_DEFINITION_NAME); + + Properties processedJobParameters = (Properties) beanDefinition.getConstructorArgumentValues().getGenericArgumentValue(Properties.class).getValue(); + assertNotNull(processedJobParameters); + assertTrue("Wrong number of job parameters", processedJobParameters.size() == 2); + assertEquals("jobParameter1Value", processedJobParameters.getProperty("jobParameter1")); + assertEquals("jobParameter2Value", processedJobParameters.getProperty("jobParameter2")); + } + + @Test + public void testGetJobProperties() { + Document document = getDocument("/META-INF/batch-jobs/jsrPropertyPreparseTestJob.xml"); + + @SuppressWarnings("resource") + JsrXmlApplicationContext applicationContext = new JsrXmlApplicationContext(); + JsrBeanDefinitionDocumentReader documentReader = new JsrBeanDefinitionDocumentReader(applicationContext); + documentReader.initProperties(document.getDocumentElement()); + + Properties documentJobProperties = documentReader.getJobProperties(); + assertNotNull(documentJobProperties); + assertTrue("Wrong number of job properties", documentJobProperties.size() == 3); + assertEquals("jobProperty1Value", documentJobProperties.getProperty("jobProperty1")); + assertEquals("jobProperty1Value", documentJobProperties.getProperty("jobProperty2")); + assertEquals("", documentJobProperties.getProperty("jobProperty3")); + } + + @Test + public void testJobParametersResolution() { + Properties jobParameters = new Properties(); + jobParameters.setProperty("jobParameter1", "myfile.txt"); + jobParameters.setProperty("jobParameter2", "#{jobProperties['jobProperty2']}"); + jobParameters.setProperty("jobParameter3", "#{jobParameters['jobParameter1']}"); + + @SuppressWarnings("resource") + JsrXmlApplicationContext applicationContext = new JsrXmlApplicationContext(jobParameters); + applicationContext.setValidating(false); + applicationContext.load(new ClassPathResource("jsrBaseContext.xml"), + new ClassPathResource("/META-INF/batch.xml"), + new ClassPathResource("/META-INF/batch-jobs/jsrPropertyPreparseTestJob.xml")); + applicationContext.refresh(); + + Document document = getDocument("/META-INF/batch-jobs/jsrPropertyPreparseTestJob.xml"); + + JsrBeanDefinitionDocumentReader documentReader = new JsrBeanDefinitionDocumentReader(applicationContext); + documentReader.initProperties(document.getDocumentElement()); + + Properties resolvedParameters = documentReader.getJobParameters(); + + assertNotNull(resolvedParameters); + assertTrue("Wrong number of job parameters", resolvedParameters.size() == 3); + assertEquals("myfile.txt", resolvedParameters.getProperty("jobParameter1")); + assertEquals("jobProperty1Value", resolvedParameters.getProperty("jobParameter2")); + assertEquals("myfile.txt", resolvedParameters.getProperty("jobParameter3")); + } + + @Test + public void testJobPropertyResolution() { + Properties jobParameters = new Properties(); + jobParameters.setProperty("file.name", "myfile.txt"); + + @SuppressWarnings("resource") + JsrXmlApplicationContext applicationContext = new JsrXmlApplicationContext(jobParameters); + applicationContext.setValidating(false); + applicationContext.load(new ClassPathResource("jsrBaseContext.xml"), + new ClassPathResource("/META-INF/batch.xml"), + new ClassPathResource("/META-INF/batch-jobs/jsrPropertyPreparseTestJob.xml")); + applicationContext.refresh(); + + Document document = getDocument("/META-INF/batch-jobs/jsrPropertyPreparseTestJob.xml"); + + JsrBeanDefinitionDocumentReader documentReader = new JsrBeanDefinitionDocumentReader(applicationContext); + documentReader.initProperties(document.getDocumentElement()); + + Properties resolvedProperties = documentReader.getJobProperties(); + assertNotNull(resolvedProperties); + assertTrue("Wrong number of job properties", resolvedProperties.size() == 3); + assertEquals("jobProperty1Value", resolvedProperties.getProperty("jobProperty1")); + assertEquals("jobProperty1Value", resolvedProperties.getProperty("jobProperty2")); + assertEquals("myfile.txt", resolvedProperties.getProperty("jobProperty3")); + } + + @SuppressWarnings("resource") + @Test + public void testGenerationOfBeanDefinitionsForMultipleReferences() throws Exception { + JsrXmlApplicationContext applicationContext = new JsrXmlApplicationContext(new Properties()); + applicationContext.setValidating(false); + applicationContext.load(new ClassPathResource("jsrBaseContext.xml"), + new ClassPathResource("/META-INF/batch.xml"), + new ClassPathResource("/META-INF/batch-jobs/jsrUniqueInstanceTests.xml")); + applicationContext.refresh(); + + assertTrue("exitStatusSettingStepListener bean definition not found", applicationContext.containsBeanDefinition("exitStatusSettingStepListener")); + assertTrue("exitStatusSettingStepListener1 bean definition not found", applicationContext.containsBeanDefinition("exitStatusSettingStepListener1")); + assertTrue("exitStatusSettingStepListener2 bean definition not found", applicationContext.containsBeanDefinition("exitStatusSettingStepListener2")); + assertTrue("exitStatusSettingStepListener3 bean definition not found", applicationContext.containsBeanDefinition("exitStatusSettingStepListener3")); + assertTrue("exitStatusSettingStepListenerClassBeanDefinition bean definition not found", applicationContext.containsBeanDefinition("org.springframework.batch.core.jsr.step.listener.ExitStatusSettingStepListener")); + assertTrue("exitStatusSettingStepListener1ClassBeanDefinition bean definition not found", applicationContext.containsBeanDefinition("org.springframework.batch.core.jsr.step.listener.ExitStatusSettingStepListener1")); + assertTrue("exitStatusSettingStepListener2ClassBeanDefinition bean definition not found", applicationContext.containsBeanDefinition("org.springframework.batch.core.jsr.step.listener.ExitStatusSettingStepListener2")); + assertTrue("exitStatusSettingStepListener3ClassBeanDefinition bean definition not found", applicationContext.containsBeanDefinition("org.springframework.batch.core.jsr.step.listener.ExitStatusSettingStepListener3")); + assertTrue("testBatchlet bean definition not found", applicationContext.containsBeanDefinition("testBatchlet")); + assertTrue("testBatchlet1 bean definition not found", applicationContext.containsBeanDefinition("testBatchlet1")); + } + + @Test + public void testArtifactUniqueness() throws Exception { + JobExecution jobExecution = runJob("jsrUniqueInstanceTests", new Properties(), 10000L); + String exitStatus = jobExecution.getExitStatus(); + + assertTrue("Exit status must contain listener3", exitStatus.contains("listener3")); + exitStatus = exitStatus.replace("listener3", ""); + + assertTrue("Exit status must contain listener2", exitStatus.contains("listener2")); + exitStatus = exitStatus.replace("listener2", ""); + + assertTrue("Exit status must contain listener1", exitStatus.contains("listener1")); + exitStatus = exitStatus.replace("listener1", ""); + + assertTrue("Exit status must contain listener0", exitStatus.contains("listener0")); + exitStatus = exitStatus.replace("listener0", ""); + + assertTrue("Exit status must contain listener7", exitStatus.contains("listener7")); + exitStatus = exitStatus.replace("listener7", ""); + + assertTrue("Exit status must contain listener6", exitStatus.contains("listener6")); + exitStatus = exitStatus.replace("listener6", ""); + + assertTrue("Exit status must contain listener5", exitStatus.contains("listener5")); + exitStatus = exitStatus.replace("listener5", ""); + + assertTrue("Exit status must contain listener4", exitStatus.contains("listener4")); + exitStatus = exitStatus.replace("listener4", ""); + + assertTrue("exitStatus must be empty", "".equals(exitStatus)); + } + + @Test + @SuppressWarnings("resource") + public void testGenerationOfSpringBeanDefinitionsForMultipleReferences() { + JsrXmlApplicationContext applicationContext = new JsrXmlApplicationContext(new Properties()); + applicationContext.setValidating(false); + applicationContext.load(new ClassPathResource("jsrBaseContext.xml"), + new ClassPathResource("/META-INF/batch-jobs/jsrSpringInstanceTests.xml")); + + applicationContext.refresh(); + + assertTrue("exitStatusSettingStepListener bean definition not found", applicationContext.containsBeanDefinition("exitStatusSettingStepListener")); + assertTrue("scopedTarget.exitStatusSettingStepListener bean definition not found", applicationContext.containsBeanDefinition("scopedTarget.exitStatusSettingStepListener")); + + BeanDefinition exitStatusSettingStepListenerBeanDefinition = applicationContext.getBeanDefinition("scopedTarget.exitStatusSettingStepListener"); + assertTrue("step".equals(exitStatusSettingStepListenerBeanDefinition.getScope())); + + assertTrue("Should not contain bean definition for exitStatusSettingStepListener1", !applicationContext.containsBeanDefinition("exitStatusSettingStepListener1")); + assertTrue("Should not contain bean definition for exitStatusSettingStepListener2", !applicationContext.containsBeanDefinition("exitStatusSettingStepListener2")); + assertTrue("Should not contain bean definition for exitStatusSettingStepListener3", !applicationContext.containsBeanDefinition("exitStatusSettingStepListener3")); + + assertTrue("Should not contain bean definition for testBatchlet1", !applicationContext.containsBeanDefinition("testBatchlet1")); + assertTrue("Should not contain bean definition for testBatchlet2", !applicationContext.containsBeanDefinition("testBatchlet2")); + + assertTrue("testBatchlet bean definition not found", applicationContext.containsBeanDefinition("testBatchlet")); + + BeanDefinition testBatchletBeanDefinition = applicationContext.getBeanDefinition("testBatchlet"); + assertTrue("singleton".equals(testBatchletBeanDefinition.getScope())); + } + + @Test + public void testSpringArtifactUniqueness() throws Exception { + JobExecution jobExecution = runJob("jsrSpringInstanceTests", new Properties(), 10000L); + String exitStatus = jobExecution.getExitStatus(); + + assertTrue("Exit status must contain listener1", exitStatus.contains("listener1")); + assertTrue("exitStatus must contain 2 listener1 values", StringUtils.countOccurrencesOf(exitStatus, "listener1") == 2); + + exitStatus = exitStatus.replace("listener1", ""); + + assertTrue("Exit status must contain listener4", exitStatus.contains("listener4")); + assertTrue("exitStatus must contain 2 listener4 values", StringUtils.countOccurrencesOf(exitStatus, "listener4") == 2); + exitStatus = exitStatus.replace("listener4", ""); + + assertTrue("exitStatus must be empty", "".equals(exitStatus)); + } + + private Document getDocument(String location) { + InputStream inputStream = this.getClass().getResourceAsStream(location); + + try { + return documentLoader.loadDocument(new InputSource(inputStream), + new DelegatingEntityResolver(getClass().getClassLoader()), errorHandler, 0, true); + } catch (Exception e) { + throw new RuntimeException(e); + } finally { + try { + inputStream.close(); + } catch (IOException e) { } + } + } + + public static class TestBatchlet implements Batchlet { + @Override + public String process() throws Exception { + return null; + } + + @Override + public void stop() throws Exception { + + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrDecisionParsingTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrDecisionParsingTests.java new file mode 100644 index 0000000000..075b60cebf --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrDecisionParsingTests.java @@ -0,0 +1,58 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import static org.junit.Assert.assertEquals; + +import javax.batch.api.Decider; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class JsrDecisionParsingTests { + + @Autowired + public Job job; + + @Autowired + public JobLauncher jobLauncher; + + @Test + public void test() throws Exception { + JobExecution execution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(3, execution.getStepExecutions().size()); + } + + public static class JsrDecider implements Decider { + + @Override + public String decide(javax.batch.runtime.StepExecution[] executions) + throws Exception { + return "next"; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrSplitParsingTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrSplitParsingTests.java new file mode 100644 index 0000000000..609a081d27 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrSplitParsingTests.java @@ -0,0 +1,100 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.junit.Assert; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; +import org.springframework.batch.core.jsr.AbstractJsrTestCase; +import org.springframework.beans.PropertyValue; +import org.springframework.beans.factory.config.RuntimeBeanReference; +import org.springframework.beans.factory.parsing.BeanDefinitionParsingException; +import org.springframework.beans.factory.support.BeanDefinitionRegistry; +import org.springframework.context.support.ClassPathXmlApplicationContext; +import org.springframework.core.task.SimpleAsyncTaskExecutor; + +import javax.batch.api.AbstractBatchlet; +import javax.batch.runtime.BatchRuntime; +import javax.batch.runtime.StepExecution; +import javax.batch.runtime.context.JobContext; +import javax.inject.Inject; +import java.util.List; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +public class JsrSplitParsingTests extends AbstractJsrTestCase { + + @Rule + public ExpectedException expectedException = ExpectedException.none(); + + @Test + public void test() throws Exception { + javax.batch.runtime.JobExecution execution = runJob("JsrSplitParsingTests-context", null, 10000L); + assertEquals(javax.batch.runtime.BatchStatus.COMPLETED, execution.getBatchStatus()); + assertEquals("COMPLETED", execution.getExitStatus()); + + List stepExecutions = BatchRuntime.getJobOperator().getStepExecutions(execution.getExecutionId()); + assertEquals(5, stepExecutions.size()); + } + + @Test + public void testOneFlowInSplit() { + try { + new ClassPathXmlApplicationContext("/org/springframework/batch/core/jsr/configuration/xml/invalid-split-context.xml"); + } catch (BeanDefinitionParsingException bdpe) { + assertTrue(bdpe.getMessage().contains("A must contain at least two 'flow' elements.")); + return; + } + + fail("Expected exception was not thrown"); + } + + @Test + public void testUserSpecifiedTaskExecutor() { + ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext("/org/springframework/batch/core/jsr/configuration/xml/user-specified-split-task-executor-context.xml"); + BeanDefinitionRegistry registry = (BeanDefinitionRegistry) context.getBeanFactory(); + PropertyValue propertyValue = new JsrSplitParser(null).getSplitTaskExecutorPropertyValue(registry); + + RuntimeBeanReference runtimeBeanReferenceValue = (RuntimeBeanReference) propertyValue.getValue(); + + Assert.assertTrue("RuntimeBeanReference should have a name of jsr352splitTaskExecutor" , "jsr352splitTaskExecutor".equals(runtimeBeanReferenceValue.getBeanName())); + context.close(); + } + + @Test + public void testDefaultTaskExecutor() { + ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext("/org/springframework/batch/core/jsr/configuration/xml/default-split-task-executor-context.xml"); + BeanDefinitionRegistry registry = (BeanDefinitionRegistry) context.getBeanFactory(); + PropertyValue propertyValue = new JsrSplitParser(null).getSplitTaskExecutorPropertyValue(registry); + Assert.assertTrue("Task executor not an instance of SimpleAsyncTaskExecutor" , (propertyValue.getValue() instanceof SimpleAsyncTaskExecutor)); + context.close(); + } + + public static class ExitStatusSettingBatchlet extends AbstractBatchlet { + + @Inject + JobContext jobContext; + + @Override + public String process() throws Exception { + jobContext.setExitStatus("Should be ignored"); + return null; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrXmlApplicationContextTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrXmlApplicationContextTests.java new file mode 100644 index 0000000000..e6770c050e --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrXmlApplicationContextTests.java @@ -0,0 +1,65 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +import java.util.Properties; + +import org.junit.Test; +import org.springframework.beans.factory.config.BeanDefinition; + +/** + *

      + * Test cases around {@link JsrXmlApplicationContext}. + *

      + * + * @author Chris Schaefer + */ +public class JsrXmlApplicationContextTests { + private static final String JOB_PARAMETERS_BEAN_DEFINITION_NAME = "jsr_jobParameters"; + + @Test + @SuppressWarnings("resource") + public void testNullProperties() { + JsrXmlApplicationContext applicationContext = new JsrXmlApplicationContext(null); + + BeanDefinition beanDefinition = applicationContext.getBeanDefinition(JOB_PARAMETERS_BEAN_DEFINITION_NAME); + Properties properties = (Properties) beanDefinition.getConstructorArgumentValues().getGenericArgumentValue(Properties.class).getValue(); + + assertNotNull("Properties should not be null", properties); + assertTrue("Properties should be empty", properties.isEmpty()); + } + + @Test + @SuppressWarnings("resource") + public void testWithProperties() { + Properties properties = new Properties(); + properties.put("prop1key", "prop1val"); + + JsrXmlApplicationContext applicationContext = new JsrXmlApplicationContext(properties); + + BeanDefinition beanDefinition = applicationContext.getBeanDefinition(JOB_PARAMETERS_BEAN_DEFINITION_NAME); + Properties storedProperties = (Properties) beanDefinition.getConstructorArgumentValues().getGenericArgumentValue(Properties.class).getValue(); + + assertNotNull("Properties should not be null", storedProperties); + assertFalse("Properties not be empty", storedProperties.isEmpty()); + assertEquals("prop1val", storedProperties.getProperty("prop1key")); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ListenerParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ListenerParserTests.java new file mode 100644 index 0000000000..331a4cd024 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ListenerParserTests.java @@ -0,0 +1,67 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.junit.Test; +import org.springframework.batch.core.listener.StepListenerFactoryBean; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.context.support.GenericApplicationContext; +import static org.junit.Assert.assertEquals; + +/** + *

      + * Test cases around scoping of job/step listeners when building their bean definitions. + *

      + * + * @author Chris Schaefer + */ +public class ListenerParserTests { + @Test + public void testStepListenerStepScoped() { + @SuppressWarnings("resource") + GenericApplicationContext applicationContext = new GenericApplicationContext(); + + AbstractBeanDefinition newBeanDefinition = BeanDefinitionBuilder.genericBeanDefinition("stepListener").getBeanDefinition(); + newBeanDefinition.setScope("step"); + + applicationContext.registerBeanDefinition("stepListener", newBeanDefinition); + + ListenerParser listenerParser = new ListenerParser(StepListenerFactoryBean.class, "listeners"); + listenerParser.applyListenerScope("stepListener", applicationContext); + + BeanDefinition beanDefinition = applicationContext.getBeanDefinition("stepListener"); + assertEquals("step", beanDefinition.getScope()); + } + + @Test + public void testJobListenerSingletonScoped() { + @SuppressWarnings("resource") + GenericApplicationContext applicationContext = new GenericApplicationContext(); + + AbstractBeanDefinition newBeanDefinition = BeanDefinitionBuilder.genericBeanDefinition("jobListener").getBeanDefinition(); + newBeanDefinition.setScope("step"); + + applicationContext.registerBeanDefinition("jobListener", newBeanDefinition); + + ListenerParser listenerParser = new ListenerParser(JsrJobListenerFactoryBean.class, "jobExecutionListeners"); + listenerParser.applyListenerScope("jobListener", applicationContext); + + BeanDefinition beanDefinition = applicationContext.getBeanDefinition("jobListener"); + assertEquals("job", beanDefinition.getScope()); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/PartitionParserTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/PartitionParserTests.java new file mode 100644 index 0000000000..65eb6afd46 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/PartitionParserTests.java @@ -0,0 +1,366 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.io.Serializable; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashSet; +import java.util.List; +import java.util.Properties; +import java.util.Set; +import java.util.Vector; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import javax.batch.api.BatchProperty; +import javax.batch.api.Batchlet; +import javax.batch.api.chunk.AbstractItemReader; +import javax.batch.api.chunk.AbstractItemWriter; +import javax.batch.api.partition.PartitionPlan; +import javax.batch.api.partition.PartitionPlanImpl; +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.JobExecution; +import javax.batch.runtime.context.JobContext; +import javax.batch.runtime.context.StepContext; +import javax.inject.Inject; + +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.core.jsr.AbstractJsrTestCase; +import org.springframework.util.Assert; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +public class PartitionParserTests extends AbstractJsrTestCase { + private Pattern caPattern = Pattern.compile("ca"); + private Pattern asPattern = Pattern.compile("AS"); + private static final long TIMEOUT = 10000L; + + @Before + public void before() { + MyBatchlet.processed = 0; + MyBatchlet.threadNames = Collections.synchronizedSet(new HashSet<>()); + MyBatchlet.artifactNames = Collections.synchronizedSet(new HashSet<>()); + PartitionCollector.artifactNames = Collections.synchronizedSet(new HashSet<>()); + } + + @Test + public void testBatchletNoProperties() throws Exception { + BatchStatus curBatchStatus = runJob("partitionParserTestsBatchlet", new Properties(), TIMEOUT).getBatchStatus(); + + assertEquals(BatchStatus.COMPLETED, curBatchStatus); + assertEquals(10, MyBatchlet.processed); + assertEquals(10, MyBatchlet.threadNames.size()); + } + + @Test + public void testChunkNoProperties() throws Exception { + JobExecution execution = runJob("partitionParserTestsChunk", new Properties(), TIMEOUT); + + assertEquals(BatchStatus.COMPLETED, execution.getBatchStatus()); + assertEquals(30, ItemReader.processedItems.size()); + assertEquals(10, ItemReader.threadNames.size()); + assertEquals(30, ItemWriter.processedItems.size()); + assertEquals(10, ItemWriter.threadNames.size()); + } + + @Test + public void testFullPartitionConfiguration() throws Exception { + JobExecution execution = runJob("fullPartitionParserTests", new Properties(), TIMEOUT); + + assertEquals(BatchStatus.COMPLETED, execution.getBatchStatus()); + assertTrue(execution.getExitStatus().startsWith("BPS_")); + assertTrue(execution.getExitStatus().endsWith("BPSC_APSC")); + assertEquals(3, countMatches(execution.getExitStatus(), caPattern)); + assertEquals(3, countMatches(execution.getExitStatus(), asPattern)); + assertEquals(3, MyBatchlet.processed); + assertEquals(3, MyBatchlet.threadNames.size()); + } + + @Test + public void testFullPartitionConfigurationWithProperties() throws Exception { + JobExecution execution = runJob("fullPartitionParserWithPropertiesTests", new Properties(), TIMEOUT); + + assertEquals(BatchStatus.COMPLETED, execution.getBatchStatus()); + assertTrue(execution.getExitStatus().startsWith("BPS_")); + assertTrue(execution.getExitStatus().endsWith("BPSC_APSC")); + assertEquals(3, countMatches(execution.getExitStatus(), caPattern)); + assertEquals(3, countMatches(execution.getExitStatus(), asPattern)); + assertEquals(3, MyBatchlet.processed); + assertEquals(3, MyBatchlet.threadNames.size()); + assertEquals(MyBatchlet.artifactNames.iterator().next(), "batchlet"); + assertEquals(PartitionMapper.name, "mapper"); + assertEquals(PartitionAnalyzer.name, "analyzer"); + assertEquals(PartitionReducer.name, "reducer"); + assertEquals(PartitionCollector.artifactNames.size(), 1); + assertTrue(PartitionCollector.artifactNames.contains("collector")); + } + + @Test + public void testFullPartitionConfigurationWithMapperSuppliedProperties() throws Exception { + JobExecution execution = runJob("fullPartitionParserWithMapperPropertiesTests", new Properties(), TIMEOUT); + + assertEquals(BatchStatus.COMPLETED, execution.getBatchStatus()); + assertTrue(execution.getExitStatus().startsWith("BPS_")); + assertTrue(execution.getExitStatus().endsWith("BPSC_APSC")); + assertEquals(3, countMatches(execution.getExitStatus(), caPattern)); + assertEquals(3, countMatches(execution.getExitStatus(), asPattern)); + assertEquals(3, MyBatchlet.processed); + assertEquals(3, MyBatchlet.threadNames.size()); + + assertEquals(MyBatchlet.artifactNames.size(), 3); + assertTrue(MyBatchlet.artifactNames.contains("batchlet0")); + assertTrue(MyBatchlet.artifactNames.contains("batchlet1")); + assertTrue(MyBatchlet.artifactNames.contains("batchlet2")); + assertEquals(PartitionCollector.artifactNames.size(), 3); + assertTrue(PartitionCollector.artifactNames.contains("collector0")); + assertTrue(PartitionCollector.artifactNames.contains("collector1")); + assertTrue(PartitionCollector.artifactNames.contains("collector2")); + + assertEquals(PartitionAnalyzer.name, "analyzer"); + assertEquals(PartitionReducer.name, "reducer"); + } + + @Test + public void testFullPartitionConfigurationWithHardcodedProperties() throws Exception { + JobExecution execution = runJob("fullPartitionParserWithHardcodedPropertiesTests", new Properties(), TIMEOUT); + + assertEquals(BatchStatus.COMPLETED, execution.getBatchStatus()); + assertTrue(execution.getExitStatus().startsWith("BPS_")); + assertTrue(execution.getExitStatus().endsWith("BPSC_APSC")); + assertEquals(3, countMatches(execution.getExitStatus(), caPattern)); + assertEquals(3, countMatches(execution.getExitStatus(), asPattern)); + assertEquals(3, MyBatchlet.processed); + assertEquals(3, MyBatchlet.threadNames.size()); + + assertEquals(MyBatchlet.artifactNames.size(), 3); + assertTrue(MyBatchlet.artifactNames.contains("batchlet0")); + assertTrue(MyBatchlet.artifactNames.contains("batchlet1")); + assertTrue(MyBatchlet.artifactNames.contains("batchlet2")); + assertEquals(PartitionCollector.artifactNames.size(), 3); + assertTrue(PartitionCollector.artifactNames.contains("collector0")); + assertTrue(PartitionCollector.artifactNames.contains("collector1")); + assertTrue(PartitionCollector.artifactNames.contains("collector2")); + + assertEquals(PartitionMapper.name, "mapper"); + assertEquals(PartitionAnalyzer.name, "analyzer"); + assertEquals(PartitionReducer.name, "reducer"); + } + + private int countMatches(String string, Pattern pattern) { + Matcher matcher = pattern.matcher(string); + + int count = 0; + while(matcher.find()) { + count++; + } + + return count; + } + + public static class PartitionReducer implements javax.batch.api.partition.PartitionReducer { + + public static String name; + + @Inject + @BatchProperty + String artifactName; + + @Inject + protected JobContext jobContext; + + @Override + public void beginPartitionedStep() throws Exception { + name = artifactName; + jobContext.setExitStatus("BPS_"); + } + + @Override + public void beforePartitionedStepCompletion() throws Exception { + jobContext.setExitStatus(jobContext.getExitStatus() + "BPSC_"); + } + + @Override + public void rollbackPartitionedStep() throws Exception { + jobContext.setExitStatus(jobContext.getExitStatus() + "RPS"); + } + + @Override + public void afterPartitionedStepCompletion(PartitionStatus status) + throws Exception { + jobContext.setExitStatus(jobContext.getExitStatus() + "APSC"); + } + } + + public static class PartitionAnalyzer implements javax.batch.api.partition.PartitionAnalyzer { + + public static String name; + + @Inject + @BatchProperty + String artifactName; + + @Inject + protected JobContext jobContext; + + @Override + public void analyzeCollectorData(Serializable data) throws Exception { + name = artifactName; + + Assert.isTrue(data.equals("c"), "Expected c but was " + data); + jobContext.setExitStatus(jobContext.getExitStatus() + data + "a"); + } + + @Override + public void analyzeStatus(BatchStatus batchStatus, String exitStatus) + throws Exception { + Assert.isTrue(batchStatus.equals(BatchStatus.COMPLETED), String.format("expected %s but received %s", BatchStatus.COMPLETED, batchStatus)); + jobContext.setExitStatus(jobContext.getExitStatus() + "AS"); + } + } + + public static class PartitionCollector implements javax.batch.api.partition.PartitionCollector { + + protected static Set artifactNames = Collections.synchronizedSet(new HashSet<>()); + + @Inject + @BatchProperty + String artifactName; + + @Override + public Serializable collectPartitionData() throws Exception { + artifactNames.add(artifactName); + return "c"; + } + } + + public static class PropertyPartitionMapper implements javax.batch.api.partition.PartitionMapper { + + @Override + public PartitionPlan mapPartitions() throws Exception { + Properties[] props = new Properties[3]; + + for(int i = 0; i < props.length; i++) { + props[i] = new Properties(); + props[i].put("collectorName", "collector" + i); + props[i].put("batchletName", "batchlet" + i); + } + + PartitionPlan plan = new PartitionPlanImpl(); + plan.setPartitions(3); + plan.setThreads(3); + plan.setPartitionProperties(props); + + return plan; + } + } + + public static class PartitionMapper implements javax.batch.api.partition.PartitionMapper { + + public static String name; + + @Inject + @BatchProperty + public String artifactName; + + @Override + public PartitionPlan mapPartitions() throws Exception { + name = artifactName; + + PartitionPlan plan = new PartitionPlanImpl(); + plan.setPartitions(3); + plan.setThreads(3); + + return plan; + } + } + + public static class MyBatchlet implements Batchlet { + + protected static int processed = 0; + protected static Set threadNames = Collections.synchronizedSet(new HashSet<>()); + protected static Set artifactNames = Collections.synchronizedSet(new HashSet<>()); + + @Inject + @BatchProperty + String artifactName; + + @Inject + StepContext stepContext; + + @Inject + JobContext jobContext; + + @Override + public String process() throws Exception { + artifactNames.add(artifactName); + threadNames.add(Thread.currentThread().getName()); + processed++; + + stepContext.setExitStatus("bad step exit status"); + jobContext.setExitStatus("bad job exit status"); + + return null; + } + + @Override + public void stop() throws Exception { + } + } + + public static class ItemReader extends AbstractItemReader { + + private List items; + protected static Vector processedItems = new Vector<>(); + protected static Set threadNames = Collections.synchronizedSet(new HashSet<>()); + + @Override + public void open(Serializable checkpoint) throws Exception { + items = new ArrayList<>(); + items.add(1); + items.add(2); + items.add(3); + } + + @Override + public Object readItem() throws Exception { + threadNames.add(Thread.currentThread().getName()); + if(items.size() > 0) { + Integer curItem = items.remove(0); + processedItems.add(curItem); + return curItem; + } else { + return null; + } + } + } + + public static class ItemWriter extends AbstractItemWriter { + + protected static Vector processedItems = new Vector<>(); + protected static Set threadNames = Collections.synchronizedSet(new HashSet<>()); + + @Override + public void writeItems(List items) throws Exception { + threadNames.add(Thread.currentThread().getName()); + for (Object object : items) { + processedItems.add((Integer) object); + } + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/RetryListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/RetryListenerTests.java new file mode 100644 index 0000000000..0a7097490e --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/RetryListenerTests.java @@ -0,0 +1,247 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import java.io.Serializable; +import java.util.Collection; +import java.util.List; +import javax.batch.api.chunk.ItemProcessor; +import javax.batch.api.chunk.ItemReader; +import javax.batch.api.chunk.ItemWriter; +import javax.batch.api.chunk.listener.RetryProcessListener; +import javax.batch.api.chunk.listener.RetryReadListener; +import javax.batch.api.chunk.listener.RetryWriteListener; +import javax.batch.operations.BatchRuntimeException; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Test; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.context.ApplicationContext; +import org.springframework.context.support.ClassPathXmlApplicationContext; +import org.springframework.retry.RetryException; +import org.springframework.util.Assert; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +/** + *

      + * Test cases around JSR-352 retry listeners. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class RetryListenerTests { + private static final Log LOG = LogFactory.getLog(RetryListenerTests.class); + + @Test + @SuppressWarnings("resource") + public void testReadRetryExhausted() throws Exception { + ApplicationContext context = new ClassPathXmlApplicationContext("org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerExhausted.xml"); + + JobLauncher jobLauncher = context.getBean(JobLauncher.class); + JobExecution jobExecution = jobLauncher.run(context.getBean(Job.class), new JobParameters()); + + List failureExceptions = jobExecution.getAllFailureExceptions(); + assertTrue("Expected 1 failure exceptions", failureExceptions.size() == 1); + assertTrue("Failure exception must be of type RetryException", (failureExceptions.get(0) instanceof RetryException)); + assertTrue("Exception cause must be of type IllegalArgumentException", (failureExceptions.get(0).getCause() instanceof IllegalArgumentException)); + + assertEquals(ExitStatus.FAILED, jobExecution.getExitStatus()); + } + + @Test + @SuppressWarnings("resource") + public void testReadRetryOnce() throws Exception { + ApplicationContext context = new ClassPathXmlApplicationContext("org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerRetryOnce.xml"); + + JobLauncher jobLauncher = context.getBean(JobLauncher.class); + JobExecution jobExecution = jobLauncher.run(context.getBean(Job.class), new JobParameters()); + + Collection stepExecutions = jobExecution.getStepExecutions(); + assertEquals(1, stepExecutions.size()); + + StepExecution stepExecution = stepExecutions.iterator().next(); + assertEquals(1, stepExecution.getCommitCount()); + assertEquals(2, stepExecution.getReadCount()); + + assertEquals(ExitStatus.COMPLETED, jobExecution.getExitStatus()); + } + + @Test + @SuppressWarnings("resource") + public void testReadRetryExceptionInListener() throws Exception { + ApplicationContext context = new ClassPathXmlApplicationContext("org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerListenerException.xml"); + + JobLauncher jobLauncher = context.getBean(JobLauncher.class); + JobExecution jobExecution = jobLauncher.run(context.getBean(Job.class), new JobParameters()); + + List failureExceptions = jobExecution.getAllFailureExceptions(); + assertTrue("Failure exceptions must equal one", failureExceptions.size() == 1); + assertTrue("Failure exception must be of type RetryException", (failureExceptions.get(0) instanceof RetryException)); + assertTrue("Exception cause must be of type BatchRuntimeException", (failureExceptions.get(0).getCause() instanceof BatchRuntimeException)); + + assertEquals(ExitStatus.FAILED, jobExecution.getExitStatus()); + } + + public static class ExceptionThrowingRetryReadListener implements RetryReadListener { + @Override + public void onRetryReadException(Exception ex) throws Exception { + Assert.isInstanceOf(IllegalArgumentException.class, ex); + throw new IllegalStateException(); + } + } + + public static class TestRetryReadListener implements RetryReadListener { + @Override + public void onRetryReadException(Exception ex) throws Exception { + Assert.isInstanceOf(IllegalArgumentException.class, ex); + } + } + + public static class TestRetryProcessListener implements RetryProcessListener { + @Override + public void onRetryProcessException(Object item, Exception ex) throws Exception { + Assert.isInstanceOf(String.class, item); + + String currentItem = (String) item; + + Assert.isTrue("three".equals(currentItem), "currentItem was expected to be three but was not " + currentItem); + Assert.isInstanceOf(IllegalArgumentException.class, ex); + } + } + + public static class TestRetryWriteListener implements RetryWriteListener { + @Override + public void onRetryWriteException(List items, Exception ex) throws Exception { + Assert.isTrue(items.size() == 2, "Must be two items to write"); + Assert.isTrue(items.contains("three"), "Items must contain the string 'three'"); + Assert.isTrue(items.contains("one"), "Items must contain the string 'one'"); + Assert.isInstanceOf(IllegalArgumentException.class, ex); + } + } + + public static class AlwaysFailItemReader implements ItemReader { + @Override + public void open(Serializable checkpoint) throws Exception { + } + + @Override + public void close() throws Exception { + } + + @Override + public Object readItem() throws Exception { + throw new IllegalArgumentException(); + } + + @Override + public Serializable checkpointInfo() throws Exception { + return null; + } + } + + public static class FailOnceItemReader implements ItemReader { + private int cnt; + + @Override + public void open(Serializable checkpoint) throws Exception { + } + + @Override + public void close() throws Exception { + } + + @Override + public Object readItem() throws Exception { + if(cnt == 0) { + cnt++; + return "one"; + } else if (cnt == 1) { + cnt++; + throw new IllegalArgumentException(); + } else if (cnt == 2) { + cnt++; + return "three"; + } + + return null; + } + + @Override + public Serializable checkpointInfo() throws Exception { + return null; + } + } + + public static class FailOnceItemProcessor implements ItemProcessor { + private int cnt; + + @Override + public Object processItem(Object item) throws Exception { + if(cnt == 0) { + cnt++; + return "one"; + } else if (cnt == 1) { + cnt++; + throw new IllegalArgumentException(); + } else if (cnt == 2) { + cnt++; + return "three"; + } + + return null; + } + } + + public static class FailOnceItemWriter implements ItemWriter { + private int cnt; + + @Override + public void open(Serializable checkpoint) throws Exception { + } + + @Override + public void close() throws Exception { + } + + @Override + public void writeItems(List items) throws Exception { + for(@SuppressWarnings("unused") Object item : items) { + if(cnt == 0) { + cnt++; + LOG.info("one"); + } else if (cnt == 1) { + cnt++; + throw new IllegalArgumentException(); + } else if (cnt == 2) { + cnt++; + LOG.info("three"); + } + } + } + + @Override + public Serializable checkpointInfo() throws Exception { + return null; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/SimpleItemBasedJobParsingTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/SimpleItemBasedJobParsingTests.java new file mode 100644 index 0000000000..6f73ddd376 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/SimpleItemBasedJobParsingTests.java @@ -0,0 +1,122 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import static org.junit.Assert.assertEquals; + +import java.io.Serializable; +import java.util.List; + +import javax.batch.api.chunk.CheckpointAlgorithm; +import javax.batch.api.chunk.ItemWriter; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class SimpleItemBasedJobParsingTests { + + @Autowired + public Job job; + + @Autowired + public Step step1; + + @Autowired + public CountingItemProcessor processor; + + @Autowired + public CountingCompletionPolicy policy; + + @Autowired + public CountingItemWriter writer; + + @Autowired + public JobLauncher jobLauncher; + + @Test + public void test() throws Exception { + JobExecution execution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(4, execution.getStepExecutions().size()); + assertEquals(27, processor.count); + assertEquals(1, policy.checkpointCount); + assertEquals(7, writer.writeCount); + assertEquals(27, writer.itemCount); + } + + public static class CountingItemWriter implements ItemWriter { + + protected int writeCount = 0; + protected int itemCount = 0; + + @Override + public void open(Serializable checkpoint) throws Exception { + } + + @Override + public void close() throws Exception { + } + + @Override + public void writeItems(List items) throws Exception { + System.err.println("Items to be written: " + items); + writeCount++; + itemCount += items.size(); + } + + @Override + public Serializable checkpointInfo() throws Exception { + return null; + } + } + + public static class CountingCompletionPolicy implements CheckpointAlgorithm { + + protected int itemCount = 0; + protected int checkpointCount = 0; + + @Override + public int checkpointTimeout() throws Exception { + return 0; + } + + @Override + public void beginCheckpoint() throws Exception { + } + + @Override + public boolean isReadyToCheckpoint() throws Exception { + itemCount++; + return itemCount % 3 == 0; + } + + @Override + public void endCheckpoint() throws Exception { + checkpointCount++; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/SimpleJobParsingTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/SimpleJobParsingTests.java new file mode 100644 index 0000000000..764c9d0605 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/SimpleJobParsingTests.java @@ -0,0 +1,77 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import javax.batch.api.Batchlet; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class SimpleJobParsingTests { + + @Autowired + public Job job; + + @Autowired + @Qualifier("step1") + public Step step1; + + @Autowired + @Qualifier("step2") + public Step step2; + + @Autowired + @Qualifier("step3") + public Step step3; + + @Autowired + public JobLauncher jobLauncher; + + @Autowired + public Batchlet batchlet; + + @Test + public void test() throws Exception { + assertNotNull(job); + assertEquals("job1", job.getName()); + assertNotNull(step1); + assertEquals("step1", step1.getName()); + assertNotNull(step2); + assertEquals("step2", step2.getName()); + assertNotNull(step3); + assertEquals("step3", step3.getName()); + assertNotNull(batchlet); + + JobExecution execution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(3, execution.getStepExecutions().size()); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/StepListenerParsingTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/StepListenerParsingTests.java new file mode 100644 index 0000000000..40471eb941 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/StepListenerParsingTests.java @@ -0,0 +1,95 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import static org.junit.Assert.assertEquals; + +import javax.batch.api.listener.StepListener; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.StepExecutionListener; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.lang.Nullable; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class StepListenerParsingTests { + + @Autowired + public Job job; + + @Autowired + public JobLauncher jobLauncher; + + @Autowired + public SpringStepListener springStepListener; + + @Autowired + public JsrStepListener jsrStepListener; + + @Test + public void test() throws Exception { + JobExecution execution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(3, execution.getStepExecutions().size()); + assertEquals(2, springStepListener.countBeforeStep); + assertEquals(2, springStepListener.countAfterStep); + assertEquals(2, jsrStepListener.countBeforeStep); + assertEquals(2, jsrStepListener.countAfterStep); + } + + public static class SpringStepListener implements StepExecutionListener { + protected int countBeforeStep = 0; + protected int countAfterStep = 0; + + @Override + public void beforeStep(StepExecution stepExecution) { + countBeforeStep++; + } + + @Nullable + @Override + public ExitStatus afterStep(StepExecution stepExecution) { + countAfterStep++; + return null; + } + } + + public static class JsrStepListener implements StepListener { + protected int countBeforeStep = 0; + protected int countAfterStep = 0; + + @Override + public void beforeStep() throws Exception { + countBeforeStep++; + } + + @Override + public void afterStep() throws Exception { + countAfterStep++; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ThreadLocalClassloaderBeanPostProcessorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ThreadLocalClassloaderBeanPostProcessorTests.java new file mode 100644 index 0000000000..c7c9665bb7 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ThreadLocalClassloaderBeanPostProcessorTests.java @@ -0,0 +1,34 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import org.junit.Test; +import org.springframework.batch.core.jsr.AbstractJsrTestCase; + +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.JobExecution; + +import static org.junit.Assert.assertEquals; + +public class ThreadLocalClassloaderBeanPostProcessorTests extends AbstractJsrTestCase { + + @Test + public void test() throws Exception { + JobExecution execution = runJob("threadLocalClassloaderBeanPostProcessorTestsJob", null, 10000); + + assertEquals(BatchStatus.COMPLETED, execution.getBatchStatus()); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ThreadLocalClassloaderBeanPostProcessorTestsBatchlet.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ThreadLocalClassloaderBeanPostProcessorTestsBatchlet.java new file mode 100644 index 0000000000..944c369958 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/ThreadLocalClassloaderBeanPostProcessorTestsBatchlet.java @@ -0,0 +1,50 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.configuration.xml; + +import javax.batch.api.BatchProperty; +import javax.batch.api.Batchlet; +import javax.batch.runtime.context.JobContext; +import javax.batch.runtime.context.StepContext; +import javax.inject.Inject; + +import org.springframework.util.Assert; + +public class ThreadLocalClassloaderBeanPostProcessorTestsBatchlet implements Batchlet { + @Inject + @BatchProperty + public String jobParam1; + + @Inject + public JobContext jobContext; + + @Inject + public StepContext stepContext; + + @Override + public String process() throws Exception { + Assert.isTrue("someParameter".equals(jobParam1), jobParam1 + " does not equal someParameter"); + Assert.isTrue("threadLocalClassloaderBeanPostProcessorTestsJob".equals(jobContext.getJobName()), + "jobName does not equal threadLocalClassloaderBeanPostProcessorTestsJob"); + Assert.isTrue("step1".equals(stepContext.getStepName()), "stepName does not equal step1"); + + return null; + } + + @Override + public void stop() throws Exception { + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/job/flow/JsrFlowJobTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/job/flow/JsrFlowJobTests.java new file mode 100644 index 0000000000..38f7b1aea4 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/job/flow/JsrFlowJobTests.java @@ -0,0 +1,764 @@ +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.job.flow; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobInterruptedException; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.explore.support.MapJobExplorerFactoryBean; +import org.springframework.batch.core.job.flow.Flow; +import org.springframework.batch.core.job.flow.FlowExecutionStatus; +import org.springframework.batch.core.job.flow.FlowExecutor; +import org.springframework.batch.core.job.flow.JobExecutionDecider; +import org.springframework.batch.core.job.flow.State; +import org.springframework.batch.core.job.flow.StateSupport; +import org.springframework.batch.core.job.flow.support.SimpleFlow; +import org.springframework.batch.core.job.flow.support.StateTransition; +import org.springframework.batch.core.job.flow.support.state.DecisionState; +import org.springframework.batch.core.job.flow.support.state.EndState; +import org.springframework.batch.core.job.flow.support.state.FlowState; +import org.springframework.batch.core.job.flow.support.state.SplitState; +import org.springframework.batch.core.job.flow.support.state.StepState; +import org.springframework.batch.core.jsr.JsrStepExecution; +import org.springframework.batch.core.jsr.job.flow.support.JsrFlow; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.repository.dao.JobExecutionDao; +import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; +import org.springframework.batch.core.step.StepSupport; +import org.springframework.lang.Nullable; + +import javax.batch.api.Decider; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collections; +import java.util.List; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; + +/** + * @author Dave Syer + * @author Michael Minella + */ +public class JsrFlowJobTests { + + private JsrFlowJob job; + + private JobExecution jobExecution; + + private JobRepository jobRepository; + + private JobExplorer jobExplorer; + + private boolean fail = false; + + private JobExecutionDao jobExecutionDao; + + @Before + public void setUp() throws Exception { + job = new JsrFlowJob(); + MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean(); + jobRepositoryFactory.afterPropertiesSet(); + jobExecutionDao = jobRepositoryFactory.getJobExecutionDao(); + jobRepository = jobRepositoryFactory.getObject(); + job.setJobRepository(jobRepository); + jobExecution = jobRepository.createJobExecution("job", new JobParameters()); + + MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory); + jobExplorerFactory.afterPropertiesSet(); + jobExplorer = jobExplorerFactory.getObject(); + job.setJobExplorer(jobExplorer); + } + + @Test + public void testGetSteps() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + job.setFlow(flow); + job.afterPropertiesSet(); + assertEquals(2, job.getStepNames().size()); + } + + @Test + public void testTwoSteps() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); + StepState step2 = new StepState(new StubStep("step2")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.FAILED.getExitCode(), "end0")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.COMPLETED.getExitCode(), "end1")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end0"))); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end1"))); + flow.setStateTransitions(transitions); + job.setFlow(flow); + job.afterPropertiesSet(); + job.doExecute(jobExecution); + StepExecution stepExecution = getStepExecution(jobExecution, "step2"); + assertEquals(ExitStatus.COMPLETED, stepExecution.getExitStatus()); + assertEquals(2, jobExecution.getStepExecutions().size()); + } + + @Test + public void testFailedStep() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StateSupport("step1", FlowExecutionStatus.FAILED), + "step2")); + StepState step2 = new StepState(new StubStep("step2")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.FAILED.getExitCode(), "end0")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.COMPLETED.getExitCode(), "end1")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end0"))); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end1"))); + flow.setStateTransitions(transitions); + job.setFlow(flow); + job.afterPropertiesSet(); + job.doExecute(jobExecution); + StepExecution stepExecution = getStepExecution(jobExecution, "step2"); + assertEquals(ExitStatus.COMPLETED, stepExecution.getExitStatus()); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(2, jobExecution.getStepExecutions().size()); + } + + @Test + public void testFailedStepRestarted() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); + State step2State = new StateSupport("step2") { + @Override + public FlowExecutionStatus handle(FlowExecutor executor) throws Exception { + JobExecution jobExecution = executor.getJobExecution(); + StepExecution stepExecution = jobExecution.createStepExecution(getName()); + jobRepository.add(stepExecution); + if (fail) { + return FlowExecutionStatus.FAILED; + } + else { + return FlowExecutionStatus.COMPLETED; + } + } + }; + transitions.add(StateTransition.createStateTransition(step2State, ExitStatus.COMPLETED.getExitCode(), "end0")); + transitions.add(StateTransition.createStateTransition(step2State, ExitStatus.FAILED.getExitCode(), "end1")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end1"))); + flow.setStateTransitions(transitions); + job.setFlow(flow); + job.afterPropertiesSet(); + fail = true; + job.execute(jobExecution); + assertEquals(ExitStatus.FAILED, jobExecution.getExitStatus()); + assertEquals(2, jobExecution.getStepExecutions().size()); + jobRepository.update(jobExecution); + jobExecution = jobRepository.createJobExecution("job", new JobParameters()); + fail = false; + job.execute(jobExecution); + assertEquals(ExitStatus.COMPLETED, jobExecution.getExitStatus()); + assertEquals(1, jobExecution.getStepExecutions().size()); + } + + @Test + public void testStoppingStep() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); + State state2 = new StateSupport("step2", FlowExecutionStatus.FAILED); + transitions.add(StateTransition.createStateTransition(state2, ExitStatus.FAILED.getExitCode(), "end0")); + transitions.add(StateTransition.createStateTransition(state2, ExitStatus.COMPLETED.getExitCode(), "end1")); + transitions.add(StateTransition.createStateTransition(new EndState(FlowExecutionStatus.STOPPED, "end0"), + "step3")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end1"))); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step3")), "end2")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end2"))); + flow.setStateTransitions(transitions); + job.setFlow(flow); + job.afterPropertiesSet(); + job.doExecute(jobExecution); + assertEquals(2, jobExecution.getStepExecutions().size()); + assertEquals(BatchStatus.STOPPED, jobExecution.getStatus()); + } + + @Test + public void testInterrupted() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1") { + @Override + public void execute(StepExecution stepExecution) throws JobInterruptedException { + stepExecution.setStatus(BatchStatus.STOPPING); + jobRepository.update(stepExecution); + } + }), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + job.setFlow(flow); + job.afterPropertiesSet(); + job.execute(jobExecution); + assertEquals(BatchStatus.STOPPED, jobExecution.getStatus()); + checkRepository(BatchStatus.STOPPED, ExitStatus.STOPPED); + assertEquals(1, jobExecution.getAllFailureExceptions().size()); + assertEquals(JobInterruptedException.class, jobExecution.getFailureExceptions().get(0).getClass()); + } + + @Test + public void testUnknownStatusStopsJob() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1") { + @Override + public void execute(StepExecution stepExecution) throws JobInterruptedException { + stepExecution.setStatus(BatchStatus.UNKNOWN); + stepExecution.setTerminateOnly(); + jobRepository.update(stepExecution); + } + }), "step2")); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + job.setFlow(flow); + job.afterPropertiesSet(); + job.execute(jobExecution); + assertEquals(BatchStatus.UNKNOWN, jobExecution.getStatus()); + checkRepository(BatchStatus.UNKNOWN, ExitStatus.STOPPED); + assertEquals(1, jobExecution.getAllFailureExceptions().size()); + assertEquals(JobInterruptedException.class, jobExecution.getFailureExceptions().get(0).getClass()); + } + + @Test + public void testInterruptedSplit() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + SimpleFlow flow1 = new JsrFlow("flow1"); + SimpleFlow flow2 = new JsrFlow("flow2"); + + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1") { + @Override + public void execute(StepExecution stepExecution) throws JobInterruptedException { + if (!stepExecution.getJobExecution().getExecutionContext().containsKey("STOPPED")) { + stepExecution.getJobExecution().getExecutionContext().put("STOPPED", true); + stepExecution.setStatus(BatchStatus.STOPPED); + jobRepository.update(stepExecution); + } + else { + fail("The Job should have stopped by now"); + } + } + }), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow1.setStateTransitions(new ArrayList<>(transitions)); + flow1.afterPropertiesSet(); + flow2.setStateTransitions(new ArrayList<>(transitions)); + flow2.afterPropertiesSet(); + + transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new SplitState(Arrays. asList(flow1, flow2), + "split"), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + + job.setFlow(flow); + job.afterPropertiesSet(); + job.execute(jobExecution); + assertEquals(BatchStatus.STOPPED, jobExecution.getStatus()); + checkRepository(BatchStatus.STOPPED, ExitStatus.STOPPED); + assertEquals(1, jobExecution.getAllFailureExceptions().size()); + assertEquals(JobInterruptedException.class, jobExecution.getFailureExceptions().get(0).getClass()); + assertEquals(1, jobExecution.getStepExecutions().size()); + for (StepExecution stepExecution : jobExecution.getStepExecutions()) { + assertEquals(BatchStatus.STOPPED, stepExecution.getStatus()); + } + } + + @Test + public void testInterruptedException() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1") { + @Override + public void execute(StepExecution stepExecution) throws JobInterruptedException { + throw new JobInterruptedException("Stopped"); + } + }), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + job.setFlow(flow); + job.afterPropertiesSet(); + job.execute(jobExecution); + assertEquals(BatchStatus.STOPPED, jobExecution.getStatus()); + checkRepository(BatchStatus.STOPPED, ExitStatus.STOPPED); + assertEquals(1, jobExecution.getAllFailureExceptions().size()); + assertEquals(JobInterruptedException.class, jobExecution.getFailureExceptions().get(0).getClass()); + } + + @Test + public void testInterruptedSplitException() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + SimpleFlow flow1 = new JsrFlow("flow1"); + SimpleFlow flow2 = new JsrFlow("flow2"); + + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1") { + @Override + public void execute(StepExecution stepExecution) throws JobInterruptedException { + throw new JobInterruptedException("Stopped"); + } + }), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow1.setStateTransitions(new ArrayList<>(transitions)); + flow1.afterPropertiesSet(); + flow2.setStateTransitions(new ArrayList<>(transitions)); + flow2.afterPropertiesSet(); + + transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new SplitState(Arrays. asList(flow1, flow2), + "split"), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + + job.setFlow(flow); + job.afterPropertiesSet(); + job.execute(jobExecution); + assertEquals(BatchStatus.STOPPED, jobExecution.getStatus()); + checkRepository(BatchStatus.STOPPED, ExitStatus.STOPPED); + assertEquals(1, jobExecution.getAllFailureExceptions().size()); + assertEquals(JobInterruptedException.class, jobExecution.getFailureExceptions().get(0).getClass()); + } + + @Test + public void testEndStateStopped() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "end")); + transitions.add(StateTransition + .createStateTransition(new EndState(FlowExecutionStatus.STOPPED, "end"), "step2")); + StepState step2 = new StepState(new StubStep("step2")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.FAILED.getExitCode(), "end0")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.COMPLETED.getExitCode(), "end1")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end0"))); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end1"))); + flow.setStateTransitions(transitions); + job.setFlow(flow); + job.afterPropertiesSet(); + job.doExecute(jobExecution); + assertEquals(1, jobExecution.getStepExecutions().size()); + assertEquals(BatchStatus.STOPPED, jobExecution.getStatus()); + } + + public void testEndStateFailed() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "end")); + transitions + .add(StateTransition.createStateTransition(new EndState(FlowExecutionStatus.FAILED, "end"), "step2")); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), ExitStatus.FAILED + .getExitCode(), "end0")); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), + ExitStatus.COMPLETED.getExitCode(), "end1")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end0"))); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end1"))); + flow.setStateTransitions(transitions); + job.setFlow(flow); + job.afterPropertiesSet(); + job.doExecute(jobExecution); + assertEquals(BatchStatus.FAILED, jobExecution.getStatus()); + assertEquals(1, jobExecution.getStepExecutions().size()); + } + + @Test + public void testEndStateStoppedWithRestart() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "end")); + transitions.add(StateTransition + .createStateTransition(new EndState(FlowExecutionStatus.STOPPED, "end"), "step2")); + StepState step2 = new StepState(new StubStep("step2")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.COMPLETED.getExitCode(), "end0")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.FAILED.getExitCode(), "end1")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end1"))); + flow.setStateTransitions(transitions); + job.setFlow(flow); + job.afterPropertiesSet(); + + // To test a restart we have to use the AbstractJob.execute()... + job.execute(jobExecution); + assertEquals(BatchStatus.STOPPED, jobExecution.getStatus()); + assertEquals(1, jobExecution.getStepExecutions().size()); + + jobExecution = jobRepository.createJobExecution("job", new JobParameters()); + job.execute(jobExecution); + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + assertEquals(1, jobExecution.getStepExecutions().size()); + + } + + @Test + public void testBranching() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + StepState step1 = new StepState(new StubStep("step1")); + transitions.add(StateTransition.createStateTransition(step1, "step2")); + transitions.add(StateTransition.createStateTransition(step1, "COMPLETED", "step3")); + StepState step2 = new StepState(new StubStep("step2")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.COMPLETED.getExitCode(), "end0")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.FAILED.getExitCode(), "end1")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end1"))); + StepState step3 = new StepState(new StubStep("step3")); + transitions.add(StateTransition.createStateTransition(step3, ExitStatus.FAILED.getExitCode(), "end2")); + transitions.add(StateTransition.createStateTransition(step3, ExitStatus.COMPLETED.getExitCode(), "end3")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end2"))); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end3"))); + flow.setStateTransitions(transitions); + job.setFlow(flow); + job.afterPropertiesSet(); + job.doExecute(jobExecution); + StepExecution stepExecution = getStepExecution(jobExecution, "step2"); + assertEquals(ExitStatus.COMPLETED, stepExecution.getExitStatus()); + assertEquals(2, jobExecution.getStepExecutions().size()); + } + + @Test + public void testBasicFlow() throws Throwable { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step")), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + job.setFlow(flow); + job.execute(jobExecution); + if (!jobExecution.getAllFailureExceptions().isEmpty()) { + throw jobExecution.getAllFailureExceptions().get(0); + } + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + } + + @Test + public void testDecisionFlow() throws Throwable { + + SimpleFlow flow = new JsrFlow("job"); + Decider decider = new Decider() { + + @Override + public String decide(javax.batch.runtime.StepExecution[] executions) + throws Exception { + assertNotNull(executions); + return "SWITCH"; + } + }; + + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "decision")); + StepState decision = new StepState(new StubDecisionStep("decision", decider)); + transitions.add(StateTransition.createStateTransition(decision, "SWITCH", "step3")); + transitions.add(StateTransition.createStateTransition(decision, "step2")); + StepState step2 = new StepState(new StubStep("step2")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.COMPLETED.getExitCode(), "end0")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.FAILED.getExitCode(), "end1")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end1"))); + StepState step3 = new StepState(new StubStep("step3")); + transitions.add(StateTransition.createStateTransition(step3, ExitStatus.FAILED.getExitCode(), "end2")); + transitions.add(StateTransition.createStateTransition(step3, ExitStatus.COMPLETED.getExitCode(), "end3")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end2"))); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end3"))); + flow.setStateTransitions(transitions); + + job.setFlow(flow); + job.doExecute(jobExecution); + + StepExecution stepExecution = getStepExecution(jobExecution, "step3"); + if (!jobExecution.getAllFailureExceptions().isEmpty()) { + throw jobExecution.getAllFailureExceptions().get(0); + } + + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(3, jobExecution.getStepExecutions().size()); + + } + + @Test + public void testDecisionFlowWithExceptionInDecider() throws Throwable { + + SimpleFlow flow = new JsrFlow("job"); + JobExecutionDecider decider = new JobExecutionDecider() { + @Override + public FlowExecutionStatus decide(JobExecution jobExecution, @Nullable StepExecution stepExecution) { + assertNotNull(stepExecution); + throw new RuntimeException("Foo"); + } + }; + + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "decision")); + DecisionState decision = new DecisionState(decider, "decision"); + transitions.add(StateTransition.createStateTransition(decision, "step2")); + transitions.add(StateTransition.createStateTransition(decision, "SWITCH", "step3")); + StepState step2 = new StepState(new StubStep("step2")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.COMPLETED.getExitCode(), "end0")); + transitions.add(StateTransition.createStateTransition(step2, ExitStatus.FAILED.getExitCode(), "end1")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end1"))); + StepState step3 = new StepState(new StubStep("step3")); + transitions.add(StateTransition.createStateTransition(step3, ExitStatus.FAILED.getExitCode(), "end2")); + transitions.add(StateTransition.createStateTransition(step3, ExitStatus.COMPLETED.getExitCode(), "end3")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.FAILED, "end2"))); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end3"))); + flow.setStateTransitions(transitions); + + job.setFlow(flow); + try { + job.execute(jobExecution); + } + finally { + + assertEquals(BatchStatus.FAILED, jobExecution.getStatus()); + assertEquals(1, jobExecution.getStepExecutions().size()); + + assertEquals(1, jobExecution.getAllFailureExceptions().size()); + assertEquals("Foo", jobExecution.getAllFailureExceptions().get(0).getCause().getCause().getMessage()); + + } + } + + @Test + public void testGetStepExists() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + job.setFlow(flow); + job.afterPropertiesSet(); + + Step step = job.getStep("step2"); + assertNotNull(step); + assertEquals("step2", step.getName()); + } + + @Test + public void testGetStepExistsWithPrefix() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState("job.step", new StubStep("step")), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + job.setFlow(flow); + job.setName(flow.getName()); + job.afterPropertiesSet(); + + Step step = job.getStep("step"); + assertNotNull(step); + assertEquals("step", step.getName()); + } + + @Test + public void testGetStepNamesWithPrefix() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState("job.step", new StubStep("step")), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + job.setFlow(flow); + job.setName(flow.getName()); + job.afterPropertiesSet(); + + assertEquals("[step]", job.getStepNames().toString()); + } + + @Test + public void testGetStepNotExists() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + job.setFlow(flow); + job.afterPropertiesSet(); + + Step step = job.getStep("foo"); + assertNull(step); + } + + @Test + public void testGetStepNotStepState() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "step2")); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + job.setFlow(flow); + job.afterPropertiesSet(); + + Step step = job.getStep("end0"); + assertNull(step); + } + + @Test + public void testGetStepNestedFlow() throws Exception { + SimpleFlow nested = new JsrFlow("nested"); + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end1")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end1"))); + nested.setStateTransitions(transitions); + nested.afterPropertiesSet(); + + SimpleFlow flow = new JsrFlow("job"); + transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "nested")); + transitions.add(StateTransition.createStateTransition(new FlowState(nested, "nested"), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + job.setFlow(flow); + job.afterPropertiesSet(); + + List names = new ArrayList<>(job.getStepNames()); + Collections.sort(names); + assertEquals("[step1, step2]", names.toString()); + } + + @Test + public void testGetStepSplitFlow() throws Exception { + SimpleFlow flow = new JsrFlow("job"); + SimpleFlow flow1 = new JsrFlow("flow1"); + SimpleFlow flow2 = new JsrFlow("flow2"); + + List transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step1")), "end0")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end0"))); + flow1.setStateTransitions(new ArrayList<>(transitions)); + flow1.afterPropertiesSet(); + transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new StepState(new StubStep("step2")), "end1")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end1"))); + flow2.setStateTransitions(new ArrayList<>(transitions)); + flow2.afterPropertiesSet(); + + transitions = new ArrayList<>(); + transitions.add(StateTransition.createStateTransition(new SplitState(Arrays. asList(flow1, flow2), + "split"), "end2")); + transitions.add(StateTransition.createEndStateTransition(new EndState(FlowExecutionStatus.COMPLETED, "end2"))); + flow.setStateTransitions(transitions); + flow.afterPropertiesSet(); + + job.setFlow(flow); + job.afterPropertiesSet(); + List names = new ArrayList<>(job.getStepNames()); + Collections.sort(names); + assertEquals("[step1, step2]", names.toString()); + } + + /** + * @author Dave Syer + * + */ + private class StubStep extends StepSupport { + + private StubStep(String name) { + super(name); + } + + @Override + public void execute(StepExecution stepExecution) throws JobInterruptedException { + stepExecution.setStatus(BatchStatus.COMPLETED); + stepExecution.setExitStatus(ExitStatus.COMPLETED); + jobRepository.update(stepExecution); + } + + } + + /** + * @author Michael Minella + * + */ + private class StubDecisionStep extends StepSupport { + + private Decider decider; + + private StubDecisionStep(String name, Decider decider) { + super(name); + this.decider = decider; + } + + @Override + public void execute(StepExecution stepExecution) throws JobInterruptedException { + stepExecution.setStatus(BatchStatus.COMPLETED); + try { + stepExecution.setExitStatus(new ExitStatus(decider.decide(new javax.batch.runtime.StepExecution [] {new JsrStepExecution(stepExecution)}))); + } catch (Exception e) { + throw new RuntimeException(e); + } + + jobRepository.update(stepExecution); + } + } + + /** + * @param jobExecution + * @param stepName + * @return the StepExecution corresponding to the specified step + */ + private StepExecution getStepExecution(JobExecution jobExecution, String stepName) { + for (StepExecution stepExecution : jobExecution.getStepExecutions()) { + if (stepExecution.getStepName().equals(stepName)) { + return stepExecution; + } + } + fail("No stepExecution found with name: [" + stepName + "]"); + return null; + } + + private void checkRepository(BatchStatus status, ExitStatus exitStatus) { + // because map DAO stores in memory, it can be checked directly + JobInstance jobInstance = jobExecution.getJobInstance(); + JobExecution other = jobExecutionDao.findJobExecutions(jobInstance).get(0); + assertEquals(jobInstance.getId(), other.getJobId()); + assertEquals(status, other.getStatus()); + if (exitStatus != null) { + assertEquals(exitStatus.getExitCode(), other.getExitStatus().getExitCode()); + } + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/job/flow/support/JsrFlowTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/job/flow/support/JsrFlowTests.java new file mode 100644 index 0000000000..621abf291e --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/job/flow/support/JsrFlowTests.java @@ -0,0 +1,76 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.job.flow.support; + +import static org.junit.Assert.assertEquals; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.job.flow.FlowExecution; +import org.springframework.batch.core.job.flow.FlowExecutionStatus; +import org.springframework.batch.core.job.flow.State; +import org.springframework.batch.core.job.flow.StateSupport; +import org.springframework.batch.core.job.flow.support.JobFlowExecutorSupport; +import org.springframework.batch.core.job.flow.support.SimpleFlowTests; +import org.springframework.batch.core.job.flow.support.StateTransition; +import org.springframework.lang.Nullable; + +public class JsrFlowTests extends SimpleFlowTests { + + @Override + @Before + public void setUp() { + flow = new JsrFlow("flow1"); + } + + @Test + public void testNextBasedOnBatchStatus() throws Exception { + StepExecution stepExecution = new StepExecution("step1", new JobExecution(5L)); + stepExecution.setExitStatus(new ExitStatus("unmapped exit code")); + stepExecution.setStatus(BatchStatus.FAILED); + executor = new FlowExecutor(stepExecution); + + State startState = new StateSupport("step1", new FlowExecutionStatus("unmapped exit code")); + State endState = new StateSupport("failed", FlowExecutionStatus.FAILED); + + StateTransition failureTransition = StateTransition.createStateTransition(startState, "FAILED", "failed"); + StateTransition endTransition = StateTransition.createEndStateTransition(endState); + flow.setStateTransitions(collect(failureTransition, endTransition)); + flow.afterPropertiesSet(); + FlowExecution execution = flow.start(executor); + assertEquals(FlowExecutionStatus.FAILED, execution.getStatus()); + assertEquals("failed", execution.getName()); + } + + public static class FlowExecutor extends JobFlowExecutorSupport { + + private StepExecution stepExecution; + + public FlowExecutor(StepExecution stepExecution) { + this.stepExecution = stepExecution; + } + + @Nullable + @Override + public StepExecution getStepExecution() { + return stepExecution; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/job/flow/support/state/JsrEndStateTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/job/flow/support/state/JsrEndStateTests.java new file mode 100644 index 0000000000..f2d2f9d38e --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/job/flow/support/state/JsrEndStateTests.java @@ -0,0 +1,51 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.job.flow.support.state; + +import static org.junit.Assert.assertEquals; + +import javax.batch.api.AbstractBatchlet; +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.JobExecution; + +import org.junit.Test; + +import org.springframework.batch.core.jsr.AbstractJsrTestCase; + +/** + * Tests for the JSR-352 version of {@link JsrEndState} + * + * @author Michael Minella + */ +public class JsrEndStateTests extends AbstractJsrTestCase { + + @Test + public void test() throws Exception { + JobExecution jobExecution = runJob("jobWithEndTransition", null, 10000L); + + assertEquals(BatchStatus.COMPLETED, jobExecution.getBatchStatus()); + assertEquals("SUCCESS", jobExecution.getExitStatus()); + assertEquals(1, operator.getStepExecutions(jobExecution.getExecutionId()).size()); + } + + public static class EndStateBatchlet extends AbstractBatchlet { + + @Override + public String process() throws Exception { + return "GOOD"; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/launch/JsrJobOperatorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/launch/JsrJobOperatorTests.java new file mode 100644 index 0000000000..31fd8341b5 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/launch/JsrJobOperatorTests.java @@ -0,0 +1,685 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.launch; + +import java.util.ArrayList; +import java.util.Date; +import java.util.HashSet; +import java.util.List; +import java.util.Properties; +import java.util.Set; +import javax.batch.api.AbstractBatchlet; +import javax.batch.api.Batchlet; +import javax.batch.operations.JobExecutionIsRunningException; +import javax.batch.operations.JobOperator; +import javax.batch.operations.JobRestartException; +import javax.batch.operations.JobStartException; +import javax.batch.operations.NoSuchJobException; +import javax.batch.operations.NoSuchJobExecutionException; +import javax.batch.operations.NoSuchJobInstanceException; +import javax.batch.runtime.BatchRuntime; +import javax.batch.runtime.BatchStatus; +import javax.sql.DataSource; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.ArgumentCaptor; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.configuration.annotation.DataSourceConfiguration; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.converter.JobParametersConverter; +import org.springframework.batch.core.converter.JobParametersConverterSupport; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.explore.support.SimpleJobExplorer; +import org.springframework.batch.core.jsr.AbstractJsrTestCase; +import org.springframework.batch.core.jsr.JsrJobParametersConverter; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.step.JobRepositorySupport; +import org.springframework.batch.support.transaction.ResourcelessTransactionManager; +import org.springframework.beans.factory.BeanCreationException; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; +import org.springframework.context.support.GenericApplicationContext; +import org.springframework.core.task.AsyncTaskExecutor; +import org.springframework.core.task.SyncTaskExecutor; +import org.springframework.test.util.ReflectionTestUtils; +import org.springframework.transaction.PlatformTransactionManager; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +/** + * Tests for {@link JsrJobOperator}. + */ +public class JsrJobOperatorTests extends AbstractJsrTestCase { + + private JobOperator jsrJobOperator; + @Mock + private JobExplorer jobExplorer; + @Mock + private JobRepository jobRepository; + private JobParametersConverter parameterConverter; + private static final long TIMEOUT = 10000L; + + @Before + public void setup() throws Exception { + + MockitoAnnotations.initMocks(this); + parameterConverter = new JobParametersConverterSupport(); + jsrJobOperator = new JsrJobOperator(jobExplorer, jobRepository, parameterConverter, new ResourcelessTransactionManager()); + } + + @Test + public void testLoadingWithBatchRuntime() { + jsrJobOperator = BatchRuntime.getJobOperator(); + assertNotNull(jsrJobOperator); + } + + @Test + public void testNullsInConstructor() { + try { + new JsrJobOperator(null, new JobRepositorySupport(), parameterConverter, null); + fail("JobExplorer should be required"); + } catch (IllegalArgumentException correct) { + } + + try { + new JsrJobOperator(new SimpleJobExplorer(null, null, null, null), null, parameterConverter, null); + fail("JobRepository should be required"); + } catch (IllegalArgumentException correct) { + } + + try { + new JsrJobOperator(new SimpleJobExplorer(null, null, null, null), new JobRepositorySupport(), null, null); + fail("ParameterConverter should be required"); + } catch (IllegalArgumentException correct) { + } + + try { + new JsrJobOperator(new SimpleJobExplorer(null, null, null, null), new JobRepositorySupport(), parameterConverter, null); + } + catch (IllegalArgumentException correct) { + } + + new JsrJobOperator(new SimpleJobExplorer(null, null, null, null), new JobRepositorySupport(), parameterConverter, new ResourcelessTransactionManager()); + } + + @Test + public void testCustomBaseContextJsrCompliant() throws Exception { + System.setProperty("JSR-352-BASE-CONTEXT", "META-INF/alternativeJsrBaseContext.xml"); + + ReflectionTestUtils.setField(JsrJobOperator.BaseContextHolder.class, "instance", null); + + JobOperator jobOperator = BatchRuntime.getJobOperator(); + + Object transactionManager = ReflectionTestUtils.getField(jobOperator, "transactionManager"); + assertTrue(transactionManager instanceof ResourcelessTransactionManager); + + long executionId = jobOperator.start("longRunningJob", null); + // Give the job a chance to get started + Thread.sleep(1000L); + jobOperator.stop(executionId); + // Give the job the chance to finish stopping + Thread.sleep(1000L); + + assertEquals(BatchStatus.STOPPED, jobOperator.getJobExecution(executionId).getBatchStatus()); + + System.getProperties().remove("JSR-352-BASE-CONTEXT"); + } + + @Test + public void testCustomBaseContextCustomWired() throws Exception { + + GenericApplicationContext context = new AnnotationConfigApplicationContext(BatchConfgiuration.class); + + JobOperator jobOperator = (JobOperator) context.getBean("jobOperator"); + + assertEquals(context, ReflectionTestUtils.getField(jobOperator, "baseContext")); + + long executionId = jobOperator.start("longRunningJob", null); + // Give the job a chance to get started + Thread.sleep(1000L); + jobOperator.stop(executionId); + // Give the job the chance to finish stopping + Thread.sleep(1000L); + + assertEquals(BatchStatus.STOPPED, jobOperator.getJobExecution(executionId).getBatchStatus()); + + System.getProperties().remove("JSR-352-BASE-CONTEXT"); + } + + @Test + public void testDefaultTaskExecutor() throws Exception { + JsrJobOperator jsrJobOperatorImpl = (JsrJobOperator) jsrJobOperator; + jsrJobOperatorImpl.afterPropertiesSet(); + assertNotNull(jsrJobOperatorImpl.getTaskExecutor()); + assertTrue((jsrJobOperatorImpl.getTaskExecutor() instanceof AsyncTaskExecutor)); + } + + @Test + public void testCustomTaskExecutor() throws Exception { + JsrJobOperator jsrJobOperatorImpl = (JsrJobOperator) jsrJobOperator; + jsrJobOperatorImpl.setTaskExecutor(new SyncTaskExecutor()); + jsrJobOperatorImpl.afterPropertiesSet(); + assertNotNull(jsrJobOperatorImpl.getTaskExecutor()); + assertTrue((jsrJobOperatorImpl.getTaskExecutor() instanceof SyncTaskExecutor)); + } + + @Test + public void testAbandonRoseyScenario() throws Exception { + JobExecution jobExecution = new JobExecution(5L); + jobExecution.setEndTime(new Date()); + when(jobExplorer.getJobExecution(5L)).thenReturn(jobExecution); + + jsrJobOperator.abandon(5L); + + ArgumentCaptor executionCaptor = ArgumentCaptor.forClass(JobExecution.class); + verify(jobRepository).update(executionCaptor.capture()); + assertEquals(org.springframework.batch.core.BatchStatus.ABANDONED, executionCaptor.getValue().getStatus()); + + } + + @Test(expected=NoSuchJobExecutionException.class) + public void testAbandonNoSuchJob() throws Exception { + jsrJobOperator.abandon(5L); + } + + @Test(expected=JobExecutionIsRunningException.class) + public void testAbandonJobRunning() throws Exception { + JobExecution jobExecution = new JobExecution(5L); + jobExecution.setStartTime(new Date(1L)); + + when(jobExplorer.getJobExecution(5L)).thenReturn(jobExecution); + + jsrJobOperator.abandon(5L); + } + + @Test + public void testGetJobExecutionRoseyScenario() { + when(jobExplorer.getJobExecution(5L)).thenReturn(new JobExecution(5L)); + + assertEquals(5L, jsrJobOperator.getJobExecution(5L).getExecutionId()); + } + + @Test(expected=NoSuchJobExecutionException.class) + public void testGetJobExecutionNoExecutionFound() { + jsrJobOperator.getJobExecution(5L); + } + + @Test + public void testGetJobExecutionsRoseyScenario() { + org.springframework.batch.core.JobInstance jobInstance = new org.springframework.batch.core.JobInstance(5L, "my job"); + List executions = new ArrayList<>(); + executions.add(new JobExecution(2L)); + + when(jobExplorer.getJobExecutions(jobInstance)).thenReturn(executions); + + List jobExecutions = jsrJobOperator.getJobExecutions(jobInstance); + assertEquals(1, jobExecutions.size()); + assertEquals(2L, executions.get(0).getId().longValue()); + } + + @Test(expected=NoSuchJobInstanceException.class) + public void testGetJobExecutionsNullJobInstance() { + jsrJobOperator.getJobExecutions(null); + } + + @Test(expected=NoSuchJobInstanceException.class) + public void testGetJobExecutionsNullReturned() { + org.springframework.batch.core.JobInstance jobInstance = new org.springframework.batch.core.JobInstance(5L, "my job"); + + jsrJobOperator.getJobExecutions(jobInstance); + } + + @Test(expected=NoSuchJobInstanceException.class) + public void testGetJobExecutionsNoneReturned() { + org.springframework.batch.core.JobInstance jobInstance = new org.springframework.batch.core.JobInstance(5L, "my job"); + List executions = new ArrayList<>(); + + when(jobExplorer.getJobExecutions(jobInstance)).thenReturn(executions); + + jsrJobOperator.getJobExecutions(jobInstance); + } + + @Test + public void testGetJobInstanceRoseyScenario() { + JobInstance instance = new JobInstance(1L, "my job"); + JobExecution execution = new JobExecution(5L); + execution.setJobInstance(instance); + + when(jobExplorer.getJobExecution(5L)).thenReturn(execution); + when(jobExplorer.getJobInstance(1L)).thenReturn(instance); + + javax.batch.runtime.JobInstance jobInstance = jsrJobOperator.getJobInstance(5L); + + assertEquals(1L, jobInstance.getInstanceId()); + assertEquals("my job", jobInstance.getJobName()); + } + + @Test(expected=NoSuchJobExecutionException.class) + public void testGetJobInstanceNoExecution() { + JobInstance instance = new JobInstance(1L, "my job"); + JobExecution execution = new JobExecution(5L); + execution.setJobInstance(instance); + + jsrJobOperator.getJobInstance(5L); + } + + @Test + public void testGetJobInstanceCount() throws Exception { + when(jobExplorer.getJobInstanceCount("myJob")).thenReturn(4); + + assertEquals(4, jsrJobOperator.getJobInstanceCount("myJob")); + } + + @Test(expected=NoSuchJobException.class) + public void testGetJobInstanceCountNoSuchJob() throws Exception { + when(jobExplorer.getJobInstanceCount("myJob")).thenThrow(new org.springframework.batch.core.launch.NoSuchJobException("expected")); + + jsrJobOperator.getJobInstanceCount("myJob"); + } + + @Test(expected=NoSuchJobException.class) + public void testGetJobInstanceCountZeroInstancesReturned() throws Exception { + when(jobExplorer.getJobInstanceCount("myJob")).thenReturn(0); + + jsrJobOperator.getJobInstanceCount("myJob"); + } + + @Test + public void testGetJobInstancesRoseyScenario() { + List instances = new ArrayList<>(); + instances.add(new JobInstance(1L, "myJob")); + instances.add(new JobInstance(2L, "myJob")); + instances.add(new JobInstance(3L, "myJob")); + + when(jobExplorer.getJobInstances("myJob", 0, 3)).thenReturn(instances); + + List jobInstances = jsrJobOperator.getJobInstances("myJob", 0, 3); + + assertEquals(3, jobInstances.size()); + assertEquals(1L, jobInstances.get(0).getInstanceId()); + assertEquals(2L, jobInstances.get(1).getInstanceId()); + assertEquals(3L, jobInstances.get(2).getInstanceId()); + } + + @Test(expected=NoSuchJobException.class) + public void testGetJobInstancesNullInstancesReturned() { + jsrJobOperator.getJobInstances("myJob", 0, 3); + } + + @Test(expected=NoSuchJobException.class) + public void testGetJobInstancesZeroInstancesReturned() { + List instances = new ArrayList<>(); + + when(jobExplorer.getJobInstances("myJob", 0, 3)).thenReturn(instances); + + jsrJobOperator.getJobInstances("myJob", 0, 3); + } + + @Test + public void testGetJobNames() { + List jobNames = new ArrayList<>(); + jobNames.add("job1"); + jobNames.add("job2"); + + when(jobExplorer.getJobNames()).thenReturn(jobNames); + + Set result = jsrJobOperator.getJobNames(); + + assertEquals(2, result.size()); + assertTrue(result.contains("job1")); + assertTrue(result.contains("job2")); + } + + @Test + public void testGetParametersRoseyScenario() { + JobExecution jobExecution = new JobExecution(5L, new JobParametersBuilder().addString("key1", "value1").addLong(JsrJobParametersConverter.JOB_RUN_ID, 5L).toJobParameters()); + + when(jobExplorer.getJobExecution(5L)).thenReturn(jobExecution); + + Properties params = jsrJobOperator.getParameters(5L); + + assertEquals("value1", params.get("key1")); + assertNull(params.get(JsrJobParametersConverter.JOB_RUN_ID)); + } + + @Test(expected=NoSuchJobExecutionException.class) + public void testGetParametersNoExecution() { + jsrJobOperator.getParameters(5L); + } + + @Test(expected=NoSuchJobException.class) + public void testGetNoRunningExecutions() { + Set executions = new HashSet<>(); + + when(jobExplorer.findRunningJobExecutions("myJob")).thenReturn(executions); + + jsrJobOperator.getRunningExecutions("myJob"); + } + + @Test + public void testGetRunningExecutions() { + Set executions = new HashSet<>(); + executions.add(new JobExecution(5L)); + + when(jobExplorer.findRunningJobExecutions("myJob")).thenReturn(executions); + + assertEquals(5L, jsrJobOperator.getRunningExecutions("myJob").get(0).longValue()); + } + + @Test + public void testGetStepExecutionsRoseyScenario() { + JobExecution jobExecution = new JobExecution(5L); + List stepExecutions = new ArrayList<>(); + stepExecutions.add(new StepExecution("step1", jobExecution, 1L)); + stepExecutions.add(new StepExecution("step2", jobExecution, 2L)); + jobExecution.addStepExecutions(stepExecutions); + + when(jobExplorer.getJobExecution(5L)).thenReturn(jobExecution); + when(jobExplorer.getStepExecution(5L, 1L)).thenReturn(new StepExecution("step1", jobExecution, 1L)); + when(jobExplorer.getStepExecution(5L, 2L)).thenReturn(new StepExecution("step2", jobExecution, 2L)); + + List results = jsrJobOperator.getStepExecutions(5L); + + assertEquals("step1", results.get(0).getStepName()); + assertEquals("step2", results.get(1).getStepName()); + } + + @Test(expected=NoSuchJobException.class) + public void testGetStepExecutionsNoExecutionReturned() { + jsrJobOperator.getStepExecutions(5L); + } + + @Test + public void testGetStepExecutionsPartitionedStepScenario() { + JobExecution jobExecution = new JobExecution(5L); + List stepExecutions = new ArrayList<>(); + stepExecutions.add(new StepExecution("step1", jobExecution, 1L)); + stepExecutions.add(new StepExecution("step2", jobExecution, 2L)); + stepExecutions.add(new StepExecution("step2:partition0", jobExecution, 2L)); + stepExecutions.add(new StepExecution("step2:partition1", jobExecution, 2L)); + stepExecutions.add(new StepExecution("step2:partition2", jobExecution, 2L)); + jobExecution.addStepExecutions(stepExecutions); + + when(jobExplorer.getJobExecution(5L)).thenReturn(jobExecution); + when(jobExplorer.getStepExecution(5L, 1L)).thenReturn(new StepExecution("step1", jobExecution, 1L)); + when(jobExplorer.getStepExecution(5L, 2L)).thenReturn(new StepExecution("step2", jobExecution, 2L)); + + List results = jsrJobOperator.getStepExecutions(5L); + + assertEquals("step1", results.get(0).getStepName()); + assertEquals("step2", results.get(1).getStepName()); + } + + @Test + public void testGetStepExecutionsNoStepExecutions() { + JobExecution jobExecution = new JobExecution(5L); + + when(jobExplorer.getJobExecution(5L)).thenReturn(jobExecution); + + List results = jsrJobOperator.getStepExecutions(5L); + + assertEquals(0, results.size()); + } + + @Test + public void testStartRoseyScenario() throws Exception { + javax.batch.runtime.JobExecution execution = runJob("jsrJobOperatorTestJob", new Properties(), TIMEOUT); + + assertEquals(BatchStatus.COMPLETED, execution.getBatchStatus()); + } + + @Test + public void testStartMultipleTimesSameParameters() throws Exception { + jsrJobOperator = BatchRuntime.getJobOperator(); + + int jobInstanceCountBefore = 0; + + try { + jobInstanceCountBefore = jsrJobOperator.getJobInstanceCount("myJob3"); + } catch (NoSuchJobException ignore) { + } + + javax.batch.runtime.JobExecution execution1 = runJob("jsrJobOperatorTestJob", new Properties(), TIMEOUT); + javax.batch.runtime.JobExecution execution2 = runJob("jsrJobOperatorTestJob", new Properties(), TIMEOUT); + javax.batch.runtime.JobExecution execution3 = runJob("jsrJobOperatorTestJob", new Properties(), TIMEOUT); + + assertEquals(BatchStatus.COMPLETED, execution1.getBatchStatus()); + assertEquals(BatchStatus.COMPLETED, execution2.getBatchStatus()); + assertEquals(BatchStatus.COMPLETED, execution3.getBatchStatus()); + + int jobInstanceCountAfter = jsrJobOperator.getJobInstanceCount("myJob3"); + + assertTrue((jobInstanceCountAfter - jobInstanceCountBefore) == 3); + } + + @Test + public void testRestartRoseyScenario() throws Exception { + javax.batch.runtime.JobExecution execution = runJob("jsrJobOperatorTestRestartJob", new Properties(), TIMEOUT); + + assertEquals(BatchStatus.FAILED, execution.getBatchStatus()); + + execution = restartJob(execution.getExecutionId(), null, TIMEOUT); + + assertEquals(BatchStatus.COMPLETED, execution.getBatchStatus()); + } + + @Test(expected = JobRestartException.class) + public void testNonRestartableJob() throws Exception { + javax.batch.runtime.JobExecution jobExecutionStart = runJob("jsrJobOperatorTestNonRestartableJob", new Properties(), TIMEOUT); + assertEquals(BatchStatus.FAILED, jobExecutionStart.getBatchStatus()); + + restartJob(jobExecutionStart.getExecutionId(), null, TIMEOUT); + } + + @Test(expected = JobRestartException.class) + public void testRestartAbandoned() throws Exception { + jsrJobOperator = BatchRuntime.getJobOperator(); + javax.batch.runtime.JobExecution execution = runJob("jsrJobOperatorTestRestartAbandonJob", null, TIMEOUT); + + assertEquals(BatchStatus.FAILED, execution.getBatchStatus()); + + jsrJobOperator.abandon(execution.getExecutionId()); + jsrJobOperator.restart(execution.getExecutionId(), null); + } + + @Test + public void testGetNoRestartJobParameters() { + JsrJobOperator jobOperator = (JsrJobOperator) jsrJobOperator; + Properties properties = jobOperator.getJobRestartProperties(null, null); + assertTrue(properties.isEmpty()); + } + + @Test + public void testGetRestartJobParameters() { + JsrJobOperator jobOperator = (JsrJobOperator) jsrJobOperator; + + JobExecution jobExecution = new JobExecution(1L, + new JobParametersBuilder().addString("prevKey1", "prevVal1").toJobParameters()); + + Properties userProperties = new Properties(); + userProperties.put("userKey1", "userVal1"); + + Properties properties = jobOperator.getJobRestartProperties(userProperties, jobExecution); + + assertTrue(properties.size() == 2); + assertTrue(properties.getProperty("prevKey1").equals("prevVal1")); + assertTrue(properties.getProperty("userKey1").equals("userVal1")); + } + + @Test + public void testGetRestartJobParametersWithDefaults() { + JsrJobOperator jobOperator = (JsrJobOperator) jsrJobOperator; + + JobExecution jobExecution = new JobExecution(1L, + new JobParametersBuilder().addString("prevKey1", "prevVal1").addString("prevKey2", "prevVal2").toJobParameters()); + + Properties defaultProperties = new Properties(); + defaultProperties.setProperty("prevKey2", "not value 2"); + Properties userProperties = new Properties(defaultProperties); + + Properties properties = jobOperator.getJobRestartProperties(userProperties, jobExecution); + + assertTrue(properties.size() == 2); + assertTrue(properties.getProperty("prevKey1").equals("prevVal1")); + assertTrue("prevKey2 = " + properties.getProperty("prevKey2"), properties.getProperty("prevKey2").equals("not value 2")); + } + + @Test + public void testNewJobParametersOverridePreviousRestartParameters() { + JsrJobOperator jobOperator = (JsrJobOperator) jsrJobOperator; + + JobExecution jobExecution = new JobExecution(1L, + new JobParametersBuilder() + .addString("prevKey1", "prevVal1") + .addString("overrideTest", "jobExecution") + .toJobParameters()); + + Properties userProperties = new Properties(); + userProperties.put("userKey1", "userVal1"); + userProperties.put("overrideTest", "userProperties"); + + Properties properties = jobOperator.getJobRestartProperties(userProperties, jobExecution); + + assertTrue(properties.size() == 3); + assertTrue(properties.getProperty("prevKey1").equals("prevVal1")); + assertTrue(properties.getProperty("userKey1").equals("userVal1")); + assertTrue(properties.getProperty("overrideTest").equals("userProperties")); + } + + @Test(expected = JobStartException.class) + public void testBeanCreationExceptionOnStart() throws Exception { + jsrJobOperator = BatchRuntime.getJobOperator(); + + try { + jsrJobOperator.start("jsrJobOperatorTestBeanCreationException", null); + } catch (JobStartException e) { + assertTrue(e.getCause() instanceof BeanCreationException); + throw e; + } + + fail("Should have failed"); + } + + @SuppressWarnings("unchecked") + @Test(expected=JobStartException.class) + public void testStartUnableToCreateJobExecution() throws Exception { + when(jobRepository.createJobExecution("myJob", null)).thenThrow(RuntimeException.class); + + jsrJobOperator.start("myJob", null); + } + + @Test + public void testJobStopRoseyScenario() throws Exception { + jsrJobOperator = BatchRuntime.getJobOperator(); + long executionId = jsrJobOperator.start("longRunningJob", null); + // Give the job a chance to get started + Thread.sleep(1000L); + jsrJobOperator.stop(executionId); + // Give the job the chance to finish stopping + Thread.sleep(1000L); + + assertEquals(BatchStatus.STOPPED, jsrJobOperator.getJobExecution(executionId).getBatchStatus()); + + } + + @Test + public void testApplicationContextClosingAfterJobSuccessful() throws Exception { + for(int i = 0; i < 3; i++) { + javax.batch.runtime.JobExecution execution = runJob("contextClosingTests", new Properties(), TIMEOUT); + + assertEquals(BatchStatus.COMPLETED, execution.getBatchStatus()); + + // Added to allow time for the context to finish closing before running the job again + Thread.sleep(1000l); + } + } + + public static class LongRunningBatchlet implements Batchlet { + + private boolean stopped = false; + + @Override + public String process() throws Exception { + while(!stopped) { + Thread.sleep(250); + } + return null; + } + + @Override + public void stop() throws Exception { + stopped = true; + } + } + + public static class FailingBatchlet extends AbstractBatchlet { + @Override + public String process() throws Exception { + throw new RuntimeException("blah"); + } + } + + public static class MustBeClosedBatchlet extends AbstractBatchlet { + + public static boolean closed = true; + + public MustBeClosedBatchlet() { + if(!closed) { + throw new RuntimeException("Batchlet wasn't closed last time"); + } + } + + public void close() { + closed = true; + } + + @Override + public String process() throws Exception { + closed = false; + return null; + } + } + + @Configuration + @Import(DataSourceConfiguration.class) + @EnableBatchProcessing + public static class BatchConfgiuration { + + @Bean + public JsrJobOperator jobOperator(JobExplorer jobExplorer, JobRepository jobrepository, DataSource dataSource, + PlatformTransactionManager transactionManager) throws Exception{ + + JsrJobParametersConverter jobParametersConverter = new JsrJobParametersConverter(dataSource); + jobParametersConverter.afterPropertiesSet(); + return new JsrJobOperator(jobExplorer, jobrepository, jobParametersConverter, transactionManager); + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/partition/JsrPartitionHandlerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/partition/JsrPartitionHandlerTests.java new file mode 100644 index 0000000000..7f9b2ee216 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/partition/JsrPartitionHandlerTests.java @@ -0,0 +1,419 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.partition; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobInterruptedException; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.jsr.AbstractJsrTestCase; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.jsr.step.batchlet.BatchletSupport; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; +import org.springframework.batch.core.step.JobRepositorySupport; +import org.springframework.batch.core.step.StepSupport; +import org.springframework.util.StopWatch; + +import javax.batch.api.BatchProperty; +import javax.batch.api.partition.PartitionAnalyzer; +import javax.batch.api.partition.PartitionCollector; +import javax.batch.api.partition.PartitionMapper; +import javax.batch.api.partition.PartitionPlan; +import javax.batch.api.partition.PartitionPlanImpl; +import javax.batch.api.partition.PartitionReducer; +import javax.batch.runtime.BatchStatus; +import javax.inject.Inject; +import java.io.Serializable; +import java.util.Collection; +import java.util.Properties; +import java.util.Queue; +import java.util.concurrent.ConcurrentLinkedQueue; +import java.util.concurrent.atomic.AtomicInteger; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +public class JsrPartitionHandlerTests extends AbstractJsrTestCase { + + private JsrPartitionHandler handler; + private JobRepository repository = new JobRepositorySupport(); + private StepExecution stepExecution; + private AtomicInteger count; + private BatchPropertyContext propertyContext; + private JsrStepExecutionSplitter stepSplitter; + + @Before + public void setUp() throws Exception { + JobExecution jobExecution = new JobExecution(1L); + jobExecution.setJobInstance(new JobInstance(1L, "job")); + stepExecution = new StepExecution("step1", jobExecution); + stepSplitter = new JsrStepExecutionSplitter(repository, false, "step1", true); + Analyzer.collectorData = ""; + Analyzer.status = ""; + count = new AtomicInteger(0); + handler = new JsrPartitionHandler(); + handler.setStep(new StepSupport() { + @Override + public void execute(StepExecution stepExecution) throws JobInterruptedException { + count.incrementAndGet(); + stepExecution.setStatus(org.springframework.batch.core.BatchStatus.COMPLETED); + stepExecution.setExitStatus(new ExitStatus("done")); + } + }); + propertyContext = new BatchPropertyContext(); + handler.setPropertyContext(propertyContext); + repository = new MapJobRepositoryFactoryBean().getObject(); + handler.setJobRepository(repository); + MyPartitionReducer.reset(); + CountingPartitionCollector.reset(); + } + + @Test + public void testAfterPropertiesSet() throws Exception { + handler = new JsrPartitionHandler(); + + try { + handler.afterPropertiesSet(); + fail("PropertyContext was not checked for"); + } catch(IllegalArgumentException iae) { + assertEquals("A BatchPropertyContext is required", iae.getMessage()); + } + + handler.setPropertyContext(new BatchPropertyContext()); + + try { + handler.afterPropertiesSet(); + fail("Threads or mapper was not checked for"); + } catch(IllegalArgumentException iae) { + assertEquals("Either a mapper implementation or the number of partitions/threads is required", iae.getMessage()); + } + + handler.setThreads(3); + + try { + handler.afterPropertiesSet(); + fail("JobRepository was not checked for"); + } catch(IllegalArgumentException iae) { + assertEquals("A JobRepository is required", iae.getMessage()); + } + + handler.setJobRepository(repository); + handler.afterPropertiesSet(); + + handler.setPollingInterval(-1); + try { + handler.afterPropertiesSet(); + fail("Polling interval was not checked for"); + } catch(IllegalArgumentException iae) { + assertEquals("The polling interval must be positive", iae.getMessage()); + } + } + + @Test + public void testHardcodedNumberOfPartitions() throws Exception { + handler.setThreads(3); + handler.setPartitions(3); + handler.afterPropertiesSet(); + + Collection executions = handler.handle(stepSplitter, stepExecution); + + assertEquals(3, executions.size()); + assertEquals(3, count.get()); + } + + @Test + public void testPollingPartitionsCompletion() throws Exception { + handler.setThreads(3); + handler.setPartitions(3); + handler.setPollingInterval(1000); + handler.afterPropertiesSet(); + + StopWatch stopWatch = new StopWatch(); + stopWatch.start(); + Collection executions = handler.handle(stepSplitter, stepExecution); + stopWatch.stop(); + + assertEquals(3, executions.size()); + assertEquals(3, count.get()); + assertTrue(stopWatch.getLastTaskTimeMillis() >= 1000); + } + + @Test + public void testMapperProvidesPartitions() throws Exception { + handler.setPartitionMapper(new PartitionMapper() { + + @Override + public PartitionPlan mapPartitions() throws Exception { + PartitionPlan plan = new PartitionPlanImpl(); + plan.setPartitions(3); + plan.setThreads(0); + return plan; + } + }); + + handler.afterPropertiesSet(); + + Collection executions = handler.handle(new JsrStepExecutionSplitter(repository, false, "step1", true), stepExecution); + + assertEquals(3, executions.size()); + assertEquals(3, count.get()); + } + + @Test + public void testMapperProvidesPartitionsAndThreads() throws Exception { + handler.setPartitionMapper(new PartitionMapper() { + + @Override + public PartitionPlan mapPartitions() throws Exception { + PartitionPlan plan = new PartitionPlanImpl(); + plan.setPartitions(3); + plan.setThreads(1); + return plan; + } + }); + + handler.afterPropertiesSet(); + + Collection executions = handler.handle(new JsrStepExecutionSplitter(repository, false, "step1", true), stepExecution); + + assertEquals(3, executions.size()); + assertEquals(3, count.get()); + } + + @Test + public void testMapperWithProperties() throws Exception { + handler.setPartitionMapper(new PartitionMapper() { + + @Override + public PartitionPlan mapPartitions() throws Exception { + PartitionPlan plan = new PartitionPlanImpl(); + Properties [] props = new Properties[2]; + props[0] = new Properties(); + props[0].put("key1", "value1"); + props[1] = new Properties(); + props[1].put("key1", "value2"); + plan.setPartitionProperties(props); + plan.setPartitions(3); + plan.setThreads(1); + return plan; + } + }); + + handler.afterPropertiesSet(); + + Collection executions = handler.handle(new JsrStepExecutionSplitter(repository, false, "step1", true), stepExecution); + + assertEquals(3, executions.size()); + assertEquals(3, count.get()); + assertEquals("value1", propertyContext.getStepProperties("step1:partition0").get("key1")); + assertEquals("value2", propertyContext.getStepProperties("step1:partition1").get("key1")); + } + + @Test + public void testAnalyzer() throws Exception { + Queue queue = new ConcurrentLinkedQueue<>(); + queue.add("foo"); + queue.add("bar"); + + handler.setPartitionDataQueue(queue); + handler.setThreads(2); + handler.setPartitions(2); + handler.setPartitionAnalyzer(new Analyzer()); + handler.afterPropertiesSet(); + + Collection executions = handler.handle(new JsrStepExecutionSplitter(repository, false, "step1", true), stepExecution); + + assertEquals(2, executions.size()); + assertEquals(2, count.get()); + assertEquals("foobar", Analyzer.collectorData); + assertEquals("COMPLETEDdone", Analyzer.status); + } + + @Test + public void testRestartNoOverride() throws Exception { + javax.batch.runtime.JobExecution execution1 = runJob("jsrPartitionHandlerRestartWithOverrideJob", null, 1000000L); + assertEquals(BatchStatus.FAILED, execution1.getBatchStatus()); + assertEquals(1, MyPartitionReducer.beginCount); + assertEquals(0, MyPartitionReducer.beforeCount); + assertEquals(1, MyPartitionReducer.rollbackCount); + assertEquals(1, MyPartitionReducer.afterCount); + assertEquals(3, CountingPartitionCollector.collected); + + MyPartitionReducer.reset(); + CountingPartitionCollector.reset(); + + javax.batch.runtime.JobExecution execution2 = restartJob(execution1.getExecutionId(), null, 1000000L); + assertEquals(BatchStatus.COMPLETED, execution2.getBatchStatus()); + assertEquals(1, MyPartitionReducer.beginCount); + assertEquals(1, MyPartitionReducer.beforeCount); + assertEquals(0, MyPartitionReducer.rollbackCount); + assertEquals(1, MyPartitionReducer.afterCount); + assertEquals(1, CountingPartitionCollector.collected); + } + + + @Test + public void testRestartOverride() throws Exception { + Properties jobParameters = new Properties(); + jobParameters.put("mapper.override", "true"); + + javax.batch.runtime.JobExecution execution1 = runJob("jsrPartitionHandlerRestartWithOverrideJob", jobParameters, 1000000L); + assertEquals(BatchStatus.FAILED, execution1.getBatchStatus()); + assertEquals(1, MyPartitionReducer.beginCount); + assertEquals(0, MyPartitionReducer.beforeCount); + assertEquals(1, MyPartitionReducer.rollbackCount); + assertEquals(1, MyPartitionReducer.afterCount); + assertEquals(3, CountingPartitionCollector.collected); + + MyPartitionReducer.reset(); + CountingPartitionCollector.reset(); + + javax.batch.runtime.JobExecution execution2 = restartJob(execution1.getExecutionId(), jobParameters, 1000000L); + assertEquals(BatchStatus.COMPLETED, execution2.getBatchStatus()); + assertEquals(1, MyPartitionReducer.beginCount); + assertEquals(1, MyPartitionReducer.beforeCount); + assertEquals(0, MyPartitionReducer.rollbackCount); + assertEquals(1, MyPartitionReducer.afterCount); + assertEquals(5, CountingPartitionCollector.collected); + } + + public static class CountingPartitionCollector implements PartitionCollector { + + public static int collected = 0; + + public static void reset() { + collected = 0; + } + + @Override + public Serializable collectPartitionData() throws Exception { + collected++; + + return null; + } + } + + public static class MyPartitionReducer implements PartitionReducer { + + public static int beginCount = 0; + public static int beforeCount = 0; + public static int rollbackCount = 0; + public static int afterCount = 0; + + public static void reset() { + beginCount = 0; + beforeCount = 0; + rollbackCount = 0; + afterCount = 0; + } + + @Override + public void beginPartitionedStep() throws Exception { + beginCount++; + } + + @Override + public void beforePartitionedStepCompletion() throws Exception { + beforeCount++; + } + + @Override + public void rollbackPartitionedStep() throws Exception { + rollbackCount++; + } + + @Override + public void afterPartitionedStepCompletion(PartitionStatus status) + throws Exception { + afterCount++; + } + } + + public static class MyPartitionMapper implements PartitionMapper { + + private static int count = 0; + + @Inject + @BatchProperty + String overrideString = "false"; + + @Override + public PartitionPlan mapPartitions() throws Exception { + count++; + + PartitionPlan plan = new PartitionPlanImpl(); + + if(count % 2 == 1) { + plan.setPartitions(3); + plan.setThreads(3); + } else { + plan.setPartitions(5); + plan.setThreads(5); + } + + plan.setPartitionsOverride(Boolean.valueOf(overrideString)); + + Properties[] props = new Properties[3]; + props[0] = new Properties(); + props[1] = new Properties(); + props[2] = new Properties(); + + if(count % 2 == 1) { + props[1].put("fail", "true"); + } + + plan.setPartitionProperties(props); + return plan; + } + } + + public static class MyBatchlet extends BatchletSupport { + @Inject + @BatchProperty + String fail; + + @Override + public String process() { + if("true".equalsIgnoreCase(fail)) { + throw new RuntimeException("Expected"); + } + + return null; + } + } + + public static class Analyzer implements PartitionAnalyzer { + + public static String collectorData; + public static String status; + + @Override + public void analyzeCollectorData(Serializable data) throws Exception { + collectorData = collectorData + data; + } + + @Override + public void analyzeStatus(BatchStatus batchStatus, String exitStatus) + throws Exception { + status = batchStatus + exitStatus; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/partition/JsrStepExecutionSplitterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/partition/JsrStepExecutionSplitterTests.java new file mode 100644 index 0000000000..646d4eed7e --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/partition/JsrStepExecutionSplitterTests.java @@ -0,0 +1,53 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.partition; + +import static org.junit.Assert.assertEquals; + +import java.util.Iterator; +import java.util.Set; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.step.JobRepositorySupport; + +public class JsrStepExecutionSplitterTests { + + private JsrStepExecutionSplitter splitter; + + @Before + public void setUp() throws Exception { + splitter = new JsrStepExecutionSplitter(new JobRepositorySupport(), false, "step1", true); + } + + @Test + public void test() throws Exception { + Set executions = splitter.split(new StepExecution("step1", new JobExecution(5L)), 3); + + assertEquals(3, executions.size()); + + Iterator stepExecutions = executions.iterator(); + + int count = 0; + while(stepExecutions.hasNext()) { + StepExecution curExecution = stepExecutions.next(); + assertEquals("step1:partition" + count, curExecution.getStepName()); + count++; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/partition/PartitionCollectorAdapterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/partition/PartitionCollectorAdapterTests.java new file mode 100644 index 0000000000..6f95a5e05b --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/partition/PartitionCollectorAdapterTests.java @@ -0,0 +1,62 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.partition; + +import static org.junit.Assert.assertEquals; + +import java.io.Serializable; +import java.util.Queue; +import java.util.concurrent.ConcurrentLinkedQueue; +import java.util.concurrent.locks.ReentrantLock; + +import javax.batch.api.partition.PartitionCollector; + +import org.junit.Test; +import org.springframework.batch.core.scope.context.ChunkContext; + +public class PartitionCollectorAdapterTests { + + private PartitionCollectorAdapter adapter; + + @Test + public void testAfterChunkSuccessful() throws Exception { + Queue dataQueue = new ConcurrentLinkedQueue<>(); + + adapter = new PartitionCollectorAdapter(dataQueue, new PartitionCollector() { + + private int count = 0; + + @Override + public Serializable collectPartitionData() throws Exception { + return String.valueOf(count++); + } + }); + + adapter.setPartitionLock(new ReentrantLock()); + + ChunkContext context = new ChunkContext(null); + context.setComplete(); + + adapter.afterChunk(context); + adapter.afterChunkError(context); + adapter.afterChunk(context); + + assertEquals(3, dataQueue.size()); + assertEquals("0", dataQueue.remove()); + assertEquals("1", dataQueue.remove()); + assertEquals("2", dataQueue.remove()); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/DecisionStepTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/DecisionStepTests.java new file mode 100644 index 0000000000..956fd483fe --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/DecisionStepTests.java @@ -0,0 +1,189 @@ +/* + * Copyright 2013-2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step; + +import java.util.List; +import java.util.Properties; +import javax.batch.api.Decider; +import javax.batch.runtime.BatchRuntime; +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.JobExecution; +import javax.batch.runtime.StepExecution; + +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.jsr.AbstractJsrTestCase; +import org.springframework.beans.factory.config.AutowireCapableBeanFactory; +import org.springframework.context.ApplicationContext; +import org.springframework.context.support.GenericXmlApplicationContext; +import org.springframework.test.util.ReflectionTestUtils; +import org.springframework.util.Assert; + +import static org.junit.Assert.assertEquals; + +public class DecisionStepTests extends AbstractJsrTestCase { + + private static ApplicationContext baseContext; + + private JobExplorer jobExplorer; + + @Before + public void setUp() { + StepExecutionCountingDecider.previousStepCount = 0; + + if(jobExplorer == null) { + baseContext = new GenericXmlApplicationContext("jsrBaseContext.xml"); + + baseContext.getAutowireCapableBeanFactory().autowireBeanProperties(this, + AutowireCapableBeanFactory.AUTOWIRE_BY_NAME, false); + } + } + + public void setJobExplorer(JobExplorer jobExplorer) { + this.jobExplorer = jobExplorer; + } + + @Test + public void testDecisionAsFirstStepOfJob() throws Exception { + JobExecution execution = runJob("DecisionStepTests-decisionAsFirstStep-context", new Properties(), 10000L); + assertEquals(BatchStatus.FAILED, execution.getBatchStatus()); + assertEquals(0, BatchRuntime.getJobOperator().getStepExecutions(execution.getExecutionId()).size()); + } + + @Test + public void testDecisionThrowsException() throws Exception { + JobExecution execution = runJob("DecisionStepTests-decisionThrowsException-context", new Properties(), 10000L); + assertEquals(BatchStatus.FAILED, execution.getBatchStatus()); + assertEquals(2, BatchRuntime.getJobOperator().getStepExecutions(execution.getExecutionId()).size()); + } + + @Test + public void testDecisionValidExitStatus() throws Exception { + JobExecution execution = runJob("DecisionStepTests-decisionValidExitStatus-context", new Properties(), 10000L); + assertEquals(BatchStatus.COMPLETED, execution.getBatchStatus()); + assertEquals(3, BatchRuntime.getJobOperator().getStepExecutions(execution.getExecutionId()).size()); + } + + @Test + public void testDecisionUnmappedExitStatus() throws Exception { + JobExecution execution = runJob("DecisionStepTests-decisionInvalidExitStatus-context", new Properties(), 10000L); + assertEquals(BatchStatus.COMPLETED, execution.getBatchStatus()); + List stepExecutions = BatchRuntime.getJobOperator().getStepExecutions(execution.getExecutionId()); + assertEquals(2, stepExecutions.size()); + + for (StepExecution curExecution : stepExecutions) { + assertEquals(BatchStatus.COMPLETED, curExecution.getBatchStatus()); + } + } + + @Test + public void testDecisionCustomExitStatus() throws Exception { + JobExecution execution = runJob("DecisionStepTests-decisionCustomExitStatus-context", new Properties(), 10000L); + assertEquals(BatchStatus.FAILED, execution.getBatchStatus()); + assertEquals(2, BatchRuntime.getJobOperator().getStepExecutions(execution.getExecutionId()).size()); + assertEquals("CustomFail", execution.getExitStatus()); + } + + @Test + public void testDecisionAfterFlow() throws Exception { + JobExecution execution = runJob("DecisionStepTests-decisionAfterFlow-context", new Properties(), 10000L); + assertEquals(execution.getExitStatus(), BatchStatus.COMPLETED, execution.getBatchStatus()); + assertEquals(3, BatchRuntime.getJobOperator().getStepExecutions(execution.getExecutionId()).size()); + } + + @Test + public void testDecisionAfterSplit() throws Exception { + JobExecution execution = runJob("DecisionStepTests-decisionAfterSplit-context", new Properties(), 10000L); + org.springframework.batch.core.JobExecution jobExecution = (org.springframework.batch.core.JobExecution) ReflectionTestUtils.getField(execution, "execution"); + assertEquals(String.format("Received a %s because of %s", execution.getBatchStatus(), jobExecution.getExitStatus().getExitDescription()), BatchStatus.COMPLETED, execution.getBatchStatus()); + assertEquals(4, BatchRuntime.getJobOperator().getStepExecutions(execution.getExecutionId()).size()); + assertEquals(2, StepExecutionCountingDecider.previousStepCount); + } + + @Test + public void testDecisionRestart() throws Exception { + JobExecution execution = runJob("DecisionStepTests-restart-context", new Properties(), 10000L); + assertEquals(BatchStatus.STOPPED, execution.getBatchStatus()); + + List stepExecutions = BatchRuntime.getJobOperator().getStepExecutions(execution.getExecutionId()); + assertEquals(2, stepExecutions.size()); + + assertEquals("step1", stepExecutions.get(0).getStepName()); + assertEquals("decision1", stepExecutions.get(1).getStepName()); + + JobExecution execution2 = restartJob(execution.getExecutionId(), new Properties(), 10000L); + assertEquals(BatchStatus.COMPLETED, execution2.getBatchStatus()); + + List stepExecutions2 = BatchRuntime.getJobOperator().getStepExecutions(execution2.getExecutionId()); + assertEquals(2, stepExecutions2.size()); + + assertEquals("decision1", stepExecutions2.get(0).getStepName()); + assertEquals("step2", stepExecutions2.get(1).getStepName()); + } + + public static class RestartDecider implements Decider { + + private static int runs = 0; + + @Override + public String decide(StepExecution[] executions) throws Exception { + Assert.isTrue(executions.length == 1, "Invalid array length"); + Assert.isTrue(executions[0].getStepName().equals("step1"), "Incorrect step name"); + + if(runs == 0) { + runs++; + return "STOP_HERE"; + } else { + return "CONTINUE"; + } + } + } + + public static class StepExecutionCountingDecider implements Decider { + + static int previousStepCount = 0; + + @Override + public String decide(StepExecution[] executions) throws Exception { + previousStepCount = executions.length; + return "next"; + } + } + + public static class NextDecider implements Decider { + + @Override + public String decide(StepExecution[] executions) throws Exception { + for(StepExecution stepExecution : executions) { + if ("customFailTest".equals(stepExecution.getStepName())) { + return "CustomFail"; + } + } + + return "next"; + } + } + + public static class FailureDecider implements Decider { + + @Override + public String decide(StepExecution[] executions) throws Exception { + throw new RuntimeException("Expected"); + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/batchlet/BatchletAdapterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/batchlet/BatchletAdapterTests.java new file mode 100644 index 0000000000..7713e77259 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/batchlet/BatchletAdapterTests.java @@ -0,0 +1,83 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.batchlet; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.doThrow; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import javax.batch.api.Batchlet; +import javax.batch.operations.BatchRuntimeException; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.repeat.RepeatStatus; + +public class BatchletAdapterTests { + + private BatchletAdapter adapter; + @Mock + private Batchlet delegate; + @Mock + private StepContribution contribution; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + + adapter = new BatchletAdapter(delegate); + } + + @Test(expected=IllegalArgumentException.class) + public void testCreateWithNull() { + adapter = new BatchletAdapter(null); + } + + @Test + public void testExecuteNoExitStatus() throws Exception { + assertEquals(RepeatStatus.FINISHED, adapter.execute(contribution, new ChunkContext(null))); + + verify(delegate).process(); + } + + @Test + public void testExecuteWithExitStatus() throws Exception { + when(delegate.process()).thenReturn("my exit status"); + + assertEquals(RepeatStatus.FINISHED, adapter.execute(contribution, new ChunkContext(null))); + + verify(delegate).process(); + verify(contribution).setExitStatus(new ExitStatus("my exit status")); + } + + @Test + public void testStop() throws Exception{ + adapter.stop(); + verify(delegate).stop(); + } + + @Test(expected=BatchRuntimeException.class) + public void testStopException() throws Exception{ + doThrow(new Exception("expected")).when(delegate).stop(); + adapter.stop(); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/batchlet/BatchletSupport.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/batchlet/BatchletSupport.java new file mode 100644 index 0000000000..3e0b40f1a3 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/batchlet/BatchletSupport.java @@ -0,0 +1,31 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.batchlet; + +import javax.batch.api.Batchlet; + +public class BatchletSupport implements Batchlet { + + @Override + public String process() throws Exception { + return null; + } + + @Override + public void stop() throws Exception { + // no-op + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/batchlet/FailingBatchlet.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/batchlet/FailingBatchlet.java new file mode 100644 index 0000000000..ca7273f521 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/batchlet/FailingBatchlet.java @@ -0,0 +1,36 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.batchlet; + +import javax.batch.api.Batchlet; + +/** + *

      + * Test batchlet that always fails. + *

      + * + * @author Chris Schaefer + */ +public class FailingBatchlet implements Batchlet { + @Override + public String process() throws Exception { + throw new RuntimeException("process failed"); + } + + @Override + public void stop() throws Exception { + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/batchlet/RestartBatchlet.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/batchlet/RestartBatchlet.java new file mode 100644 index 0000000000..85895f06fa --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/batchlet/RestartBatchlet.java @@ -0,0 +1,38 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.batchlet; + +import javax.batch.api.Batchlet; + +public class RestartBatchlet implements Batchlet { + + private static int runCount = 0; + + @Override + public String process() throws Exception { + runCount++; + + if(runCount == 1) { + throw new RuntimeException("This is expected"); + } + + return null; + } + + @Override + public void stop() throws Exception { + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/item/JsrChunkProcessorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/item/JsrChunkProcessorTests.java new file mode 100644 index 0000000000..72214df7bb --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/item/JsrChunkProcessorTests.java @@ -0,0 +1,423 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.item; + +import static org.junit.Assert.assertEquals; + +import java.util.ArrayList; +import java.util.List; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ItemProcessListener; +import org.springframework.batch.core.ItemReadListener; +import org.springframework.batch.core.ItemWriteListener; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInterruptedException; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.jsr.step.builder.JsrSimpleStepBuilder; +import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; +import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; +import org.springframework.batch.core.step.builder.StepBuilder; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.ListItemReader; +import org.springframework.batch.support.transaction.ResourcelessTransactionManager; +import org.springframework.lang.Nullable; + +public class JsrChunkProcessorTests { + + private FailingListItemReader reader; + private FailingCountingItemProcessor processor; + private StoringItemWriter writer; + private CountingListener readListener; + private JsrSimpleStepBuilder builder; + private JobRepository repository; + private StepExecution stepExecution; + + @Before + public void setUp() throws Exception { + + List items = new ArrayList<>(); + + for (int i = 0; i < 25; i++) { + items.add("item " + i); + } + + reader = new FailingListItemReader(items); + processor = new FailingCountingItemProcessor(); + writer = new StoringItemWriter(); + readListener = new CountingListener(); + + builder = new JsrSimpleStepBuilder<>(new StepBuilder("step1")); + builder.setBatchPropertyContext(new BatchPropertyContext()); + repository = new MapJobRepositoryFactoryBean().getObject(); + builder.repository(repository); + builder.transactionManager(new ResourcelessTransactionManager()); + stepExecution = null; + } + + @Test + public void testNoInputNoListeners() throws Exception{ + reader = new FailingListItemReader(new ArrayList<>()); + Step step = builder.chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) readListener).build(); + + runStep(step); + + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(0, processor.count); + assertEquals(0, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(0, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(0, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + } + + @Test + public void testSimpleScenarioNoListeners() throws Exception{ + Step step = builder.chunk(25).reader(reader).processor(processor).writer(writer).build(); + + runStep(step); + + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(25, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(25, writer.results.size()); + assertEquals(25, processor.count); + + int count = 0; + for (String curItem : writer.results) { + assertEquals("item " + count, curItem); + count++; + } + } + + @Test + public void testSimpleScenarioNoProcessor() throws Exception{ + Step step = builder.chunk(25).reader(reader).writer(writer).listener((ItemReadListener) readListener).build(); + + runStep(step); + + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(25, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(0, readListener.afterProcess); + assertEquals(25, readListener.afterRead); + assertEquals(1, readListener.afterWrite); + assertEquals(0, readListener.beforeProcess); + assertEquals(26, readListener.beforeRead); + assertEquals(1, readListener.beforeWriteCount); + assertEquals(0, readListener.onProcessError); + assertEquals(0, readListener.onReadError); + assertEquals(0, readListener.onWriteError); + assertEquals(0, processor.count); + + int count = 0; + for (String curItem : writer.results) { + assertEquals("item " + count, curItem); + count++; + } + } + + @Test + public void testProcessorFilteringNoListeners() throws Exception{ + processor.filter = true; + Step step = builder.chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) readListener).build(); + + runStep(step); + + int count = 0; + for (String curItem : writer.results) { + assertEquals("item " + count, curItem); + count += 2; + } + + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(13, stepExecution.getWriteCount()); + assertEquals(12, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(25, processor.count); + } + + @Test + public void testReadError() throws Exception{ + reader.failCount = 10; + + Step step = builder.chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) readListener).build(); + + runStep(step); + + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals(9, processor.count); + assertEquals(0, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(9, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(0, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(1, stepExecution.getFailureExceptions().size()); + assertEquals("expected at read index 10", stepExecution.getFailureExceptions().get(0).getMessage()); + assertEquals(9, readListener.afterProcess); + assertEquals(9, readListener.afterRead); + assertEquals(0, readListener.afterWrite); + assertEquals(9, readListener.beforeProcess); + assertEquals(10, readListener.beforeRead); + assertEquals(0, readListener.beforeWriteCount); + assertEquals(0, readListener.onProcessError); + assertEquals(1, readListener.onReadError); + assertEquals(0, readListener.onWriteError); + } + + @Test + public void testProcessError() throws Exception{ + processor.failCount = 10; + + Step step = builder.chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) readListener).build(); + + runStep(step); + + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals(10, processor.count); + assertEquals(0, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(10, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(0, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals("expected at process index 10", stepExecution.getFailureExceptions().get(0).getMessage()); + assertEquals(9, readListener.afterProcess); + assertEquals(10, readListener.afterRead); + assertEquals(0, readListener.afterWrite); + assertEquals(10, readListener.beforeProcess); + assertEquals(10, readListener.beforeRead); + assertEquals(0, readListener.beforeWriteCount); + assertEquals(1, readListener.onProcessError); + assertEquals(0, readListener.onReadError); + assertEquals(0, readListener.onWriteError); + } + + @Test + public void testWriteError() throws Exception{ + writer.fail = true; + + Step step = builder.chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) readListener).build(); + + runStep(step); + + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals(25, processor.count); + assertEquals(0, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(0, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals("expected in write", stepExecution.getFailureExceptions().get(0).getMessage()); + assertEquals(25, readListener.afterProcess); + assertEquals(25, readListener.afterRead); + assertEquals(0, readListener.afterWrite); + assertEquals(25, readListener.beforeProcess); + assertEquals(25, readListener.beforeRead); + assertEquals(1, readListener.beforeWriteCount); + assertEquals(0, readListener.onProcessError); + assertEquals(0, readListener.onReadError); + assertEquals(1, readListener.onWriteError); + } + + @Test + public void testMultipleChunks() throws Exception{ + + Step step = builder.chunk(10).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) readListener).build(); + + runStep(step); + + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(25, processor.count); + assertEquals(25, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(25, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(25, readListener.afterProcess); + assertEquals(25, readListener.afterRead); + assertEquals(3, readListener.afterWrite); + assertEquals(25, readListener.beforeProcess); + assertEquals(26, readListener.beforeRead); + assertEquals(3, readListener.beforeWriteCount); + assertEquals(0, readListener.onProcessError); + assertEquals(0, readListener.onReadError); + assertEquals(0, readListener.onWriteError); + } + + protected void runStep(Step step) + throws JobExecutionAlreadyRunningException, JobRestartException, + JobInstanceAlreadyCompleteException, JobInterruptedException { + JobExecution jobExecution = repository.createJobExecution("job1", new JobParameters()); + stepExecution = new StepExecution("step1", jobExecution); + repository.add(stepExecution); + + step.execute(stepExecution); + } + + public static class FailingListItemReader extends ListItemReader { + + protected int failCount = -1; + protected int count = 0; + + public FailingListItemReader(List list) { + super(list); + } + + @Nullable + @Override + public String read() { + count++; + + if(failCount == count) { + throw new RuntimeException("expected at read index " + failCount); + } else { + return super.read(); + } + } + } + + public static class FailingCountingItemProcessor implements ItemProcessor{ + protected int count = 0; + protected int failCount = -1; + protected boolean filter = false; + + @Nullable + @Override + public String process(String item) throws Exception { + count++; + + if(filter && count % 2 == 0) { + return null; + } else if(count == failCount){ + throw new RuntimeException("expected at process index " + failCount); + } else { + return item; + } + } + } + + public static class StoringItemWriter implements ItemWriter{ + + protected List results = new ArrayList<>(); + protected boolean fail = false; + + @Override + public void write(List items) throws Exception { + if(fail) { + throw new RuntimeException("expected in write"); + } + + results.addAll(items); + } + } + + public static class CountingListener implements ItemReadListener, ItemProcessListener, ItemWriteListener { + + protected int beforeWriteCount = 0; + protected int afterWrite = 0; + protected int onWriteError = 0; + protected int beforeProcess = 0; + protected int afterProcess = 0; + protected int onProcessError = 0; + protected int beforeRead = 0; + protected int afterRead = 0; + protected int onReadError = 0; + + @Override + public void beforeWrite(List items) { + beforeWriteCount++; + } + + @Override + public void afterWrite(List items) { + afterWrite++; + } + + @Override + public void onWriteError(Exception exception, + List items) { + onWriteError++; + } + + @Override + public void beforeProcess(String item) { + beforeProcess++; + } + + @Override + public void afterProcess(String item, @Nullable String result) { + afterProcess++; + } + + @Override + public void onProcessError(String item, Exception e) { + onProcessError++; + } + + @Override + public void beforeRead() { + beforeRead++; + } + + @Override + public void afterRead(String item) { + afterRead++; + } + + @Override + public void onReadError(Exception ex) { + onReadError++; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/item/JsrChunkProviderTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/item/JsrChunkProviderTests.java new file mode 100644 index 0000000000..a76c120881 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/item/JsrChunkProviderTests.java @@ -0,0 +1,40 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.item; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.step.item.Chunk; + +public class JsrChunkProviderTests { + + private JsrChunkProvider provider; + + @Before + public void setUp() throws Exception { + provider = new JsrChunkProvider<>(); + } + + @Test + public void test() throws Exception { + Chunk chunk = provider.provide(null); + assertNotNull(chunk); + assertEquals(0, chunk.getItems().size()); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/item/JsrFaultTolerantChunkProcessorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/item/JsrFaultTolerantChunkProcessorTests.java new file mode 100644 index 0000000000..95c530af0d --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/item/JsrFaultTolerantChunkProcessorTests.java @@ -0,0 +1,628 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.item; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import java.util.ArrayList; +import java.util.List; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ItemProcessListener; +import org.springframework.batch.core.ItemReadListener; +import org.springframework.batch.core.ItemWriteListener; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInterruptedException; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.SkipListener; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.core.jsr.step.builder.JsrFaultTolerantStepBuilder; +import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; +import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; +import org.springframework.batch.core.step.builder.StepBuilder; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.ListItemReader; +import org.springframework.batch.support.transaction.ResourcelessTransactionManager; +import org.springframework.lang.Nullable; + +public class JsrFaultTolerantChunkProcessorTests { + + private FailingListItemReader reader; + private FailingCountingItemProcessor processor; + private StoringItemWriter writer; + private CountingListener listener; + private JsrFaultTolerantStepBuilder builder; + private JobRepository repository; + private StepExecution stepExecution; + + @Before + public void setUp() throws Exception { + + List items = new ArrayList<>(); + + for (int i = 0; i < 25; i++) { + items.add("item " + i); + } + + reader = new FailingListItemReader(items); + processor = new FailingCountingItemProcessor(); + writer = new StoringItemWriter(); + listener = new CountingListener(); + + builder = new JsrFaultTolerantStepBuilder<>(new StepBuilder("step1")); + builder.setBatchPropertyContext(new BatchPropertyContext()); + repository = new MapJobRepositoryFactoryBean().getObject(); + builder.repository(repository); + builder.transactionManager(new ResourcelessTransactionManager()); + stepExecution = null; + } + + @Test + public void testNoInputNoListeners() throws Exception{ + reader = new FailingListItemReader(new ArrayList<>()); + Step step = builder.chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(0, processor.count); + assertEquals(0, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(0, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(0, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + } + + @Test + public void testSimpleScenarioNoListeners() throws Exception{ + Step step = builder.chunk(25).reader(reader).processor(processor).writer(writer).build(); + + runStep(step); + + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(25, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(25, writer.results.size()); + assertEquals(25, processor.count); + + int count = 0; + for (String curItem : writer.results) { + assertEquals("item " + count, curItem); + count++; + } + } + + @Test + public void testSimpleScenarioNoProcessor() throws Exception{ + Step step = builder.chunk(25).reader(reader).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(25, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(0, listener.afterProcess); + assertEquals(25, listener.afterRead); + assertEquals(1, listener.afterWrite); + assertEquals(0, listener.beforeProcess); + assertEquals(26, listener.beforeRead); + assertEquals(1, listener.beforeWriteCount); + assertEquals(0, listener.onProcessError); + assertEquals(0, listener.onReadError); + assertEquals(0, listener.onWriteError); + assertEquals(0, processor.count); + + int count = 0; + for (String curItem : writer.results) { + assertEquals("item " + count, curItem); + count++; + } + } + + @Test + public void testProcessorFilteringNoListeners() throws Exception{ + processor.filter = true; + Step step = builder.chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + int count = 0; + for (String curItem : writer.results) { + assertEquals("item " + count, curItem); + count += 2; + } + + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(13, stepExecution.getWriteCount()); + assertEquals(12, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(25, processor.count); + } + + @Test + public void testSkipReadError() throws Exception{ + reader.failCount = 10; + + Step step = builder.faultTolerant().skip(RuntimeException.class).skipLimit(20).chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + assertNotNull(stepExecution); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(25, processor.count); + assertEquals(25, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(1, stepExecution.getReadSkipCount()); + assertEquals(1, stepExecution.getSkipCount()); + assertEquals(25, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(0, stepExecution.getFailureExceptions().size()); + assertEquals(25, listener.afterProcess); + assertEquals(25, listener.afterRead); + assertEquals(1, listener.afterWrite); + assertEquals(25, listener.beforeProcess); + assertEquals(27, listener.beforeRead); + assertEquals(1, listener.beforeWriteCount); + assertEquals(0, listener.onProcessError); + assertEquals(1, listener.onReadError); + assertEquals(0, listener.onWriteError); + } + + @Test + public void testRetryReadError() throws Exception{ + reader.failCount = 10; + + Step step = builder.faultTolerant().retry(RuntimeException.class).retryLimit(20).chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(25, processor.count); + assertEquals(25, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(25, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(0, stepExecution.getFailureExceptions().size()); + assertEquals(25, listener.afterProcess); + assertEquals(25, listener.afterRead); + assertEquals(1, listener.afterWrite); + assertEquals(25, listener.beforeProcess); + assertEquals(27, listener.beforeRead); + assertEquals(1, listener.beforeWriteCount); + assertEquals(0, listener.onProcessError); + assertEquals(1, listener.onReadError); + assertEquals(0, listener.onWriteError); + } + + @Test + public void testReadError() throws Exception{ + reader.failCount = 10; + + Step step = builder.chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + assertNotNull(stepExecution); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals(9, processor.count); + assertEquals(0, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(9, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(0, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(1, stepExecution.getFailureExceptions().size()); + assertEquals("expected at read index 10", stepExecution.getFailureExceptions().get(0).getCause().getMessage()); + assertEquals(9, listener.afterProcess); + assertEquals(9, listener.afterRead); + assertEquals(0, listener.afterWrite); + assertEquals(9, listener.beforeProcess); + assertEquals(10, listener.beforeRead); + assertEquals(0, listener.beforeWriteCount); + assertEquals(0, listener.onProcessError); + assertEquals(1, listener.onReadError); + assertEquals(0, listener.onWriteError); + } + + @Test + public void testProcessError() throws Exception{ + processor.failCount = 10; + + Step step = builder.chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + assertEquals(10, processor.count); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals(0, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(10, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(0, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals("expected at process index 10", stepExecution.getFailureExceptions().get(0).getCause().getMessage()); + assertEquals(9, listener.afterProcess); + assertEquals(10, listener.afterRead); + assertEquals(0, listener.afterWrite); + assertEquals(10, listener.beforeProcess); + assertEquals(10, listener.beforeRead); + assertEquals(0, listener.beforeWriteCount); + assertEquals(1, listener.onProcessError); + assertEquals(0, listener.onReadError); + assertEquals(0, listener.onWriteError); + } + + @Test + public void testSkipProcessError() throws Exception{ + processor.failCount = 10; + + Step step = builder.faultTolerant().skip(RuntimeException.class).skipLimit(20).chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + assertNotNull(stepExecution); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(25, processor.count); + assertEquals(24, writer.results.size()); + assertEquals(1, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(1, stepExecution.getSkipCount()); + assertEquals(24, stepExecution.getWriteCount()); + assertEquals(1, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(0, stepExecution.getFailureExceptions().size()); + assertEquals(24, listener.afterProcess); + assertEquals(25, listener.afterRead); + assertEquals(1, listener.afterWrite); + assertEquals(25, listener.beforeProcess); + assertEquals(26, listener.beforeRead); + assertEquals(1, listener.beforeWriteCount); + assertEquals(1, listener.onProcessError); + assertEquals(0, listener.onReadError); + assertEquals(0, listener.onWriteError); + } + + @Test + public void testRetryProcessError() throws Exception{ + processor.failCount = 10; + + Step step = builder.faultTolerant().retry(RuntimeException.class).retryLimit(20).chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + assertNotNull(stepExecution); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(26, processor.count); + assertEquals(25, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(25, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(0, stepExecution.getFailureExceptions().size()); + assertEquals(25, listener.afterProcess); + assertEquals(25, listener.afterRead); + assertEquals(1, listener.afterWrite); + assertEquals(26, listener.beforeProcess); + assertEquals(26, listener.beforeRead); + assertEquals(1, listener.beforeWriteCount); + assertEquals(1, listener.onProcessError); + assertEquals(0, listener.onReadError); + assertEquals(0, listener.onWriteError); + } + + @Test + public void testWriteError() throws Exception{ + writer.fail = true; + + Step step = builder.chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + assertEquals(25, processor.count); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals(0, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(0, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(25, listener.afterProcess); + assertEquals(25, listener.afterRead); + assertEquals(0, listener.afterWrite); + assertEquals(25, listener.beforeProcess); + assertEquals(25, listener.beforeRead); + assertEquals(1, listener.beforeWriteCount); + assertEquals(0, listener.onProcessError); + assertEquals(0, listener.onReadError); + assertEquals(1, listener.onWriteError); + } + + @Test + public void testRetryWriteError() throws Exception{ + writer.fail = true; + + Step step = builder.faultTolerant().retry(RuntimeException.class).retryLimit(25).chunk(25).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + assertEquals(25, processor.count); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(25, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(25, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(25, listener.afterProcess); + assertEquals(25, listener.afterRead); + assertEquals(1, listener.afterWrite); + assertEquals(25, listener.beforeProcess); + assertEquals(26, listener.beforeRead); + assertEquals(2, listener.beforeWriteCount); + assertEquals(0, listener.onProcessError); + assertEquals(0, listener.onReadError); + assertEquals(1, listener.onWriteError); + } + + @Test + public void testSkipWriteError() throws Exception{ + writer.fail = true; + + Step step = builder.faultTolerant().skip(RuntimeException.class).skipLimit(25).chunk(7).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(25, processor.count); + assertEquals(18, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(1, stepExecution.getSkipCount()); + assertEquals(18, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(1, stepExecution.getWriteSkipCount()); + assertEquals(25, listener.afterProcess); + assertEquals(25, listener.afterRead); + assertEquals(3, listener.afterWrite); + assertEquals(25, listener.beforeProcess); + assertEquals(26, listener.beforeRead); + assertEquals(4, listener.beforeWriteCount); + assertEquals(0, listener.onProcessError); + assertEquals(0, listener.onReadError); + assertEquals(1, listener.onWriteError); + assertEquals(0, listener.onSkipInRead); + assertEquals(0, listener.onSkipInProcess); + assertEquals(1, listener.onSkipInWrite); + } + + @Test + public void testMultipleChunks() throws Exception{ + + Step step = builder.chunk(10).reader(reader).processor(processor).writer(writer).listener((ItemReadListener) listener).build(); + + runStep(step); + + assertEquals(25, processor.count); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals(25, writer.results.size()); + assertEquals(0, stepExecution.getProcessSkipCount()); + assertEquals(25, stepExecution.getReadCount()); + assertEquals(0, stepExecution.getReadSkipCount()); + assertEquals(0, stepExecution.getSkipCount()); + assertEquals(25, stepExecution.getWriteCount()); + assertEquals(0, stepExecution.getFilterCount()); + assertEquals(0, stepExecution.getWriteSkipCount()); + assertEquals(25, listener.afterProcess); + assertEquals(25, listener.afterRead); + assertEquals(3, listener.afterWrite); + assertEquals(25, listener.beforeProcess); + assertEquals(26, listener.beforeRead); + assertEquals(3, listener.beforeWriteCount); + assertEquals(0, listener.onProcessError); + assertEquals(0, listener.onReadError); + assertEquals(0, listener.onWriteError); + } + + protected void runStep(Step step) + throws JobExecutionAlreadyRunningException, JobRestartException, + JobInstanceAlreadyCompleteException, JobInterruptedException { + JobExecution jobExecution = repository.createJobExecution("job1", new JobParameters()); + stepExecution = new StepExecution("step1", jobExecution); + repository.add(stepExecution); + + step.execute(stepExecution); + } + + public static class FailingListItemReader extends ListItemReader { + + protected int failCount = -1; + protected int count = 0; + + public FailingListItemReader(List list) { + super(list); + } + + @Nullable + @Override + public String read() { + count++; + + if(failCount == count) { + throw new RuntimeException("expected at read index " + failCount); + } else { + return super.read(); + } + } + } + + public static class FailingCountingItemProcessor implements ItemProcessor{ + protected int count = 0; + protected int failCount = -1; + protected boolean filter = false; + + @Nullable + @Override + public String process(String item) throws Exception { + count++; + + if(filter && count % 2 == 0) { + return null; + } else if(count == failCount){ + throw new RuntimeException("expected at process index " + failCount); + } else { + return item; + } + } + } + + public static class StoringItemWriter implements ItemWriter{ + + protected List results = new ArrayList<>(); + protected boolean fail = false; + + @Override + public void write(List items) throws Exception { + if(fail) { + fail = false; + throw new RuntimeException("expected in write"); + } + + results.addAll(items); + } + } + + public static class CountingListener implements ItemReadListener, ItemProcessListener, ItemWriteListener, SkipListener> { + + protected int beforeWriteCount = 0; + protected int afterWrite = 0; + protected int onWriteError = 0; + protected int beforeProcess = 0; + protected int afterProcess = 0; + protected int onProcessError = 0; + protected int beforeRead = 0; + protected int afterRead = 0; + protected int onReadError = 0; + protected int onSkipInRead = 0; + protected int onSkipInProcess = 0; + protected int onSkipInWrite = 0; + + @Override + public void beforeWrite(List items) { + beforeWriteCount++; + } + + @Override + public void afterWrite(List items) { + afterWrite++; + } + + @Override + public void onWriteError(Exception exception, + List items) { + onWriteError++; + } + + @Override + public void beforeProcess(String item) { + beforeProcess++; + } + + @Override + public void afterProcess(String item, @Nullable String result) { + afterProcess++; + } + + @Override + public void onProcessError(String item, Exception e) { + onProcessError++; + } + + @Override + public void beforeRead() { + beforeRead++; + } + + @Override + public void afterRead(String item) { + afterRead++; + } + + @Override + public void onReadError(Exception ex) { + onReadError++; + } + + @Override + public void onSkipInRead(Throwable t) { + onSkipInRead++; + } + + @Override + public void onSkipInWrite(List items, Throwable t) { + onSkipInWrite++; + } + + @Override + public void onSkipInProcess(String item, Throwable t) { + onSkipInProcess++; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/listener/ExitStatusSettingStepListener.java b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/listener/ExitStatusSettingStepListener.java new file mode 100644 index 0000000000..de9c372eef --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/listener/ExitStatusSettingStepListener.java @@ -0,0 +1,55 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.jsr.step.listener; + +import javax.batch.api.BatchProperty; +import javax.batch.api.listener.StepListener; +import javax.batch.runtime.context.JobContext; +import javax.inject.Inject; + +/** + *

      + * {@link StepListener} for testing. Sets or appends the value of the + * testProperty field to the {@link JobContext} exit status on afterStep. + *

      + * + * @author Chris Schaefer + * @since 3.0 + */ +public class ExitStatusSettingStepListener implements StepListener { + @Inject + @BatchProperty + private String testProperty; + + @Inject + private JobContext jobContext; + + @Override + public void beforeStep() throws Exception { + + } + + @Override + public void afterStep() throws Exception { + String exitStatus = jobContext.getExitStatus(); + + if("".equals(exitStatus) || exitStatus == null) { + jobContext.setExitStatus(testProperty); + } else { + jobContext.setExitStatus(exitStatus + testProperty); + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/EmptyItemWriter.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/EmptyItemWriter.java index 3a49d2a608..18c4f36c01 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/EmptyItemWriter.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/EmptyItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobExecutionNotFailedExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobExecutionNotFailedExceptionTests.java index 7de6f10dee..a1f1a13073 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobExecutionNotFailedExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobExecutionNotFailedExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobExecutionNotRunningExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobExecutionNotRunningExceptionTests.java index 29a7321650..b81c5db8d3 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobExecutionNotRunningExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobExecutionNotRunningExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobExecutionNotStoppedExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobExecutionNotStoppedExceptionTests.java index db6e99f1d1..5102c1036f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobExecutionNotStoppedExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobExecutionNotStoppedExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobInstanceAlreadyExistsExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobInstanceAlreadyExistsExceptionTests.java index 3921e4d898..2453e8e98d 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobInstanceAlreadyExistsExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobInstanceAlreadyExistsExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobLauncherIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobLauncherIntegrationTests.java index e1c255bf2c..c0910b9f90 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobLauncherIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobLauncherIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.launch; import static org.junit.Assert.assertEquals; @@ -38,13 +53,13 @@ public void setDataSource(DataSource dataSource) { @Test public void testLaunchAndRelaunch() throws Exception { - int before = jdbcTemplate.queryForInt("select count(*) from BATCH_JOB_INSTANCE"); + int before = jdbcTemplate.queryForObject("select count(*) from BATCH_JOB_INSTANCE", Integer.class); JobExecution jobExecution = launch(true,0); launch(false, jobExecution.getId()); launch(false, jobExecution.getId()); - int after = jdbcTemplate.queryForInt("select count(*) from BATCH_JOB_INSTANCE"); + int after = jdbcTemplate.queryForObject("select count(*) from BATCH_JOB_INSTANCE", Integer.class); assertEquals(before+1, after); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobParametersNotFoundExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobParametersNotFoundExceptionTests.java index 003e33e0be..dca7604bdc 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobParametersNotFoundExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/JobParametersNotFoundExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/NoSuchJobExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/NoSuchJobExceptionTests.java index 3ad7fec123..8bcafdb373 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/NoSuchJobExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/NoSuchJobExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/NoSuchJobExecutionExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/NoSuchJobExecutionExceptionTests.java index 46fdab4d25..6d8e62c137 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/NoSuchJobExecutionExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/NoSuchJobExecutionExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/NoSuchJobInstanceExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/NoSuchJobInstanceExceptionTests.java index 1a1a4e3006..fbb55694ce 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/NoSuchJobInstanceExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/NoSuchJobInstanceExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/SimpleJobLauncherTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/SimpleJobLauncherTests.java index e8b7d2242b..6f164f1782 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/SimpleJobLauncherTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/SimpleJobLauncherTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -41,6 +41,7 @@ import org.springframework.batch.core.job.DefaultJobParametersValidator; import org.springframework.batch.core.job.JobSupport; import org.springframework.batch.core.launch.support.SimpleJobLauncher; +import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.JobRestartException; import org.springframework.core.task.TaskExecutor; @@ -150,7 +151,7 @@ public void execute(JobExecution execution) { @Test public void testTaskExecutor() throws Exception { - final List list = new ArrayList(); + final List list = new ArrayList<>(); jobLauncher.setTaskExecutor(new TaskExecutor() { @Override public void execute(Runnable task) { @@ -165,7 +166,7 @@ public void execute(Runnable task) { @Test public void testTaskExecutorRejects() throws Exception { - final List list = new ArrayList(); + final List list = new ArrayList<>(); jobLauncher.setTaskExecutor(new TaskExecutor() { @Override public void execute(Runnable task) { @@ -274,8 +275,25 @@ private boolean contains(String str, String searchStr) { */ @Test(expected=JobRestartException.class) public void testRunStepStatusUnknown() throws Exception { - //try and restart a job where the step execution is UNKNOWN - //setup + testRestartStepExecutionInvalidStatus(BatchStatus.UNKNOWN); + } + + @Test(expected = JobExecutionAlreadyRunningException.class) + public void testRunStepStatusStarting() throws Exception { + testRestartStepExecutionInvalidStatus(BatchStatus.STARTING); + } + + @Test(expected = JobExecutionAlreadyRunningException.class) + public void testRunStepStatusStarted() throws Exception { + testRestartStepExecutionInvalidStatus(BatchStatus.STARTED); + } + + @Test(expected = JobExecutionAlreadyRunningException.class) + public void testRunStepStatusStopping() throws Exception { + testRestartStepExecutionInvalidStatus(BatchStatus.STOPPING); + } + + private void testRestartStepExecutionInvalidStatus(BatchStatus status) throws Exception { String jobName = "test_job"; JobRepository jobRepository = mock(JobRepository.class); JobParameters parameters = new JobParametersBuilder().addLong("runtime", System.currentTimeMillis()).toJobParameters(); @@ -288,7 +306,7 @@ public void testRunStepStatusUnknown() throws Exception { when(job.isRestartable()).thenReturn(true); when(job.getJobParametersValidator()).thenReturn(validator); when(jobRepository.getLastJobExecution(jobName, parameters)).thenReturn(jobExecution); - when(stepExecution.getStatus()).thenReturn(BatchStatus.UNKNOWN); + when(stepExecution.getStatus()).thenReturn(status); when(jobExecution.getStepExecutions()).thenReturn(Arrays.asList(stepExecution)); //setup launcher @@ -297,6 +315,5 @@ public void testRunStepStatusUnknown() throws Exception { //run jobLauncher.run(job, parameters); - } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/CommandLineJobRunnerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/CommandLineJobRunnerTests.java index 7e2a85e386..8b068fc2bf 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/CommandLineJobRunnerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/CommandLineJobRunnerTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,14 +15,12 @@ */ package org.springframework.batch.core.launch.support; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; - import java.io.IOException; import java.io.InputStream; import java.util.ArrayList; import java.util.Arrays; import java.util.Date; +import java.util.HashSet; import java.util.List; import java.util.Properties; import java.util.Set; @@ -30,6 +28,7 @@ import org.junit.After; import org.junit.Before; import org.junit.Test; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.Job; @@ -42,12 +41,20 @@ import org.springframework.batch.core.converter.JobParametersConverter; import org.springframework.batch.core.explore.JobExplorer; import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.launch.NoSuchJobException; import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; import org.springframework.batch.core.step.JobRepositorySupport; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.lang.Nullable; import org.springframework.util.ClassUtils; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + /** * @author Lucas Ward + * @author Mahmoud Ben Hassine * */ public class CommandLineJobRunnerTests { @@ -69,7 +76,7 @@ public class CommandLineJobRunnerTests { @Before public void setUp() throws Exception { - JobExecution jobExecution = new JobExecution(null, new Long(1), null); + JobExecution jobExecution = new JobExecution(null, new Long(1), null, null); ExitStatus exitStatus = ExitStatus.COMPLETED; jobExecution.setExitStatus(exitStatus); StubJobLauncher.jobExecution = jobExecution; @@ -117,7 +124,7 @@ public void testInvalidArgs() throws Exception { CommandLineJobRunner.main(args); assertEquals(1, StubSystemExiter.status); String errorMessage = CommandLineJobRunner.getErrorMessage(); - assertTrue("Wrong error message: " + errorMessage, errorMessage.contains("Config locations must not be null")); + assertTrue("Wrong error message: " + errorMessage, errorMessage.contains("At least 2 arguments are required: JobPath/JobClass and jobIdentifier.")); } @Test @@ -126,8 +133,9 @@ public void testWrongJobName() throws Exception { CommandLineJobRunner.main(args); assertEquals(1, StubSystemExiter.status); String errorMessage = CommandLineJobRunner.getErrorMessage(); - assertTrue("Wrong error message: " + errorMessage, errorMessage - .contains("No bean named 'no-such-job' is defined")); + assertTrue("Wrong error message: " + errorMessage, (errorMessage + .contains("No bean named 'no-such-job' is defined") || (errorMessage + .contains("No bean named 'no-such-job' available")))); } @Test @@ -177,6 +185,28 @@ public int read() { assertEquals(0, StubSystemExiter.status); assertEquals(2, StubJobLauncher.jobParameters.getParameters().size()); } + + @Test + public void testWithStdinCommandLineWithEmptyLines() throws Throwable { + System.setIn(new InputStream() { + char[] input = (jobPath+"\n"+jobName+"\nfoo=bar\n\nspam=bucket\n\n").toCharArray(); + + int index = 0; + + @Override + public int available() { + return input.length - index; + } + + @Override + public int read() { + return index(); + StubJobExplorer.jobInstances = new ArrayList<>(); CommandLineJobRunner.main(args); assertEquals(0, StubSystemExiter.status); JobParameters jobParameters = new JobParametersBuilder().addString("foo", "spam").toJobParameters(); @@ -354,6 +384,19 @@ public void testDestroyCallback() throws Throwable { assertTrue(StubJobLauncher.destroyed); } + @Test + public void testJavaConfig() throws Exception { + String[] args = + new String[] { "org.springframework.batch.core.launch.support.CommandLineJobRunnerTests$Configuration1", + "invalidJobName"}; + CommandLineJobRunner.presetSystemExiter(new StubSystemExiter()); + CommandLineJobRunner.main(args); + assertEquals(1, StubSystemExiter.status); + String errorMessage = CommandLineJobRunner.getErrorMessage(); + assertTrue("Wrong error message: " + errorMessage, + errorMessage.contains("A JobLauncher must be provided. Please add one to the configuration.")); + } + public static class StubSystemExiter implements SystemExiter { private static int status; @@ -407,19 +450,20 @@ public static class StubJobRepository extends JobRepositorySupport { public static class StubJobExplorer implements JobExplorer { - static List jobInstances = new ArrayList(); + static List jobInstances = new ArrayList<>(); static JobExecution jobExecution; static JobParameters jobParameters = new JobParameters(); @Override - public Set findRunningJobExecutions(String jobName) { - throw new UnsupportedOperationException(); + public Set findRunningJobExecutions(@Nullable String jobName) { + return new HashSet<>(); } + @Nullable @Override - public JobExecution getJobExecution(Long executionId) { + public JobExecution getJobExecution(@Nullable Long executionId) { if (jobExecution != null) { return jobExecution; } @@ -451,7 +495,7 @@ public List getJobExecutions(JobInstance jobInstance) { } private JobExecution createJobExecution(JobInstance jobInstance, BatchStatus status) { - JobExecution jobExecution = new JobExecution(jobInstance, 1L, jobParameters); + JobExecution jobExecution = new JobExecution(jobInstance, 1L, jobParameters, null); jobExecution.setStatus(status); jobExecution.setStartTime(new Date()); if (status != BatchStatus.STARTED) { @@ -460,23 +504,37 @@ private JobExecution createJobExecution(JobInstance jobInstance, BatchStatus sta return jobExecution; } + @Nullable @Override - public JobInstance getJobInstance(Long instanceId) { + public JobInstance getJobInstance(@Nullable Long instanceId) { throw new UnsupportedOperationException(); } + @Nullable + @Override + public JobInstance getLastJobInstance(String jobName) { + return null; + } + + @Nullable + @Override + public JobExecution getLastJobExecution(JobInstance jobInstance) { + return null; + } + @Override public List getJobInstances(String jobName, int start, int count) { if (jobInstances == null) { - return new ArrayList(); + return new ArrayList<>(); } List result = jobInstances; jobInstances = null; return result; } + @Nullable @Override - public StepExecution getStepExecution(Long jobExecutionId, Long stepExecutionId) { + public StepExecution getStepExecution(@Nullable Long jobExecutionId, @Nullable Long stepExecutionId) { throw new UnsupportedOperationException(); } @@ -485,6 +543,29 @@ public List getJobNames() { throw new UnsupportedOperationException(); } + @Override + public List findJobInstancesByJobName(String jobName, int start, int count) { + throw new UnsupportedOperationException(); + } + + @Override + public int getJobInstanceCount(@Nullable String jobName) + throws NoSuchJobException { + int count = 0; + + for (JobInstance jobInstance : jobInstances) { + if(jobInstance.getJobName().equals(jobName)) { + count++; + } + } + + if(count == 0) { + throw new NoSuchJobException("Unable to find job instances for " + jobName); + } else { + return count; + } + } + } public static class StubJobParametersConverter implements JobParametersConverter { @@ -494,16 +575,24 @@ public static class StubJobParametersConverter implements JobParametersConverter static boolean called = false; @Override - public JobParameters getJobParameters(Properties properties) { + public JobParameters getJobParameters(@Nullable Properties properties) { called = true; return delegate.getJobParameters(properties); } @Override - public Properties getProperties(JobParameters params) { + public Properties getProperties(@Nullable JobParameters params) { throw new UnsupportedOperationException(); } } + @Configuration + public static class Configuration1 { + @Bean + public String bean1() { + return "bean1"; + } + } + } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/JobRegistryBackgroundJobRunnerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/JobRegistryBackgroundJobRunnerTests.java index 43db9ca37f..70ce0fc2d8 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/JobRegistryBackgroundJobRunnerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/JobRegistryBackgroundJobRunnerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/RunIdIncrementerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/RunIdIncrementerTests.java index 4b2ab2e7f8..9e85f3a24d 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/RunIdIncrementerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/RunIdIncrementerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/ScheduledJobParametersFactoryTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/ScheduledJobParametersFactoryTests.java index 32521bd16c..111bb47da6 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/ScheduledJobParametersFactoryTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/ScheduledJobParametersFactoryTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/SimpleJobOperatorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/SimpleJobOperatorTests.java index 197153ac9d..052ceeb79b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/SimpleJobOperatorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/SimpleJobOperatorTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,15 +15,8 @@ */ package org.springframework.batch.core.launch.support; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - import java.util.Arrays; +import java.util.Collection; import java.util.Collections; import java.util.HashSet; import java.util.List; @@ -33,30 +26,49 @@ import org.junit.Before; import org.junit.Test; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobExecutionException; import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.JobParametersIncrementer; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.configuration.JobRegistry; import org.springframework.batch.core.configuration.support.MapJobRegistry; import org.springframework.batch.core.converter.DefaultJobParametersConverter; import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.job.AbstractJob; import org.springframework.batch.core.job.JobSupport; import org.springframework.batch.core.launch.JobInstanceAlreadyExistsException; -import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.launch.NoSuchJobException; import org.springframework.batch.core.launch.NoSuchJobExecutionException; import org.springframework.batch.core.launch.NoSuchJobInstanceException; import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; -import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; import org.springframework.batch.core.repository.JobRepository; -import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.StoppableTasklet; +import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.batch.repeat.RepeatStatus; import org.springframework.batch.support.PropertiesConverter; +import org.springframework.lang.Nullable; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; /** * @author Dave Syer * @author Will Schipp + * @author Mahmoud Ben Hassine * */ public class SimpleJobOperatorTests { @@ -71,22 +83,14 @@ public class SimpleJobOperatorTests { private JobParameters jobParameters; - /** - * @throws Exception - * - */ @Before public void setUp() throws Exception { job = new JobSupport("foo") { + @Nullable @Override public JobParametersIncrementer getJobParametersIncrementer() { - return new JobParametersIncrementer() { - @Override - public JobParameters getNext(JobParameters parameters) { - return jobParameters; - } - }; + return parameters -> jobParameters; } }; @@ -94,27 +98,20 @@ public JobParameters getNext(JobParameters parameters) { jobOperator.setJobRegistry(new MapJobRegistry() { @Override - public Job getJob(String name) throws NoSuchJobException { + public Job getJob(@Nullable String name) throws NoSuchJobException { if (name.equals("foo")) { return job; } throw new NoSuchJobException("foo"); } - @SuppressWarnings({ "unchecked", "rawtypes" }) @Override public Set getJobNames() { - return new HashSet(Arrays.asList(new String[] { "foo", "bar" })); + return new HashSet<>(Arrays.asList(new String[] { "foo", "bar" })); } }); - jobOperator.setJobLauncher(new JobLauncher() { - @Override - public JobExecution run(Job job, JobParameters jobParameters) throws JobExecutionAlreadyRunningException, - JobRestartException, JobInstanceAlreadyCompleteException { - return new JobExecution(new JobInstance(123L, job.getName()), 999L, jobParameters); - } - }); + jobOperator.setJobLauncher((job, jobParameters) -> new JobExecution(new JobInstance(123L, job.getName()), 999L, jobParameters, null)); jobExplorer = mock(JobExplorer.class); @@ -125,13 +122,13 @@ public JobExecution run(Job job, JobParameters jobParameters) throws JobExecutio jobOperator.setJobParametersConverter(new DefaultJobParametersConverter() { @Override - public JobParameters getJobParameters(Properties props) { + public JobParameters getJobParameters(@Nullable Properties props) { assertTrue("Wrong properties", props.containsKey("a")); return jobParameters; } @Override - public Properties getProperties(JobParameters params) { + public Properties getProperties(@Nullable JobParameters params) { return PropertiesConverter.stringToProperties("a=b"); } }); @@ -156,10 +153,10 @@ public void testMandatoryProperties() throws Exception { * Test method for * {@link org.springframework.batch.core.launch.support.SimpleJobOperator#startNextInstance(java.lang.String)} * . - * @throws Exception */ @Test public void testStartNextInstanceSunnyDay() throws Exception { + jobParameters = new JobParameters(); JobInstance jobInstance = new JobInstance(321L, "foo"); when(jobExplorer.getJobInstances("foo", 0, 1)).thenReturn(Collections.singletonList(jobInstance)); when(jobExplorer.getJobExecutions(jobInstance)).thenReturn(Collections.singletonList(new JobExecution(jobInstance, new JobParameters()))); @@ -192,7 +189,7 @@ public void testStartNewInstanceAlreadyExists() throws Exception { @Test public void testResumeSunnyDay() throws Exception { jobParameters = new JobParameters(); - when(jobExplorer.getJobExecution(111l)).thenReturn(new JobExecution(new JobInstance(123L, job.getName()), 111L, jobParameters)); + when(jobExplorer.getJobExecution(111L)).thenReturn(new JobExecution(new JobInstance(123L, job.getName()), 111L, jobParameters, null)); jobExplorer.getJobExecution(111L); Long value = jobOperator.restart(111L); assertEquals(999, value.longValue()); @@ -201,7 +198,7 @@ public void testResumeSunnyDay() throws Exception { @Test public void testGetSummarySunnyDay() throws Exception { jobParameters = new JobParameters(); - JobExecution jobExecution = new JobExecution(new JobInstance(123L, job.getName()), 111L, jobParameters); + JobExecution jobExecution = new JobExecution(new JobInstance(123L, job.getName()), 111L, jobParameters, null); when(jobExplorer.getJobExecution(111L)).thenReturn(jobExecution); jobExplorer.getJobExecution(111L); String value = jobOperator.getSummary(111L); @@ -224,7 +221,7 @@ public void testGetSummaryNoSuchExecution() throws Exception { public void testGetStepExecutionSummariesSunnyDay() throws Exception { jobParameters = new JobParameters(); - JobExecution jobExecution = new JobExecution(new JobInstance(123L, job.getName()), 111L, jobParameters); + JobExecution jobExecution = new JobExecution(new JobInstance(123L, job.getName()), 111L, jobParameters, null); jobExecution.createStepExecution("step1"); jobExecution.createStepExecution("step2"); jobExecution.getStepExecutions().iterator().next().setId(21L); @@ -248,13 +245,14 @@ public void testGetStepExecutionSummariesNoSuchExecution() throws Exception { @Test public void testFindRunningExecutionsSunnyDay() throws Exception { jobParameters = new JobParameters(); - JobExecution jobExecution = new JobExecution(new JobInstance(123L, job.getName()), 111L, jobParameters); + JobExecution jobExecution = new JobExecution(new JobInstance(123L, job.getName()), 111L, jobParameters, null); when(jobExplorer.findRunningJobExecutions("foo")).thenReturn(Collections.singleton(jobExecution)); Set value = jobOperator.getRunningExecutions("foo"); assertEquals(111L, value.iterator().next().longValue()); } @Test + @SuppressWarnings("unchecked") public void testFindRunningExecutionsNoSuchJob() throws Exception { jobParameters = new JobParameters(); when(jobExplorer.findRunningJobExecutions("no-such-job")).thenReturn(Collections.EMPTY_SET); @@ -269,7 +267,7 @@ public void testFindRunningExecutionsNoSuchJob() throws Exception { @Test public void testGetJobParametersSunnyDay() throws Exception { final JobParameters jobParameters = new JobParameters(); - when(jobExplorer.getJobExecution(111L)).thenReturn(new JobExecution(new JobInstance(123L, job.getName()), 111L, jobParameters)); + when(jobExplorer.getJobExecution(111L)).thenReturn(new JobExecution(new JobInstance(123L, job.getName()), 111L, jobParameters, null)); String value = jobOperator.getParameters(111L); assertEquals("a=b", value); } @@ -319,8 +317,8 @@ public void testGetJobNames() throws Exception { public void testGetExecutionsSunnyDay() throws Exception { JobInstance jobInstance = new JobInstance(123L, job.getName()); when(jobExplorer.getJobInstance(123L)).thenReturn(jobInstance); - - JobExecution jobExecution = new JobExecution(jobInstance, 111L, jobParameters); + + JobExecution jobExecution = new JobExecution(jobInstance, 111L, jobParameters, null); when(jobExplorer.getJobExecutions(jobInstance)).thenReturn(Collections.singletonList(jobExecution)); List value = jobOperator.getExecutions(123L); assertEquals(111L, value.iterator().next().longValue()); @@ -341,18 +339,98 @@ public void testGetExecutionsNoSuchInstance() throws Exception { @Test public void testStop() throws Exception{ JobInstance jobInstance = new JobInstance(123L, job.getName()); - JobExecution jobExecution = new JobExecution(jobInstance, 111L, jobParameters); + JobExecution jobExecution = new JobExecution(jobInstance, 111L, jobParameters, null); + when(jobExplorer.getJobExecution(111L)).thenReturn(jobExecution); + jobExplorer.getJobExecution(111L); + jobRepository.update(jobExecution); + jobOperator.stop(111L); + assertEquals(BatchStatus.STOPPING, jobExecution.getStatus()); + } + + @Test + public void testStopTasklet() throws Exception { + JobInstance jobInstance = new JobInstance(123L, job.getName()); + JobExecution jobExecution = new JobExecution(jobInstance, 111L, jobParameters, null); + StoppableTasklet tasklet = mock(StoppableTasklet.class); + TaskletStep taskletStep = new TaskletStep(); + taskletStep.setTasklet(tasklet); + MockJob job = new MockJob(); + job.taskletStep = taskletStep; + + JobRegistry jobRegistry = mock(JobRegistry.class); + TaskletStep step = mock(TaskletStep.class); + + when(step.getTasklet()).thenReturn(tasklet); + when(step.getName()).thenReturn("test_job.step1"); + when(jobRegistry.getJob(any(String.class))).thenReturn(job); when(jobExplorer.getJobExecution(111L)).thenReturn(jobExecution); + + jobOperator.setJobRegistry(jobRegistry); jobExplorer.getJobExecution(111L); jobRepository.update(jobExecution); jobOperator.stop(111L); assertEquals(BatchStatus.STOPPING, jobExecution.getStatus()); } + + @Test + public void testStopTaskletWhenJobNotRegistered() throws Exception { + JobInstance jobInstance = new JobInstance(123L, job.getName()); + JobExecution jobExecution = new JobExecution(jobInstance, 111L, jobParameters, null); + StoppableTasklet tasklet = mock(StoppableTasklet.class); + JobRegistry jobRegistry = mock(JobRegistry.class); + TaskletStep step = mock(TaskletStep.class); + + when(step.getTasklet()).thenReturn(tasklet); + when(jobRegistry.getJob(job.getName())).thenThrow(new NoSuchJobException("Unable to find job")); + when(jobExplorer.getJobExecution(111L)).thenReturn(jobExecution); + + jobOperator.setJobRegistry(jobRegistry); + jobOperator.stop(111L); + assertEquals(BatchStatus.STOPPING, jobExecution.getStatus()); + verify(tasklet, never()).stop(); + } + + @Test + public void testStopTaskletException() throws Exception { + JobInstance jobInstance = new JobInstance(123L, job.getName()); + JobExecution jobExecution = new JobExecution(jobInstance, 111L, jobParameters, null); + StoppableTasklet tasklet = new StoppableTasklet() { + + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, + ChunkContext chunkContext) throws Exception { + return null; + } + + @Override + public void stop() { + throw new IllegalStateException(); + }}; + TaskletStep taskletStep = new TaskletStep(); + taskletStep.setTasklet(tasklet); + MockJob job = new MockJob(); + job.taskletStep = taskletStep; + + JobRegistry jobRegistry = mock(JobRegistry.class); + TaskletStep step = mock(TaskletStep.class); + + when(step.getTasklet()).thenReturn(tasklet); + when(step.getName()).thenReturn("test_job.step1"); + when(jobRegistry.getJob(any(String.class))).thenReturn(job); + when(jobExplorer.getJobExecution(111L)).thenReturn(jobExecution); + + jobOperator.setJobRegistry(jobRegistry); + jobExplorer.getJobExecution(111L); + jobRepository.update(jobExecution); + jobOperator.stop(111L); + assertEquals(BatchStatus.STOPPING, jobExecution.getStatus()); + } @Test public void testAbort() throws Exception { JobInstance jobInstance = new JobInstance(123L, job.getName()); - JobExecution jobExecution = new JobExecution(jobInstance, 111L, jobParameters); + JobExecution jobExecution = new JobExecution(jobInstance, 111L, jobParameters, null); jobExecution.setStatus(BatchStatus.STOPPING); when(jobExplorer.getJobExecution(123L)).thenReturn(jobExecution); jobRepository.update(jobExecution); @@ -364,10 +442,31 @@ public void testAbort() throws Exception { @Test(expected = JobExecutionAlreadyRunningException.class) public void testAbortNonStopping() throws Exception { JobInstance jobInstance = new JobInstance(123L, job.getName()); - JobExecution jobExecution = new JobExecution(jobInstance, 111L, jobParameters); + JobExecution jobExecution = new JobExecution(jobInstance, 111L, jobParameters, null); jobExecution.setStatus(BatchStatus.STARTED); when(jobExplorer.getJobExecution(123L)).thenReturn(jobExecution); jobRepository.update(jobExecution); jobOperator.abandon(123L); } + + class MockJob extends AbstractJob { + + private TaskletStep taskletStep; + + @Override + public Step getStep(String stepName) { + return taskletStep; + } + + @Override + public Collection getStepNames() { + return Collections.singletonList("test_job.step1"); + } + + @Override + protected void doExecute(JobExecution execution) throws JobExecutionException { + + } + + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/SimpleJvmExitCodeMapperTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/SimpleJvmExitCodeMapperTests.java index 0546cbd985..cbf37cc002 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/SimpleJvmExitCodeMapperTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/SimpleJvmExitCodeMapperTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,12 +31,12 @@ public class SimpleJvmExitCodeMapperTests extends TestCase { @Override protected void setUp() throws Exception { ecm = new SimpleJvmExitCodeMapper(); - Map ecmMap = new HashMap(); + Map ecmMap = new HashMap<>(); ecmMap.put("MY_CUSTOM_CODE", new Integer(3)); ecm.setMapping(ecmMap); ecm2 = new SimpleJvmExitCodeMapper(); - Map ecm2Map = new HashMap(); + Map ecm2Map = new HashMap<>(); ecm2Map.put(ExitStatus.COMPLETED.getExitCode(), new Integer(-1)); ecm2Map.put(ExitStatus.FAILED.getExitCode(), new Integer(-2)); ecm2Map.put(ExitCodeMapper.JOB_NOT_PROVIDED, new Integer(-3)); @@ -49,7 +49,7 @@ protected void tearDown() throws Exception { super.tearDown(); } - public void testGetExitCodeWithpPredefinedCodes() { + public void testGetExitCodeWithPredefinedCodes() { assertEquals( ecm.intValue(ExitStatus.COMPLETED.getExitCode()), ExitCodeMapper.JVM_EXITCODE_COMPLETED); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/StubJobLauncher.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/StubJobLauncher.java index a28fa153f1..5ba5ac2fb0 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/StubJobLauncher.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/StubJobLauncher.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.launch.support; import org.springframework.batch.core.Job; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/TestJobParametersIncrementer.java b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/TestJobParametersIncrementer.java index bde47fef7c..41008f2fae 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/TestJobParametersIncrementer.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/launch/support/TestJobParametersIncrementer.java @@ -1,13 +1,29 @@ +/* + * Copyright 2009-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.launch.support; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.JobParametersBuilder; import org.springframework.batch.core.JobParametersIncrementer; +import org.springframework.lang.Nullable; public class TestJobParametersIncrementer implements JobParametersIncrementer { @Override - public JobParameters getNext(JobParameters parameters) { + public JobParameters getNext(@Nullable JobParameters parameters) { return new JobParametersBuilder().addString("foo", "spam").toJobParameters(); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/AbstractDoubleExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/AbstractDoubleExceptionTests.java index f91ab22996..7da346d3fa 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/AbstractDoubleExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/AbstractDoubleExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeChunkListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeChunkListenerTests.java index 2ff0a27fff..f9c42d73cb 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeChunkListenerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeChunkListenerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeItemProcessListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeItemProcessListenerTests.java index a6469d50b5..f9b58d9e8a 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeItemProcessListenerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeItemProcessListenerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -38,7 +38,7 @@ public class CompositeItemProcessListenerTests { @Before public void setUp() throws Exception { listener = mock(ItemProcessListener.class); - compositeListener = new CompositeItemProcessListener(); + compositeListener = new CompositeItemProcessListener<>(); compositeListener.register(listener); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeItemReadListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeItemReadListenerTests.java index 09b42daf33..13e21a2f8c 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeItemReadListenerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeItemReadListenerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -37,7 +37,7 @@ public class CompositeItemReadListenerTests { @Before public void setUp() throws Exception { listener = mock(ItemReadListener.class); - compositeListener = new CompositeItemReadListener(); + compositeListener = new CompositeItemReadListener<>(); compositeListener.register(listener); } @@ -63,6 +63,7 @@ public void testOnReadError(){ compositeListener.onReadError(ex); } + @SuppressWarnings("serial") @Test public void testSetListeners() throws Exception { compositeListener.setListeners(new ArrayList>() { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeItemWriteListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeItemWriteListenerTests.java index 15980e932f..1fe68e8010 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeItemWriteListenerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeItemWriteListenerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,7 +40,7 @@ public class CompositeItemWriteListenerTests { @Before public void setUp() throws Exception { listener = mock(ItemWriteListener.class); - compositeListener = new CompositeItemWriteListener(); + compositeListener = new CompositeItemWriteListener<>(); compositeListener.register(listener); } @@ -66,6 +66,7 @@ public void testOnWriteError() { compositeListener.onWriteError(ex, item); } + @SuppressWarnings("serial") @Test public void testSetListeners() throws Exception { compositeListener.setListeners(new ArrayList>() { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeJobExecutionListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeJobExecutionListenerTests.java index 2abc4feaa8..30580fed7e 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeJobExecutionListenerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeJobExecutionListenerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,7 +32,7 @@ public class CompositeJobExecutionListenerTests extends TestCase { private CompositeJobExecutionListener listener = new CompositeJobExecutionListener(); - private List list = new ArrayList(); + private List list = new ArrayList<>(); /** * Test method for diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeStepExecutionListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeStepExecutionListenerTests.java index 750e19a1ca..7aad3ea6e3 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeStepExecutionListenerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/CompositeStepExecutionListenerTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,6 +24,7 @@ import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.StepExecutionListener; +import org.springframework.lang.Nullable; /** * @author Dave Syer @@ -33,7 +34,7 @@ public class CompositeStepExecutionListenerTests extends TestCase { private CompositeStepExecutionListener listener = new CompositeStepExecutionListener(); - private List list = new ArrayList(); + private List list = new ArrayList<>(); /** * Test method for @@ -41,9 +42,10 @@ public class CompositeStepExecutionListenerTests extends TestCase { * . */ public void testSetListeners() { - JobExecution jobExecution = new JobExecution(1l); + JobExecution jobExecution = new JobExecution(1L); StepExecution stepExecution = new StepExecution("s1", jobExecution); listener.setListeners(new StepExecutionListener[] { new StepExecutionListenerSupport() { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { assertEquals(ExitStatus.STOPPED, stepExecution.getExitStatus()); @@ -51,6 +53,7 @@ public ExitStatus afterStep(StepExecution stepExecution) { return ExitStatus.FAILED; } }, new StepExecutionListenerSupport() { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { list.add("continue"); @@ -67,9 +70,10 @@ public ExitStatus afterStep(StepExecution stepExecution) { * . */ public void testSetListener() { - JobExecution jobExecution = new JobExecution(1l); + JobExecution jobExecution = new JobExecution(1L); StepExecution stepExecution = new StepExecution("s1", jobExecution); listener.register(new StepExecutionListenerSupport() { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { list.add("fail"); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/ExecutionContextPromotionListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/ExecutionContextPromotionListenerTests.java index cf76e5b1d3..673a32628a 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/ExecutionContextPromotionListenerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/ExecutionContextPromotionListenerTests.java @@ -1,14 +1,30 @@ +/* + * Copyright 2009-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.listener; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; - import org.junit.Test; + import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.StepExecution; import org.springframework.util.Assert; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; + /** * Tests for {@link ExecutionContextPromotionListener}. */ @@ -42,8 +58,8 @@ public void promoteEntryNullStatuses() throws Exception { StepExecution stepExecution = jobExecution.createStepExecution("step1"); stepExecution.setExitStatus(ExitStatus.COMPLETED); - Assert.state(jobExecution.getExecutionContext().isEmpty()); - Assert.state(stepExecution.getExecutionContext().isEmpty()); + Assert.state(jobExecution.getExecutionContext().isEmpty(), "Job ExecutionContext is not empty"); + Assert.state(stepExecution.getExecutionContext().isEmpty(), "Step ExecutionContext is not empty"); stepExecution.getExecutionContext().putString(key, value); stepExecution.getExecutionContext().putString(key2, value2); @@ -72,8 +88,8 @@ public void promoteEntryStatusFound() throws Exception { StepExecution stepExecution = jobExecution.createStepExecution("step1"); stepExecution.setExitStatus(new ExitStatus(status)); - Assert.state(jobExecution.getExecutionContext().isEmpty()); - Assert.state(stepExecution.getExecutionContext().isEmpty()); + Assert.state(jobExecution.getExecutionContext().isEmpty(), "Job ExecutionContext is not empty"); + Assert.state(stepExecution.getExecutionContext().isEmpty(), "Step ExecutionContext is not empty"); stepExecution.getExecutionContext().putString(key, value); stepExecution.getExecutionContext().putString(key2, value2); @@ -102,8 +118,8 @@ public void promoteEntryStatusNotFound() throws Exception { StepExecution stepExecution = jobExecution.createStepExecution("step1"); stepExecution.setExitStatus(new ExitStatus(status2)); - Assert.state(jobExecution.getExecutionContext().isEmpty()); - Assert.state(stepExecution.getExecutionContext().isEmpty()); + Assert.state(jobExecution.getExecutionContext().isEmpty(), "Job ExecutionContext is not empty"); + Assert.state(stepExecution.getExecutionContext().isEmpty(), "Step ExecutionContext is not empty"); stepExecution.getExecutionContext().putString(key, value); stepExecution.getExecutionContext().putString(key2, value2); @@ -132,8 +148,8 @@ public void promoteEntryStatusWildcardFound() throws Exception { StepExecution stepExecution = jobExecution.createStepExecution("step1"); stepExecution.setExitStatus(new ExitStatus(status)); - Assert.state(jobExecution.getExecutionContext().isEmpty()); - Assert.state(stepExecution.getExecutionContext().isEmpty()); + Assert.state(jobExecution.getExecutionContext().isEmpty(), "Job ExecutionContext is not empty"); + Assert.state(stepExecution.getExecutionContext().isEmpty(), "Step ExecutionContext is not empty"); stepExecution.getExecutionContext().putString(key, value); stepExecution.getExecutionContext().putString(key2, value2); @@ -161,8 +177,8 @@ public void promoteEntriesKeyNotFound() throws Exception { StepExecution stepExecution = jobExecution.createStepExecution("step1"); stepExecution.setExitStatus(ExitStatus.COMPLETED); - Assert.state(jobExecution.getExecutionContext().isEmpty()); - Assert.state(stepExecution.getExecutionContext().isEmpty()); + Assert.state(jobExecution.getExecutionContext().isEmpty(), "Job ExecutionContext is not empty"); + Assert.state(stepExecution.getExecutionContext().isEmpty(), "Step ExecutionContext is not empty"); stepExecution.getExecutionContext().putString(key, value); @@ -188,8 +204,8 @@ public void promoteEntriesKeyNotFoundInStep() throws Exception { StepExecution stepExecution = jobExecution.createStepExecution("step1"); stepExecution.setExitStatus(ExitStatus.COMPLETED); - Assert.state(jobExecution.getExecutionContext().isEmpty()); - Assert.state(stepExecution.getExecutionContext().isEmpty()); + Assert.state(jobExecution.getExecutionContext().isEmpty(), "Job ExecutionContext is not empty"); + Assert.state(stepExecution.getExecutionContext().isEmpty(), "Step ExecutionContext is not empty"); jobExecution.getExecutionContext().putString(key, value); @@ -216,8 +232,8 @@ public void promoteEntriesKeyNotFoundStrict() throws Exception { StepExecution stepExecution = jobExecution.createStepExecution("step1"); stepExecution.setExitStatus(ExitStatus.COMPLETED); - Assert.state(jobExecution.getExecutionContext().isEmpty()); - Assert.state(stepExecution.getExecutionContext().isEmpty()); + Assert.state(jobExecution.getExecutionContext().isEmpty(), "Job ExecutionContext is not empty"); + Assert.state(stepExecution.getExecutionContext().isEmpty(), "Step ExecutionContext is not empty"); stepExecution.getExecutionContext().putString(key, value); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/ItemListenerErrorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/ItemListenerErrorTests.java new file mode 100644 index 0000000000..c79ed1c14b --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/ItemListenerErrorTests.java @@ -0,0 +1,307 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.listener; + +import static org.junit.Assert.assertEquals; + +import java.util.Collections; +import java.util.List; + +import org.junit.Before; +import org.junit.Ignore; +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ItemProcessListener; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.launch.support.RunIdIncrementer; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.ListItemReader; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.lang.Nullable; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +/** + * BATCH-2322 + * + * @author Michael Minella + */ +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(classes = {ItemListenerErrorTests.BatchConfiguration.class}) +public class ItemListenerErrorTests { + + @Autowired + private FailingListener listener; + + @Autowired + private FailingItemReader reader; + + @Autowired + private FailingItemProcessor processor; + + @Autowired + private FailingItemWriter writer; + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private Job job; + + @Before + public void setUp() { + listener.setMethodToThrowExceptionFrom(""); + reader.setGoingToFail(false); + processor.setGoingToFail(false); + writer.setGoingToFail(false); + } + + @Test + @DirtiesContext + public void testOnWriteError() throws Exception { + listener.setMethodToThrowExceptionFrom("onWriteError"); + writer.setGoingToFail(true); + + JobExecution execution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + } + + @Ignore + @Test + @DirtiesContext + public void testOnReadError() throws Exception { + listener.setMethodToThrowExceptionFrom("onReadError"); + reader.setGoingToFail(true); + + JobExecution execution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + } + + @Test + @DirtiesContext + public void testOnProcessError() throws Exception { + listener.setMethodToThrowExceptionFrom("onProcessError"); + processor.setGoingToFail(true); + + JobExecution execution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + } + + @Configuration + @EnableBatchProcessing + public static class BatchConfiguration { + + @Bean + public Job testJob(JobBuilderFactory jobs, Step testStep) { + return jobs.get("testJob") + .incrementer(new RunIdIncrementer()) + .start(testStep) + .build(); + } + + @Bean + public Step step1(StepBuilderFactory stepBuilderFactory, + ItemReader fakeItemReader, + ItemProcessor fakeProcessor, + ItemWriter fakeItemWriter, + ItemProcessListener itemProcessListener) { + + return stepBuilderFactory.get("testStep").chunk(10) + .reader(fakeItemReader) + .processor(fakeProcessor) + .writer(fakeItemWriter) + .listener(itemProcessListener) + .faultTolerant().skipLimit(50).skip(RuntimeException.class) + .build(); + } + + @Bean + public FailingListener itemListener() { + return new FailingListener(); + } + + @Bean + public FailingItemReader fakeReader() { + return new FailingItemReader(); + } + + @Bean + public FailingItemProcessor fakeProcessor() { + return new FailingItemProcessor(); + } + + @Bean + public FailingItemWriter fakeItemWriter() { + return new FailingItemWriter(); + } + } + + public static class FailingItemWriter implements ItemWriter { + + private boolean goingToFail = false; + + @Override + public void write(List items) throws Exception { + if(goingToFail) { + throw new RuntimeException("failure in the writer"); + } + else { + for (String item : items) { + System.out.println(item); + } + } + } + + public void setGoingToFail(boolean goingToFail) { + this.goingToFail = goingToFail; + } + } + + public static class FailingItemProcessor implements ItemProcessor { + + private boolean goingToFail = false; + + @Nullable + @Override + public String process(String item) throws Exception { + if(goingToFail) { + throw new RuntimeException("failure in the processor"); + } + else { + return item; + } + } + + public void setGoingToFail(boolean goingToFail) { + this.goingToFail = goingToFail; + } + } + + public static class FailingItemReader implements ItemReader { + + private boolean goingToFail = false; + + private ItemReader delegate = new ListItemReader<>(Collections.singletonList("1")); + + private int count = 0; + + @Nullable + @Override + public String read() throws Exception { + count++; + if(goingToFail) { + throw new RuntimeException("failure in the reader"); + } + else { + return delegate.read(); + } + } + + public void setGoingToFail(boolean goingToFail) { + this.goingToFail = goingToFail; + } + + public int getCount() { + return count; + } + } + + public static class FailingListener extends ItemListenerSupport { + + private String methodToThrowExceptionFrom; + + public void setMethodToThrowExceptionFrom(String methodToThrowExceptionFrom) { + this.methodToThrowExceptionFrom = methodToThrowExceptionFrom; + } + + @Override + public void beforeRead() { + if (methodToThrowExceptionFrom.equals("beforeRead")) { + throw new RuntimeException("beforeRead caused this Exception"); + } + } + + @Override + public void afterRead(String item) { + if (methodToThrowExceptionFrom.equals("afterRead")) { + throw new RuntimeException("afterRead caused this Exception"); + } + } + + @Override + public void onReadError(Exception ex) { + if (methodToThrowExceptionFrom.equals("onReadError")) { + throw new RuntimeException("onReadError caused this Exception"); + } + } + + @Override + public void beforeProcess(String item) { + if (methodToThrowExceptionFrom.equals("beforeProcess")) { + throw new RuntimeException("beforeProcess caused this Exception"); + } + } + + @Override + public void afterProcess(String item, @Nullable String result) { + if (methodToThrowExceptionFrom.equals("afterProcess")) { + throw new RuntimeException("afterProcess caused this Exception"); + } + } + + @Override + public void onProcessError(String item, Exception ex) { + if (methodToThrowExceptionFrom.equals("onProcessError")) { + throw new RuntimeException("onProcessError caused this Exception"); + } + } + + @Override + public void beforeWrite(List items) { + if (methodToThrowExceptionFrom.equals("beforeWrite")) { + throw new RuntimeException("beforeWrite caused this Exception"); + } + } + + @Override + public void afterWrite(List items) { + if (methodToThrowExceptionFrom.equals("afterWrite")) { + throw new RuntimeException("afterWrite caused this Exception"); + } + } + + @Override + public void onWriteError(Exception ex, List item) { + if (methodToThrowExceptionFrom.equals("onWriteError")) { + throw new RuntimeException("onWriteError caused this Exception"); + } + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/JobListenerFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/JobListenerFactoryBeanTests.java index d6323ac62e..e1d4628686 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/JobListenerFactoryBeanTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/JobListenerFactoryBeanTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,8 +15,8 @@ */ package org.springframework.batch.core.listener; -import static junit.framework.Assert.assertTrue; import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; import static org.springframework.batch.core.listener.JobListenerMetaData.AFTER_JOB; import java.util.HashMap; @@ -95,7 +95,7 @@ public void testUseInHashSet() throws Exception { Object listener = JobListenerFactoryBean.getListener(delegate); Object other = JobListenerFactoryBean.getListener(delegate); assertTrue(listener instanceof JobExecutionListener); - Set listeners = new HashSet(); + Set listeners = new HashSet<>(); listeners.add((JobExecutionListener) listener); listeners.add((JobExecutionListener) other); assertTrue(listeners.contains(listener)); @@ -191,7 +191,7 @@ public void aMethod() { } }; factoryBean.setDelegate(delegate); - Map metaDataMap = new HashMap(); + Map metaDataMap = new HashMap<>(); metaDataMap.put(AFTER_JOB.getPropertyName(), "aMethod"); factoryBean.setMetaDataMap(metaDataMap); JobExecutionListener listener = (JobExecutionListener) factoryBean.getObject(); @@ -209,7 +209,7 @@ public void aMethod(JobExecution jobExecution) { } }; factoryBean.setDelegate(delegate); - Map metaDataMap = new HashMap(); + Map metaDataMap = new HashMap<>(); metaDataMap.put(AFTER_JOB.getPropertyName(), "aMethod"); factoryBean.setMetaDataMap(metaDataMap); JobExecutionListener listener = (JobExecutionListener) factoryBean.getObject(); @@ -226,7 +226,7 @@ public void aMethod(Integer item) { } }; factoryBean.setDelegate(delegate); - Map metaDataMap = new HashMap(); + Map metaDataMap = new HashMap<>(); metaDataMap.put(AFTER_JOB.getPropertyName(), "aMethod"); factoryBean.setMetaDataMap(metaDataMap); factoryBean.getObject(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/JobParameterExecutionContextCopyListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/JobParameterExecutionContextCopyListenerTests.java index 5d0c8f41da..84fc480365 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/JobParameterExecutionContextCopyListenerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/JobParameterExecutionContextCopyListenerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/MulticasterBatchListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/MulticasterBatchListenerTests.java index 5bb5fc0fef..2996d17daf 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/MulticasterBatchListenerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/MulticasterBatchListenerTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,6 +16,7 @@ package org.springframework.batch.core.listener; import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; import static org.junit.Assert.fail; import java.util.Arrays; @@ -26,15 +27,26 @@ import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.StepListener; +import org.springframework.batch.core.annotation.AfterChunk; +import org.springframework.batch.core.annotation.AfterProcess; +import org.springframework.batch.core.annotation.AfterRead; +import org.springframework.batch.core.annotation.AfterWrite; +import org.springframework.batch.core.annotation.BeforeChunk; +import org.springframework.batch.core.annotation.BeforeProcess; +import org.springframework.batch.core.annotation.BeforeRead; +import org.springframework.batch.core.annotation.BeforeWrite; import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.lang.Nullable; /** * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public class MulticasterBatchListenerTests { - private MulticasterBatchListener multicast = new MulticasterBatchListener(); + private MulticasterBatchListener multicast = new MulticasterBatchListener<>(); private int count = 0; @@ -47,9 +59,10 @@ public void setUp() { @Test public void testSetListeners() { - JobExecution jobExecution = new JobExecution(1l); + JobExecution jobExecution = new JobExecution(1L); StepExecution stepExecution = new StepExecution("s1", jobExecution); multicast.setListeners(Arrays.asList(new StepListenerSupport() { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { count++; @@ -68,9 +81,10 @@ public ExitStatus afterStep(StepExecution stepExecution) { */ @Test public void testRegister() { - JobExecution jobExecution = new JobExecution(1l); + JobExecution jobExecution = new JobExecution(1L); StepExecution stepExecution = new StepExecution("s1", jobExecution); multicast.register(new StepListenerSupport() { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { count++; @@ -512,6 +526,188 @@ public void onSkipInProcess(Object item, Throwable t) { assertEquals(1, count); } + @Test + public void testBeforeReadFails_withAnnotatedListener() { + StepListener listener = StepListenerFactoryBean.getListener(new AnnotationBasedStepListener()); + multicast.register(listener); + + try { + multicast.beforeRead(); + fail("Expected StepListenerFailedException"); + } catch (StepListenerFailedException e) { + // expected + Throwable cause = e.getCause(); + String message = cause.getMessage(); + assertTrue(cause instanceof IllegalStateException); + assertEquals("Wrong message: " + message, "listener error", message); + } + } + + @Test + public void testAfterReadFails_withAnnotatedListener() { + StepListener listener = StepListenerFactoryBean.getListener(new AnnotationBasedStepListener()); + multicast.register(listener); + + try { + multicast.afterRead(null); + fail("Expected StepListenerFailedException"); + } catch (StepListenerFailedException e) { + // expected + Throwable cause = e.getCause(); + String message = cause.getMessage(); + assertTrue(cause instanceof IllegalStateException); + assertEquals("Wrong message: " + message, "listener error", message); + } + } + + @Test + public void testBeforeProcessFails_withAnnotatedListener() { + StepListener listener = StepListenerFactoryBean.getListener(new AnnotationBasedStepListener()); + multicast.register(listener); + + try { + multicast.beforeProcess(null); + fail("Expected StepListenerFailedException"); + } catch (StepListenerFailedException e) { + // expected + Throwable cause = e.getCause(); + String message = cause.getMessage(); + assertTrue(cause instanceof IllegalStateException); + assertEquals("Wrong message: " + message, "listener error", message); + } + } + + @Test + public void testAfterProcessFails_withAnnotatedListener() { + StepListener listener = StepListenerFactoryBean.getListener(new AnnotationBasedStepListener()); + multicast.register(listener); + + try { + multicast.afterProcess(null, null); + fail("Expected StepListenerFailedException"); + } catch (StepListenerFailedException e) { + // expected + Throwable cause = e.getCause(); + String message = cause.getMessage(); + assertTrue(cause instanceof IllegalStateException); + assertEquals("Wrong message: " + message, "listener error", message); + } + } + + @Test + public void testBeforeWriteFails_withAnnotatedListener() { + StepListener listener = StepListenerFactoryBean.getListener(new AnnotationBasedStepListener()); + multicast.register(listener); + + try { + multicast.beforeWrite(null); + fail("Expected StepListenerFailedException"); + } catch (StepListenerFailedException e) { + // expected + Throwable cause = e.getCause(); + String message = cause.getMessage(); + assertTrue(cause instanceof IllegalStateException); + assertEquals("Wrong message: " + message, "listener error", message); + } + } + + @Test + public void testAfterWriteFails_withAnnotatedListener() { + StepListener listener = StepListenerFactoryBean.getListener(new AnnotationBasedStepListener()); + multicast.register(listener); + + try { + multicast.afterWrite(null); + fail("Expected StepListenerFailedException"); + } catch (StepListenerFailedException e) { + // expected + Throwable cause = e.getCause(); + String message = cause.getMessage(); + assertTrue(cause instanceof IllegalStateException); + assertEquals("Wrong message: " + message, "listener error", message); + } + } + + @Test + public void testBeforeChunkFails_withAnnotatedListener() { + StepListener listener = StepListenerFactoryBean.getListener(new AnnotationBasedStepListener()); + multicast.register(listener); + + try { + multicast.beforeChunk(null); + fail("Expected StepListenerFailedException"); + } catch (StepListenerFailedException e) { + // expected + Throwable cause = e.getCause(); + String message = cause.getMessage(); + assertTrue(cause instanceof IllegalStateException); + assertEquals("Wrong message: " + message, "listener error", message); + } + } + + @Test + public void testAfterChunkFails_withAnnotatedListener() { + StepListener listener = StepListenerFactoryBean.getListener(new AnnotationBasedStepListener()); + multicast.register(listener); + + try { + multicast.afterChunk(null); + fail("Expected StepListenerFailedException"); + } catch (StepListenerFailedException e) { + // expected + Throwable cause = e.getCause(); + String message = cause.getMessage(); + assertTrue(cause instanceof IllegalStateException); + assertEquals("Wrong message: " + message, "listener error", message); + } + } + + private final class AnnotationBasedStepListener { + + private IllegalStateException exception = new IllegalStateException("listener error"); + + @BeforeRead + public void beforeRead() { + throw exception; + } + + @AfterRead + public void afterRead() { + throw exception; + } + + @BeforeProcess + public void beforeProcess() { + throw exception; + } + + @AfterProcess + public void afterProcess() { + throw exception; + } + + @BeforeWrite + public void beforeWrite() { + throw exception; + } + + @AfterWrite + public void afterWrite() { + throw exception; + } + + @BeforeChunk + public void beforeChunk() { + throw exception; + } + + @AfterChunk + public void afterChunk() { + throw exception; + } + + } + /** * @author Dave Syer * @@ -565,6 +761,7 @@ public void afterRead(Integer item) { * org.springframework.batch.core.listener.StepListenerSupport#afterStep * (org.springframework.batch.core.StepExecution) */ + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { count++; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/OrderedCompositeTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/OrderedCompositeTests.java index 99ef203eea..2648b343fb 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/OrderedCompositeTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/OrderedCompositeTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,7 +31,7 @@ */ public class OrderedCompositeTests { - private OrderedComposite list = new OrderedComposite(); + private OrderedComposite list = new OrderedComposite<>(); @Test public void testSetItems() { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/StepListenerFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/StepListenerFactoryBeanTests.java index a29654f8c1..9bd1a3dd7e 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/StepListenerFactoryBeanTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/StepListenerFactoryBeanTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2002-2013 the original author or authors. + * Copyright 2002-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,23 +15,17 @@ */ package org.springframework.batch.core.listener; -import static junit.framework.Assert.assertTrue; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.springframework.batch.core.listener.StepListenerMetaData.AFTER_STEP; -import static org.springframework.batch.core.listener.StepListenerMetaData.AFTER_WRITE; - import java.util.Arrays; import java.util.HashMap; import java.util.List; import java.util.Map; - import javax.sql.DataSource; import org.aopalliance.intercept.MethodInterceptor; import org.aopalliance.intercept.MethodInvocation; import org.junit.Before; import org.junit.Test; + import org.springframework.aop.framework.ProxyFactory; import org.springframework.batch.core.ChunkListener; import org.springframework.batch.core.ExitStatus; @@ -61,8 +55,15 @@ import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.beans.factory.InitializingBean; import org.springframework.core.Ordered; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; +import static org.springframework.batch.core.listener.StepListenerMetaData.AFTER_STEP; +import static org.springframework.batch.core.listener.StepListenerMetaData.AFTER_WRITE; + /** * @author Lucas Ward * @@ -135,7 +136,7 @@ public void testAllThreeTypes() throws Exception { // method name, that all three will be called ThreeStepExecutionListener delegate = new ThreeStepExecutionListener(); factoryBean.setDelegate(delegate); - Map metaDataMap = new HashMap(); + Map metaDataMap = new HashMap<>(); metaDataMap.put(AFTER_STEP.getPropertyName(), "destroy"); factoryBean.setMetaDataMap(metaDataMap); StepListener listener = (StepListener) factoryBean.getObject(); @@ -147,7 +148,7 @@ public void testAllThreeTypes() throws Exception { public void testAnnotatingInterfaceResultsInOneCall() throws Exception { MultipleAfterStep delegate = new MultipleAfterStep(); factoryBean.setDelegate(delegate); - Map metaDataMap = new HashMap(); + Map metaDataMap = new HashMap<>(); metaDataMap.put(AFTER_STEP.getPropertyName(), "afterStep"); factoryBean.setMetaDataMap(metaDataMap); StepListener listener = (StepListener) factoryBean.getObject(); @@ -329,7 +330,7 @@ public void aMethod() { } }; factoryBean.setDelegate(delegate); - Map metaDataMap = new HashMap(); + Map metaDataMap = new HashMap<>(); metaDataMap.put(AFTER_WRITE.getPropertyName(), "aMethod"); factoryBean.setMetaDataMap(metaDataMap); @SuppressWarnings("unchecked") @@ -349,7 +350,7 @@ public void aMethod(List items) { } }; factoryBean.setDelegate(delegate); - Map metaDataMap = new HashMap(); + Map metaDataMap = new HashMap<>(); metaDataMap.put(AFTER_WRITE.getPropertyName(), "aMethod"); factoryBean.setMetaDataMap(metaDataMap); @SuppressWarnings("unchecked") @@ -367,7 +368,7 @@ public void aMethod(Integer item) { } }; factoryBean.setDelegate(delegate); - Map metaDataMap = new HashMap(); + Map metaDataMap = new HashMap<>(); metaDataMap.put(AFTER_WRITE.getPropertyName(), "aMethod"); factoryBean.setMetaDataMap(metaDataMap); factoryBean.getObject(); @@ -377,10 +378,11 @@ private class MultipleAfterStep implements StepExecutionListener { int callcount = 0; + @Nullable @Override @AfterStep public ExitStatus afterStep(StepExecution stepExecution) { - Assert.notNull(stepExecution); + Assert.notNull(stepExecution, "A stepExecution is required"); callcount++; return null; } @@ -397,9 +399,10 @@ private class ThreeStepExecutionListener implements StepExecutionListener { int callcount = 0; + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { - Assert.notNull(stepExecution); + Assert.notNull(stepExecution, "A stepExecution is required"); callcount++; return null; } @@ -489,7 +492,7 @@ public void beforeReadMethod() { @AfterRead public void afterReadMethod(Object item) { - Assert.notNull(item); + Assert.notNull(item, "An item is required"); afterReadCalled = true; } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/StepListenerFailedExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/StepListenerFailedExceptionTests.java index 785fcebd64..3ec79ba5be 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/StepListenerFailedExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/StepListenerFailedExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/StepListenerMethodInterceptorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/StepListenerMethodInterceptorTests.java index 297c9cea04..fbbc36eb3f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/listener/StepListenerMethodInterceptorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/listener/StepListenerMethodInterceptorTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.listener; import static org.junit.Assert.assertEquals; @@ -30,7 +45,7 @@ public void setUp(){ @Test public void testNormalCase() throws Throwable{ - Map> invokerMap = new HashMap>(); + Map> invokerMap = new HashMap<>(); for(Method method : TestClass.class.getMethods()){ invokerMap.put(method.getName(), asSet( new SimpleMethodInvoker(testClass, method))); } @@ -44,7 +59,7 @@ public void testNormalCase() throws Throwable{ @Test public void testMultipleInvokersPerName() throws Throwable{ - Map> invokerMap = new HashMap>(); + Map> invokerMap = new HashMap<>(); Set invokers = asSet(MethodInvokerUtils.getMethodInvokerByName(testClass, "method1", false)); invokers.add(MethodInvokerUtils.getMethodInvokerByName(testClass, "method2", false)); invokerMap.put("method1", invokers); @@ -57,7 +72,7 @@ public void testMultipleInvokersPerName() throws Throwable{ @Test public void testExitStatusReturn() throws Throwable{ - Map> invokerMap = new HashMap>(); + Map> invokerMap = new HashMap<>(); Set invokers = asSet(MethodInvokerUtils.getMethodInvokerByName(testClass, "method3", false)); invokers.add(MethodInvokerUtils.getMethodInvokerByName(testClass, "method3", false)); invokerMap.put("method3", invokers); @@ -66,7 +81,7 @@ public void testExitStatusReturn() throws Throwable{ } public Set asSet(MethodInvoker methodInvoker){ - Set invokerSet = new HashSet(); + Set invokerSet = new HashSet<>(); invokerSet.add(methodInvoker); return invokerSet; } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/ExampleItemReader.java b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/ExampleItemReader.java index 131bcf32c2..a2e071889f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/ExampleItemReader.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/ExampleItemReader.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.partition; import org.apache.commons.logging.Log; @@ -6,6 +21,7 @@ import org.springframework.batch.item.ItemStreamException; import org.springframework.batch.item.ItemStreamReader; import org.springframework.batch.item.support.AbstractItemStreamItemReader; +import org.springframework.lang.Nullable; import org.springframework.util.ClassUtils; /** @@ -46,6 +62,7 @@ public void setMax(int max) { /** * Reads next record from input */ + @Nullable @Override public String read() throws Exception { if (index >= input.length || index >= max) { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/ExampleItemReaderTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/ExampleItemReaderTests.java index e78e4b5a1d..0d2a0e5397 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/ExampleItemReaderTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/ExampleItemReaderTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.partition; import static org.junit.Assert.*; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/ExampleItemWriter.java b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/ExampleItemWriter.java index 9026a8e04b..2f5df55c4c 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/ExampleItemWriter.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/ExampleItemWriter.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.partition; import java.util.ArrayList; @@ -14,7 +29,7 @@ public class ExampleItemWriter implements ItemWriter { private static final Log log = LogFactory.getLog(ExampleItemWriter.class); - private static List items = new ArrayList(); + private static List items = new ArrayList<>(); public static void clear() { items.clear(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/MinMaxPartitioner.java b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/MinMaxPartitioner.java index d52f20508d..4aa8489f12 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/MinMaxPartitioner.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/MinMaxPartitioner.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/RestartIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/RestartIntegrationTests.java index 9e6f3b6197..6f32ae18e3 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/RestartIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/RestartIntegrationTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -74,8 +74,8 @@ public void testLaunchJob() throws Exception { ExampleItemReader.fail = true; JobParameters jobParameters = new JobParametersBuilder().addString("restart", "yes").toJobParameters(); - int beforeMaster = jdbcTemplate.queryForInt("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME='step1:master'"); - int beforePartition = jdbcTemplate.queryForInt("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME like 'step1:partition%'"); + int beforeMaster = jdbcTemplate.queryForObject("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME='step1:master'", Integer.class); + int beforePartition = jdbcTemplate.queryForObject("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME like 'step1:partition%'", Integer.class); ExampleItemWriter.clear(); JobExecution execution = jobLauncher.run(job, jobParameters); @@ -88,8 +88,8 @@ public void testLaunchJob() throws Exception { // Only 4 because the others were processed in the first attempt assertEquals(4, ExampleItemWriter.getItems().size()); - int afterMaster = jdbcTemplate.queryForInt("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME='step1:master'"); - int afterPartition = jdbcTemplate.queryForInt("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME like 'step1:partition%'"); + int afterMaster = jdbcTemplate.queryForObject("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME='step1:master'", Integer.class); + int afterPartition = jdbcTemplate.queryForObject("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME like 'step1:partition%'", Integer.class); // Two attempts assertEquals(2, afterMaster-beforeMaster); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/VanillaIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/VanillaIntegrationTests.java index 1f3619f45f..d502ab797e 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/VanillaIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/VanillaIntegrationTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -58,11 +58,11 @@ public void testSimpleProperties() throws Exception { @Test public void testLaunchJob() throws Exception { - int beforeMaster = jdbcTemplate.queryForInt("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME='step1:master'"); - int beforePartition = jdbcTemplate.queryForInt("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME like 'step1:partition%'"); + int beforeMaster = jdbcTemplate.queryForObject("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME='step1:master'", Integer.class); + int beforePartition = jdbcTemplate.queryForObject("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME like 'step1:partition%'", Integer.class); assertNotNull(jobLauncher.run(job, new JobParameters())); - int afterMaster = jdbcTemplate.queryForInt("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME='step1:master'"); - int afterPartition = jdbcTemplate.queryForInt("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME like 'step1:partition%'"); + int afterMaster = jdbcTemplate.queryForObject("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME='step1:master'", Integer.class); + int afterPartition = jdbcTemplate.queryForObject("SELECT COUNT(*) from BATCH_STEP_EXECUTION where STEP_NAME like 'step1:partition%'", Integer.class); assertEquals(1, afterMaster-beforeMaster); // Should be same as grid size in step splitter assertEquals(2, afterPartition-beforePartition); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/DefaultStepExecutionAggregatorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/DefaultStepExecutionAggregatorTests.java index 3111f7d3b2..8a9e6f6dcc 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/DefaultStepExecutionAggregatorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/DefaultStepExecutionAggregatorTests.java @@ -1,17 +1,32 @@ +/* + * Copyright 2009-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.partition.support; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; - -import java.util.Arrays; -import java.util.Collections; - import org.junit.Test; import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.StepExecution; +import java.util.Arrays; +import java.util.Collections; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + public class DefaultStepExecutionAggregatorTests { private StepExecutionAggregator aggregator = new DefaultStepExecutionAggregator(); @@ -72,10 +87,31 @@ public void testAggregateExitStatusSunnyDay() { } @Test - public void testAggregateCommitCountSunnyDay() { - stepExecution1.setCommitCount(10); - stepExecution2.setCommitCount(5); + public void testAggregateCountsSunnyDay() { + stepExecution1.setCommitCount(1); + stepExecution1.setFilterCount(2); + stepExecution1.setProcessSkipCount(3); + stepExecution1.setReadCount(4); + stepExecution1.setReadSkipCount(5); + stepExecution1.setRollbackCount(6); + stepExecution1.setWriteCount(7); + stepExecution1.setWriteSkipCount(8); + stepExecution2.setCommitCount(11); + stepExecution2.setFilterCount(12); + stepExecution2.setProcessSkipCount(13); + stepExecution2.setReadCount(14); + stepExecution2.setReadSkipCount(15); + stepExecution2.setRollbackCount(16); + stepExecution2.setWriteCount(17); + stepExecution2.setWriteSkipCount(18); aggregator.aggregate(result, Arrays. asList(stepExecution1, stepExecution2)); - assertEquals(15, result.getCommitCount()); + assertEquals(12, result.getCommitCount()); + assertEquals(14, result.getFilterCount()); + assertEquals(16, result.getProcessSkipCount()); + assertEquals(18, result.getReadCount()); + assertEquals(20, result.getReadSkipCount()); + assertEquals(22, result.getRollbackCount()); + assertEquals(24, result.getWriteCount()); + assertEquals(26, result.getWriteSkipCount()); } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/MultiResourcePartitionerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/MultiResourcePartitionerTests.java index 34a323d975..19e99d8caf 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/MultiResourcePartitionerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/MultiResourcePartitionerTests.java @@ -1,18 +1,34 @@ +/* + * Copyright 2009-2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.partition.support; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; - import java.util.Map; import org.junit.Before; import org.junit.Test; + import org.springframework.batch.item.ExecutionContext; import org.springframework.core.io.FileSystemResource; import org.springframework.core.io.Resource; import org.springframework.core.io.UrlResource; import org.springframework.core.io.support.ResourceArrayPropertyEditor; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + public class MultiResourcePartitionerTests { private MultiResourcePartitioner partitioner = new MultiResourcePartitioner(); @@ -20,7 +36,7 @@ public class MultiResourcePartitionerTests { @Before public void setUp() { ResourceArrayPropertyEditor editor = new ResourceArrayPropertyEditor(); - editor.setAsText("classpath:log4j*"); + editor.setAsText("classpath:jsrBaseContext.xml"); partitioner.setResources((Resource[]) editor.getValue()); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/PartitionStepTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/PartitionStepTests.java index 5e2118e15c..2c6883e07a 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/PartitionStepTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/PartitionStepTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,8 +15,6 @@ */ package org.springframework.batch.core.partition.support; -import static org.junit.Assert.assertEquals; - import java.util.Arrays; import java.util.Collection; import java.util.Date; @@ -25,6 +23,7 @@ import org.junit.Before; import org.junit.Test; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.JobExecution; @@ -35,6 +34,8 @@ import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; +import static org.junit.Assert.assertEquals; + /** * @author Dave Syer * @@ -48,7 +49,7 @@ public class PartitionStepTests { @Before public void setUp() throws Exception { MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(); - jobRepository = (JobRepository) factory.getObject(); + jobRepository = factory.getObject(); step.setJobRepository(jobRepository); step.setName("partitioned"); } @@ -73,7 +74,7 @@ public Collection handle(StepExecutionSplitter stepSplitter, Step StepExecution stepExecution = jobExecution.createStepExecution("foo"); jobRepository.add(stepExecution); step.execute(stepExecution); - // one master and two workers + // one manager and two workers assertEquals(3, stepExecution.getJobExecution().getStepExecutions().size()); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); } @@ -98,7 +99,7 @@ public Collection handle(StepExecutionSplitter stepSplitter, Step StepExecution stepExecution = jobExecution.createStepExecution("foo"); jobRepository.add(stepExecution); step.execute(stepExecution); - // one master and two workers + // one manager and two workers assertEquals(3, stepExecution.getJobExecution().getStepExecutions().size()); assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); } @@ -146,7 +147,7 @@ public Collection handle(StepExecutionSplitter stepSplitter, Step stepExecution = jobExecution.createStepExecution("foo"); jobRepository.add(stepExecution); step.execute(stepExecution); - // one master and two workers + // one manager and two workers assertEquals(3, stepExecution.getJobExecution().getStepExecutions().size()); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); } @@ -171,7 +172,7 @@ public Collection handle(StepExecutionSplitter stepSplitter, Step StepExecution stepExecution = jobExecution.createStepExecution("foo"); jobRepository.add(stepExecution); step.execute(stepExecution); - // one master and two workers + // one manager and two workers assertEquals(3, stepExecution.getJobExecution().getStepExecutions().size()); assertEquals(BatchStatus.STOPPED, stepExecution.getStatus()); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/RemoteStepExecutionAggregatorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/RemoteStepExecutionAggregatorTests.java index 321b733945..653b655d8b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/RemoteStepExecutionAggregatorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/RemoteStepExecutionAggregatorTests.java @@ -1,22 +1,36 @@ +/* + * Copyright 2011-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.partition.support; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; - -import java.util.Arrays; -import java.util.Collections; - import org.junit.Before; import org.junit.Test; import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.StepExecution; -import org.springframework.batch.core.explore.JobExplorer; import org.springframework.batch.core.explore.support.MapJobExplorerFactoryBean; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; +import java.util.Arrays; +import java.util.Collections; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + public class RemoteStepExecutionAggregatorTests { private RemoteStepExecutionAggregator aggregator = new RemoteStepExecutionAggregator(); @@ -32,8 +46,8 @@ public class RemoteStepExecutionAggregatorTests { @Before public void init() throws Exception { MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(); - JobRepository jobRepository = (JobRepository) factory.getObject(); - aggregator.setJobExplorer((JobExplorer) new MapJobExplorerFactoryBean(factory).getObject()); + JobRepository jobRepository = factory.getObject(); + aggregator.setJobExplorer(new MapJobExplorerFactoryBean(factory).getObject()); jobExecution = jobRepository.createJobExecution("job", new JobParameters()); result = jobExecution.createStepExecution("aggregate"); stepExecution1 = jobExecution.createStepExecution("foo:1"); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/SimpleStepExecutionSplitterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/SimpleStepExecutionSplitterTests.java index 15d6213269..a39d7e699b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/SimpleStepExecutionSplitterTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/SimpleStepExecutionSplitterTests.java @@ -1,9 +1,20 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.partition.support; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertTrue; - import java.util.Arrays; import java.util.Collection; import java.util.Collections; @@ -13,6 +24,7 @@ import org.junit.Before; import org.junit.Test; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobExecutionException; @@ -25,6 +37,10 @@ import org.springframework.batch.core.step.tasklet.TaskletStep; import org.springframework.batch.item.ExecutionContext; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + public class SimpleStepExecutionSplitterTests { private Step step; @@ -37,7 +53,7 @@ public class SimpleStepExecutionSplitterTests { public void setUp() throws Exception { step = new TaskletStep("step"); MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(); - jobRepository = (JobRepository) factory.getObject(); + jobRepository = factory.getObject(); stepExecution = jobRepository.createJobExecution("job", new JobParameters()).createStepExecution("bar"); jobRepository.add(stepExecution); } @@ -54,6 +70,22 @@ public void testSimpleStepExecutionProviderJobRepositoryStep() throws Exception } } + /** + * Tests the results of BATCH-2490 + * @throws Exception + */ + @Test + public void testAddressabilityOfSetResults() throws Exception { + SimpleStepExecutionSplitter splitter = new SimpleStepExecutionSplitter(jobRepository, true, step.getName(), + new SimplePartitioner()); + Set execs = splitter.split(stepExecution, 2); + assertEquals(2, execs.size()); + + StepExecution execution = execs.iterator().next(); + execs.remove(execution); + assertEquals(1, execs.size()); + } + @Test public void testSimpleStepExecutionProviderJobRepositoryStepPartitioner() throws Exception { final Map map = Collections.singletonMap("foo", new ExecutionContext()); @@ -108,7 +140,7 @@ public void testGetStepName() { } @Test - public void testUnkownStatus() throws Exception { + public void testUnknownStatus() throws Exception { SimpleStepExecutionSplitter provider = new SimpleStepExecutionSplitter(jobRepository, true, step.getName(), new SimplePartitioner()); Set split = provider.split(stepExecution, 2); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/TaskExecutorPartitionHandlerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/TaskExecutorPartitionHandlerTests.java index b6287cdd1e..aa25a7e347 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/TaskExecutorPartitionHandlerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/partition/support/TaskExecutorPartitionHandlerTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.partition.support; import static org.junit.Assert.assertEquals; @@ -28,7 +43,7 @@ public class TaskExecutorPartitionHandlerTests { private int count = 0; - private Collection stepExecutions = new TreeSet(); + private Collection stepExecutions = new TreeSet<>(); private StepExecution stepExecution = new StepExecution("step", new JobExecution(1L)); @@ -41,7 +56,7 @@ public String getStepName() { @Override public Set split(StepExecution stepExecution, int gridSize) throws JobExecutionException { - HashSet result = new HashSet(); + HashSet result = new HashSet<>(); for (int i = gridSize; i-- > 0;) { result.add(stepExecution.getJobExecution().createStepExecution("foo" + i)); } @@ -61,6 +76,20 @@ public void execute(StepExecution stepExecution) throws JobInterruptedException handler.afterPropertiesSet(); } + @Test + public void testConfiguration() throws Exception { + handler = new TaskExecutorPartitionHandler(); + try { + handler.afterPropertiesSet(); + fail("Expected IllegalStateException when no step is set"); + } + catch (IllegalStateException e) { + // expected + String message = e.getMessage(); + assertEquals("Wrong message: " + message, "A Step must be provided.", message); + } + } + @Test public void testNullStep() throws Exception { handler = new TaskExecutorPartitionHandler(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/JobExecutionAlreadyRunningExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/JobExecutionAlreadyRunningExceptionTests.java index ab2d705112..77eceddd82 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/JobExecutionAlreadyRunningExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/JobExecutionAlreadyRunningExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/JobInstanceAlreadyCompleteExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/JobInstanceAlreadyCompleteExceptionTests.java index 3d824c05aa..84fa41cfef 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/JobInstanceAlreadyCompleteExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/JobInstanceAlreadyCompleteExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/JobRestartExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/JobRestartExceptionTests.java index 20d7db314e..54d835be38 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/JobRestartExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/JobRestartExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractExecutionContextDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractExecutionContextDaoTests.java index 5f5ce9ab63..7aae0704d8 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractExecutionContextDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractExecutionContextDaoTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; import static org.junit.Assert.assertEquals; @@ -90,7 +105,7 @@ public void testSaveAndFindJobContext() { @Test public void testSaveAndFindExecutionContexts() { - List stepExecutions = new ArrayList(); + List stepExecutions = new ArrayList<>(); for (int i = 0; i < 3; i++) { JobInstance ji = jobInstanceDao.createJobInstance("testJob" + i, new JobParameters()); JobExecution je = new JobExecution(ji, new JobParameters()); @@ -126,7 +141,7 @@ public void testSaveNullExecutionContexts() { @Transactional @Test public void testSaveEmptyExecutionContexts() { - contextDao.saveExecutionContexts(new ArrayList()); + contextDao.saveExecutionContexts(new ArrayList<>()); } @Transactional diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractExecutionContextSerializerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractExecutionContextSerializerTests.java new file mode 100644 index 0000000000..2462ed4ef1 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractExecutionContextSerializerTests.java @@ -0,0 +1,261 @@ +/* + * Copyright 2012-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.repository.dao; + +import org.junit.Test; +import org.springframework.batch.core.JobParameter; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.repository.ExecutionContextSerializer; + +import java.io.*; +import java.math.BigDecimal; +import java.util.Date; +import java.util.HashMap; +import java.util.Map; + +import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.hasEntry; + +/** + * Abstract test class for {@code ExecutionContextSerializer} implementations. Provides a minimum on test methods + * that should pass for each {@code ExecutionContextSerializer} implementation. + * + * @author Thomas Risberg + * @author Michael Minella + * @author Marten Deinum + * @author Mahmoud Ben Hassine + */ +public abstract class AbstractExecutionContextSerializerTests { + + @Test + public void testSerializeAMap() throws Exception { + Map m1 = new HashMap<>(); + m1.put("object1", Long.valueOf(12345L)); + m1.put("object2", "OBJECT TWO"); + // Use a date after 1971 (otherwise daylight saving screws up)... + m1.put("object3", new Date(123456790123L)); + m1.put("object4", new Double(1234567.1234D)); + + Map m2 = serializationRoundTrip(m1); + + compareContexts(m1, m2); + } + + @Test + public void testSerializeStringJobParameter() throws Exception { + Map m1 = new HashMap<>(); + m1.put("name", new JobParameter("foo")); + + Map m2 = serializationRoundTrip(m1); + + compareContexts(m1, m2); + } + + @Test + public void testSerializeDateJobParameter() throws Exception { + Map m1 = new HashMap<>(); + m1.put("birthDate", new JobParameter(new Date(123456790123L))); + + Map m2 = serializationRoundTrip(m1); + + compareContexts(m1, m2); + } + + @Test + public void testSerializeDoubleJobParameter() throws Exception { + Map m1 = new HashMap<>(); + m1.put("weight", new JobParameter(80.5D)); + + Map m2 = serializationRoundTrip(m1); + + compareContexts(m1, m2); + } + + @Test + public void testSerializeLongJobParameter() throws Exception { + Map m1 = new HashMap<>(); + m1.put("age", new JobParameter(20L)); + + Map m2 = serializationRoundTrip(m1); + + compareContexts(m1, m2); + } + + @Test + public void testSerializeNonIdentifyingJobParameter() throws Exception { + Map m1 = new HashMap<>(); + m1.put("name", new JobParameter("foo", false)); + + Map m2 = serializationRoundTrip(m1); + + compareContexts(m1, m2); + } + + @Test + public void testSerializeJobParameters() throws Exception { + Map jobParametersMap = new HashMap<>(); + jobParametersMap.put("paramName", new JobParameter("paramValue")); + + Map m1 = new HashMap<>(); + m1.put("params", new JobParameters(jobParametersMap)); + + Map m2 = serializationRoundTrip(m1); + + compareContexts(m1, m2); + } + + @Test + public void testSerializeEmptyJobParameters() throws IOException { + Map m1 = new HashMap<>(); + m1.put("params", new JobParameters()); + + Map m2 = serializationRoundTrip(m1); + + compareContexts(m1, m2); + } + + @Test + public void testComplexObject() throws Exception { + Map m1 = new HashMap<>(); + ComplexObject o1 = new ComplexObject(); + o1.setName("02345"); + Map m = new HashMap<>(); + m.put("object1", Long.valueOf(12345L)); + m.put("object2", "OBJECT TWO"); + o1.setMap(m); + o1.setNumber(new BigDecimal("12345.67")); + ComplexObject o2 = new ComplexObject(); + o2.setName("Inner Object"); + o2.setMap(m); + o2.setNumber(new BigDecimal("98765.43")); + o1.setObj(o2); + m1.put("co", o1); + + Map m2 = serializationRoundTrip(m1); + + compareContexts(m1, m2); + } + + @Test (expected=IllegalArgumentException.class) + public void testNullSerialization() throws Exception { + getSerializer().serialize(null, null); + } + + protected void compareContexts(Map m1, Map m2) { + + for (Map.Entry entry : m1.entrySet()) { + assertThat(m2, hasEntry(entry.getKey(), entry.getValue())); + } + } + + protected Map serializationRoundTrip(Map m1) throws IOException { + ByteArrayOutputStream out = new ByteArrayOutputStream(); + getSerializer().serialize(m1, out); + + String s = new String(out.toByteArray(), "ISO-8859-1"); + + InputStream in = new ByteArrayInputStream(s.getBytes("ISO-8859-1")); + Map m2 = getSerializer().deserialize(in); + return m2; + } + + + protected abstract ExecutionContextSerializer getSerializer(); + + public static class ComplexObject implements Serializable { + private static final long serialVersionUID = 1L; + private String name; + private BigDecimal number; + private ComplexObject obj; + private Map map; + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public BigDecimal getNumber() { + return number; + } + + public void setNumber(BigDecimal number) { + this.number = number; + } + + public ComplexObject getObj() { + return obj; + } + + public void setObj(ComplexObject obj) { + this.obj = obj; + } + + public Map getMap() { + return map; + } + + public void setMap(Map map) { + this.map = map; + } + + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + + ComplexObject that = (ComplexObject) o; + + if (map != null ? !map.equals(that.map) : that.map != null) { + return false; + } + if (name != null ? !name.equals(that.name) : that.name != null) { + return false; + } + if (number != null ? !number.equals(that.number) : that.number != null) { + return false; + } + if (obj != null ? !obj.equals(that.obj) : that.obj != null) { + return false; + } + + return true; + } + + @Override + public int hashCode() { + int result; + result = (name != null ? name.hashCode() : 0); + result = 31 * result + (number != null ? number.hashCode() : 0); + result = 31 * result + (obj != null ? obj.hashCode() : 0); + result = 31 * result + (map != null ? map.hashCode() : 0); + return result; + } + + @Override + public String toString() { + return "ComplexObject [name=" + name + ", number=" + number + "]"; + } + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractJobDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractJobDaoTests.java index d4397ee460..18522d0c46 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractJobDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractJobDaoTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -84,7 +84,6 @@ public void setJobExecutionDao(JobExecutionDao jobExecutionDao) { @Before public void onSetUpInTransaction() throws Exception { - // Create job. jobInstance = jobInstanceDao.createJobInstance(jobName, jobParameters); @@ -98,15 +97,15 @@ public void onSetUpInTransaction() throws Exception { @Transactional @Test public void testVersionIsNotNullForJob() throws Exception { - int version = jdbcTemplate.queryForInt("select version from BATCH_JOB_INSTANCE where JOB_INSTANCE_ID=" - + jobInstance.getId()); + int version = jdbcTemplate.queryForObject("select version from BATCH_JOB_INSTANCE where JOB_INSTANCE_ID=" + + jobInstance.getId(), Integer.class); assertEquals(0, version); } @Transactional @Test public void testVersionIsNotNullForJobExecution() throws Exception { - int version = jdbcTemplate.queryForInt("select version from BATCH_JOB_EXECUTION where JOB_EXECUTION_ID=" - + jobExecution.getId()); + int version = jdbcTemplate.queryForObject("select version from BATCH_JOB_EXECUTION where JOB_EXECUTION_ID=" + + jobExecution.getId(), Integer.class); assertEquals(0, version); } @@ -119,7 +118,6 @@ public void testFindNonExistentJob() { @Transactional @Test public void testFindJob() { - JobInstance instance = jobInstanceDao.getJobInstance(jobName, jobParameters); assertNotNull(instance); assertTrue(jobInstance.equals(instance)); @@ -187,7 +185,7 @@ public void testSaveJobExecution() { public void testUpdateInvalidJobExecution() { // id is invalid - JobExecution execution = new JobExecution(jobInstance, (long) 29432, jobParameters); + JobExecution execution = new JobExecution(jobInstance, (long) 29432, jobParameters, null); execution.incrementVersion(); try { jobExecutionDao.updateJobExecution(execution); @@ -199,7 +197,7 @@ public void testUpdateInvalidJobExecution() { } @Transactional @Test - public void testUpdateNullIdJobExection() { + public void testUpdateNullIdJobExecution() { JobExecution execution = new JobExecution(jobInstance, jobParameters); try { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractJobExecutionDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractJobExecutionDaoTests.java index b7d84e212f..61b645458b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractJobExecutionDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractJobExecutionDaoTests.java @@ -1,10 +1,20 @@ +/* + * Copyright 2008-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.fail; - import java.util.ArrayList; import java.util.Collections; import java.util.Date; @@ -13,6 +23,7 @@ import org.junit.Before; import org.junit.Test; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.JobExecution; @@ -21,8 +32,16 @@ import org.springframework.batch.core.StepExecution; import org.springframework.dao.OptimisticLockingFailureException; import org.springframework.transaction.annotation.Transactional; -import org.springframework.util.Assert; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * Parent Test Class for {@link JdbcJobExecutionDao} and {@link MapJobExecutionDao}. + */ public abstract class AbstractJobExecutionDaoTests { protected JobExecutionDao dao; @@ -81,7 +100,7 @@ public void testSaveAndFind() { @Test public void testFindExecutionsOrdering() { - List execs = new ArrayList(); + List execs = new ArrayList<>(); for (int i = 0; i < 10; i++) { JobExecution exec = new JobExecution(jobInstance, jobParameters); @@ -178,14 +197,24 @@ public void testGetMissingLastExecution() { @Transactional @Test public void testFindRunningExecutions() { - + //Normally completed JobExecution as EndTime is populated JobExecution exec = new JobExecution(jobInstance, jobParameters); exec.setCreateTime(new Date(0)); - exec.setEndTime(new Date(1L)); + exec.setStartTime(new Date(1L)); + exec.setEndTime(new Date(2L)); + exec.setLastUpdated(new Date(5L)); + dao.saveJobExecution(exec); + + //BATCH-2675 + //Abnormal JobExecution as both StartTime and EndTime are null + //This can occur when SimpleJobLauncher#run() submission to taskExecutor throws a TaskRejectedException + exec = new JobExecution(jobInstance, jobParameters); exec.setLastUpdated(new Date(5L)); dao.saveJobExecution(exec); + //Running JobExecution as StartTime is populated but EndTime is null exec = new JobExecution(jobInstance, jobParameters); + exec.setStartTime(new Date(2L)); exec.setLastUpdated(new Date(5L)); exec.createStepExecution("step"); dao.saveJobExecution(exec); @@ -292,13 +321,13 @@ public void testSynchronizeStatusUpgrade() { dao.saveJobExecution(exec1); JobExecution exec2 = new JobExecution(jobInstance, jobParameters); - Assert.state(exec1.getId() != null); + assertTrue(exec1.getId() != null); exec2.setId(exec1.getId()); exec2.setStatus(BatchStatus.STARTED); exec2.setVersion(7); - Assert.state(exec1.getVersion() != exec2.getVersion()); - Assert.state(exec1.getStatus() != exec2.getStatus()); + assertTrue(exec1.getVersion() != exec2.getVersion()); + assertTrue(exec1.getStatus() != exec2.getStatus()); dao.synchronizeStatus(exec2); @@ -319,13 +348,13 @@ public void testSynchronizeStatusDowngrade() { dao.saveJobExecution(exec1); JobExecution exec2 = new JobExecution(jobInstance, jobParameters); - Assert.state(exec1.getId() != null); + assertTrue(exec1.getId() != null); exec2.setId(exec1.getId()); exec2.setStatus(BatchStatus.UNKNOWN); exec2.setVersion(7); - Assert.state(exec1.getVersion() != exec2.getVersion()); - Assert.state(exec1.getStatus().isLessThan(exec2.getStatus())); + assertTrue(exec1.getVersion() != exec2.getVersion()); + assertTrue(exec1.getStatus().isLessThan(exec2.getStatus())); dao.synchronizeStatus(exec2); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractJobInstanceDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractJobInstanceDaoTests.java index c48ceff0ca..87b32711b8 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractJobInstanceDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractJobInstanceDaoTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; import static org.junit.Assert.assertEquals; @@ -139,6 +154,33 @@ public void testGetLastInstances() throws Exception { } + @Transactional + @Test + public void testGetLastInstance() throws Exception { + testCreateAndRetrieve(); + + // unrelated job instance that should be ignored by the query + dao.createJobInstance("anotherJob", new JobParameters()); + + // we need two instances of the same job to check ordering + dao.createJobInstance(fooJob, new JobParameters()); + + List jobInstances = dao.getJobInstances(fooJob, 0, 2); + assertEquals(2, jobInstances.size()); + JobInstance lastJobInstance = dao.getLastJobInstance(fooJob); + assertNotNull(lastJobInstance); + assertEquals(fooJob, lastJobInstance.getJobName()); + assertEquals("Last instance should be first on the list", + jobInstances.get(0), lastJobInstance); + } + + @Transactional + @Test + public void testGetLastInstanceWhenNoInstance() { + JobInstance lastJobInstance = dao.getLastJobInstance("NonExistingJob"); + assertNull(lastJobInstance); + } + /** * Create and retrieve a job instance. */ diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractStepExecutionDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractStepExecutionDaoTests.java index 5c4f8389e9..fbcaba3d80 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractStepExecutionDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/AbstractStepExecutionDaoTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,7 +21,9 @@ import static org.junit.Assert.assertNull; import static org.junit.Assert.fail; +import java.time.Instant; import java.util.ArrayList; +import java.util.Arrays; import java.util.Collection; import java.util.Date; import java.util.List; @@ -65,7 +67,7 @@ public abstract class AbstractStepExecutionDaoTests extends AbstractTransactiona protected abstract StepExecutionDao getStepExecutionDao(); /** - * @return {@link JobRepository} that uses the stepExecution dao. + * @return {@link JobRepository} that uses the stepExecution DAO. */ protected abstract JobRepository getJobRepository(); @@ -121,7 +123,7 @@ public void testSaveAndGetExecution() { @Test public void testSaveAndGetExecutions() { - List stepExecutions = new ArrayList(); + List stepExecutions = new ArrayList<>(); for (int i = 0; i < 3; i++) { StepExecution se = new StepExecution("step" + i, jobExecution); se.setStatus(BatchStatus.STARTED); @@ -152,6 +154,38 @@ public void testSaveAndGetExecutions() { } } + @Transactional + @Test + public void testSaveAndGetLastExecution() { + Instant now = Instant.now(); + StepExecution stepExecution1 = new StepExecution("step1", jobExecution); + stepExecution1.setStartTime(Date.from(now)); + StepExecution stepExecution2 = new StepExecution("step1", jobExecution); + stepExecution2.setStartTime(Date.from(now.plusMillis(500))); + + dao.saveStepExecutions(Arrays.asList(stepExecution1, stepExecution2)); + + StepExecution lastStepExecution = dao.getLastStepExecution(jobInstance, "step1"); + assertNotNull(lastStepExecution); + assertEquals(stepExecution2.getId(), lastStepExecution.getId()); + } + + @Transactional + @Test + public void testSaveAndGetLastExecutionWhenSameStartTime() { + Instant now = Instant.now(); + StepExecution stepExecution1 = new StepExecution("step1", jobExecution); + stepExecution1.setStartTime(Date.from(now)); + StepExecution stepExecution2 = new StepExecution("step1", jobExecution); + stepExecution2.setStartTime(Date.from(now)); + + dao.saveStepExecutions(Arrays.asList(stepExecution1, stepExecution2)); + StepExecution lastStepExecution = stepExecution1.getId() > stepExecution2.getId() ? stepExecution1 : stepExecution2; + StepExecution retrieved = dao.getLastStepExecution(jobInstance, "step1"); + assertNotNull(retrieved); + assertEquals(lastStepExecution.getId(), retrieved.getId()); + } + @Transactional @Test(expected = IllegalArgumentException.class) public void testSaveNullCollectionThrowsException() { @@ -161,7 +195,7 @@ public void testSaveNullCollectionThrowsException() { @Transactional @Test public void testSaveEmptyCollection() { - dao.saveStepExecutions(new ArrayList()); + dao.saveStepExecutions(new ArrayList<>()); } @Transactional @@ -188,7 +222,7 @@ public void testSaveAndFindExecution() { @Transactional @Test public void testGetForNotExistingJobExecution() { - assertNull(dao.getStepExecution(new JobExecution(jobInstance, (long) 777, new JobParameters()), 11L)); + assertNull(dao.getStepExecution(new JobExecution(jobInstance, (long) 777, new JobParameters(), null), 11L)); } /** diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/DateFormatTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/DateFormatTests.java index e5b0c1d18f..1348af337d 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/DateFormatTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/DateFormatTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -81,7 +81,7 @@ public void testDateFormat() throws Exception { @Parameters public static List data() { - List params = new ArrayList(); + List params = new ArrayList<>(); String format = "yyyy-MM-dd HH:mm:ss.S z"; /* diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/DefaultExecutionContextSerializerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/DefaultExecutionContextSerializerTests.java index 9973f22dff..2d3214562f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/DefaultExecutionContextSerializerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/DefaultExecutionContextSerializerTests.java @@ -1,28 +1,33 @@ -/** +/* + * Copyright 2012-2016 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ package org.springframework.batch.core.repository.dao; -import static org.junit.Assert.assertEquals; +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.repository.ExecutionContextSerializer; -import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; -import java.io.IOException; -import java.io.InputStream; -import java.io.Serializable; -import java.math.BigDecimal; -import java.util.Date; import java.util.HashMap; import java.util.Map; -import org.junit.Before; -import org.junit.Test; - /** * @author Michael Minella * */ -public class DefaultExecutionContextSerializerTests { +public class DefaultExecutionContextSerializerTests extends AbstractExecutionContextSerializerTests { private DefaultExecutionContextSerializer serializer; @@ -34,135 +39,17 @@ public void setUp() throws Exception { serializer = new DefaultExecutionContextSerializer(); } - @Test - public void testSerializeAMap() throws Exception { - Map m1 = new HashMap(); - m1.put("object1", Long.valueOf(12345L)); - m1.put("object2", "OBJECT TWO"); - // Use a date after 1971 (otherwise daylight saving screws up)... - m1.put("object3", new Date(123456790123L)); - m1.put("object4", new Double(1234567.1234D)); - - Map m2 = serializationRoundTrip(m1); - - compareContexts(m1, m2); - } - - @Test - public void testComplexObject() throws Exception { - Map m1 = new HashMap(); - ComplexObject o1 = new ComplexObject(); - o1.setName("02345"); - Map m = new HashMap(); - m.put("object1", Long.valueOf(12345L)); - m.put("object2", "OBJECT TWO"); - o1.setMap(m); - o1.setNumber(new BigDecimal("12345.67")); - ComplexObject o2 = new ComplexObject(); - o2.setName("Inner Object"); - o2.setMap(m); - o2.setNumber(new BigDecimal("98765.43")); - o1.setObj(o2); - m1.put("co", o1); - - Map m2 = serializationRoundTrip(m1); - - compareContexts(m1, m2); - } - - @Test (expected=IllegalArgumentException.class) - public void testNullSerialization() throws Exception { - serializer.serialize(null, null); - } - - private void compareContexts(Map m1, Map m2) { - for (String key : m1.keySet()) { - assertEquals("Bad key/value for " + key, m1.get(key), m2.get(key)); - } - } - - @SuppressWarnings("unchecked") - private Map serializationRoundTrip(Map m1) throws IOException { - ByteArrayOutputStream out = new ByteArrayOutputStream(); - serializer.serialize(m1, out); - String s = new String(out.toByteArray(), "ISO-8859-1"); + @Test(expected = IllegalArgumentException.class) + public void testSerializeNonSerializable() throws Exception { + Map m1 = new HashMap<>(); + m1.put("object1", new Object()); - InputStream in = new ByteArrayInputStream(s.getBytes("ISO-8859-1")); - Map m2 = (Map) serializer.deserialize(in); - return m2; + serializer.serialize(m1, new ByteArrayOutputStream()); } - @SuppressWarnings("unused") - private static class ComplexObject implements Serializable { - private static final long serialVersionUID = 1L; - private String name; - private BigDecimal number; - private ComplexObject obj; - private Map map; - - public String getName() { - return name; - } - - public void setName(String name) { - this.name = name; - } - - public BigDecimal getNumber() { - return number; - } - - public void setNumber(BigDecimal number) { - this.number = number; - } - - public ComplexObject getObj() { - return obj; - } - - public void setObj(ComplexObject obj) { - this.obj = obj; - } - - public Map getMap() { - return map; - } - - public void setMap(Map map) { - this.map = map; - } - - - @Override - public boolean equals(Object o) { - if (this == o) return true; - if (o == null || getClass() != o.getClass()) return false; - - ComplexObject that = (ComplexObject) o; - - if (map != null ? !map.equals(that.map) : that.map != null) return false; - if (name != null ? !name.equals(that.name) : that.name != null) return false; - if (number != null ? !number.equals(that.number) : that.number != null) return false; - if (obj != null ? !obj.equals(that.obj) : that.obj != null) return false; - - return true; - } - - @Override - public int hashCode() { - int result; - result = (name != null ? name.hashCode() : 0); - result = 31 * result + (number != null ? number.hashCode() : 0); - result = 31 * result + (obj != null ? obj.hashCode() : 0); - result = 31 * result + (map != null ? map.hashCode() : 0); - return result; - } - - @Override - public String toString() { - return "ComplexObject [name=" + name + ", number=" + number + "]"; - } - + @Override + protected ExecutionContextSerializer getSerializer() { + return this.serializer; } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/Jackson2ExecutionContextStringSerializerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/Jackson2ExecutionContextStringSerializerTests.java new file mode 100644 index 0000000000..0987a42035 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/Jackson2ExecutionContextStringSerializerTests.java @@ -0,0 +1,39 @@ +/* + * Copyright 2008-2016 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.repository.dao; + +import org.junit.Before; +import org.springframework.batch.core.repository.ExecutionContextSerializer; + +/** + * @author Marten Deinum + */ +public class Jackson2ExecutionContextStringSerializerTests extends AbstractExecutionContextSerializerTests { + + ExecutionContextSerializer serializer; + + @Before + public void onSetUp() throws Exception { + Jackson2ExecutionContextStringSerializer serializerDeserializer = new Jackson2ExecutionContextStringSerializer(); + + serializer = serializerDeserializer; + } + + @Override + protected ExecutionContextSerializer getSerializer() { + return this.serializer; + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcExecutionContextDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcExecutionContextDaoTests.java index 52d61eac4c..5868ef36d3 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcExecutionContextDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcExecutionContextDaoTests.java @@ -1,31 +1,77 @@ +/* + * Copyright 2008-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; +import org.junit.Assert; +import org.junit.Test; import org.junit.runner.RunWith; + +import org.springframework.jdbc.core.JdbcOperations; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import static org.mockito.Mockito.mock; + @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = {"sql-dao-test.xml"}) public class JdbcExecutionContextDaoTests extends AbstractExecutionContextDaoTests { + @Test + public void testNoSerializer() { + try { + JdbcExecutionContextDao jdbcExecutionContextDao = new JdbcExecutionContextDao(); + jdbcExecutionContextDao.setJdbcTemplate(mock(JdbcOperations.class)); + jdbcExecutionContextDao.afterPropertiesSet(); + } catch (Exception e) { + Assert.assertTrue(e instanceof IllegalStateException); + Assert.assertEquals("ExecutionContextSerializer is required", e.getMessage()); + } + } + + @Test + public void testNullSerializer() { + try { + JdbcExecutionContextDao jdbcExecutionContextDao = new JdbcExecutionContextDao(); + jdbcExecutionContextDao.setJdbcTemplate(mock(JdbcOperations.class)); + jdbcExecutionContextDao.setSerializer(null); + jdbcExecutionContextDao.afterPropertiesSet(); + } catch (Exception e) { + Assert.assertTrue(e instanceof IllegalArgumentException); + Assert.assertEquals("Serializer must not be null", e.getMessage()); + } + } + @Override protected JobInstanceDao getJobInstanceDao() { - return (JobInstanceDao) applicationContext.getBean("jobInstanceDao", JobInstanceDao.class); + return applicationContext.getBean("jobInstanceDao", JobInstanceDao.class); } @Override protected JobExecutionDao getJobExecutionDao() { - return (JobExecutionDao) applicationContext.getBean("jobExecutionDao", JdbcJobExecutionDao.class); + return applicationContext.getBean("jobExecutionDao", JdbcJobExecutionDao.class); } @Override protected StepExecutionDao getStepExecutionDao() { - return (StepExecutionDao) applicationContext.getBean("stepExecutionDao", StepExecutionDao.class); + return applicationContext.getBean("stepExecutionDao", StepExecutionDao.class); } @Override protected ExecutionContextDao getExecutionContextDao() { - return (ExecutionContextDao) applicationContext.getBean("executionContextDao", JdbcExecutionContextDao.class); + return applicationContext.getBean("executionContextDao", JdbcExecutionContextDao.class); } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobDaoQueryTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobDaoQueryTests.java index 1a24e99c88..9c2bdc5d5b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobDaoQueryTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobDaoQueryTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,7 +35,7 @@ public class JdbcJobDaoQueryTests extends TestCase { JdbcJobExecutionDao jobExecutionDao; - List list = new ArrayList(); + List list = new ArrayList<>(); /* * (non-Javadoc) diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobDaoTests.java index 1ad220481e..3bf89ddc0d 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobDaoTests.java @@ -1,16 +1,32 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; -import static org.junit.Assert.*; -import org.junit.runner.RunWith; -import org.junit.Before; -import org.junit.Test; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; import java.util.List; import java.util.Map; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; import org.springframework.batch.core.ExitStatus; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.transaction.annotation.Transactional; @RunWith(SpringJUnit4ClassRunner.class) diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobExecutionDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobExecutionDaoTests.java index 339ba22fc1..a0172bbfa4 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobExecutionDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobExecutionDaoTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; import javax.sql.DataSource; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobInstanceDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobInstanceDaoTests.java index d0ffa096b3..0a235d74de 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobInstanceDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcJobInstanceDaoTests.java @@ -1,9 +1,26 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; import java.math.BigInteger; import java.security.MessageDigest; +import java.util.List; import javax.sql.DataSource; @@ -39,7 +56,7 @@ public void setDataSource(DataSource dataSource) { @Override protected JobInstanceDao getJobInstanceDao() { JdbcTestUtils.deleteFromTables(jdbcTemplate, "BATCH_JOB_EXECUTION_CONTEXT", - "BATCH_STEP_EXECUTION_CONTEXT", "BATCH_STEP_EXECUTION", "BATCH_JOB_EXECUTION_PARAMS", + "BATCH_STEP_EXECUTION_CONTEXT", "BATCH_STEP_EXECUTION", "BATCH_JOB_EXECUTION_PARAMS", "BATCH_JOB_EXECUTION", "BATCH_JOB_INSTANCE"); return jobInstanceDao; } @@ -51,7 +68,7 @@ public void testFindJobInstanceByExecution() { JobParameters jobParameters = new JobParameters(); JobInstance jobInstance = dao.createJobInstance("testInstance", jobParameters); - JobExecution jobExecution = new JobExecution(jobInstance, 2L, jobParameters); + JobExecution jobExecution = new JobExecution(jobInstance, 2L, jobParameters, null); jobExecutionDao.saveJobExecution(jobExecution); JobInstance returnedInstance = dao.getJobInstance(jobExecution); @@ -62,7 +79,7 @@ public void testFindJobInstanceByExecution() { public void testHexing() throws Exception { MessageDigest digest = MessageDigest.getInstance("MD5"); byte[] bytes = digest.digest("f78spx".getBytes("UTF-8")); - StringBuffer output = new StringBuffer(); + StringBuilder output = new StringBuilder(); for (byte bite : bytes) { output.append(String.format("%02x", bite)); } @@ -71,4 +88,20 @@ public void testHexing() throws Exception { assertEquals("Wrong hash: " + value, 32, value.length()); assertEquals(value, output.toString()); } + + @Test + public void testJobInstanceWildcard() { + dao.createJobInstance("anotherJob", new JobParameters()); + dao.createJobInstance("someJob", new JobParameters()); + + List jobInstances = dao.findJobInstancesByName("*Job", 0, 2); + assertEquals(2, jobInstances.size()); + + for (JobInstance instance : jobInstances) { + assertTrue(instance.getJobName().contains("Job")); + } + + jobInstances = dao.getJobInstances("Job*", 0, 2); + assertTrue(jobInstances.isEmpty()); + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcStepExecutionDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcStepExecutionDaoTests.java index 223ebbf0d9..a913852a4b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcStepExecutionDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/JdbcStepExecutionDaoTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; import static org.junit.Assert.assertTrue; @@ -34,7 +49,7 @@ protected JobRepository getJobRepository() { @Test public void testTruncateExitDescription() { - StringBuffer sb = new StringBuffer(); + StringBuilder sb = new StringBuilder(); for (int i = 0; i < 100; i++) { sb.append("too long exit description"); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapExecutionContextDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapExecutionContextDaoTests.java index eba9470a5d..effdd0b2e6 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapExecutionContextDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapExecutionContextDaoTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; import static org.junit.Assert.*; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapJobExecutionDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapJobExecutionDaoTests.java index 21d5df1d22..27abba4772 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapJobExecutionDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapJobExecutionDaoTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; import static org.junit.Assert.assertNull; @@ -63,8 +78,8 @@ public void testConcurrentSaveJobExecution() throws Exception { // Support objects for this testing final CountDownLatch latch = new CountDownLatch(1); - final SortedSet ids = Collections.synchronizedSortedSet(new TreeSet()); // TODO Change to SkipList w/JDK6 - final AtomicReference exception = new AtomicReference(null); + final SortedSet ids = Collections.synchronizedSortedSet(new TreeSet<>()); // TODO Change to SkipList w/JDK6 + final AtomicReference exception = new AtomicReference<>(null); // Implementation of the high-concurrency code final Runnable codeUnderTest = new Runnable() { @@ -98,7 +113,7 @@ public void run() { // Ensure no general exceptions arose if(exception.get() != null) { - throw new RuntimeException("Excepion occurred under high concurrency usage", exception.get()); + throw new RuntimeException("Exception occurred under high concurrency usage", exception.get()); } // Validate the ids: we'd expect one of these three things to fail diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapJobInstanceDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapJobInstanceDaoTests.java index 4314df6ec0..668769e08b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapJobInstanceDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapJobInstanceDaoTests.java @@ -1,7 +1,29 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; +import org.junit.Test; import org.junit.runner.RunWith; import org.junit.runners.JUnit4; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameters; + +import java.util.List; + +import static org.junit.Assert.assertTrue; @RunWith(JUnit4.class) public class MapJobInstanceDaoTests extends AbstractJobInstanceDaoTests { @@ -11,4 +33,30 @@ protected JobInstanceDao getJobInstanceDao() { return new MapJobInstanceDao(); } + @Test + public void testWildcardPrefix() { + MapJobInstanceDao mapJobInstanceDao = new MapJobInstanceDao(); + mapJobInstanceDao.createJobInstance("testJob", new JobParameters()); + mapJobInstanceDao.createJobInstance("Jobtest", new JobParameters()); + List jobInstances = mapJobInstanceDao.findJobInstancesByName("*Job", 0, 2); + assertTrue("Invalid matching job instances found, expected 1, got: " + jobInstances.size(), jobInstances.size() == 1); + } + + @Test + public void testWildcardSuffix() { + MapJobInstanceDao mapJobInstanceDao = new MapJobInstanceDao(); + mapJobInstanceDao.createJobInstance("testJob", new JobParameters()); + mapJobInstanceDao.createJobInstance("Jobtest", new JobParameters()); + List jobInstances = mapJobInstanceDao.findJobInstancesByName("Job*", 0, 2); + assertTrue("No matching job instances found, expected 1, got: " + jobInstances.size(), jobInstances.size() == 1); + } + + @Test + public void testWildcardRange() { + MapJobInstanceDao mapJobInstanceDao = new MapJobInstanceDao(); + mapJobInstanceDao.createJobInstance("testJob", new JobParameters()); + mapJobInstanceDao.createJobInstance("Jobtest", new JobParameters()); + List jobInstances = mapJobInstanceDao.findJobInstancesByName("*Job*", 0, 2); + assertTrue("No matching job instances found, expected 2, got: " + jobInstances.size(), jobInstances.size() == 2); + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapStepExecutionDaoTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapStepExecutionDaoTests.java index 5f2d8422c6..a881bf55e9 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapStepExecutionDaoTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/MapStepExecutionDaoTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/NoSuchBatchDomainObjectExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/NoSuchBatchDomainObjectExceptionTests.java index f42f63d16f..21a0753b97 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/NoSuchBatchDomainObjectExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/NoSuchBatchDomainObjectExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/OptimisticLockingFailureTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/OptimisticLockingFailureTests.java new file mode 100644 index 0000000000..410ac17742 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/OptimisticLockingFailureTests.java @@ -0,0 +1,115 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.core.repository.dao; + +import static org.junit.Assert.assertTrue; + +import java.util.List; + +import org.junit.Test; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.launch.JobOperator; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.context.ApplicationContext; +import org.springframework.context.support.ClassPathXmlApplicationContext; +import org.springframework.lang.Nullable; + +public class OptimisticLockingFailureTests { + @Test + public void testAsyncStopOfStartingJob() throws Exception { + ApplicationContext applicationContext = + new ClassPathXmlApplicationContext("org/springframework/batch/core/repository/dao/OptimisticLockingFailureTests-context.xml"); + Job job = applicationContext.getBean(Job.class); + JobLauncher jobLauncher = applicationContext.getBean(JobLauncher.class); + JobOperator jobOperator = applicationContext.getBean(JobOperator.class); + + JobExecution jobExecution = jobLauncher.run(job, new JobParametersBuilder() + .addLong("test", 1L) + .toJobParameters()); + + Thread.sleep(1000); + + jobOperator.stop(jobExecution.getId()); + + while(jobExecution.isRunning()) { + // wait for async launched job to complete execution + } + + int numStepExecutions = jobExecution.getStepExecutions().size(); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + String stepName = stepExecution.getStepName(); + BatchStatus stepExecutionStatus = stepExecution.getStatus(); + BatchStatus jobExecutionStatus = jobExecution.getStatus(); + + assertTrue("Should only be one StepExecution but got: " + numStepExecutions, numStepExecutions == 1); + assertTrue("Step name for execution should be step1 but got: " + stepName, "step1".equals(stepName)); + assertTrue("Step execution status should be STOPPED but got: " + stepExecutionStatus, stepExecutionStatus.equals(BatchStatus.STOPPED)); + assertTrue("Job execution status should be STOPPED but got:" + jobExecutionStatus, jobExecutionStatus.equals(BatchStatus.STOPPED)); + + JobExecution restartJobExecution = jobLauncher.run(job, new JobParametersBuilder() + .addLong("test", 1L) + .toJobParameters()); + + Thread.sleep(1000); + + while(restartJobExecution.isRunning()) { + // wait for async launched job to complete execution + } + + int restartNumStepExecutions = restartJobExecution.getStepExecutions().size(); + assertTrue("Should be two StepExecution's on restart but got: " + restartNumStepExecutions, restartNumStepExecutions == 2); + + for(StepExecution restartStepExecution : restartJobExecution.getStepExecutions()) { + BatchStatus restartStepExecutionStatus = restartStepExecution.getStatus(); + + assertTrue("Step execution status should be COMPLETED but got: " + restartStepExecutionStatus, + restartStepExecutionStatus.equals(BatchStatus.COMPLETED)); + } + + BatchStatus restartJobExecutionStatus = restartJobExecution.getStatus(); + assertTrue("Job execution status should be COMPLETED but got:" + restartJobExecutionStatus, + restartJobExecutionStatus.equals(BatchStatus.COMPLETED)); + } + + public static class Writer implements ItemWriter { + @Override + public void write(List items) throws Exception { + for(String item : items) { + System.out.println(item); + } + } + } + + public static class SleepingTasklet implements Tasklet { + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + Thread.sleep(2000L); + return RepeatStatus.FINISHED; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/TablePrefixTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/TablePrefixTests.java index 4f9e6c3185..d69fce5996 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/TablePrefixTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/TablePrefixTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,6 +32,7 @@ import org.springframework.batch.repeat.RepeatStatus; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.lang.Nullable; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.test.jdbc.JdbcTestUtils; @@ -62,6 +63,7 @@ public void testJobLaunch() throws Exception { public static class TestTasklet implements Tasklet { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { return RepeatStatus.FINISHED; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/XStreamExecutionContextStringSerializerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/XStreamExecutionContextStringSerializerTests.java index ba50714995..e5cd4a856f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/XStreamExecutionContextStringSerializerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/XStreamExecutionContextStringSerializerTests.java @@ -1,165 +1,42 @@ +/* + * Copyright 2008-2016 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.dao; -import static org.junit.Assert.assertEquals; - -import java.io.ByteArrayInputStream; -import java.io.ByteArrayOutputStream; -import java.io.IOException; -import java.math.BigDecimal; -import java.util.Date; -import java.util.HashMap; -import java.util.Map; - import org.junit.Before; -import org.junit.Test; import org.springframework.batch.core.repository.ExecutionContextSerializer; /** * @author Thomas Risberg * @author Michael Minella */ -public class XStreamExecutionContextStringSerializerTests { +public class XStreamExecutionContextStringSerializerTests extends AbstractExecutionContextSerializerTests { ExecutionContextSerializer serializer; @Before public void onSetUp() throws Exception { + @SuppressWarnings("deprecation") XStreamExecutionContextStringSerializer serializerDeserializer = new XStreamExecutionContextStringSerializer(); (serializerDeserializer).afterPropertiesSet(); serializer = serializerDeserializer; } - @Test - public void testSerializeAMap() throws Exception { - Map m1 = new HashMap(); - m1.put("object1", Long.valueOf(12345L)); - m1.put("object2", "OBJECT TWO"); - // Use a date after 1971 (otherwise daylight saving screws up)... - m1.put("object3", new Date(123456790123L)); - m1.put("object4", new Double(1234567.1234D)); - - Map m2 = serializationRoundTrip(m1); - - compareContexts(m1, m2); - } - - @Test - public void testComplexObject() throws Exception { - Map m1 = new HashMap(); - ComplexObject o1 = new ComplexObject(); - o1.setName("02345"); - Map m = new HashMap(); - m.put("object1", Long.valueOf(12345L)); - m.put("object2", "OBJECT TWO"); - o1.setMap(m); - o1.setNumber(new BigDecimal("12345.67")); - ComplexObject o2 = new ComplexObject(); - o2.setName("Inner Object"); - o2.setMap(m); - o2.setNumber(new BigDecimal("98765.43")); - o1.setObj(o2); - m1.put("co", o1); - - Map m2 = serializationRoundTrip(m1); - - compareContexts(m1, m2); - } - - @Test (expected=IllegalArgumentException.class) - @SuppressWarnings("unchecked") - public void testNullSerialization() throws Exception { - serializer.serialize(null, null); - } - - private void compareContexts(Map m1, Map m2) { - for (String key : m1.keySet()) { - System.out.println("m1 = " + m1 + " m2 = " + m2); - assertEquals("Bad key/value for " + key, m1.get(key), m2.get(key)); - } - } - - @SuppressWarnings("unchecked") - private Map serializationRoundTrip(Map m1) throws IOException { - ByteArrayOutputStream out = new ByteArrayOutputStream(); - serializer.serialize(m1, out); - - String s = out.toString(); - - ByteArrayInputStream in = new ByteArrayInputStream(s.getBytes()); - Map m2 = (Map) serializer.deserialize(in); - return m2; - } - - @SuppressWarnings("unused") - private static class ComplexObject { - private String name; - private BigDecimal number; - private ComplexObject obj; - private Map map; - - public String getName() { - return name; - } - - public void setName(String name) { - this.name = name; - } - - public BigDecimal getNumber() { - return number; - } - - public void setNumber(BigDecimal number) { - this.number = number; - } - - public ComplexObject getObj() { - return obj; - } - - public void setObj(ComplexObject obj) { - this.obj = obj; - } - - public Map getMap() { - return map; - } - - public void setMap(Map map) { - this.map = map; - } - - - @Override - public boolean equals(Object o) { - if (this == o) return true; - if (o == null || getClass() != o.getClass()) return false; - - ComplexObject that = (ComplexObject) o; - - if (map != null ? !map.equals(that.map) : that.map != null) return false; - if (name != null ? !name.equals(that.name) : that.name != null) return false; - if (number != null ? !number.equals(that.number) : that.number != null) return false; - if (obj != null ? !obj.equals(that.obj) : that.obj != null) return false; - - return true; - } - - @Override - public int hashCode() { - int result; - result = (name != null ? name.hashCode() : 0); - result = 31 * result + (number != null ? number.hashCode() : 0); - result = 31 * result + (obj != null ? obj.hashCode() : 0); - result = 31 * result + (map != null ? map.hashCode() : 0); - return result; - } - - @Override - public String toString() { - return "ComplexObject [name=" + name + ", number=" + number + "]"; - } - + @Override + protected ExecutionContextSerializer getSerializer() { + return this.serializer; } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/JobRepositoryFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/JobRepositoryFactoryBeanTests.java index 7285707be6..bdede582b8 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/JobRepositoryFactoryBeanTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/JobRepositoryFactoryBeanTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,37 +15,40 @@ */ package org.springframework.batch.core.repository.support; -import static junit.framework.Assert.assertEquals; -import static junit.framework.Assert.assertTrue; -import static junit.framework.Assert.fail; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - import java.sql.Connection; import java.sql.DatabaseMetaData; +import java.sql.Types; import java.util.Map; - import javax.sql.DataSource; import org.junit.Before; import org.junit.Ignore; import org.junit.Test; + import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.repository.ExecutionContextSerializer; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.dao.DefaultExecutionContextSerializer; -import org.springframework.batch.core.repository.dao.XStreamExecutionContextStringSerializer; +import org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer; import org.springframework.batch.item.database.support.DataFieldMaxValueIncrementerFactory; import org.springframework.core.serializer.Serializer; import org.springframework.dao.DataAccessException; +import org.springframework.jdbc.core.JdbcOperations; +import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jdbc.support.incrementer.DataFieldMaxValueIncrementer; import org.springframework.jdbc.support.lob.DefaultLobHandler; import org.springframework.jdbc.support.lob.LobHandler; -import org.springframework.jdbc.support.lob.OracleLobHandler; import org.springframework.test.util.ReflectionTestUtils; import org.springframework.transaction.PlatformTransactionManager; import org.springframework.transaction.support.DefaultTransactionDefinition; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + /** * @author Lucas Ward * @author Will Schipp @@ -111,7 +114,7 @@ public void testOracleLobHandler() throws Exception { factory.afterPropertiesSet(); LobHandler lobHandler = (LobHandler) ReflectionTestUtils.getField(factory, "lobHandler"); - assertTrue(lobHandler instanceof OracleLobHandler); + assertTrue(lobHandler instanceof DefaultLobHandler); } @@ -150,7 +153,7 @@ public void tesDefaultSerializer() throws Exception { factory.afterPropertiesSet(); Serializer> serializer = (Serializer>) ReflectionTestUtils.getField(factory, "serializer"); - assertTrue(serializer instanceof XStreamExecutionContextStringSerializer); + assertTrue(serializer instanceof Jackson2ExecutionContextStringSerializer); } @Test @@ -171,7 +174,45 @@ public void testCustomSerializer() throws Exception { factory.afterPropertiesSet(); assertEquals(customSerializer, ReflectionTestUtils.getField(factory, "serializer")); } + + @Test + public void testDefaultJdbcOperations() throws Exception { + factory.setDatabaseType("ORACLE"); + + incrementerFactory = mock(DataFieldMaxValueIncrementerFactory.class); + when(incrementerFactory.isSupportedIncrementerType("ORACLE")).thenReturn(true); + when(incrementerFactory.getIncrementer("ORACLE", tablePrefix + "JOB_SEQ")).thenReturn(new StubIncrementer()); + when(incrementerFactory.getIncrementer("ORACLE", tablePrefix + "JOB_EXECUTION_SEQ")).thenReturn(new StubIncrementer()); + when(incrementerFactory.getIncrementer("ORACLE", tablePrefix + "STEP_EXECUTION_SEQ")).thenReturn(new StubIncrementer()); + factory.setIncrementerFactory(incrementerFactory); + + factory.afterPropertiesSet(); + + JdbcOperations jdbcOperations = (JdbcOperations) ReflectionTestUtils.getField(factory, "jdbcOperations"); + assertTrue(jdbcOperations instanceof JdbcTemplate); + } + + @Test + public void testCustomJdbcOperations() throws Exception { + + factory.setDatabaseType("ORACLE"); + + incrementerFactory = mock(DataFieldMaxValueIncrementerFactory.class); + when(incrementerFactory.isSupportedIncrementerType("ORACLE")).thenReturn(true); + when(incrementerFactory.getIncrementer("ORACLE", tablePrefix + "JOB_SEQ")).thenReturn(new StubIncrementer()); + when(incrementerFactory.getIncrementer("ORACLE", tablePrefix + "JOB_EXECUTION_SEQ")).thenReturn(new StubIncrementer()); + when(incrementerFactory.getIncrementer("ORACLE", tablePrefix + "STEP_EXECUTION_SEQ")).thenReturn(new StubIncrementer()); + factory.setIncrementerFactory(incrementerFactory); + + JdbcOperations customJdbcOperations = mock(JdbcOperations.class); + factory.setJdbcOperations(customJdbcOperations); + + factory.afterPropertiesSet(); + + assertEquals(customJdbcOperations, ReflectionTestUtils.getField(factory, "jdbcOperations")); + } + @Test public void testMissingDataSource() throws Exception { @@ -183,7 +224,7 @@ public void testMissingDataSource() throws Exception { catch (IllegalArgumentException ex) { // expected String message = ex.getMessage(); - assertTrue("Wrong message: " + message, message.indexOf("DataSource") >= 0); + assertTrue("Wrong message: " + message, message.contains("DataSource")); } } @@ -203,7 +244,7 @@ public void testMissingTransactionManager() throws Exception { catch (IllegalArgumentException ex) { // expected String message = ex.getMessage(); - assertTrue("Wrong message: " + message, message.indexOf("TransactionManager") >= 0); + assertTrue("Wrong message: " + message, message.contains("TransactionManager")); } } @@ -221,7 +262,7 @@ public void testInvalidDatabaseType() throws Exception { catch (IllegalArgumentException ex) { // expected String message = ex.getMessage(); - assertTrue("Wrong message: " + message, message.indexOf("foo") >= 0); + assertTrue("Wrong message: " + message, message.contains("foo")); } } @@ -241,11 +282,11 @@ public void testCreateRepository() throws Exception { factory.getObject(); } - @Ignore //TODO - fix this test + @Ignore @Test public void testTransactionAttributesForCreateMethodNullHypothesis() throws Exception { testCreateRepository(); - JobRepository repository = (JobRepository) factory.getObject(); + JobRepository repository = factory.getObject(); DefaultTransactionDefinition transactionDefinition = new DefaultTransactionDefinition( DefaultTransactionDefinition.PROPAGATION_REQUIRES_NEW); when(transactionManager.getTransaction(transactionDefinition)).thenReturn(null); @@ -267,7 +308,7 @@ public void testTransactionAttributesForCreateMethodNullHypothesis() throws Exce public void testTransactionAttributesForCreateMethod() throws Exception { testCreateRepository(); - JobRepository repository = (JobRepository) factory.getObject(); + JobRepository repository = factory.getObject(); DefaultTransactionDefinition transactionDefinition = new DefaultTransactionDefinition( DefaultTransactionDefinition.PROPAGATION_REQUIRES_NEW); transactionDefinition.setIsolationLevel(DefaultTransactionDefinition.ISOLATION_SERIALIZABLE); @@ -292,7 +333,7 @@ public void testSetTransactionAttributesForCreateMethod() throws Exception { factory.setIsolationLevelForCreate("ISOLATION_READ_UNCOMMITTED"); testCreateRepository(); - JobRepository repository = (JobRepository) factory.getObject(); + JobRepository repository = factory.getObject(); DefaultTransactionDefinition transactionDefinition = new DefaultTransactionDefinition( DefaultTransactionDefinition.PROPAGATION_REQUIRES_NEW); transactionDefinition.setIsolationLevel(DefaultTransactionDefinition.ISOLATION_READ_UNCOMMITTED); @@ -309,7 +350,20 @@ public void testSetTransactionAttributesForCreateMethod() throws Exception { // expected exception from DataSourceUtils assertEquals("No Statement specified", e.getMessage()); } + } + @Test(expected=IllegalArgumentException.class) + public void testInvalidCustomLobType() throws Exception { + factory.setClobType(Integer.MAX_VALUE); + testCreateRepository(); + } + + @Test + public void testCustomLobType() throws Exception { + factory.setClobType(Types.ARRAY); + testCreateRepository(); + JobRepository repository = factory.getObject(); + assertNotNull(repository); } private static class StubIncrementer implements DataFieldMaxValueIncrementer { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/MapJobRepositoryFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/MapJobRepositoryFactoryBeanTests.java index 5e04a83036..6eff4b12aa 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/MapJobRepositoryFactoryBeanTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/MapJobRepositoryFactoryBeanTests.java @@ -1,14 +1,32 @@ +/* + * Copyright 2008-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.support; -import static org.junit.Assert.fail; - import org.junit.Test; import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.job.JobSupport; import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; import org.springframework.batch.core.repository.JobRepository; +import java.util.Date; + +import static org.junit.Assert.fail; + /** * Tests for {@link MapJobRepositoryFactoryBean}. */ @@ -23,11 +41,15 @@ public class MapJobRepositoryFactoryBeanTests { @Test public void testCreateRepository() throws Exception { tested.afterPropertiesSet(); - JobRepository repository = (JobRepository) tested.getObject(); + JobRepository repository = tested.getObject(); Job job = new JobSupport("jobName"); JobParameters jobParameters = new JobParameters(); - repository.createJobExecution(job.getName(), jobParameters); + JobExecution jobExecution = repository.createJobExecution(job.getName(), jobParameters); + + // simulate a running execution + jobExecution.setStartTime(new Date()); + repository.update(jobExecution); try { repository.createJobExecution(job.getName(), jobParameters); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/SimpleJobRepositoryIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/SimpleJobRepositoryIntegrationTests.java index 284a05eeb1..dc5539e1fd 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/SimpleJobRepositoryIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/SimpleJobRepositoryIntegrationTests.java @@ -1,11 +1,20 @@ +/* + * Copyright 2008-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.support; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.fail; - -import java.util.Date; - import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.batch.core.BatchStatus; @@ -23,10 +32,18 @@ import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.transaction.annotation.Transactional; +import java.util.Arrays; +import java.util.Date; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.fail; + /** * Repository tests using JDBC DAOs (rather than mocks). - * + * * @author Robert Kasanicky + * @author Dimitrios Liapis */ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "/org/springframework/batch/core/repository/dao/sql-dao-test.xml") @@ -131,6 +148,7 @@ public void testGetStepExecutionCountAndLastStepExecution() throws Exception { @Transactional @Test public void testSaveExecutionContext() throws Exception { + @SuppressWarnings("serial") ExecutionContext ctx = new ExecutionContext() { { putLong("crashedPosition", 7); @@ -163,7 +181,11 @@ public void testSaveExecutionContext() throws Exception { @Test public void testOnlyOneJobExecutionAllowedRunning() throws Exception { job.setRestartable(true); - jobRepository.createJobExecution(job.getName(), jobParameters); + JobExecution jobExecution = jobRepository.createJobExecution(job.getName(), jobParameters); + + //simulating a running job execution + jobExecution.setStartTime(new Date()); + jobRepository.update(jobExecution); try { jobRepository.createJobExecution(job.getName(), jobParameters); @@ -183,7 +205,11 @@ public void testGetLastJobExecution() throws Exception { jobRepository.update(jobExecution); Thread.sleep(10); jobExecution = jobRepository.createJobExecution(job.getName(), jobParameters); + StepExecution stepExecution = new StepExecution("step1", jobExecution); + jobRepository.add(stepExecution); + jobExecution.addStepExecutions(Arrays.asList(stepExecution)); assertEquals(jobExecution, jobRepository.getLastJobExecution(job.getName(), jobParameters)); + assertEquals(stepExecution, jobExecution.getStepExecutions().iterator().next()); } -} \ No newline at end of file +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/SimpleJobRepositoryProxyTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/SimpleJobRepositoryProxyTests.java index 2e4f9b4c85..b7c4d78d8d 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/SimpleJobRepositoryProxyTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/SimpleJobRepositoryProxyTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.repository.support; import static org.junit.Assert.assertFalse; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/SimpleJobRepositoryTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/SimpleJobRepositoryTests.java index d7ff292b62..173f63183a 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/SimpleJobRepositoryTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/SimpleJobRepositoryTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,6 +25,8 @@ import static org.mockito.Mockito.when; import java.util.ArrayList; +import java.util.Arrays; +import java.util.Date; import java.util.List; import org.junit.Before; @@ -37,6 +39,9 @@ import org.springframework.batch.core.Step; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.job.JobSupport; +import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; +import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; +import org.springframework.batch.core.repository.JobRestartException; import org.springframework.batch.core.repository.dao.ExecutionContextDao; import org.springframework.batch.core.repository.dao.JobExecutionDao; import org.springframework.batch.core.repository.dao.JobInstanceDao; @@ -50,7 +55,8 @@ * * @author Lucas Ward * @author Will Schipp - * + * @author Dimitrios Liapis + * */ public class SimpleJobRepositoryTests { @@ -92,7 +98,7 @@ public void setUp() throws Exception { jobRepository = new SimpleJobRepository(jobInstanceDao, jobExecutionDao, stepExecutionDao, ecDao); - jobParameters = new JobParametersBuilder().toJobParameters(); + jobParameters = new JobParametersBuilder().addString("bar", "test").toJobParameters(); job = new JobSupport(); job.setBeanName("RepositoryTest"); @@ -102,7 +108,7 @@ public void setUp() throws Exception { stepConfiguration2 = new StepSupport("TestStep2"); - List stepConfigurations = new ArrayList(); + List stepConfigurations = new ArrayList<>(); stepConfigurations.add(stepConfiguration1); stepConfigurations.add(stepConfiguration2); @@ -113,11 +119,11 @@ public void setUp() throws Exception { databaseStep1 = "dbStep1"; databaseStep2 = "dbStep2"; - steps = new ArrayList(); + steps = new ArrayList<>(); steps.add(databaseStep1); steps.add(databaseStep2); - jobExecution = new JobExecution(new JobInstance(1L, job.getName()), 1L, jobParameters); + jobExecution = new JobExecution(new JobInstance(1L, job.getName()), 1L, jobParameters, null); } @Test @@ -137,8 +143,8 @@ public void testSaveOrUpdateInvalidJobExecution() { @Test public void testUpdateValidJobExecution() throws Exception { - JobExecution jobExecution = new JobExecution(new JobInstance(1L, job.getName()), 1L, jobParameters); - // new execution - call update on job dao + JobExecution jobExecution = new JobExecution(new JobInstance(1L, job.getName()), 1L, jobParameters, null); + // new execution - call update on job DAO jobExecutionDao.updateJobExecution(jobExecution); jobRepository.update(jobExecution); assertNotNull(jobExecution.getLastUpdated()); @@ -176,7 +182,7 @@ public void testSaveStepExecutionSetsLastUpdated(){ @Test public void testSaveStepExecutions() { - List stepExecutions = new ArrayList(); + List stepExecutions = new ArrayList<>(); for (int i = 0; i < 3; i++) { StepExecution stepExecution = new StepExecution("stepName" + i, jobExecution); stepExecutions.add(stepExecution); @@ -232,4 +238,38 @@ public void testIsJobInstanceTrue() throws Exception { assertTrue(jobRepository.isJobInstanceExists("foo", new JobParameters())); } + @Test(expected = JobExecutionAlreadyRunningException.class) + public void testCreateJobExecutionAlreadyRunning() throws Exception { + jobExecution.setStatus(BatchStatus.STARTED); + jobExecution.setStartTime(new Date()); + jobExecution.setEndTime(null); + + when(jobInstanceDao.getJobInstance("foo", new JobParameters())).thenReturn(jobInstance); + when(jobExecutionDao.findJobExecutions(jobInstance)).thenReturn(Arrays.asList(jobExecution)); + + jobRepository.createJobExecution("foo", new JobParameters()); + } + + @Test(expected = JobRestartException.class) + public void testCreateJobExecutionStatusUnknown() throws Exception { + jobExecution.setStatus(BatchStatus.UNKNOWN); + jobExecution.setEndTime(new Date()); + + when(jobInstanceDao.getJobInstance("foo", new JobParameters())).thenReturn(jobInstance); + when(jobExecutionDao.findJobExecutions(jobInstance)).thenReturn(Arrays.asList(jobExecution)); + + jobRepository.createJobExecution("foo", new JobParameters()); + } + + @Test(expected = JobInstanceAlreadyCompleteException.class) + public void testCreateJobExecutionAlreadyComplete() throws Exception { + jobExecution.setStatus(BatchStatus.COMPLETED); + jobExecution.setEndTime(new Date()); + + when(jobInstanceDao.getJobInstance("foo", new JobParameters())).thenReturn(jobInstance); + when(jobExecutionDao.findJobExecutions(jobInstance)).thenReturn(Arrays.asList(jobExecution)); + + jobRepository.createJobExecution("foo", new JobParameters()); + } + } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/resource/Foo.java b/spring-batch-core/src/test/java/org/springframework/batch/core/resource/Foo.java index 6f6efec711..17ced1aeac 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/resource/Foo.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/resource/Foo.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.resource; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/resource/FooRowMapper.java b/spring-batch-core/src/test/java/org/springframework/batch/core/resource/FooRowMapper.java index 983f221351..85e309f041 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/resource/FooRowMapper.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/resource/FooRowMapper.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.resource; import java.sql.ResultSet; @@ -6,11 +21,10 @@ import org.springframework.jdbc.core.RowMapper; -@SuppressWarnings("rawtypes") -public class FooRowMapper implements RowMapper { +public class FooRowMapper implements RowMapper { @Override - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { + public Foo mapRow(ResultSet rs, int rowNum) throws SQLException { Foo foo = new Foo(); foo.setId(rs.getInt(1)); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/resource/JdbcCursorItemReaderPreparedStatementIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/resource/JdbcCursorItemReaderPreparedStatementIntegrationTests.java index 3dd4507dbf..ae26762b71 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/resource/JdbcCursorItemReaderPreparedStatementIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/resource/JdbcCursorItemReaderPreparedStatementIntegrationTests.java @@ -1,23 +1,39 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.resource; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNull; - import java.util.ArrayList; import java.util.List; - import javax.sql.DataSource; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.database.JdbcCursorItemReader; import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.jdbc.core.ArgumentPreparedStatementSetter; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.transaction.annotation.Transactional; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; + @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "/org/springframework/batch/core/repository/dao/data-source-context.xml") public class JdbcCursorItemReaderPreparedStatementIntegrationTests { @@ -34,7 +50,7 @@ public void setDataSource(DataSource dataSource) { @Before public void onSetUpInTransaction() throws Exception { - itemReader = new JdbcCursorItemReader(); + itemReader = new JdbcCursorItemReader<>(); itemReader.setDataSource(dataSource); itemReader.setSql("select ID, NAME, VALUE from T_FOOS where ID > ? and ID < ?"); itemReader.setIgnoreWarnings(true); @@ -45,12 +61,11 @@ public void onSetUpInTransaction() throws Exception { itemReader.setMaxRows(100); itemReader.setQueryTimeout(1000); itemReader.setSaveState(true); - ListPreparedStatementSetter pss = new ListPreparedStatementSetter(); - List parameters = new ArrayList(); + List parameters = new ArrayList<>(); parameters.add(1L); parameters.add(4L); - pss.setParameters(parameters); - + ArgumentPreparedStatementSetter pss = new ArgumentPreparedStatementSetter(parameters.toArray()); + itemReader.setPreparedStatementSetter(pss); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/resource/ListPreparedStatementSetterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/resource/ListPreparedStatementSetterTests.java index cbca871e8f..913a441cf1 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/resource/ListPreparedStatementSetterTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/resource/ListPreparedStatementSetterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,6 +34,7 @@ import org.springframework.batch.core.job.AbstractJob; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.database.support.ListPreparedStatementSetter; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jdbc.core.RowCallbackHandler; @@ -74,18 +75,17 @@ public void setDataSource(DataSource dataSource) { @Before public void onSetUpInTransaction() throws Exception { - pss = new ListPreparedStatementSetter(); - List parameters = new ArrayList(); + List parameters = new ArrayList<>(); parameters.add(1L); parameters.add(4L); - pss.setParameters(parameters); + pss = new ListPreparedStatementSetter(parameters); } @Transactional @Test public void testSetValues() { - final List results = new ArrayList(); + final List results = new ArrayList<>(); jdbcTemplate.query("SELECT NAME from T_FOOS where ID > ? and ID < ?", pss, new RowCallbackHandler() { @Override @@ -102,7 +102,7 @@ public void processRow(ResultSet rs) throws SQLException { @Transactional @Test(expected = IllegalArgumentException.class) public void testAfterPropertiesSet() throws Exception { - pss.setParameters(null); + pss = new ListPreparedStatementSetter(null); pss.afterPropertiesSet(); } @@ -132,7 +132,7 @@ public void testXmlConfiguration() throws Exception { } public static class FooStoringItemWriter implements ItemWriter { - private List foos = new ArrayList(); + private List foos = new ArrayList<>(); @Override public void write(List items) throws Exception { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/resource/StepExecutionSimpleCompletionPolicyTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/resource/StepExecutionSimpleCompletionPolicyTests.java index 488786ba82..13199b75bc 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/resource/StepExecutionSimpleCompletionPolicyTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/resource/StepExecutionSimpleCompletionPolicyTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/AsyncJobScopeIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/AsyncJobScopeIntegrationTests.java new file mode 100644 index 0000000000..ad344a5054 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/AsyncJobScopeIntegrationTests.java @@ -0,0 +1,166 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import static org.junit.Assert.assertEquals; + +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.Callable; +import java.util.concurrent.FutureTask; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.scope.context.JobContext; +import org.springframework.batch.core.scope.context.JobSynchronizationManager; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.beans.factory.BeanFactoryAware; +import org.springframework.beans.factory.ListableBeanFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.core.task.TaskExecutor; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class AsyncJobScopeIntegrationTests implements BeanFactoryAware { + + private Log logger = LogFactory.getLog(getClass()); + + @Autowired + @Qualifier("simple") + private Collaborator simple; + + private TaskExecutor taskExecutor = new SimpleAsyncTaskExecutor(); + + private ListableBeanFactory beanFactory; + + private int beanCount; + + @Override + public void setBeanFactory(BeanFactory beanFactory) throws BeansException { + this.beanFactory = (ListableBeanFactory) beanFactory; + } + + @Before + public void countBeans() { + JobSynchronizationManager.release(); + beanCount = beanFactory.getBeanDefinitionCount(); + } + + @After + public void cleanUp() { + JobSynchronizationManager.close(); + // Check that all temporary bean definitions are cleaned up + assertEquals(beanCount, beanFactory.getBeanDefinitionCount()); + } + + @Test + public void testSimpleProperty() throws Exception { + JobExecution jobExecution = new JobExecution(11L); + ExecutionContext executionContext = jobExecution.getExecutionContext(); + executionContext.put("foo", "bar"); + JobSynchronizationManager.register(jobExecution); + assertEquals("bar", simple.getName()); + } + + @Test + public void testGetMultipleInMultipleThreads() throws Exception { + + List> tasks = new ArrayList<>(); + + for (int i = 0; i < 12; i++) { + final String value = "foo" + i; + final Long id = 123L + i; + FutureTask task = new FutureTask<>(new Callable() { + @Override + public String call() throws Exception { + JobExecution jobExecution = new JobExecution(id); + ExecutionContext executionContext = jobExecution.getExecutionContext(); + executionContext.put("foo", value); + JobContext context = JobSynchronizationManager.register(jobExecution); + logger.debug("Registered: " + context.getJobExecutionContext()); + try { + return simple.getName(); + } + finally { + JobSynchronizationManager.close(); + } + } + }); + tasks.add(task); + taskExecutor.execute(task); + } + + int i = 0; + for (FutureTask task : tasks) { + assertEquals("foo" + i, task.get()); + i++; + } + + } + + @Test + public void testGetSameInMultipleThreads() throws Exception { + + List> tasks = new ArrayList<>(); + final JobExecution jobExecution = new JobExecution(11L); + ExecutionContext executionContext = jobExecution.getExecutionContext(); + executionContext.put("foo", "foo"); + JobSynchronizationManager.register(jobExecution); + assertEquals("foo", simple.getName()); + + for (int i = 0; i < 12; i++) { + final String value = "foo" + i; + FutureTask task = new FutureTask<>(new Callable() { + @Override + public String call() throws Exception { + ExecutionContext executionContext = jobExecution.getExecutionContext(); + executionContext.put("foo", value); + JobContext context = JobSynchronizationManager.register(jobExecution); + logger.debug("Registered: " + context.getJobExecutionContext()); + try { + return simple.getName(); + } + finally { + JobSynchronizationManager.close(); + } + } + }); + tasks.add(task); + taskExecutor.execute(task); + } + + for (FutureTask task : tasks) { + assertEquals("foo", task.get()); + } + + // Don't close the outer scope until all tasks are finished. This should + // always be the case if using an AbstractJob + JobSynchronizationManager.close(); + + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/AsyncStepScopeIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/AsyncStepScopeIntegrationTests.java index 2fa1ca891f..2e08dab1e8 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/AsyncStepScopeIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/AsyncStepScopeIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import static org.junit.Assert.assertEquals; @@ -75,27 +90,26 @@ public void testSimpleProperty() throws Exception { @Test public void testGetMultipleInMultipleThreads() throws Exception { - List> tasks = new ArrayList>(); + List> tasks = new ArrayList<>(); for (int i = 0; i < 12; i++) { final String value = "foo" + i; final Long id = 123L + i; - FutureTask task = new FutureTask(new Callable() { - @Override - public String call() throws Exception { - StepExecution stepExecution = new StepExecution(value, new JobExecution(0L), id); - ExecutionContext executionContext = stepExecution.getExecutionContext(); - executionContext.put("foo", value); - StepContext context = StepSynchronizationManager.register(stepExecution); - logger.debug("Registered: " + context.getStepExecutionContext()); - try { - return simple.getName(); - } - finally { - StepSynchronizationManager.close(); - } - } - }); + FutureTask task = new FutureTask<>(new Callable() { + @Override + public String call() throws Exception { + StepExecution stepExecution = new StepExecution(value, new JobExecution(0L), id); + ExecutionContext executionContext = stepExecution.getExecutionContext(); + executionContext.put("foo", value); + StepContext context = StepSynchronizationManager.register(stepExecution); + logger.debug("Registered: " + context.getStepExecutionContext()); + try { + return simple.getName(); + } finally { + StepSynchronizationManager.close(); + } + } + }); tasks.add(task); taskExecutor.execute(task); } @@ -111,7 +125,7 @@ public String call() throws Exception { @Test public void testGetSameInMultipleThreads() throws Exception { - List> tasks = new ArrayList>(); + List> tasks = new ArrayList<>(); final StepExecution stepExecution = new StepExecution("foo", new JobExecution(0L), 123L); ExecutionContext executionContext = stepExecution.getExecutionContext(); executionContext.put("foo", "foo"); @@ -120,7 +134,7 @@ public void testGetSameInMultipleThreads() throws Exception { for (int i = 0; i < 12; i++) { final String value = "foo" + i; - FutureTask task = new FutureTask(new Callable() { + FutureTask task = new FutureTask<>(new Callable() { @Override public String call() throws Exception { ExecutionContext executionContext = stepExecution.getExecutionContext(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/Collaborator.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/Collaborator.java index 4b99b102d4..4e9d4c3969 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/Collaborator.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/Collaborator.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import java.util.List; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeDestructionCallbackIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeDestructionCallbackIntegrationTests.java new file mode 100644 index 0000000000..5b0508bb9f --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeDestructionCallbackIntegrationTests.java @@ -0,0 +1,102 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.util.StringUtils; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class JobScopeDestructionCallbackIntegrationTests { + + @Autowired + @Qualifier("proxied") + private Job proxied; + + @Autowired + @Qualifier("nested") + private Job nested; + + @Autowired + @Qualifier("ref") + private Job ref; + + @Autowired + @Qualifier("foo") + private Collaborator foo; + + @Before + @After + public void resetMessage() throws Exception { + TestDisposableCollaborator.message = "none"; + TestAdvice.names.clear(); + } + + @Test + public void testDisposableScopedProxy() throws Exception { + assertNotNull(proxied); + proxied.execute(new JobExecution(1L)); + assertEquals(1, StringUtils.countOccurrencesOf(TestDisposableCollaborator.message, "destroyed")); + } + + @Test + public void testDisposableInnerScopedProxy() throws Exception { + assertNotNull(nested); + nested.execute(new JobExecution(1L)); + assertEquals(1, StringUtils.countOccurrencesOf(TestDisposableCollaborator.message, "destroyed")); + } + + @Test + public void testProxiedScopedProxy() throws Exception { + assertNotNull(nested); + nested.execute(new JobExecution(1L)); + assertEquals(4, TestAdvice.names.size()); + assertEquals("bar", TestAdvice.names.get(0)); + assertEquals(1, StringUtils.countOccurrencesOf(TestDisposableCollaborator.message, "destroyed")); + } + + @Test + public void testRefScopedProxy() throws Exception { + assertNotNull(ref); + ref.execute(new JobExecution(1L)); + assertEquals(4, TestAdvice.names.size()); + assertEquals("spam", TestAdvice.names.get(0)); + assertEquals(2, StringUtils.countOccurrencesOf(TestDisposableCollaborator.message, "destroyed")); + assertEquals(1, StringUtils.countOccurrencesOf(TestDisposableCollaborator.message, "bar:destroyed")); + assertEquals(1, StringUtils.countOccurrencesOf(TestDisposableCollaborator.message, "spam:destroyed")); + } + + @Test + public void testProxiedNormalBean() throws Exception { + assertNotNull(nested); + String name = foo.getName(); + assertEquals(1, TestAdvice.names.size()); + assertEquals(name, TestAdvice.names.get(0)); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeIntegrationTests.java new file mode 100644 index 0000000000..8dd863a29a --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeIntegrationTests.java @@ -0,0 +1,132 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.scope.context.JobSynchronizationManager; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class JobScopeIntegrationTests { + + private static final String PROXY_TO_STRING_REGEX = "class .*\\$Proxy\\d+"; + + @Autowired + @Qualifier("vanilla") + private Job vanilla; + + @Autowired + @Qualifier("proxied") + private Job proxied; + + @Autowired + @Qualifier("nested") + private Job nested; + + @Autowired + @Qualifier("enhanced") + private Job enhanced; + + @Autowired + @Qualifier("double") + private Job doubleEnhanced; + + @Before + @After + public void start() { + JobSynchronizationManager.close(); + TestJob.reset(); + } + + @Test + public void testScopeCreation() throws Exception { + vanilla.execute(new JobExecution(11L)); + assertNotNull(TestJob.getContext()); + assertNull(JobSynchronizationManager.getContext()); + } + + @Test + public void testScopedProxy() throws Exception { + proxied.execute(new JobExecution(11L)); + assertTrue(TestJob.getContext().attributeNames().length > 0); + String collaborator = (String) TestJob.getContext().getAttribute("collaborator"); + assertNotNull(collaborator); + assertEquals("bar", collaborator); + assertTrue("Scoped proxy not created", ((String) TestJob.getContext().getAttribute("collaborator.class")) + .matches(PROXY_TO_STRING_REGEX)); + } + + @Test + public void testNestedScopedProxy() throws Exception { + nested.execute(new JobExecution(11L)); + assertTrue(TestJob.getContext().attributeNames().length > 0); + String collaborator = (String) TestJob.getContext().getAttribute("collaborator"); + assertNotNull(collaborator); + assertEquals("foo", collaborator); + String parent = (String) TestJob.getContext().getAttribute("parent"); + assertNotNull(parent); + assertEquals("bar", parent); + assertTrue("Scoped proxy not created", ((String) TestJob.getContext().getAttribute("parent.class")) + .matches(PROXY_TO_STRING_REGEX)); + } + + @Test + public void testExecutionContext() throws Exception { + JobExecution stepExecution = new JobExecution(11L); + ExecutionContext executionContext = new ExecutionContext(); + executionContext.put("name", "spam"); + stepExecution.setExecutionContext(executionContext); + proxied.execute(stepExecution); + assertTrue(TestJob.getContext().attributeNames().length > 0); + String collaborator = (String) TestJob.getContext().getAttribute("collaborator"); + assertNotNull(collaborator); + assertEquals("bar", collaborator); + } + + @Test + public void testScopedProxyForReference() throws Exception { + enhanced.execute(new JobExecution(11L)); + assertTrue(TestJob.getContext().attributeNames().length > 0); + String collaborator = (String) TestJob.getContext().getAttribute("collaborator"); + assertNotNull(collaborator); + assertEquals("bar", collaborator); + } + + @Test + public void testScopedProxyForSecondReference() throws Exception { + doubleEnhanced.execute(new JobExecution(11L)); + assertTrue(TestJob.getContext().attributeNames().length > 0); + String collaborator = (String) TestJob.getContext().getAttribute("collaborator"); + assertNotNull(collaborator); + assertEquals("bar", collaborator); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeNestedIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeNestedIntegrationTests.java new file mode 100644 index 0000000000..f82fafb83e --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeNestedIntegrationTests.java @@ -0,0 +1,48 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.Job; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class JobScopeNestedIntegrationTests { + + @Autowired + @Qualifier("proxied") + private Job proxied; + + @Autowired + @Qualifier("parent") + private Collaborator parent; + + @Test + public void testNestedScopedProxy() throws Exception { + assertNotNull(proxied); + assertEquals("foo", parent.getName()); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopePlaceholderIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopePlaceholderIntegrationTests.java new file mode 100644 index 0000000000..dab3bc90b3 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopePlaceholderIntegrationTests.java @@ -0,0 +1,171 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import static org.junit.Assert.assertEquals; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.scope.context.JobSynchronizationManager; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.beans.factory.BeanFactoryAware; +import org.springframework.beans.factory.ListableBeanFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class JobScopePlaceholderIntegrationTests implements BeanFactoryAware { + + @Autowired + @Qualifier("simple") + private Collaborator simple; + + @Autowired + @Qualifier("compound") + private Collaborator compound; + + @Autowired + @Qualifier("value") + private Collaborator value; + + @Autowired + @Qualifier("ref") + private Collaborator ref; + + @Autowired + @Qualifier("scopedRef") + private Collaborator scopedRef; + + @Autowired + @Qualifier("list") + private Collaborator list; + + @Autowired + @Qualifier("bar") + private Collaborator bar; + + @Autowired + @Qualifier("nested") + private Collaborator nested; + + private JobExecution jobExecution; + + private ListableBeanFactory beanFactory; + + private int beanCount; + + @Override + public void setBeanFactory(BeanFactory beanFactory) throws BeansException { + this.beanFactory = (ListableBeanFactory) beanFactory; + } + + @Before + public void start() { + start("bar"); + } + + private void start(String foo) { + + JobSynchronizationManager.close(); + jobExecution = new JobExecution(123L); + + ExecutionContext executionContext = new ExecutionContext(); + executionContext.put("foo", foo); + executionContext.put("parent", bar); + + jobExecution.setExecutionContext(executionContext); + JobSynchronizationManager.register(jobExecution); + + beanCount = beanFactory.getBeanDefinitionCount(); + + } + + @After + public void stop() { + JobSynchronizationManager.close(); + // Check that all temporary bean definitions are cleaned up + assertEquals(beanCount, beanFactory.getBeanDefinitionCount()); + } + + @Test + public void testSimpleProperty() throws Exception { + assertEquals("bar", simple.getName()); + // Once the job context is set up it should be baked into the proxies + // so changing it now should have no effect + jobExecution.getExecutionContext().put("foo", "wrong!"); + assertEquals("bar", simple.getName()); + } + + @Test + public void testCompoundProperty() throws Exception { + assertEquals("bar-bar", compound.getName()); + } + + @Test + public void testCompoundPropertyTwice() throws Exception { + + assertEquals("bar-bar", compound.getName()); + + JobSynchronizationManager.close(); + jobExecution = new JobExecution(123L); + + ExecutionContext executionContext = new ExecutionContext(); + executionContext.put("foo", "spam"); + + jobExecution.setExecutionContext(executionContext); + JobSynchronizationManager.register(jobExecution); + + assertEquals("spam-bar", compound.getName()); + + } + + @Test + public void testParentByRef() throws Exception { + assertEquals("bar", ref.getParent().getName()); + } + + @Test + public void testParentByValue() throws Exception { + assertEquals("bar", value.getParent().getName()); + } + + @Test + public void testList() throws Exception { + assertEquals("[bar]", list.getList().toString()); + } + + @Test + public void testNested() throws Exception { + assertEquals("bar", nested.getParent().getName()); + } + + @Test + public void testScopedRef() throws Exception { + assertEquals("bar", scopedRef.getParent().getName()); + stop(); + start("spam"); + assertEquals("spam", scopedRef.getParent().getName()); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeProxyTargetClassIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeProxyTargetClassIntegrationTests.java new file mode 100644 index 0000000000..9005273e34 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeProxyTargetClassIntegrationTests.java @@ -0,0 +1,87 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import static org.junit.Assert.assertEquals; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.scope.context.JobSynchronizationManager; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.beans.factory.BeanFactoryAware; +import org.springframework.beans.factory.ListableBeanFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class JobScopeProxyTargetClassIntegrationTests implements BeanFactoryAware { + + @Autowired + @Qualifier("simple") + private TestCollaborator simple; + + private JobExecution jobExecution; + + private ListableBeanFactory beanFactory; + + private int beanCount; + + @Override + public void setBeanFactory(BeanFactory beanFactory) throws BeansException { + this.beanFactory = (ListableBeanFactory) beanFactory; + } + + @Before + public void start() { + + JobSynchronizationManager.close(); + jobExecution = new JobExecution(123L); + + ExecutionContext executionContext = new ExecutionContext(); + executionContext.put("foo", "bar"); + + jobExecution.setExecutionContext(executionContext); + JobSynchronizationManager.register(jobExecution); + + beanCount = beanFactory.getBeanDefinitionCount(); + + } + + @After + public void cleanUp() { + JobSynchronizationManager.close(); + // Check that all temporary bean definitions are cleaned up + assertEquals(beanCount, beanFactory.getBeanDefinitionCount()); + } + + @Test + public void testSimpleProperty() throws Exception { + assertEquals("bar", simple.getName()); + // Once the job context is set up it should be baked into the proxies + // so changing it now should have no effect + jobExecution.getExecutionContext().put("foo", "wrong!"); + assertEquals("bar", simple.getName()); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeStartupIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeStartupIntegrationTests.java new file mode 100644 index 0000000000..6119d8c354 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeStartupIntegrationTests.java @@ -0,0 +1,32 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class JobScopeStartupIntegrationTests { + + @Test + public void testScopedProxyDuringStartup() throws Exception { + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeTests.java new file mode 100644 index 0000000000..49fd282c4d --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobScopeTests.java @@ -0,0 +1,175 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +import java.util.ArrayList; +import java.util.List; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.scope.context.JobContext; +import org.springframework.batch.core.scope.context.JobSynchronizationManager; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.ObjectFactory; +import org.springframework.context.support.StaticApplicationContext; + +/** + * @author Dave Syer + * @author Jimmy Praet + */ +public class JobScopeTests { + + private JobScope scope = new JobScope(); + + private JobExecution jobExecution = new JobExecution(0L); + + private JobContext context; + + @Before + public void setUp() throws Exception { + context = JobSynchronizationManager.register(jobExecution); + } + + @After + public void tearDown() throws Exception { + JobSynchronizationManager.release(); + } + + @Test + public void testGetWithNoContext() throws Exception { + final String foo = "bar"; + JobSynchronizationManager.release(); + try { + scope.get("foo", new ObjectFactory() { + @Override + public String getObject() throws BeansException { + return foo; + } + }); + fail("Expected IllegalStateException"); + } + catch (IllegalStateException e) { + // expected + } + + } + + @Test + public void testGetWithNothingAlreadyThere() { + final String foo = "bar"; + Object value = scope.get("foo", new ObjectFactory() { + @Override + public String getObject() throws BeansException { + return foo; + } + }); + assertEquals(foo, value); + assertTrue(context.hasAttribute("foo")); + } + + @Test + public void testGetWithSomethingAlreadyThere() { + context.setAttribute("foo", "bar"); + Object value = scope.get("foo", new ObjectFactory() { + @Override + public String getObject() throws BeansException { + return null; + } + }); + assertEquals("bar", value); + assertTrue(context.hasAttribute("foo")); + } + + @Test + public void testGetConversationId() { + String id = scope.getConversationId(); + assertNotNull(id); + } + + @Test + public void testRegisterDestructionCallback() { + final List list = new ArrayList<>(); + context.setAttribute("foo", "bar"); + scope.registerDestructionCallback("foo", new Runnable() { + @Override + public void run() { + list.add("foo"); + } + }); + assertEquals(0, list.size()); + // When the context is closed, provided the attribute exists the + // callback is called... + context.close(); + assertEquals(1, list.size()); + } + + @Test + public void testRegisterAnotherDestructionCallback() { + final List list = new ArrayList<>(); + context.setAttribute("foo", "bar"); + scope.registerDestructionCallback("foo", new Runnable() { + @Override + public void run() { + list.add("foo"); + } + }); + scope.registerDestructionCallback("foo", new Runnable() { + @Override + public void run() { + list.add("bar"); + } + }); + assertEquals(0, list.size()); + // When the context is closed, provided the attribute exists the + // callback is called... + context.close(); + assertEquals(2, list.size()); + } + + @Test + public void testRemove() { + context.setAttribute("foo", "bar"); + scope.remove("foo"); + assertFalse(context.hasAttribute("foo")); + } + + @Test + public void testOrder() throws Exception { + assertEquals(Integer.MAX_VALUE, scope.getOrder()); + scope.setOrder(11); + assertEquals(11, scope.getOrder()); + } + + @Test + @SuppressWarnings("resource") + public void testName() throws Exception { + scope.setName("foo"); + StaticApplicationContext beanFactory = new StaticApplicationContext(); + scope.postProcessBeanFactory(beanFactory.getDefaultListableBeanFactory()); + String[] scopes = beanFactory.getDefaultListableBeanFactory().getRegisteredScopeNames(); + assertEquals(1, scopes.length); + assertEquals("foo", scopes[0]); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobStartupRunner.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobStartupRunner.java index d67b72a158..903899b439 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobStartupRunner.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/JobStartupRunner.java @@ -1,22 +1,36 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; +import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; -import org.springframework.batch.core.Step; -import org.springframework.batch.core.StepExecution; import org.springframework.beans.factory.InitializingBean; public class JobStartupRunner implements InitializingBean { - private Step step; + private Job job; - public void setStep(Step step) { - this.step = step; + public void setJob(Job job) { + this.job = job; } @Override public void afterPropertiesSet() throws Exception { - StepExecution stepExecution = new StepExecution("step", new JobExecution(1L), 0L); - step.execute(stepExecution); + JobExecution jobExecution = new JobExecution(11L); + job.execute(jobExecution); // expect no errors } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeClassIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeClassIntegrationTests.java index a007ae5a11..0bdbd7529e 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeClassIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeClassIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeDestructionCallbackIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeDestructionCallbackIntegrationTests.java index d1739cba9d..b6c03de6ff 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeDestructionCallbackIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeDestructionCallbackIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeIntegrationTests.java index 30468194a1..585ea02329 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeNestedIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeNestedIntegrationTests.java index 8aa13fea7e..d44cf0a73b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeNestedIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeNestedIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopePerformanceTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopePerformanceTests.java index e24b204533..d624cf6e7b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopePerformanceTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopePerformanceTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import org.apache.commons.logging.Log; @@ -37,10 +52,10 @@ public void setApplicationContext(ApplicationContext applicationContext) public void start() throws Exception { int count = doTest("vanilla", "warmup"); logger.info("Item count: "+count); + StepSynchronizationManager.close(); StepSynchronizationManager.register(new StepExecution("step", new JobExecution(0L),1L)); } - @Before @After public void cleanup() { StepSynchronizationManager.close(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopePlaceholderIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopePlaceholderIntegrationTests.java index 79a919ff80..2744cb4a9c 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopePlaceholderIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopePlaceholderIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeProxyTargetClassIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeProxyTargetClassIntegrationTests.java index a3e0944fb4..2440eb2df9 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeProxyTargetClassIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeProxyTargetClassIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeProxyTargetClassOverrideIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeProxyTargetClassOverrideIntegrationTests.java new file mode 100644 index 0000000000..4129ab1083 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeProxyTargetClassOverrideIntegrationTests.java @@ -0,0 +1,158 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.aop.support.AopUtils; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.beans.factory.BeanFactoryAware; +import org.springframework.beans.factory.ListableBeanFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class StepScopeProxyTargetClassOverrideIntegrationTests implements BeanFactoryAware { + + private static final String JDK_PROXY_TO_STRING_REGEX = "class .*\\$Proxy\\d+"; + + private static final String CGLIB_PROXY_TO_STRING_REGEX = "class .*\\$EnhancerBySpringCGLIB.*"; + + @Autowired + @Qualifier("simple") + private TestCollaborator simple; + + @Autowired + @Qualifier("simpleProxyTargetClassTrue") + private TestCollaborator simpleProxyTargetClassTrue; + + @Autowired + @Qualifier("simpleProxyTargetClassFalse") + private Collaborator simpleProxyTargetClassFalse; + + @Autowired + @Qualifier("nested") + private Step nested; + + @Autowired + @Qualifier("nestedProxyTargetClassTrue") + private Step nestedProxyTargetClassTrue; + + @Autowired + @Qualifier("nestedProxyTargetClassFalse") + private Step nestedProxyTargetClassFalse; + + private StepExecution stepExecution; + + private ListableBeanFactory beanFactory; + + private int beanCount; + + @Override + public void setBeanFactory(BeanFactory beanFactory) throws BeansException { + this.beanFactory = (ListableBeanFactory) beanFactory; + } + + @Before + public void start() { + + StepSynchronizationManager.close(); + TestStep.reset(); + stepExecution = new StepExecution("foo", new JobExecution(11L), 123L); + + ExecutionContext executionContext = new ExecutionContext(); + executionContext.put("foo", "bar"); + + stepExecution.setExecutionContext(executionContext); + StepSynchronizationManager.register(stepExecution); + + beanCount = beanFactory.getBeanDefinitionCount(); + + } + + @After + public void cleanUp() { + StepSynchronizationManager.close(); + // Check that all temporary bean definitions are cleaned up + assertEquals(beanCount, beanFactory.getBeanDefinitionCount()); + } + + @Test + public void testSimple() throws Exception { + assertTrue(AopUtils.isCglibProxy(simple)); + assertEquals("bar", simple.getName()); + } + + @Test + public void testSimpleProxyTargetClassTrue() throws Exception { + assertTrue(AopUtils.isCglibProxy(simpleProxyTargetClassTrue)); + assertEquals("bar", simpleProxyTargetClassTrue.getName()); + } + + @Test + public void testSimpleProxyTargetClassFalse() throws Exception { + assertTrue(AopUtils.isJdkDynamicProxy(simpleProxyTargetClassFalse)); + assertEquals("bar", simpleProxyTargetClassFalse.getName()); + } + + @Test + public void testNested() throws Exception { + nested.execute(new StepExecution("foo", new JobExecution(11L), 31L)); + assertTrue(TestStep.getContext().attributeNames().length > 0); + String collaborator = (String) TestStep.getContext().getAttribute("collaborator"); + assertNotNull(collaborator); + assertEquals("foo", collaborator); + String parent = (String) TestStep.getContext().getAttribute("parent"); + assertNotNull(parent); + assertEquals("bar", parent); + assertTrue("Scoped proxy not created", ((String) TestStep.getContext().getAttribute("parent.class")) + .matches(CGLIB_PROXY_TO_STRING_REGEX)); + } + + @Test + public void testNestedProxyTargetClassTrue() throws Exception { + nestedProxyTargetClassTrue.execute(new StepExecution("foo", new JobExecution(11L), 31L)); + String parent = (String) TestStep.getContext().getAttribute("parent"); + assertEquals("bar", parent); + assertTrue("Scoped proxy not created", ((String) TestStep.getContext().getAttribute("parent.class")) + .matches(CGLIB_PROXY_TO_STRING_REGEX)); + } + + @Test + public void testNestedProxyTargetClassFalse() throws Exception { + nestedProxyTargetClassFalse.execute(new StepExecution("foo", new JobExecution(11L), 31L)); + String parent = (String) TestStep.getContext().getAttribute("parent"); + assertEquals("bar", parent); + assertTrue("Scoped proxy not created", ((String) TestStep.getContext().getAttribute("parent.class")) + .matches(JDK_PROXY_TO_STRING_REGEX)); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeStartupIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeStartupIntegrationTests.java index fab5e2a69e..7815641815 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeStartupIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeStartupIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import org.junit.Test; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeTests.java index 4bd8f8de77..252c6728f7 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepScopeTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,7 +40,6 @@ * @author Dave Syer * */ -@SuppressWarnings("rawtypes") public class StepScopeTests { private StepScope scope = new StepScope(); @@ -65,7 +64,7 @@ public void testGetWithNoContext() throws Exception { final String foo = "bar"; StepSynchronizationManager.close(); try { - scope.get("foo", new ObjectFactory() { + scope.get("foo", new ObjectFactory() { @Override public Object getObject() throws BeansException { return foo; @@ -82,7 +81,7 @@ public Object getObject() throws BeansException { @Test public void testGetWithNothingAlreadyThere() { final String foo = "bar"; - Object value = scope.get("foo", new ObjectFactory() { + Object value = scope.get("foo", new ObjectFactory() { @Override public Object getObject() throws BeansException { return foo; @@ -95,7 +94,7 @@ public Object getObject() throws BeansException { @Test public void testGetWithSomethingAlreadyThere() { context.setAttribute("foo", "bar"); - Object value = scope.get("foo", new ObjectFactory() { + Object value = scope.get("foo", new ObjectFactory() { @Override public Object getObject() throws BeansException { return null; @@ -109,7 +108,7 @@ public Object getObject() throws BeansException { public void testGetWithSomethingAlreadyInParentContext() { context.setAttribute("foo", "bar"); StepContext context = StepSynchronizationManager.register(new StepExecution("bar", new JobExecution(0L))); - Object value = scope.get("foo", new ObjectFactory() { + Object value = scope.get("foo", new ObjectFactory() { @Override public Object getObject() throws BeansException { return "spam"; @@ -136,7 +135,7 @@ public void testGetConversationId() { @Test public void testRegisterDestructionCallback() { - final List list = new ArrayList(); + final List list = new ArrayList<>(); context.setAttribute("foo", "bar"); scope.registerDestructionCallback("foo", new Runnable() { @Override @@ -153,7 +152,7 @@ public void run() { @Test public void testRegisterAnotherDestructionCallback() { - final List list = new ArrayList(); + final List list = new ArrayList<>(); context.setAttribute("foo", "bar"); scope.registerDestructionCallback("foo", new Runnable() { @Override diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepStartupRunner.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepStartupRunner.java new file mode 100644 index 0000000000..09c3923257 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/StepStartupRunner.java @@ -0,0 +1,38 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.beans.factory.InitializingBean; + +public class StepStartupRunner implements InitializingBean { + + private Step step; + + public void setStep(Step step) { + this.step = step; + } + + @Override + public void afterPropertiesSet() throws Exception { + StepExecution stepExecution = new StepExecution("step", new JobExecution(1L), 0L); + step.execute(stepExecution); + // expect no errors + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestAdvice.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestAdvice.java index e434c363b3..bd5c9e0601 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestAdvice.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestAdvice.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import java.util.ArrayList; @@ -9,7 +24,7 @@ @Aspect public class TestAdvice { - public static final List names = new ArrayList(); + public static final List names = new ArrayList<>(); @AfterReturning(pointcut="execution(String org.springframework.batch.core.scope.Collaborator+.getName(..))", returning="name") public void registerCollaborator(String name) { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestCollaborator.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestCollaborator.java index 3cf7c57599..f00b6e3e1d 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestCollaborator.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestCollaborator.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import java.io.Serializable; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestDisposableCollaborator.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestDisposableCollaborator.java index b801e1337b..43d64e3331 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestDisposableCollaborator.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestDisposableCollaborator.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import org.springframework.beans.factory.DisposableBean; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestJob.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestJob.java new file mode 100644 index 0000000000..1cb4cbb0da --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestJob.java @@ -0,0 +1,83 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersIncrementer; +import org.springframework.batch.core.JobParametersValidator; +import org.springframework.batch.core.scope.context.JobContext; +import org.springframework.batch.core.scope.context.JobSynchronizationManager; +import org.springframework.lang.Nullable; + +public class TestJob implements Job { + + private static JobContext context; + + private Collaborator collaborator; + + public void setCollaborator(Collaborator collaborator) { + this.collaborator = collaborator; + } + + public static JobContext getContext() { + return context; + } + + public static void reset() { + context = null; + } + + @Override + public void execute(JobExecution stepExecution) { + context = JobSynchronizationManager.getContext(); + setContextFromCollaborator(); + stepExecution.getExecutionContext().put("foo", "changed but it shouldn't affect the collaborator"); + setContextFromCollaborator(); + } + + private void setContextFromCollaborator() { + if (context != null) { + context.setAttribute("collaborator", collaborator.getName()); + context.setAttribute("collaborator.class", collaborator.getClass().toString()); + if (collaborator.getParent()!=null) { + context.setAttribute("parent", collaborator.getParent().getName()); + context.setAttribute("parent.class", collaborator.getParent().getClass().toString()); + } + } + } + + @Override + public String getName() { + return "foo"; + } + + @Override + public boolean isRestartable() { + return false; + } + + @Nullable + @Override + public JobParametersIncrementer getJobParametersIncrementer() { + return null; + } + + @Override + public JobParametersValidator getJobParametersValidator() { + return null; + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestStep.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestStep.java index 0062d8828b..f69a6700a9 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestStep.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/TestStep.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope; import org.springframework.batch.core.JobInterruptedException; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/ChunkContextTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/ChunkContextTests.java index 8a754b9990..d6c7eb54f0 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/ChunkContextTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/ChunkContextTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,7 +35,7 @@ public class ChunkContextTests { private ChunkContext context = new ChunkContext(new StepContext(new JobExecution(new JobInstance(0L, - "job"), 1L, new JobParameters(Collections.singletonMap("foo", new JobParameter("bar")))) + "job"), 1L, new JobParameters(Collections.singletonMap("foo", new JobParameter("bar"))), null) .createStepExecution("foo"))); @Test diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/InternalBeanStepScopeIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/InternalBeanStepScopeIntegrationTests.java new file mode 100644 index 0000000000..3b899cb379 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/InternalBeanStepScopeIntegrationTests.java @@ -0,0 +1,57 @@ +/* + * Copyright 2014-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope.context; + +import org.junit.Test; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.context.ApplicationContext; +import org.springframework.context.support.ClassPathXmlApplicationContext; + +import static org.junit.Assert.assertEquals; + +/** + * @author mminella + */ +public class InternalBeanStepScopeIntegrationTests { + + @Test + public void testCommitIntervalJobParameter() throws Exception { + ApplicationContext context = new ClassPathXmlApplicationContext("/org/springframework/batch/core/scope/context/CommitIntervalJobParameter-context.xml"); + Job job = context.getBean(Job.class); + JobLauncher launcher = context.getBean(JobLauncher.class); + + JobExecution execution = launcher.run(job, new JobParametersBuilder().addLong("commit.interval", 1l).toJobParameters()); + + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(2, execution.getStepExecutions().iterator().next().getReadCount()); + assertEquals(2, execution.getStepExecutions().iterator().next().getWriteCount()); + } + + @Test + public void testInvalidCommitIntervalJobParameter() throws Exception { + ApplicationContext context = new ClassPathXmlApplicationContext("/org/springframework/batch/core/scope/context/CommitIntervalJobParameter-context.xml"); + Job job = context.getBean(Job.class); + JobLauncher launcher = context.getBean(JobLauncher.class); + + JobExecution execution = launcher.run(job, new JobParametersBuilder().addLong("commit.intervall", 1l).toJobParameters()); + + assertEquals(BatchStatus.FAILED, execution.getStatus()); + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/JobContextTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/JobContextTests.java new file mode 100644 index 0000000000..005a8dbb4c --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/JobContextTests.java @@ -0,0 +1,187 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope.context; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +import java.util.ArrayList; +import java.util.List; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.item.ExecutionContext; + +/** + * @author Dave Syer + * @author Jimmy Praet + */ +public class JobContextTests { + + private List list; + + private JobExecution jobExecution; + + private JobContext context; + + @Before + public void setUp() { + jobExecution = new JobExecution(1L); + JobInstance jobInstance = new JobInstance(2L, "job"); + jobExecution.setJobInstance(jobInstance); + context = new JobContext(jobExecution); + list = new ArrayList<>(); + } + + @Test + public void testGetJobExecution() { + context = new JobContext(jobExecution); + assertNotNull(context.getJobExecution()); + } + + @Test + public void testNullJobExecution() { + try { + context = new JobContext(null); + fail("Expected IllegalArgumentException"); + } + catch (IllegalArgumentException e) { + // expected + } + } + + @Test + public void testEqualsSelf() { + assertEquals(context, context); + } + + @Test + public void testNotEqualsNull() { + assertFalse(context.equals(null)); + } + + @Test + public void testEqualsContextWithSameJobExecution() { + assertEquals(new JobContext(jobExecution), context); + } + + @Test + public void testDestructionCallbackSunnyDay() throws Exception { + context.setAttribute("foo", "FOO"); + context.registerDestructionCallback("foo", new Runnable() { + @Override + public void run() { + list.add("bar"); + } + }); + context.close(); + assertEquals(1, list.size()); + assertEquals("bar", list.get(0)); + } + + @Test + public void testDestructionCallbackMissingAttribute() throws Exception { + context.registerDestructionCallback("foo", new Runnable() { + @Override + public void run() { + list.add("bar"); + } + }); + context.close(); + // Yes the callback should be called even if the attribute is missing - + // for inner beans + assertEquals(1, list.size()); + } + + @Test + public void testDestructionCallbackWithException() throws Exception { + context.setAttribute("foo", "FOO"); + context.setAttribute("bar", "BAR"); + context.registerDestructionCallback("bar", new Runnable() { + @Override + public void run() { + list.add("spam"); + throw new RuntimeException("fail!"); + } + }); + context.registerDestructionCallback("foo", new Runnable() { + @Override + public void run() { + list.add("bar"); + throw new RuntimeException("fail!"); + } + }); + try { + context.close(); + fail("Expected RuntimeException"); + } + catch (RuntimeException e) { + // We don't care which one was thrown... + assertEquals("fail!", e.getMessage()); + } + // ...but we do care that both were executed: + assertEquals(2, list.size()); + assertTrue(list.contains("bar")); + assertTrue(list.contains("spam")); + } + + @Test + public void testJobName() throws Exception { + assertEquals("job", context.getJobName()); + } + + @Test + public void testJobExecutionContext() throws Exception { + ExecutionContext executionContext = jobExecution.getExecutionContext(); + executionContext.put("foo", "bar"); + assertEquals("bar", context.getJobExecutionContext().get("foo")); + } + + @Test + public void testSystemProperties() throws Exception { + System.setProperty("foo", "bar"); + assertEquals("bar", context.getSystemProperties().getProperty("foo")); + } + + @Test + public void testJobParameters() throws Exception { + JobParameters jobParameters = new JobParametersBuilder().addString("foo", "bar").toJobParameters(); + JobInstance jobInstance = new JobInstance(0L, "foo"); + jobExecution = new JobExecution(5L, jobParameters); + jobExecution.setJobInstance(jobInstance); + context = new JobContext(jobExecution); + assertEquals("bar", context.getJobParameters().get("foo")); + } + + @Test + public void testContextId() throws Exception { + assertEquals("jobExecution#1", context.getId()); + } + + @Test(expected = IllegalStateException.class) + public void testIllegalContextId() throws Exception { + context = new JobContext(new JobExecution((Long) null)); + context.getId(); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/JobSynchronizationManagerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/JobSynchronizationManagerTests.java new file mode 100644 index 0000000000..74fb81725b --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/JobSynchronizationManagerTests.java @@ -0,0 +1,135 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope.context; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; + +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.Callable; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.FutureTask; +import java.util.concurrent.TimeUnit; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.JobExecution; + +/** + * JobSynchronizationManagerTests. + * + * @author Jimmy Praet + */ +public class JobSynchronizationManagerTests { + + private JobExecution jobExecution = new JobExecution(0L); + + @Before + @After + public void start() { + while (JobSynchronizationManager.getContext() != null) { + JobSynchronizationManager.close(); + } + } + + @Test + public void testGetContext() { + assertNull(JobSynchronizationManager.getContext()); + JobSynchronizationManager.register(jobExecution); + assertNotNull(JobSynchronizationManager.getContext()); + } + + @Test + public void testClose() throws Exception { + final List list = new ArrayList<>(); + JobContext context = JobSynchronizationManager.register(jobExecution); + context.registerDestructionCallback("foo", new Runnable() { + @Override + public void run() { + list.add("foo"); + } + }); + JobSynchronizationManager.close(); + assertNull(JobSynchronizationManager.getContext()); + assertEquals(0, list.size()); + } + + @Test + public void testMultithreaded() throws Exception { + JobContext context = JobSynchronizationManager.register(jobExecution); + ExecutorService executorService = Executors.newFixedThreadPool(2); + FutureTask task = new FutureTask<>(new Callable() { + @Override + public JobContext call() throws Exception { + try { + JobSynchronizationManager.register(jobExecution); + JobContext context = JobSynchronizationManager.getContext(); + context.setAttribute("foo", "bar"); + return context; + } + finally { + JobSynchronizationManager.close(); + } + } + }); + executorService.execute(task); + executorService.awaitTermination(1, TimeUnit.SECONDS); + assertEquals(context.attributeNames().length, task.get().attributeNames().length); + JobSynchronizationManager.close(); + assertNull(JobSynchronizationManager.getContext()); + } + + @Test + public void testRelease() { + JobContext context = JobSynchronizationManager.register(jobExecution); + final List list = new ArrayList<>(); + context.registerDestructionCallback("foo", new Runnable() { + @Override + public void run() { + list.add("foo"); + } + }); + // On release we expect the destruction callbacks to be called + JobSynchronizationManager.release(); + assertNull(JobSynchronizationManager.getContext()); + assertEquals(1, list.size()); + } + + @Test + public void testRegisterNull() { + assertNull(JobSynchronizationManager.getContext()); + JobSynchronizationManager.register(null); + assertNull(JobSynchronizationManager.getContext()); + } + + @Test + public void testRegisterTwice() { + JobSynchronizationManager.register(jobExecution); + JobSynchronizationManager.register(jobExecution); + JobSynchronizationManager.close(); + // if someone registers you have to assume they are going to close, so + // the last thing you want is for the close to remove another context + // that someone else has registered + assertNotNull(JobSynchronizationManager.getContext()); + JobSynchronizationManager.close(); + assertNull(JobSynchronizationManager.getContext()); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/StepContextRepeatCallbackTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/StepContextRepeatCallbackTests.java index fb30c2681d..5f190e4cba 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/StepContextRepeatCallbackTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/StepContextRepeatCallbackTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/StepContextTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/StepContextTests.java index 8e962bb3bb..e0b70a6685 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/StepContextTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/StepContextTests.java @@ -1,189 +1,214 @@ -/* - * Copyright 2006-2013 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.core.scope.context; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - -import java.util.ArrayList; -import java.util.List; - -import org.junit.Test; -import org.springframework.batch.core.JobExecution; -import org.springframework.batch.core.JobInstance; -import org.springframework.batch.core.JobParameters; -import org.springframework.batch.core.JobParametersBuilder; -import org.springframework.batch.core.StepExecution; -import org.springframework.batch.item.ExecutionContext; - -/** - * @author Dave Syer - * - */ -public class StepContextTests { - - private List list = new ArrayList(); - - private StepExecution stepExecution = new StepExecution("step", new JobExecution(new JobInstance(2L, "job"), 0L, null), 1L); - - private StepContext context = new StepContext(stepExecution); - - @Test - public void testGetStepExecution() { - context = new StepContext(stepExecution); - assertNotNull(context.getStepExecution()); - } - - @Test - public void testNullStepExecution() { - try { - context = new StepContext(null); - fail("Expected IllegalArgumentException"); - } - catch (IllegalArgumentException e) { - // expected - } - } - - @Test - public void testEqualsSelf() { - assertEquals(context, context); - } - - @Test - public void testNotEqualsNull() { - assertFalse(context.equals(null)); - } - - @Test - public void testEqualsContextWithSameStepExecution() { - assertEquals(new StepContext(stepExecution), context); - } - - @Test - public void testDestructionCallbackSunnyDay() throws Exception { - context.setAttribute("foo", "FOO"); - context.registerDestructionCallback("foo", new Runnable() { - @Override - public void run() { - list.add("bar"); - } - }); - context.close(); - assertEquals(1, list.size()); - assertEquals("bar", list.get(0)); - } - - @Test - public void testDestructionCallbackMissingAttribute() throws Exception { - context.registerDestructionCallback("foo", new Runnable() { - @Override - public void run() { - list.add("bar"); - } - }); - context.close(); - // Yes the callback should be called even if the attribute is missing - - // for inner beans - assertEquals(1, list.size()); - } - - @Test - public void testDestructionCallbackWithException() throws Exception { - context.setAttribute("foo", "FOO"); - context.setAttribute("bar", "BAR"); - context.registerDestructionCallback("bar", new Runnable() { - @Override - public void run() { - list.add("spam"); - throw new RuntimeException("fail!"); - } - }); - context.registerDestructionCallback("foo", new Runnable() { - @Override - public void run() { - list.add("bar"); - throw new RuntimeException("fail!"); - } - }); - try { - context.close(); - fail("Expected RuntimeException"); - } - catch (RuntimeException e) { - // We don't care which one was thrown... - assertEquals("fail!", e.getMessage()); - } - // ...but we do care that both were executed: - assertEquals(2, list.size()); - assertTrue(list.contains("bar")); - assertTrue(list.contains("spam")); - } - - @Test - public void testStepName() throws Exception { - assertEquals("step", context.getStepName()); - } - - @Test - public void testJobName() throws Exception { - assertEquals("job", context.getJobName()); - } - - @Test - public void testStepExecutionContext() throws Exception { - ExecutionContext executionContext = stepExecution.getExecutionContext(); - executionContext.put("foo", "bar"); - assertEquals("bar", context.getStepExecutionContext().get("foo")); - } - - @Test - public void testSystemProperties() throws Exception { - System.setProperty("foo", "bar"); - assertEquals("bar", context.getSystemProperties().getProperty("foo")); - } - - @Test - public void testJobExecutionContext() throws Exception { - ExecutionContext executionContext = stepExecution.getJobExecution().getExecutionContext(); - executionContext.put("foo", "bar"); - assertEquals("bar", context.getJobExecutionContext().get("foo")); - } - - @Test - public void testJobParameters() throws Exception { - JobParameters jobParameters = new JobParametersBuilder().addString("foo", "bar").toJobParameters(); - JobInstance instance = stepExecution.getJobExecution().getJobInstance(); - stepExecution = new StepExecution("step", new JobExecution(instance, jobParameters)); - context = new StepContext(stepExecution); - assertEquals("bar", context.getJobParameters().get("foo")); - } - - @Test - public void testContextId() throws Exception { - assertEquals("execution#1", context.getId()); - } - - @Test(expected = IllegalStateException.class) - public void testIllegalContextId() throws Exception { - context = new StepContext(new StepExecution("foo", new JobExecution(0L))); - context.getId(); - } - -} +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.scope.context; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.Properties; + +import org.junit.Test; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; +import org.springframework.batch.item.ExecutionContext; + +/** + * @author Dave Syer + * @author Nicolas Widart + * @author Mahmoud Ben Hassine + * + */ +public class StepContextTests { + + private List list = new ArrayList<>(); + + private StepExecution stepExecution = new StepExecution("step", new JobExecution(new JobInstance(2L, "job"), 0L, null, null), 1L); + + private StepContext context = new StepContext(stepExecution); + + private BatchPropertyContext propertyContext = new BatchPropertyContext(); + + @Test + public void testGetStepExecution() { + context = new StepContext(stepExecution); + assertNotNull(context.getStepExecution()); + } + + @Test + public void testNullStepExecution() { + try { + context = new StepContext(null); + fail("Expected IllegalArgumentException"); + } + catch (IllegalArgumentException e) { + // expected + } + } + + @Test + public void testGetPartitionPlan() { + Properties partitionPropertyValues = new Properties(); + partitionPropertyValues.put("key1", "value1"); + + propertyContext.setStepProperties(stepExecution.getStepName(), partitionPropertyValues); + + context = new StepContext(stepExecution, propertyContext); + + Map plan = context.getPartitionPlan(); + assertEquals("value1", plan.get("key1")); + } + + @Test + public void testEqualsSelf() { + assertEquals(context, context); + } + + @Test + public void testNotEqualsNull() { + assertFalse(context.equals(null)); + } + + @Test + public void testEqualsContextWithSameStepExecution() { + assertEquals(new StepContext(stepExecution), context); + } + + @Test + public void testDestructionCallbackSunnyDay() throws Exception { + context.setAttribute("foo", "FOO"); + context.registerDestructionCallback("foo", new Runnable() { + @Override + public void run() { + list.add("bar"); + } + }); + context.close(); + assertEquals(1, list.size()); + assertEquals("bar", list.get(0)); + } + + @Test + public void testDestructionCallbackMissingAttribute() throws Exception { + context.registerDestructionCallback("foo", new Runnable() { + @Override + public void run() { + list.add("bar"); + } + }); + context.close(); + // Yes the callback should be called even if the attribute is missing - + // for inner beans + assertEquals(1, list.size()); + } + + @Test + public void testDestructionCallbackWithException() throws Exception { + context.setAttribute("foo", "FOO"); + context.setAttribute("bar", "BAR"); + context.registerDestructionCallback("bar", new Runnable() { + @Override + public void run() { + list.add("spam"); + throw new RuntimeException("fail!"); + } + }); + context.registerDestructionCallback("foo", new Runnable() { + @Override + public void run() { + list.add("bar"); + throw new RuntimeException("fail!"); + } + }); + try { + context.close(); + fail("Expected RuntimeException"); + } + catch (RuntimeException e) { + // We don't care which one was thrown... + assertEquals("fail!", e.getMessage()); + } + // ...but we do care that both were executed: + assertEquals(2, list.size()); + assertTrue(list.contains("bar")); + assertTrue(list.contains("spam")); + } + + @Test + public void testStepName() throws Exception { + assertEquals("step", context.getStepName()); + } + + @Test + public void testJobName() throws Exception { + assertEquals("job", context.getJobName()); + } + + @Test + public void testJobInstanceId() throws Exception { + assertEquals(2L, (long)context.getJobInstanceId()); + } + + @Test + public void testStepExecutionContext() throws Exception { + ExecutionContext executionContext = stepExecution.getExecutionContext(); + executionContext.put("foo", "bar"); + assertEquals("bar", context.getStepExecutionContext().get("foo")); + } + + @Test + public void testSystemProperties() throws Exception { + System.setProperty("foo", "bar"); + assertEquals("bar", context.getSystemProperties().getProperty("foo")); + } + + @Test + public void testJobExecutionContext() throws Exception { + ExecutionContext executionContext = stepExecution.getJobExecution().getExecutionContext(); + executionContext.put("foo", "bar"); + assertEquals("bar", context.getJobExecutionContext().get("foo")); + } + + @Test + public void testJobParameters() throws Exception { + JobParameters jobParameters = new JobParametersBuilder().addString("foo", "bar").toJobParameters(); + JobInstance instance = stepExecution.getJobExecution().getJobInstance(); + stepExecution = new StepExecution("step", new JobExecution(instance, jobParameters)); + context = new StepContext(stepExecution); + assertEquals("bar", context.getJobParameters().get("foo")); + } + + @Test + public void testContextId() throws Exception { + assertEquals("execution#1", context.getId()); + } + + @Test(expected = IllegalStateException.class) + public void testIllegalContextId() throws Exception { + context = new StepContext(new StepExecution("foo", new JobExecution(0L))); + context.getId(); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/StepSynchronizationManagerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/StepSynchronizationManagerTests.java index 188c40e3b4..aee4e105eb 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/StepSynchronizationManagerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/scope/context/StepSynchronizationManagerTests.java @@ -1,13 +1,26 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.scope.context; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertNotNull; import static org.junit.Assert.assertNull; -import java.lang.reflect.Field; import java.util.ArrayList; import java.util.List; -import java.util.Map; import java.util.concurrent.Callable; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; @@ -19,11 +32,12 @@ import org.junit.Test; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.StepExecution; -import org.springframework.util.ReflectionUtils; +import org.springframework.batch.core.jsr.configuration.support.BatchPropertyContext; public class StepSynchronizationManagerTests { private StepExecution stepExecution = new StepExecution("step", new JobExecution(0L)); + private BatchPropertyContext propertyContext = new BatchPropertyContext(); @Before @After @@ -40,9 +54,19 @@ public void testGetContext() { assertNotNull(StepSynchronizationManager.getContext()); } + @Test + public void testGetContextWithBatchProperties() { + StepContext context = StepSynchronizationManager.getContext(); + assertNull(context); + StepSynchronizationManager.register(stepExecution, propertyContext); + context = StepSynchronizationManager.getContext(); + assertNotNull(context); + assertEquals(stepExecution, context.getStepExecution()); + } + @Test public void testClose() throws Exception { - final List list = new ArrayList(); + final List list = new ArrayList<>(); StepContext context = StepSynchronizationManager.register(stepExecution); context.registerDestructionCallback("foo", new Runnable() { @Override @@ -53,23 +77,13 @@ public void run() { StepSynchronizationManager.close(); assertNull(StepSynchronizationManager.getContext()); assertEquals(0, list.size()); - // check for possible memory leak - assertEquals(0, extractStaticMap("counts").size()); - assertEquals(0, extractStaticMap("contexts").size()); - } - - private Map extractStaticMap(String name) throws IllegalAccessException { - Field field = ReflectionUtils.findField(StepSynchronizationManager.class, name); - ReflectionUtils.makeAccessible(field); - Map map = (Map) field.get(StepSynchronizationManager.class); - return map; } @Test public void testMultithreaded() throws Exception { StepContext context = StepSynchronizationManager.register(stepExecution); ExecutorService executorService = Executors.newFixedThreadPool(2); - FutureTask task = new FutureTask(new Callable() { + FutureTask task = new FutureTask<>(new Callable() { @Override public StepContext call() throws Exception { try { @@ -93,7 +107,7 @@ public StepContext call() throws Exception { @Test public void testRelease() { StepContext context = StepSynchronizationManager.register(stepExecution); - final List list = new ArrayList(); + final List list = new ArrayList<>(); context.registerDestructionCallback("foo", new Runnable() { @Override public void run() { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/JobRepositorySupport.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/JobRepositorySupport.java index 8ac2eaec54..6ef540fdc0 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/JobRepositorySupport.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/JobRepositorySupport.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.repository.JobRepository; +import org.springframework.lang.Nullable; /** * @author Dave Syer @@ -36,7 +37,7 @@ public class JobRepositorySupport implements JobRepository { @Override public JobExecution createJobExecution(String jobName, JobParameters jobParameters) { JobInstance jobInstance = new JobInstance(0L, jobName); - return new JobExecution(jobInstance, 11L, jobParameters); + return new JobExecution(jobInstance, 11L, jobParameters, null); } /* (non-Javadoc) @@ -52,6 +53,7 @@ public void update(JobExecution jobExecution) { public void update(JobInstance job) { } + @Nullable @Override public StepExecution getLastStepExecution(JobInstance jobInstance, String stepName) { return null; @@ -90,6 +92,7 @@ public boolean isJobInstanceExists(String jobName, JobParameters jobParameters) return false; } + @Nullable @Override public JobExecution getLastJobExecution(String jobName, JobParameters jobParameters) { return null; @@ -103,4 +106,15 @@ public void updateExecutionContext(JobExecution jobExecution) { public void addAll(Collection stepExecutions) { } + @Override + public JobInstance createJobInstance(String jobName, + JobParameters jobParameters) { + return null; + } + + @Override + public JobExecution createJobExecution(JobInstance jobInstance, + JobParameters jobParameters, String jobConfigurationLocation) { + return null; + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/NoSuchStepExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/NoSuchStepExceptionTests.java index b708eb62d8..47c272cbbb 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/NoSuchStepExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/NoSuchStepExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/NoWorkFoundStepExecutionListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/NoWorkFoundStepExecutionListenerTests.java index 3ab938ab77..85f456feb6 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/NoWorkFoundStepExecutionListenerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/NoWorkFoundStepExecutionListenerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/NonAbstractStepTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/NonAbstractStepTests.java index 468e630ca8..f6b81e7b85 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/NonAbstractStepTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/NonAbstractStepTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step; import static org.junit.Assert.assertEquals; @@ -18,6 +33,7 @@ import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.StepExecutionListener; import org.springframework.batch.item.ExecutionContext; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -36,7 +52,7 @@ public class NonAbstractStepTests { /** * Sequence of events encountered during step execution. */ - final List events = new ArrayList(); + final List events = new ArrayList<>(); final StepExecution execution = new StepExecution(tested.getName(), new JobExecution(new JobInstance(1L, "jobName"), new JobParameters())); @@ -83,6 +99,7 @@ private String getEvent(String event) { return name + "#" + event; } + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { assertSame(execution, stepExecution); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/RestartInPriorStepTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/RestartInPriorStepTests.java index 7aa91a73a2..1c071a3672 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/RestartInPriorStepTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/RestartInPriorStepTests.java @@ -1,10 +1,20 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step; -import static org.junit.Assert.assertEquals; - -import java.util.Map; - -import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.batch.core.BatchStatus; @@ -14,7 +24,6 @@ import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.StepExecution; -import org.springframework.batch.core.explore.JobExplorer; import org.springframework.batch.core.job.flow.FlowExecutionStatus; import org.springframework.batch.core.job.flow.JobExecutionDecider; import org.springframework.batch.core.launch.JobLauncher; @@ -23,9 +32,14 @@ import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.lang.Nullable; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import java.util.Map; + +import static org.junit.Assert.assertEquals; + /** * @author Michael Minella * @@ -37,35 +51,19 @@ public class RestartInPriorStepTests { @Autowired private JobRepository jobRepository; - @Autowired - private JobExplorer jobExplorer; - @Autowired private JobLauncher jobLauncher; @Autowired private Job job; - /** - * @throws java.lang.Exception - */ - @Before - public void setUp() throws Exception { - } - @Test public void test() throws Exception { - // - // Run 1 - // JobExecution run1 = jobLauncher.run(job, new JobParameters()); assertEquals(BatchStatus.STOPPED, run1.getStatus()); assertEquals(2, run1.getStepExecutions().size()); - // - // Run 2 - // JobExecution run2 = jobLauncher.run(job, new JobParameters()); assertEquals(BatchStatus.COMPLETED, run2.getStatus()); @@ -74,6 +72,7 @@ public void test() throws Exception { public static class DecidingTasklet implements Tasklet { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { @@ -96,7 +95,7 @@ public static class CompletionDecider implements JobExecutionDecider { @Override public FlowExecutionStatus decide(JobExecution jobExecution, - StepExecution stepExecution) { + @Nullable StepExecution stepExecution) { count++; if(count > 2) { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/RestartLoopTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/RestartLoopTests.java new file mode 100644 index 0000000000..6c497ec7ba --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/RestartLoopTests.java @@ -0,0 +1,69 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.step; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.lang.Nullable; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +import static org.junit.Assert.assertEquals; + +/** + * @author Michael Minella + */ +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class RestartLoopTests { + + @Autowired + private Job job; + + @Autowired + private JobLauncher jobLauncher; + + @Test + public void test() throws Exception { + // Run 1 + JobExecution jobExecution1 = jobLauncher.run(job, new JobParameters()); + + assertEquals(BatchStatus.STOPPED, jobExecution1.getStatus()); + + // Run 2 + JobExecution jobExecution2 = jobLauncher.run(job, new JobParameters()); + + assertEquals(BatchStatus.STOPPED, jobExecution2.getStatus()); + } + + public static class DefaultTasklet implements Tasklet { + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + return RepeatStatus.FINISHED; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/StepLocatorStepFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/StepLocatorStepFactoryBeanTests.java index 0e5b74734f..1989990aac 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/StepLocatorStepFactoryBeanTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/StepLocatorStepFactoryBeanTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/StepSupport.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/StepSupport.java index f36816aeba..d235745d1e 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/StepSupport.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/StepSupport.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/ThreadStepInterruptionPolicyTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/ThreadStepInterruptionPolicyTests.java index 0f1f04d9c4..e31b7b7764 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/ThreadStepInterruptionPolicyTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/ThreadStepInterruptionPolicyTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/builder/RegisterMultiListenerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/builder/RegisterMultiListenerTests.java new file mode 100644 index 0000000000..58dc347d04 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/builder/RegisterMultiListenerTests.java @@ -0,0 +1,329 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.step.builder; + +import java.util.List; +import javax.sql.DataSource; + +import org.junit.After; +import org.junit.Test; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ChunkListener; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.ItemWriteListener; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.PooledEmbeddedDataSource; +import org.springframework.batch.core.SkipListener; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.StepExecutionListener; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.NonTransientResourceException; +import org.springframework.batch.item.ParseException; +import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.config.AutowireCapableBeanFactory; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.support.GenericApplicationContext; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabase; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType; +import org.springframework.lang.Nullable; + +import static org.junit.Assert.assertEquals; + +/** + * Test for registering a listener class that implements different listeners interfaces + * just once in java based configuration. + * + * @author Tobias Flohre + * @author Michael Minella + */ +public class RegisterMultiListenerTests { + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private Job job; + + @Autowired + private CallChecker callChecker; + + @Autowired + private EmbeddedDatabase dataSource; + + private GenericApplicationContext context; + + @After + public void tearDown() { + jobLauncher = null; + job = null; + callChecker = null; + + if(context != null) { + context.close(); + } + } + + /** + * The times the beforeChunkCalled occurs are: + * - Before chunk 1 (item1, item2) + * - Before the re-attempt of item1 (scanning) + * - Before the re-attempt of item2 (scanning) + * - Before the checking that scanning is complete + * - Before chunk 2 (item3, item4) + * - Before chunk 3 (null) + * + * @throws Exception + */ + @Test + public void testMultiListenerFaultTolerantStep() throws Exception { + bootstrap(MultiListenerFaultTolerantTestConfiguration.class); + + JobExecution execution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(1, callChecker.beforeStepCalled); + assertEquals(6, callChecker.beforeChunkCalled); + assertEquals(2, callChecker.beforeWriteCalled); + assertEquals(1, callChecker.skipInWriteCalled); + } + + @Test + public void testMultiListenerSimpleStep() throws Exception { + bootstrap(MultiListenerTestConfiguration.class); + + JobExecution execution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.FAILED, execution.getStatus()); + assertEquals(1, callChecker.beforeStepCalled); + assertEquals(1, callChecker.beforeChunkCalled); + assertEquals(1, callChecker.beforeWriteCalled); + assertEquals(0, callChecker.skipInWriteCalled); + } + + private void bootstrap(Class configurationClass) { + context = new AnnotationConfigApplicationContext(configurationClass); + context.getAutowireCapableBeanFactory().autowireBeanProperties(this, AutowireCapableBeanFactory.AUTOWIRE_BY_TYPE, false); + } + + public static abstract class MultiListenerTestConfigurationSupport { + + @Autowired + protected JobBuilderFactory jobBuilders; + + @Autowired + protected StepBuilderFactory stepBuilders; + + @Bean + public Job testJob(){ + return jobBuilders.get("testJob") + .start(step()) + .build(); + } + + @Bean + public CallChecker callChecker(){ + return new CallChecker(); + } + + @Bean + public MultiListener listener(){ + return new MultiListener(callChecker()); + } + + @Bean + public ItemReader reader(){ + return new ItemReader(){ + + private int count = 0; + + @Nullable + @Override + public String read() throws Exception, + UnexpectedInputException, ParseException, + NonTransientResourceException { + count++; + + if(count < 5) { + return "item" + count; + } else { + return null; + } + } + + }; + } + + @Bean + public ItemWriter writer(){ + return new ItemWriter(){ + + @Override + public void write(List items) + throws Exception { + if(items.contains("item2")) { + throw new MySkippableException(); + } + } + + }; + } + + public abstract Step step(); + } + + @Configuration + @EnableBatchProcessing + public static class MultiListenerFaultTolerantTestConfiguration extends MultiListenerTestConfigurationSupport{ + + @Bean + public DataSource dataSource(){ + return new PooledEmbeddedDataSource(new EmbeddedDatabaseBuilder() + .addScript("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql") + .addScript("classpath:org/springframework/batch/core/schema-hsqldb.sql") + .setType(EmbeddedDatabaseType.HSQL) + .build()); + } + + @Override + @Bean + public Step step(){ + return stepBuilders.get("step") + .listener(listener()) + .chunk(2) + .reader(reader()) + .writer(writer()) + .faultTolerant() + .skipLimit(1) + .skip(MySkippableException.class) + // ChunkListener registered twice for checking BATCH-2149 + .listener((ChunkListener) listener()) + .build(); + } + } + + @Configuration + @EnableBatchProcessing + public static class MultiListenerTestConfiguration extends MultiListenerTestConfigurationSupport{ + + @Bean + public DataSource dataSource(){ + return new PooledEmbeddedDataSource(new EmbeddedDatabaseBuilder() + .addScript("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql") + .addScript("classpath:org/springframework/batch/core/schema-hsqldb.sql") + .setType(EmbeddedDatabaseType.HSQL) + .build()); + } + + @Override + @Bean + public Step step(){ + return stepBuilders.get("step") + .listener(listener()) + .chunk(2) + .reader(reader()) + .writer(writer()) + .build(); + } + } + + private static class CallChecker { + int beforeStepCalled = 0; + int beforeChunkCalled = 0; + int beforeWriteCalled = 0; + int skipInWriteCalled = 0; + } + + private static class MultiListener implements StepExecutionListener, ChunkListener, ItemWriteListener, SkipListener{ + + private CallChecker callChecker; + + private MultiListener(CallChecker callChecker) { + super(); + this.callChecker = callChecker; + } + + @Override + public void onSkipInRead(Throwable t) { + } + + @Override + public void onSkipInWrite(String item, Throwable t) { + callChecker.skipInWriteCalled++; + } + + @Override + public void onSkipInProcess(String item, Throwable t) { + } + + @Override + public void beforeWrite(List items) { + callChecker.beforeWriteCalled++; + } + + @Override + public void afterWrite(List items) { + } + + @Override + public void onWriteError(Exception exception, + List items) { + } + + @Override + public void beforeChunk(ChunkContext context) { + callChecker.beforeChunkCalled++; + } + + @Override + public void afterChunk(ChunkContext context) { + } + + @Override + public void afterChunkError(ChunkContext context) { + } + + @Override + public void beforeStep(StepExecution stepExecution) { + callChecker.beforeStepCalled++; + } + + @Nullable + @Override + public ExitStatus afterStep(StepExecution stepExecution) { + return null; + } + + } + + private static class MySkippableException extends RuntimeException{ + + private static final long serialVersionUID = 1L; + + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/builder/StepBuilderTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/builder/StepBuilderTests.java index a5ebfca8a5..81f2f073d0 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/builder/StepBuilderTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/builder/StepBuilderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,40 +15,360 @@ */ package org.springframework.batch.core.step.builder; +import java.util.ArrayList; +import java.util.List; +import java.util.function.Function; + import org.junit.Test; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.JobParameters; -import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.StepExecutionListener; +import org.springframework.batch.core.annotation.AfterChunk; +import org.springframework.batch.core.annotation.AfterChunkError; +import org.springframework.batch.core.annotation.AfterProcess; +import org.springframework.batch.core.annotation.AfterRead; +import org.springframework.batch.core.annotation.AfterStep; +import org.springframework.batch.core.annotation.AfterWrite; +import org.springframework.batch.core.annotation.BeforeChunk; +import org.springframework.batch.core.annotation.BeforeProcess; +import org.springframework.batch.core.annotation.BeforeRead; +import org.springframework.batch.core.annotation.BeforeStep; +import org.springframework.batch.core.annotation.BeforeWrite; +import org.springframework.batch.core.configuration.xml.DummyItemReader; +import org.springframework.batch.core.configuration.xml.DummyItemWriter; +import org.springframework.batch.core.job.SimpleJob; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; -import org.springframework.batch.core.scope.context.ChunkContext; -import org.springframework.batch.core.step.tasklet.Tasklet; -import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.support.ListItemReader; +import org.springframework.batch.item.support.ListItemWriter; +import org.springframework.batch.item.support.PassThroughItemProcessor; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; +import org.springframework.lang.Nullable; import org.springframework.transaction.PlatformTransactionManager; +import static org.junit.Assert.assertEquals; + /** * @author Dave Syer + * @author Michael Minella + * @author Mahmoud Ben Hassine * */ +@SuppressWarnings("serial") public class StepBuilderTests { @Test public void test() throws Exception { - JobRepository jobRepository = new MapJobRepositoryFactoryBean().getJobRepository(); + JobRepository jobRepository = new MapJobRepositoryFactoryBean().getObject(); StepExecution execution = jobRepository.createJobExecution("foo", new JobParameters()).createStepExecution( "step"); jobRepository.add(execution); PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); TaskletStepBuilder builder = new StepBuilder("step").repository(jobRepository) - .transactionManager(transactionManager).tasklet(new Tasklet() { - @Override - public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) - throws Exception { - return null; - } - }); + .transactionManager(transactionManager).tasklet((contribution, chunkContext) -> null); + builder.build().execute(execution); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + } + + @Test + public void testListeners() throws Exception { + JobRepository jobRepository = new MapJobRepositoryFactoryBean().getObject(); + StepExecution execution = jobRepository.createJobExecution("foo", new JobParameters()).createStepExecution("step"); + jobRepository.add(execution); + PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); + TaskletStepBuilder builder = new StepBuilder("step") + .repository(jobRepository) + .transactionManager(transactionManager) + .listener(new InterfaceBasedStepExecutionListener()) + .listener(new AnnotationBasedStepExecutionListener()) + .tasklet((contribution, chunkContext) -> null); + builder.build().execute(execution); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(1, InterfaceBasedStepExecutionListener.beforeStepCount); + assertEquals(1, InterfaceBasedStepExecutionListener.afterStepCount); + assertEquals(1, AnnotationBasedStepExecutionListener.beforeStepCount); + assertEquals(1, AnnotationBasedStepExecutionListener.afterStepCount); + assertEquals(1, AnnotationBasedStepExecutionListener.beforeChunkCount); + assertEquals(1, AnnotationBasedStepExecutionListener.afterChunkCount); + } + + @Test + public void testAnnotationBasedChunkListenerForTaskletStep() throws Exception { + JobRepository jobRepository = new MapJobRepositoryFactoryBean().getObject(); + StepExecution execution = jobRepository.createJobExecution("foo", new JobParameters()).createStepExecution("step"); + jobRepository.add(execution); + PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); + TaskletStepBuilder builder = new StepBuilder("step") + .repository(jobRepository) + .transactionManager(transactionManager) + .tasklet((contribution, chunkContext) -> null) + .listener(new AnnotationBasedChunkListener()); + builder.build().execute(execution); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(1, AnnotationBasedChunkListener.beforeChunkCount); + assertEquals(1, AnnotationBasedChunkListener.afterChunkCount); + } + + @Test + public void testAnnotationBasedChunkListenerForSimpleTaskletStep() throws Exception { + JobRepository jobRepository = new MapJobRepositoryFactoryBean().getObject(); + StepExecution execution = jobRepository.createJobExecution("foo", new JobParameters()).createStepExecution("step"); + jobRepository.add(execution); + PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); + SimpleStepBuilder builder = new StepBuilder("step") + .repository(jobRepository) + .transactionManager(transactionManager) + .chunk(5) + .reader(new DummyItemReader()) + .writer(new DummyItemWriter()) + .listener(new AnnotationBasedChunkListener()); + builder.build().execute(execution); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(1, AnnotationBasedChunkListener.beforeChunkCount); + assertEquals(1, AnnotationBasedChunkListener.afterChunkCount); + } + + @Test + public void testAnnotationBasedChunkListenerForFaultTolerantTaskletStep() throws Exception { + JobRepository jobRepository = new MapJobRepositoryFactoryBean().getObject(); + StepExecution execution = jobRepository.createJobExecution("foo", new JobParameters()).createStepExecution("step"); + jobRepository.add(execution); + PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); + SimpleStepBuilder builder = new StepBuilder("step") + .repository(jobRepository) + .transactionManager(transactionManager) + .chunk(5) + .reader(new DummyItemReader()) + .writer(new DummyItemWriter()) + .faultTolerant() + .listener(new AnnotationBasedChunkListener()); // TODO should this return FaultTolerantStepBuilder? + builder.build().execute(execution); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(1, AnnotationBasedChunkListener.beforeChunkCount); + assertEquals(1, AnnotationBasedChunkListener.afterChunkCount); + } + + @Test + public void testAnnotationBasedChunkListenerForJobStepBuilder() throws Exception { + JobRepository jobRepository = new MapJobRepositoryFactoryBean().getObject(); + StepExecution execution = jobRepository.createJobExecution("foo", new JobParameters()).createStepExecution("step"); + jobRepository.add(execution); + PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); + SimpleJob job = new SimpleJob("job"); + job.setJobRepository(jobRepository); + JobStepBuilder builder = new StepBuilder("step") + .repository(jobRepository) + .transactionManager(transactionManager) + .job(job) + .listener(new AnnotationBasedChunkListener()); + builder.build().execute(execution); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + + // it makes no sense to register a ChunkListener on a step which is not of type tasklet, so it should not be invoked + assertEquals(0, AnnotationBasedChunkListener.beforeChunkCount); + assertEquals(0, AnnotationBasedChunkListener.afterChunkCount); + } + + @Test + public void testItemListeners() throws Exception { + JobRepository jobRepository = new MapJobRepositoryFactoryBean().getObject(); + StepExecution execution = jobRepository.createJobExecution("foo", new JobParameters()).createStepExecution("step"); + jobRepository.add(execution); + PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); + + List items = new ArrayList() {{ + add("1"); + add("2"); + add("3"); + }}; + + ItemReader reader = new ListItemReader<>(items); + + @SuppressWarnings("unchecked") + SimpleStepBuilder builder = new StepBuilder("step") + .repository(jobRepository) + .transactionManager(transactionManager) + .chunk(3) + .reader(reader) + .processor(new PassThroughItemProcessor<>()) + .writer(new DummyItemWriter()) + .listener(new AnnotationBasedStepExecutionListener()); + builder.build().execute(execution); + + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + assertEquals(1, AnnotationBasedStepExecutionListener.beforeStepCount); + assertEquals(1, AnnotationBasedStepExecutionListener.afterStepCount); + assertEquals(4, AnnotationBasedStepExecutionListener.beforeReadCount); + assertEquals(3, AnnotationBasedStepExecutionListener.afterReadCount); + assertEquals(3, AnnotationBasedStepExecutionListener.beforeProcessCount); + assertEquals(3, AnnotationBasedStepExecutionListener.afterProcessCount); + assertEquals(1, AnnotationBasedStepExecutionListener.beforeWriteCount); + assertEquals(1, AnnotationBasedStepExecutionListener.afterWriteCount); + assertEquals(2, AnnotationBasedStepExecutionListener.beforeChunkCount); + assertEquals(2, AnnotationBasedStepExecutionListener.afterChunkCount); + } + + @Test + public void testFunctions() throws Exception { + JobRepository jobRepository = new MapJobRepositoryFactoryBean().getObject(); + StepExecution execution = jobRepository.createJobExecution("foo", new JobParameters()).createStepExecution("step"); + jobRepository.add(execution); + PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); + + List items = new ArrayList() {{ + add(1L); + add(2L); + add(3L); + }}; + + ItemReader reader = new ListItemReader<>(items); + + ListItemWriter itemWriter = new ListItemWriter<>(); + @SuppressWarnings("unchecked") + SimpleStepBuilder builder = new StepBuilder("step") + .repository(jobRepository) + .transactionManager(transactionManager) + .chunk(3) + .reader(reader) + .processor((Function) s -> s.toString()) + .writer(itemWriter) + .listener(new AnnotationBasedStepExecutionListener()); builder.build().execute(execution); + + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + + List writtenItems = itemWriter.getWrittenItems(); + assertEquals("1", writtenItems.get(0)); + assertEquals("2", writtenItems.get(1)); + assertEquals("3", writtenItems.get(2)); + } + + public static class InterfaceBasedStepExecutionListener implements StepExecutionListener { + + static int beforeStepCount = 0; + static int afterStepCount = 0; + + @Override + public void beforeStep(StepExecution stepExecution) { + beforeStepCount++; + } + + @Nullable + @Override + public ExitStatus afterStep(StepExecution stepExecution) { + afterStepCount++; + return stepExecution.getExitStatus(); + } } + @SuppressWarnings("unused") + public static class AnnotationBasedStepExecutionListener { + + static int beforeStepCount = 0; + static int afterStepCount = 0; + static int beforeReadCount = 0; + static int afterReadCount = 0; + static int beforeProcessCount = 0; + static int afterProcessCount = 0; + static int beforeWriteCount = 0; + static int afterWriteCount = 0; + static int beforeChunkCount = 0; + static int afterChunkCount = 0; + + public AnnotationBasedStepExecutionListener() { + beforeStepCount = 0; + afterStepCount = 0; + beforeReadCount = 0; + afterReadCount = 0; + beforeProcessCount = 0; + afterProcessCount = 0; + beforeWriteCount = 0; + afterWriteCount = 0; + beforeChunkCount = 0; + afterChunkCount = 0; + } + + @BeforeStep + public void beforeStep() { + beforeStepCount++; + } + + @AfterStep + public ExitStatus afterStep(StepExecution stepExecution) { + afterStepCount++; + return stepExecution.getExitStatus(); + } + + @BeforeRead + public void beforeRead() { + beforeReadCount++; + } + + @AfterRead + public void afterRead() { + afterReadCount++; + } + + @BeforeProcess + public void beforeProcess() { + beforeProcessCount++; + } + + @AfterProcess + public void afterProcess() { + afterProcessCount++; + } + + @BeforeWrite + public void beforeWrite() { + beforeWriteCount++; + } + + @AfterWrite + public void setAfterWrite() { + afterWriteCount++; + } + + @BeforeChunk + public void beforeChunk() { + beforeChunkCount++; + } + + @AfterChunk + public void afterChunk() { + afterChunkCount++; + } + } + + public static class AnnotationBasedChunkListener { + + static int beforeChunkCount = 0; + static int afterChunkCount = 0; + static int afterChunkErrorCount = 0; + + public AnnotationBasedChunkListener() { + beforeChunkCount = 0; + afterChunkCount = 0; + afterChunkErrorCount = 0; + } + + @BeforeChunk + public void beforeChunk() { + beforeChunkCount++; + } + + @AfterChunk + public void afterChunk() { + afterChunkCount++; + } + + @AfterChunkError + public void afterChunkError() { + afterChunkErrorCount++; + } + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/AbstractExceptionThrowingItemHandlerStub.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/AbstractExceptionThrowingItemHandlerStub.java index ae3672fe8d..7c70f91142 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/AbstractExceptionThrowingItemHandlerStub.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/AbstractExceptionThrowingItemHandlerStub.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,8 +40,9 @@ public AbstractExceptionThrowingItemHandlerStub() throws Exception { exception = SkippableRuntimeException.class.getConstructor(String.class); } + @SuppressWarnings("unchecked") public void setFailures(T... failures) { - this.failures = new ArrayList(Arrays.asList(failures)); + this.failures = new ArrayList<>(Arrays.asList(failures)); } public void setExceptionType(Class exceptionType) throws Exception { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/AlmostStatefulRetryChunkTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/AlmostStatefulRetryChunkTests.java index 26b62fb17c..aa55f2009c 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/AlmostStatefulRetryChunkTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/AlmostStatefulRetryChunkTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,10 +15,6 @@ */ package org.springframework.batch.core.step.item; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - import java.util.ArrayList; import java.util.Collections; import java.util.List; @@ -30,6 +26,10 @@ import org.junit.runners.Parameterized; import org.junit.runners.Parameterized.Parameters; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + /** * @author Dave Syer * @@ -50,7 +50,7 @@ public class AlmostStatefulRetryChunkTests { private int count = 0; public AlmostStatefulRetryChunkTests(String[] args, int limit) { - chunk = new Chunk(); + chunk = new Chunk<>(); for (String string : args) { chunk.add(string); } @@ -60,7 +60,7 @@ public AlmostStatefulRetryChunkTests(String[] args, int limit) { @Test public void testRetry() throws Exception { logger.debug("Starting simple scenario"); - List items = new ArrayList(chunk.getItems()); + List items = new ArrayList<>(chunk.getItems()); int before = items.size(); items.removeAll(Collections.singleton("fail")); boolean error = true; @@ -81,8 +81,7 @@ public void testRetry() throws Exception { } /** - * @param chunk - * @throws Exception + * @param chunk Chunk to retry */ private void statefulRetry(Chunk chunk) throws Exception { if (retryAttempts <= retryLimit) { @@ -113,8 +112,7 @@ private void statefulRetry(Chunk chunk) throws Exception { } /** - * @param chunk - * @throws Exception + * @param chunk Chunk to recover */ private void recover(Chunk chunk) throws Exception { for (Chunk.ChunkIterator iterator = chunk.iterator(); iterator.hasNext();) { @@ -129,8 +127,7 @@ private void recover(Chunk chunk) throws Exception { } /** - * @param chunk - * @throws Exception + * @param items items to write */ private void doWrite(List items) throws Exception { if (items.contains("fail")) { @@ -140,7 +137,7 @@ private void doWrite(List items) throws Exception { @Parameters public static List data() { - List params = new ArrayList(); + List params = new ArrayList<>(); params.add(new Object[] { new String[] { "foo" }, 0 }); params.add(new Object[] { new String[] { "foo", "bar" }, 0 }); params.add(new Object[] { new String[] { "foo", "bar", "spam" }, 0 }); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/BatchRetryTemplateTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/BatchRetryTemplateTests.java index 31a8e34d6b..0550061a3e 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/BatchRetryTemplateTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/BatchRetryTemplateTests.java @@ -1,14 +1,20 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.item; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - -import java.util.ArrayList; -import java.util.Arrays; -import java.util.Collections; -import java.util.List; - import org.junit.Test; import org.springframework.retry.ExhaustedRetryException; import org.springframework.retry.RecoveryCallback; @@ -18,6 +24,15 @@ import org.springframework.retry.policy.SimpleRetryPolicy; import org.springframework.retry.support.DefaultRetryState; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collections; +import java.util.List; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + public class BatchRetryTemplateTests { @SuppressWarnings("serial") @@ -31,14 +46,14 @@ public RecoverableException(String message) { private int count = 0; - private List outputs = new ArrayList(); + private List outputs = new ArrayList<>(); @Test public void testSuccessfulAttempt() throws Exception { BatchRetryTemplate template = new BatchRetryTemplate(); - String result = template.execute(new RetryCallback() { + String result = template.execute(new RetryCallback() { @Override public String doWithRetry(RetryContext context) throws Exception { assertTrue("Wrong context type: " + context.getClass().getSimpleName(), context.getClass() @@ -56,7 +71,7 @@ public void testUnSuccessfulAttemptAndRetry() throws Exception { BatchRetryTemplate template = new BatchRetryTemplate(); - RetryCallback retryCallback = new RetryCallback() { + RetryCallback retryCallback = new RetryCallback() { @Override public String[] doWithRetry(RetryContext context) throws Exception { assertEquals(count, context.getRetryCount()); @@ -88,7 +103,7 @@ public void testExhaustedRetry() throws Exception { template.setRetryPolicy(new SimpleRetryPolicy(1, Collections ., Boolean> singletonMap(Exception.class, true))); - RetryCallback retryCallback = new RetryCallback() { + RetryCallback retryCallback = new RetryCallback() { @Override public String[] doWithRetry(RetryContext context) throws Exception { if (count++ < 2) { @@ -118,7 +133,7 @@ public void testExhaustedRetryAfterShuffle() throws Exception { template.setRetryPolicy(new SimpleRetryPolicy(1, Collections ., Boolean> singletonMap(Exception.class, true))); - RetryCallback retryCallback = new RetryCallback() { + RetryCallback retryCallback = new RetryCallback() { @Override public String[] doWithRetry(RetryContext context) throws Exception { if (count++ < 1) { @@ -173,7 +188,7 @@ public void testExhaustedRetryWithRecovery() throws Exception { template.setRetryPolicy(new SimpleRetryPolicy(1, Collections ., Boolean> singletonMap(Exception.class, true))); - RetryCallback retryCallback = new RetryCallback() { + RetryCallback retryCallback = new RetryCallback() { @Override public String[] doWithRetry(RetryContext context) throws Exception { if (count++ < 2) { @@ -186,7 +201,7 @@ public String[] doWithRetry(RetryContext context) throws Exception { RecoveryCallback recoveryCallback = new RecoveryCallback() { @Override public String[] recover(RetryContext context) throws Exception { - List recovered = new ArrayList(); + List recovered = new ArrayList<>(); for (String item : outputs) { recovered.add("r:" + item); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ChunkMonitorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ChunkMonitorTests.java index 4b8edf3cc8..bf9ed43ab1 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ChunkMonitorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ChunkMonitorTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,6 +26,7 @@ import org.springframework.batch.item.ItemStreamSupport; import org.springframework.batch.item.ParseException; import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.lang.Nullable; /** * @author Dave Syer @@ -33,9 +34,6 @@ */ public class ChunkMonitorTests { - /** - * - */ private static final int CHUNK_SIZE = 5; private ChunkMonitor monitor = new ChunkMonitor(); @@ -47,6 +45,7 @@ public class ChunkMonitorTests { @Before public void setUp() { monitor.setItemReader(new ItemReader() { + @Nullable @Override public String read() throws Exception, UnexpectedInputException, ParseException { return "" + (count++); @@ -55,7 +54,7 @@ public String read() throws Exception, UnexpectedInputException, ParseException monitor.registerItemStream(new ItemStreamSupport() { @Override public void close() { - super.close(); + super.close(); closed = true; } }); @@ -98,6 +97,7 @@ public void testOpen() { executionContext.putInt(ChunkMonitor.class.getName() + ".OFFSET", 2); monitor.open(executionContext); assertEquals(2, count); + assertEquals(0, monitor.getOffset()); } @Test @@ -111,6 +111,7 @@ public void testOpenWithNullReader() { @Test(expected = ItemStreamException.class) public void testOpenWithErrorInReader() { monitor.setItemReader(new ItemReader() { + @Nullable @Override public String read() throws Exception, UnexpectedInputException, ParseException { throw new IllegalStateException("Expected"); @@ -127,6 +128,10 @@ public void testUpdateOnBoundary() { ExecutionContext executionContext = new ExecutionContext(); monitor.update(executionContext); assertEquals(0, executionContext.size()); + + executionContext.put(ChunkMonitor.class.getName() + ".OFFSET", 3); + monitor.update(executionContext); + assertEquals(0, executionContext.size()); } @Test @@ -141,6 +146,7 @@ public void testUpdateVanilla() { public void testUpdateWithNoStream() throws Exception { monitor = new ChunkMonitor(); monitor.setItemReader(new ItemReader() { + @Nullable @Override public String read() throws Exception, UnexpectedInputException, ParseException { return "" + (count++); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ChunkOrientedTaskletTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ChunkOrientedTaskletTests.java index 0e5c3e6ca1..e20958759b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ChunkOrientedTaskletTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ChunkOrientedTaskletTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -37,16 +37,16 @@ public class ChunkOrientedTaskletTests { @Test public void testHandle() throws Exception { - ChunkOrientedTasklet handler = new ChunkOrientedTasklet(new ChunkProvider() { + ChunkOrientedTasklet handler = new ChunkOrientedTasklet<>(new ChunkProvider() { @Override public Chunk provide(StepContribution contribution) throws Exception { contribution.incrementReadCount(); - Chunk chunk = new Chunk(); + Chunk chunk = new Chunk<>(); chunk.add("foo"); return chunk; } @Override - public void postProcess(StepContribution contribution, Chunk chunk) {}; + public void postProcess(StepContribution contribution, Chunk chunk) {} }, new ChunkProcessor() { @Override public void process(StepContribution contribution, Chunk chunk) { @@ -63,13 +63,13 @@ public void process(StepContribution contribution, Chunk chunk) { @Test public void testFail() throws Exception { - ChunkOrientedTasklet handler = new ChunkOrientedTasklet(new ChunkProvider() { + ChunkOrientedTasklet handler = new ChunkOrientedTasklet<>(new ChunkProvider() { @Override public Chunk provide(StepContribution contribution) throws Exception { throw new RuntimeException("Foo!"); } @Override - public void postProcess(StepContribution contribution, Chunk chunk) {}; + public void postProcess(StepContribution contribution, Chunk chunk) {} }, new ChunkProcessor() { @Override public void process(StepContribution contribution, Chunk chunk) { @@ -90,17 +90,17 @@ public void process(StepContribution contribution, Chunk chunk) { @Test public void testExitCode() throws Exception { - ChunkOrientedTasklet handler = new ChunkOrientedTasklet(new ChunkProvider() { + ChunkOrientedTasklet handler = new ChunkOrientedTasklet<>(new ChunkProvider() { @Override public Chunk provide(StepContribution contribution) throws Exception { contribution.incrementReadCount(); - Chunk chunk = new Chunk(); + Chunk chunk = new Chunk<>(); chunk.add("foo"); chunk.setEnd(); return chunk; } @Override - public void postProcess(StepContribution contribution, Chunk chunk) {}; + public void postProcess(StepContribution contribution, Chunk chunk) {} }, new ChunkProcessor() { @Override public void process(StepContribution contribution, Chunk chunk) { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ExceptionThrowingTaskletStub.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ExceptionThrowingTaskletStub.java index c0fc4550a0..89bdfbb3de 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ExceptionThrowingTaskletStub.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ExceptionThrowingTaskletStub.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,6 +25,7 @@ import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; import org.springframework.batch.support.transaction.TransactionAwareProxyFactory; +import org.springframework.lang.Nullable; /** * @author Dan Garrette @@ -56,6 +57,7 @@ public void clear() { committed.clear(); } + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { committed.add(1); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FatalRuntimeException.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FatalRuntimeException.java index 7307321429..6bb16646ba 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FatalRuntimeException.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FatalRuntimeException.java @@ -1,9 +1,25 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.item; /** * @author Dan Garrette * @since 2.0.2 */ +@SuppressWarnings("serial") public class FatalRuntimeException extends SkippableRuntimeException { public FatalRuntimeException(String message) { super(message); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FatalSkippableException.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FatalSkippableException.java index d43a2b69e3..7eea1d7cdd 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FatalSkippableException.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FatalSkippableException.java @@ -1,9 +1,25 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.item; /** * @author Dan Garrette * @since 2.0.2 */ +@SuppressWarnings("serial") public class FatalSkippableException extends SkippableException { public FatalSkippableException(String message) { super(message); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantChunkProcessorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantChunkProcessorTests.java index b4e7d7b665..7f55025132 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantChunkProcessorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantChunkProcessorTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.item; import static org.junit.Assert.assertEquals; @@ -22,6 +37,7 @@ import org.springframework.batch.item.support.PassThroughItemProcessor; import org.springframework.classify.BinaryExceptionClassifier; import org.springframework.dao.DataIntegrityViolationException; +import org.springframework.lang.Nullable; import org.springframework.retry.RetryException; import org.springframework.retry.policy.NeverRetryPolicy; import org.springframework.retry.policy.SimpleRetryPolicy; @@ -30,11 +46,11 @@ public class FaultTolerantChunkProcessorTests { private BatchRetryTemplate batchRetryTemplate; - private List list = new ArrayList(); + private List list = new ArrayList<>(); - private List after = new ArrayList(); + private List after = new ArrayList<>(); - private List writeError = new ArrayList(); + private List writeError = new ArrayList<>(); private FaultTolerantChunkProcessor processor; @@ -44,8 +60,8 @@ public class FaultTolerantChunkProcessorTests { @Before public void setUp() { batchRetryTemplate = new BatchRetryTemplate(); - processor = new FaultTolerantChunkProcessor( - new PassThroughItemProcessor(), + processor = new FaultTolerantChunkProcessor<>( + new PassThroughItemProcessor<>(), new ItemWriter() { @Override public void write(List items) @@ -61,7 +77,7 @@ public void write(List items) @Test public void testWrite() throws Exception { - Chunk inputs = new Chunk(Arrays.asList("1", "2")); + Chunk inputs = new Chunk<>(Arrays.asList("1", "2")); processor.process(contribution, inputs); assertEquals(2, list.size()); } @@ -69,12 +85,13 @@ public void testWrite() throws Exception { @Test public void testTransform() throws Exception { processor.setItemProcessor(new ItemProcessor() { + @Nullable @Override public String process(String item) throws Exception { return item.equals("1") ? null : item; } }); - Chunk inputs = new Chunk(Arrays.asList("1", "2")); + Chunk inputs = new Chunk<>(Arrays.asList("1", "2")); processor.process(contribution, inputs); assertEquals(1, list.size()); assertEquals(1, contribution.getFilterCount()); @@ -84,6 +101,7 @@ public String process(String item) throws Exception { public void testFilterCountOnSkip() throws Exception { processor.setProcessSkipPolicy(new AlwaysSkipItemSkipPolicy()); processor.setItemProcessor(new ItemProcessor() { + @Nullable @Override public String process(String item) throws Exception { if (item.equals("1")) { @@ -95,7 +113,7 @@ public String process(String item) throws Exception { return item; } }); - Chunk inputs = new Chunk(Arrays.asList("3", "1", "2")); + Chunk inputs = new Chunk<>(Arrays.asList("3", "1", "2")); try { processor.process(contribution, inputs); fail("Expected Exception"); @@ -107,6 +125,63 @@ public String process(String item) throws Exception { assertEquals(1, contribution.getSkipCount()); assertEquals(1, contribution.getFilterCount()); } + + @Test + // BATCH-2663 + public void testFilterCountOnSkipInWriteWithoutRetry() throws Exception { + processor.setWriteSkipPolicy(new AlwaysSkipItemSkipPolicy()); + processor.setItemProcessor(new ItemProcessor() { + @Nullable + @Override + public String process(String item) throws Exception { + if (item.equals("1")) { + return null; + } + return item; + } + }); + Chunk inputs = new Chunk<>(Arrays.asList("fail", "1", "2")); + processAndExpectPlannedRuntimeException(inputs); // (first attempt) Process fail, 1, 2 + // item 1 is filtered out so it is removed from the chunk => now inputs = [fail, 2] + // using NeverRetryPolicy by default => now scanning + processAndExpectPlannedRuntimeException(inputs); // (scanning) Process fail + processor.process(contribution, inputs); // (scanning) Process 2 + assertEquals(1, list.size()); + assertEquals("[2]", list.toString()); + assertEquals(1, contribution.getWriteSkipCount()); + assertEquals(1, contribution.getFilterCount()); + } + + @Test + // BATCH-2663 + public void testFilterCountOnSkipInWriteWithRetry() throws Exception { + SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy(); + retryPolicy.setMaxAttempts(3); + batchRetryTemplate.setRetryPolicy(retryPolicy); + processor.setWriteSkipPolicy(new AlwaysSkipItemSkipPolicy()); + processor.setItemProcessor(new ItemProcessor() { + @Nullable + @Override + public String process(String item) throws Exception { + if (item.equals("1")) { + return null; + } + return item; + } + }); + Chunk inputs = new Chunk<>(Arrays.asList("fail", "1", "2")); + processAndExpectPlannedRuntimeException(inputs); // (first attempt) Process fail, 1, 2 + // item 1 is filtered out so it is removed from the chunk => now inputs = [fail, 2] + processAndExpectPlannedRuntimeException(inputs); // (first retry) Process fail, 2 + processAndExpectPlannedRuntimeException(inputs); // (second retry) Process fail, 2 + // retry exhausted (maxAttempts = 3) => now scanning + processAndExpectPlannedRuntimeException(inputs); // (scanning) Process fail + processor.process(contribution, inputs); // (scanning) Process 2 + assertEquals(1, list.size()); + assertEquals("[2]", list.toString()); + assertEquals(1, contribution.getWriteSkipCount()); + assertEquals(3, contribution.getFilterCount()); + } /** * An Error can be retried or skipped but by default it is just propagated @@ -124,7 +199,7 @@ public void write(List items) throws Exception { } } }); - Chunk inputs = new Chunk( + Chunk inputs = new Chunk<>( Arrays.asList("3", "fail", "2")); try { processor.process(contribution, inputs); @@ -146,7 +221,7 @@ public void write(List items) throws Exception { } } }); - Chunk inputs = new Chunk( + Chunk inputs = new Chunk<>( Arrays.asList("3", "fail", "2")); try { processor.process(contribution, inputs); @@ -177,7 +252,7 @@ public void write(List items) throws Exception { } } }); - Chunk inputs = new Chunk(Arrays.asList("fail")); + Chunk inputs = new Chunk<>(Arrays.asList("fail")); try { processor.process(contribution, inputs); fail("Expected RuntimeException"); @@ -201,6 +276,7 @@ public void write(List items) throws Exception { @Test public void testTransformWithExceptionAndNoRollback() throws Exception { processor.setItemProcessor(new ItemProcessor() { + @Nullable @Override public String process(String item) throws Exception { if (item.equals("1")) { @@ -215,14 +291,14 @@ public String process(String item) throws Exception { Collections .> singleton(DataIntegrityViolationException.class), false)); - Chunk inputs = new Chunk(Arrays.asList("1", "2")); + Chunk inputs = new Chunk<>(Arrays.asList("1", "2")); processor.process(contribution, inputs); assertEquals(1, list.size()); } @Test public void testAfterWrite() throws Exception { - Chunk chunk = new Chunk(Arrays.asList("foo", "fail", + Chunk chunk = new Chunk<>(Arrays.asList("foo", "fail", "bar")); processor.setListeners(Arrays .asList(new ItemListenerSupport() { @@ -249,9 +325,9 @@ public void afterWrite(List item) { @Test public void testAfterWriteAllPassedInRecovery() throws Exception { - Chunk chunk = new Chunk(Arrays.asList("foo", "bar")); - processor = new FaultTolerantChunkProcessor( - new PassThroughItemProcessor(), + Chunk chunk = new Chunk<>(Arrays.asList("foo", "bar")); + processor = new FaultTolerantChunkProcessor<>( + new PassThroughItemProcessor<>(), new ItemWriter() { @Override public void write(List items) @@ -282,7 +358,7 @@ public void afterWrite(List item) { @Test public void testOnErrorInWrite() throws Exception { - Chunk chunk = new Chunk(Arrays.asList("foo", "fail")); + Chunk chunk = new Chunk<>(Arrays.asList("foo", "fail")); processor.setListeners(Arrays .asList(new ItemListenerSupport() { @Override @@ -294,8 +370,7 @@ public void onWriteError(Exception e, processor.setWriteSkipPolicy(new AlwaysSkipItemSkipPolicy()); processAndExpectPlannedRuntimeException(chunk);// Process foo, fail - processor.process(contribution, chunk); - ;// Process foo + processor.process(contribution, chunk);// Process foo processAndExpectPlannedRuntimeException(chunk);// Process fail assertEquals("[foo, fail, fail]", writeError.toString()); @@ -303,9 +378,9 @@ public void onWriteError(Exception e, @Test public void testOnErrorInWriteAllItemsFail() throws Exception { - Chunk chunk = new Chunk(Arrays.asList("foo", "bar")); - processor = new FaultTolerantChunkProcessor( - new PassThroughItemProcessor(), + Chunk chunk = new Chunk<>(Arrays.asList("foo", "bar")); + processor = new FaultTolerantChunkProcessor<>( + new PassThroughItemProcessor<>(), new ItemWriter() { @Override public void write(List items) @@ -345,7 +420,7 @@ public void write(List items) throws Exception { } } }); - Chunk inputs = new Chunk( + Chunk inputs = new Chunk<>( Arrays.asList("3", "fail", "2")); try { processor.process(contribution, inputs); @@ -390,7 +465,7 @@ public void write(List items) throws Exception { } } }); - Chunk inputs = new Chunk(Arrays.asList("3", "fail", + Chunk inputs = new Chunk<>(Arrays.asList("3", "fail", "fail", "4")); try { processor.process(contribution, inputs); @@ -449,7 +524,7 @@ public void write(List items) throws Exception { } } }); - Chunk inputs = new Chunk( + Chunk inputs = new Chunk<>( Arrays.asList("3", "fail", "2")); try { processor.process(contribution, inputs); @@ -490,10 +565,11 @@ public void write(List items) throws Exception { @Test // BATCH-2036 public void testProcessFilterAndSkippableException() throws Exception { - final List processedItems = new ArrayList(); + final List processedItems = new ArrayList<>(); processor.setProcessorTransactional(false); processor.setProcessSkipPolicy(new AlwaysSkipItemSkipPolicy()); processor.setItemProcessor(new ItemProcessor() { + @Nullable @Override public String process(String item) throws Exception { processedItems.add(item); @@ -507,7 +583,7 @@ public String process(String item) throws Exception { } }); processor.afterPropertiesSet(); - Chunk inputs = new Chunk(Arrays.asList("1", "2", "skip", "skip", "3", "fail", "fail", "4", "5")); + Chunk inputs = new Chunk<>(Arrays.asList("1", "2", "skip", "skip", "3", "fail", "fail", "4", "5")); try { processor.process(contribution, inputs); fail("Expected IllegalArgumentException"); @@ -532,10 +608,11 @@ public String process(String item) throws Exception { @Test // BATCH-2036 public void testProcessFilterAndSkippableExceptionNoRollback() throws Exception { - final List processedItems = new ArrayList(); + final List processedItems = new ArrayList<>(); processor.setProcessorTransactional(false); processor.setProcessSkipPolicy(new AlwaysSkipItemSkipPolicy()); processor.setItemProcessor(new ItemProcessor() { + @Nullable @Override public String process(String item) throws Exception { processedItems.add(item); @@ -551,7 +628,7 @@ public String process(String item) throws Exception { processor.setRollbackClassifier(new BinaryExceptionClassifier(Collections .> singleton(IllegalArgumentException.class), false)); processor.afterPropertiesSet(); - Chunk inputs = new Chunk(Arrays.asList("1", "2", "skip", "skip", "3", "fail", "fail", "4", "5")); + Chunk inputs = new Chunk<>(Arrays.asList("1", "2", "skip", "skip", "3", "fail", "fail", "4", "5")); processor.process(contribution, inputs); assertEquals(5, list.size()); assertEquals("[1, 2, 3, 4, 5]", list.toString()); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantChunkProviderTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantChunkProviderTests.java index ecb09a6f7f..9daf80a9e6 100755 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantChunkProviderTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantChunkProviderTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.item; import static org.junit.Assert.assertEquals; @@ -18,6 +33,7 @@ import org.springframework.batch.item.UnexpectedInputException; import org.springframework.batch.item.support.ListItemReader; import org.springframework.batch.repeat.support.RepeatTemplate; +import org.springframework.lang.Nullable; public class FaultTolerantChunkProviderTests { @@ -28,7 +44,7 @@ public class FaultTolerantChunkProviderTests { @Test public void testProvide() throws Exception { - provider = new FaultTolerantChunkProvider(new ListItemReader(Arrays.asList("foo", "bar")), + provider = new FaultTolerantChunkProvider<>(new ListItemReader<>(Arrays.asList("foo", "bar")), new RepeatTemplate()); Chunk chunk = provider.provide(contribution); assertNotNull(chunk); @@ -37,7 +53,8 @@ public void testProvide() throws Exception { @Test public void testProvideWithOverflow() throws Exception { - provider = new FaultTolerantChunkProvider(new ItemReader() { + provider = new FaultTolerantChunkProvider<>(new ItemReader() { + @Nullable @Override public String read() throws Exception, UnexpectedInputException, ParseException { throw new RuntimeException("Planned"); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantExceptionClassesTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantExceptionClassesTests.java index 12760b2be1..e55ed62950 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantExceptionClassesTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantExceptionClassesTests.java @@ -1,333 +1,356 @@ -package org.springframework.batch.core.step.item; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; - -import java.util.ArrayList; -import java.util.Date; -import java.util.List; - -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.springframework.batch.core.BatchStatus; -import org.springframework.batch.core.JobExecution; -import org.springframework.batch.core.JobParametersBuilder; -import org.springframework.batch.core.Step; -import org.springframework.batch.core.StepExecution; -import org.springframework.batch.core.job.SimpleJob; -import org.springframework.batch.core.launch.JobLauncher; -import org.springframework.batch.core.repository.JobRepository; -import org.springframework.beans.BeansException; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.context.ApplicationContext; -import org.springframework.context.ApplicationContextAware; -import org.springframework.test.context.ContextConfiguration; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; -import org.springframework.transaction.UnexpectedRollbackException; - -/** - * @author Dan Garrette - * @since 2.0.2 - */ -@ContextConfiguration -@RunWith(SpringJUnit4ClassRunner.class) -public class FaultTolerantExceptionClassesTests implements ApplicationContextAware { - - @Autowired - private JobRepository jobRepository; - - @Autowired - private JobLauncher jobLauncher; - - @Autowired - private SkipReaderStub reader; - - @Autowired - private SkipWriterStub writer; - - @Autowired - private ExceptionThrowingTaskletStub tasklet; - - private ApplicationContext applicationContext; - - @Override - public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { - this.applicationContext = applicationContext; - } - - @Before - public void setup() { - reader.clear(); - writer.clear(); - } - - @Test - public void testNonSkippable() throws Exception { - writer.setExceptionType(RuntimeException.class); - StepExecution stepExecution = launchStep("nonSkippableStep"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - assertEquals("[1, 2, 3]", writer.getWritten().toString()); - assertEquals("[]", writer.getCommitted().toString()); - } - - @Test - public void testNonSkippableChecked() throws Exception { - writer.setExceptionType(Exception.class); - StepExecution stepExecution = launchStep("nonSkippableStep"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - assertEquals("[1, 2, 3]", writer.getWritten().toString()); - assertEquals("[]", writer.getCommitted().toString()); - } - - @Test - public void testSkippable() throws Exception { - writer.setExceptionType(SkippableRuntimeException.class); - StepExecution stepExecution = launchStep("skippableStep"); - assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); - assertEquals("[1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); - assertEquals("[1, 2, 4]", writer.getCommitted().toString()); - } - - @Test - public void testRegularRuntimeExceptionNotSkipped() throws Exception { - writer.setExceptionType(RuntimeException.class); - StepExecution stepExecution = launchStep("skippableStep"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - // BATCH-1327: - assertEquals("[1, 2, 3]", writer.getWritten().toString()); - // BATCH-1327: - assertEquals("[]", writer.getCommitted().toString()); - } - - @Test - public void testFatalOverridesSkippable() throws Exception { - writer.setExceptionType(FatalRuntimeException.class); - StepExecution stepExecution = launchStep("skippableFatalStep"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - assertEquals("[1, 2, 3]", writer.getWritten().toString()); - assertEquals("[]", writer.getCommitted().toString()); - } - - @Test - public void testDefaultFatalChecked() throws Exception { - writer.setExceptionType(Exception.class); - StepExecution stepExecution = launchStep("skippableFatalStep"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - // BATCH-1327: - assertEquals("[1, 2, 3]", writer.getWritten().toString()); - // BATCH-1327: - assertEquals("[]", writer.getCommitted().toString()); - } - - @Test - public void testSkippableChecked() throws Exception { - writer.setExceptionType(SkippableException.class); - StepExecution stepExecution = launchStep("skippableStep"); - assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); - assertEquals("[1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); - assertEquals("[1, 2, 4]", writer.getCommitted().toString()); - } - - @Test - public void testNonSkippableUnchecked() throws Exception { - writer.setExceptionType(UnexpectedRollbackException.class); - StepExecution stepExecution = launchStep("skippableStep"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - assertEquals("[1, 2, 3]", writer.getWritten().toString()); - assertEquals("[]", writer.getCommitted().toString()); - } - - @Test - public void testFatalChecked() throws Exception { - writer.setExceptionType(FatalSkippableException.class); - StepExecution stepExecution = launchStep("skippableFatalStep"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - assertEquals("[1, 2, 3]", writer.getWritten().toString()); - assertEquals("[]", writer.getCommitted().toString()); - } - - @Test - public void testRetryableButNotSkippable() throws Exception { - writer.setExceptionType(RuntimeException.class); - StepExecution stepExecution = launchStep("retryable"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - assertEquals("[1, 2, 3, 1, 2, 3]", writer.getWritten().toString()); - // BATCH-1327: - assertEquals("[]", writer.getCommitted().toString()); - } - - @Test - public void testRetryableSkippable() throws Exception { - writer.setExceptionType(SkippableRuntimeException.class); - StepExecution stepExecution = launchStep("retryable"); - assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); - assertEquals("[1, 2, 3, 1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); - assertEquals("[1, 2, 4]", writer.getCommitted().toString()); - } - - @Test - public void testRetryableFatal() throws Exception { - // User wants all exceptions to be retried, but only some are skippable - // FatalRuntimeException is not skippable because it is fatal, but is a - // subclass of another skippable - writer.setExceptionType(FatalRuntimeException.class); - StepExecution stepExecution = launchStep("retryable"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - // BATCH-1333: - assertEquals("[1, 2, 3, 1, 2, 3]", writer.getWritten().toString()); - assertEquals("[]", writer.getCommitted().toString()); - } - - @Test - public void testRetryableButNotSkippableChecked() throws Exception { - writer.setExceptionType(Exception.class); - StepExecution stepExecution = launchStep("retryable"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - assertEquals("[1, 2, 3, 1, 2, 3]", writer.getWritten().toString()); - // BATCH-1327: - assertEquals("[]", writer.getCommitted().toString()); - } - - @Test - public void testRetryableSkippableChecked() throws Exception { - writer.setExceptionType(SkippableException.class); - StepExecution stepExecution = launchStep("retryable"); - assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); - assertEquals("[1, 2, 3, 1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); - assertEquals("[1, 2, 4]", writer.getCommitted().toString()); - } - - @Test - public void testRetryableFatalChecked() throws Exception { - writer.setExceptionType(FatalSkippableException.class); - StepExecution stepExecution = launchStep("retryable"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - // BATCH-1333: - assertEquals("[1, 2, 3, 1, 2, 3]", writer.getWritten().toString()); - assertEquals("[]", writer.getCommitted().toString()); - assertEquals(0, stepExecution.getWriteSkipCount()); - } - - @Test - public void testNoRollbackDefaultRollbackException() throws Exception { - // Exception is neither no-rollback nor skippable - writer.setExceptionType(Exception.class); - StepExecution stepExecution = launchStep("noRollbackDefault"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - // BATCH-1318: - assertEquals("[1, 2, 3]", writer.getWritten().toString()); - // BATCH-1318: - assertEquals("[]", writer.getCommitted().toString()); - assertEquals(0, stepExecution.getWriteSkipCount()); - } - - @Test - public void testNoRollbackDefaultNoRollbackException() throws Exception { - // Exception is no-rollback and not skippable - writer.setExceptionType(IllegalStateException.class); - StepExecution stepExecution = launchStep("noRollbackDefault"); - assertNotNull(stepExecution); - assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); - // BATCH-1334: - assertEquals("[1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); - // BATCH-1334: - assertEquals("[1, 2, 3, 4]", writer.getCommitted().toString()); - // BATCH-1334: - assertEquals(0, stepExecution.getWriteSkipCount()); - } - - @Test - public void testNoRollbackPathology() throws Exception { - // Exception is neither no-rollback nor skippable and no-rollback is - // RuntimeException (potentially pathological because other obviously - // rollback signalling Exceptions also extend RuntimeException) - writer.setExceptionType(Exception.class); - StepExecution stepExecution = launchStep("noRollbackPathology"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - // BATCH-1335: - assertEquals("[1, 2, 3]", writer.getWritten().toString()); - // BATCH-1335: - assertEquals("[]", writer.getCommitted().toString()); - } - - @Test - public void testNoRollbackSkippableRollbackException() throws Exception { - writer.setExceptionType(SkippableRuntimeException.class); - StepExecution stepExecution = launchStep("noRollbackSkippable"); - assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); - assertEquals("[1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); - assertEquals("[1, 2, 4]", writer.getCommitted().toString()); - } - - @Test - public void testNoRollbackSkippableNoRollbackException() throws Exception { - writer.setExceptionType(FatalRuntimeException.class); - StepExecution stepExecution = launchStep("noRollbackSkippable"); - assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); - // BATCH-1332: - assertEquals("[1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); - // BATCH-1334: - // Skipped but also committed (because it was marked as no-rollback) - assertEquals("[1, 2, 3, 4]", writer.getCommitted().toString()); - assertEquals(1, stepExecution.getWriteSkipCount()); - } - - @Test - public void testNoRollbackFatalRollbackException() throws Exception { - writer.setExceptionType(SkippableRuntimeException.class); - StepExecution stepExecution = launchStep("noRollbackFatal"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - assertEquals("[1, 2, 3]", writer.getWritten().toString()); - assertEquals("[]", writer.getCommitted().toString()); - } - - @Test - public void testNoRollbackFatalNoRollbackException() throws Exception { - // User has asked for no rollback on a fatal exception. What should the - // outcome be? As per BATCH-1333 it is interpreted as not skippable, but - // retryable if requested. Here it was not requested to be retried, but - // it was marked as no-rollback. As per BATCH-1334 this has to be ignored - // so that the failed item can be isolated. - writer.setExceptionType(FatalRuntimeException.class); - StepExecution stepExecution = launchStep("noRollbackFatal"); - assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); - // BATCH-1331: - assertEquals("[1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); - // BATCH-1331: - assertEquals("[1, 2, 3, 4]", writer.getCommitted().toString()); - } - - @Test - public void testNoRollbackTaskletRollbackException() throws Exception { - tasklet.setExceptionType(RuntimeException.class); - StepExecution stepExecution = launchStep("noRollbackTasklet"); - assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - assertEquals("[]", tasklet.getCommitted().toString()); - } - - @Test - public void testNoRollbackTaskletNoRollbackException() throws Exception { - tasklet.setExceptionType(SkippableRuntimeException.class); - StepExecution stepExecution = launchStep("noRollbackTasklet"); - // assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); - // BATCH-1298: - assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); - assertEquals("[1, 1, 1, 1]", tasklet.getCommitted().toString()); - } - - private StepExecution launchStep(String stepName) throws Exception { - SimpleJob job = new SimpleJob(); - job.setName("job"); - job.setJobRepository(jobRepository); - - List stepsToExecute = new ArrayList(); - stepsToExecute.add((Step) applicationContext.getBean(stepName)); - job.setSteps(stepsToExecute); - - JobExecution jobExecution = jobLauncher.run(job, new JobParametersBuilder().addLong("timestamp", - new Date().getTime()).toJobParameters()); - return jobExecution.getStepExecutions().iterator().next(); - } - -} +/* + * Copyright 2009-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.step.item; + +import org.junit.Before; +import org.junit.FixMethodOrder; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.MethodSorters; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.job.SimpleJob; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.ApplicationContext; +import org.springframework.context.ApplicationContextAware; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.transaction.UnexpectedRollbackException; + +import java.util.ArrayList; +import java.util.List; +import java.util.UUID; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +/** + * @author Dan Garrette + * @author Mahmoud Ben Hassine + * @since 2.0.2 + */ +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +@FixMethodOrder(MethodSorters.JVM) +public class FaultTolerantExceptionClassesTests implements ApplicationContextAware { + + @Autowired + private JobRepository jobRepository; + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private SkipReaderStub reader; + + @Autowired + private SkipWriterStub writer; + + @Autowired + private ExceptionThrowingTaskletStub tasklet; + + private ApplicationContext applicationContext; + + @Override + public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { + this.applicationContext = applicationContext; + } + + @Before + public void setup() { + reader.clear(); + writer.clear(); + tasklet.clear(); + } + + @Test + public void testNonSkippable() throws Exception { + writer.setExceptionType(RuntimeException.class); + StepExecution stepExecution = launchStep("nonSkippableStep"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals("[1, 2, 3]", writer.getWritten().toString()); + assertEquals("[]", writer.getCommitted().toString()); + } + + @Test + public void testNonSkippableChecked() throws Exception { + writer.setExceptionType(Exception.class); + StepExecution stepExecution = launchStep("nonSkippableStep"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals("[1, 2, 3]", writer.getWritten().toString()); + assertEquals("[]", writer.getCommitted().toString()); + } + + @Test + public void testSkippable() throws Exception { + writer.setExceptionType(SkippableRuntimeException.class); + StepExecution stepExecution = launchStep("skippableStep"); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals("[1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); + assertEquals("[1, 2, 4]", writer.getCommitted().toString()); + } + + @Test + public void testRegularRuntimeExceptionNotSkipped() throws Exception { + writer.setExceptionType(RuntimeException.class); + StepExecution stepExecution = launchStep("skippableStep"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + // BATCH-1327: + assertEquals("[1, 2, 3]", writer.getWritten().toString()); + // BATCH-1327: + assertEquals("[]", writer.getCommitted().toString()); + } + + @Test + public void testFatalOverridesSkippable() throws Exception { + writer.setExceptionType(FatalRuntimeException.class); + StepExecution stepExecution = launchStep("skippableFatalStep"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals("[1, 2, 3]", writer.getWritten().toString()); + assertEquals("[]", writer.getCommitted().toString()); + } + + @Test + public void testDefaultFatalChecked() throws Exception { + writer.setExceptionType(Exception.class); + StepExecution stepExecution = launchStep("skippableFatalStep"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + // BATCH-1327: + assertEquals("[1, 2, 3]", writer.getWritten().toString()); + // BATCH-1327: + assertEquals("[]", writer.getCommitted().toString()); + } + + @Test + public void testSkippableChecked() throws Exception { + writer.setExceptionType(SkippableException.class); + StepExecution stepExecution = launchStep("skippableStep"); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals("[1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); + assertEquals("[1, 2, 4]", writer.getCommitted().toString()); + } + + @Test + public void testNonSkippableUnchecked() throws Exception { + writer.setExceptionType(UnexpectedRollbackException.class); + StepExecution stepExecution = launchStep("skippableStep"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals("[1, 2, 3]", writer.getWritten().toString()); + assertEquals("[]", writer.getCommitted().toString()); + } + + @Test + public void testFatalChecked() throws Exception { + writer.setExceptionType(FatalSkippableException.class); + StepExecution stepExecution = launchStep("skippableFatalStep"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals("[1, 2, 3]", writer.getWritten().toString()); + assertEquals("[]", writer.getCommitted().toString()); + } + + @Test + public void testRetryableButNotSkippable() throws Exception { + writer.setExceptionType(RuntimeException.class); + StepExecution stepExecution = launchStep("retryable"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals("[1, 2, 3, 1, 2, 3]", writer.getWritten().toString()); + // BATCH-1327: + assertEquals("[]", writer.getCommitted().toString()); + } + + @Test + public void testRetryableSkippable() throws Exception { + writer.setExceptionType(SkippableRuntimeException.class); + StepExecution stepExecution = launchStep("retryable"); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals("[1, 2, 3, 1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); + assertEquals("[1, 2, 4]", writer.getCommitted().toString()); + } + + @Test + public void testRetryableFatal() throws Exception { + // User wants all exceptions to be retried, but only some are skippable + // FatalRuntimeException is not skippable because it is fatal, but is a + // subclass of another skippable + writer.setExceptionType(FatalRuntimeException.class); + StepExecution stepExecution = launchStep("retryable"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + // BATCH-1333: + assertEquals("[1, 2, 3, 1, 2, 3]", writer.getWritten().toString()); + assertEquals("[]", writer.getCommitted().toString()); + } + + @Test + public void testRetryableButNotSkippableChecked() throws Exception { + writer.setExceptionType(Exception.class); + StepExecution stepExecution = launchStep("retryable"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals("[1, 2, 3, 1, 2, 3]", writer.getWritten().toString()); + // BATCH-1327: + assertEquals("[]", writer.getCommitted().toString()); + } + + @Test + public void testRetryableSkippableChecked() throws Exception { + writer.setExceptionType(SkippableException.class); + StepExecution stepExecution = launchStep("retryable"); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals("[1, 2, 3, 1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); + assertEquals("[1, 2, 4]", writer.getCommitted().toString()); + } + + @Test + public void testRetryableFatalChecked() throws Exception { + writer.setExceptionType(FatalSkippableException.class); + StepExecution stepExecution = launchStep("retryable"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + // BATCH-1333: + assertEquals("[1, 2, 3, 1, 2, 3]", writer.getWritten().toString()); + assertEquals("[]", writer.getCommitted().toString()); + assertEquals(0, stepExecution.getWriteSkipCount()); + } + + @Test + public void testNoRollbackDefaultRollbackException() throws Exception { + // Exception is neither no-rollback nor skippable + writer.setExceptionType(Exception.class); + StepExecution stepExecution = launchStep("noRollbackDefault"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + // BATCH-1318: + assertEquals("[1, 2, 3]", writer.getWritten().toString()); + // BATCH-1318: + assertEquals("[]", writer.getCommitted().toString()); + assertEquals(0, stepExecution.getWriteSkipCount()); + } + + @Test + public void testNoRollbackDefaultNoRollbackException() throws Exception { + // Exception is no-rollback and not skippable + writer.setExceptionType(IllegalStateException.class); + StepExecution stepExecution = launchStep("noRollbackDefault"); + assertNotNull(stepExecution); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + // BATCH-1334: + assertEquals("[1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); + // BATCH-1334: + assertEquals("[1, 2, 3, 4]", writer.getCommitted().toString()); + // BATCH-1334: + assertEquals(0, stepExecution.getWriteSkipCount()); + } + + @Test + public void testNoRollbackPathology() throws Exception { + // Exception is neither no-rollback nor skippable and no-rollback is + // RuntimeException (potentially pathological because other obviously + // rollback signalling Exceptions also extend RuntimeException) + writer.setExceptionType(Exception.class); + StepExecution stepExecution = launchStep("noRollbackPathology"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + // BATCH-1335: + assertEquals("[1, 2, 3]", writer.getWritten().toString()); + // BATCH-1335: + assertEquals("[]", writer.getCommitted().toString()); + } + + @Test + public void testNoRollbackSkippableRollbackException() throws Exception { + writer.setExceptionType(SkippableRuntimeException.class); + StepExecution stepExecution = launchStep("noRollbackSkippable"); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals("[1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); + assertEquals("[1, 2, 4]", writer.getCommitted().toString()); + } + + @Test + public void testNoRollbackSkippableNoRollbackException() throws Exception { + writer.setExceptionType(FatalRuntimeException.class); + StepExecution stepExecution = launchStep("noRollbackSkippable"); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + // BATCH-1332: + assertEquals("[1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); + // BATCH-1334: + // Skipped but also committed (because it was marked as no-rollback) + assertEquals("[1, 2, 3, 4]", writer.getCommitted().toString()); + assertEquals(1, stepExecution.getWriteSkipCount()); + } + + @Test + public void testNoRollbackFatalRollbackException() throws Exception { + writer.setExceptionType(SkippableRuntimeException.class); + StepExecution stepExecution = launchStep("noRollbackFatal"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals("[1, 2, 3]", writer.getWritten().toString()); + assertEquals("[]", writer.getCommitted().toString()); + } + + @Test + public void testNoRollbackFatalNoRollbackException() throws Exception { + // User has asked for no rollback on a fatal exception. What should the + // outcome be? As per BATCH-1333 it is interpreted as not skippable, but + // retryable if requested. Here it was not requested to be retried, but + // it was marked as no-rollback. As per BATCH-1334 this has to be ignored + // so that the failed item can be isolated. + writer.setExceptionType(FatalRuntimeException.class); + StepExecution stepExecution = launchStep("noRollbackFatal"); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + // BATCH-1331: + assertEquals("[1, 2, 3, 1, 2, 3, 4]", writer.getWritten().toString()); + // BATCH-1331: + assertEquals("[1, 2, 3, 4]", writer.getCommitted().toString()); + } + + @Test + @DirtiesContext + public void testNoRollbackTaskletRollbackException() throws Exception { + tasklet.setExceptionType(RuntimeException.class); + StepExecution stepExecution = launchStep("noRollbackTasklet"); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals("[]", tasklet.getCommitted().toString()); + } + + @Test + @DirtiesContext + public void testNoRollbackTaskletNoRollbackException() throws Exception { + tasklet.setExceptionType(SkippableRuntimeException.class); + StepExecution stepExecution = launchStep("noRollbackTasklet"); + // assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + // BATCH-1298: + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + assertEquals("[1, 1, 1, 1]", tasklet.getCommitted().toString()); + } + + private StepExecution launchStep(String stepName) throws Exception { + SimpleJob job = new SimpleJob(); + job.setName("job"); + job.setJobRepository(jobRepository); + + List stepsToExecute = new ArrayList<>(); + stepsToExecute.add((Step) applicationContext.getBean(stepName)); + job.setSteps(stepsToExecute); + + JobExecution jobExecution = jobLauncher.run(job, new JobParametersBuilder().addString("uuid", + UUID.randomUUID().toString()).toJobParameters()); + return jobExecution.getStepExecutions().iterator().next(); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanNonBufferingTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanNonBufferingTests.java index b379f6eb80..d0eade3308 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanNonBufferingTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanNonBufferingTests.java @@ -1,10 +1,20 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.item; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; - import java.util.Arrays; import java.util.Collection; import java.util.HashMap; @@ -15,6 +25,7 @@ import org.apache.commons.logging.LogFactory; import org.junit.Before; import org.junit.Test; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobInstance; @@ -30,15 +41,19 @@ import org.springframework.batch.support.transaction.TransactionAwareProxyFactory; import org.springframework.util.StringUtils; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.mockito.Mockito.mock; + public class FaultTolerantStepFactoryBeanNonBufferingTests { protected final Log logger = LogFactory.getLog(getClass()); - private FaultTolerantStepFactoryBean factory = new FaultTolerantStepFactoryBean(); + private FaultTolerantStepFactoryBean factory = new FaultTolerantStepFactoryBean<>(); - private List items = Arrays.asList(new String[] { "1", "2", "3", "4", "5" }); + private List items = Arrays.asList("1", "2", "3", "4", "5"); - private ListItemReader reader = new ListItemReader(TransactionAwareProxyFactory + private ListItemReader reader = new ListItemReader<>(TransactionAwareProxyFactory .createTransactionalList(items)); private SkipWriterStub writer = new SkipWriterStub(); @@ -57,7 +72,7 @@ public void setUp() throws Exception { factory.setCommitInterval(2); factory.setItemReader(reader); factory.setItemWriter(writer); - Map, Boolean> skippableExceptions = new HashMap, Boolean>(); + Map, Boolean> skippableExceptions = new HashMap<>(); skippableExceptions.put(SkippableException.class, true); skippableExceptions.put(SkippableRuntimeException.class, true); factory.setSkippableExceptionClasses(skippableExceptions); @@ -72,6 +87,7 @@ public void setUp() throws Exception { * Check items causing errors are skipped as expected. */ @Test + @SuppressWarnings("rawtypes") public void testSkip() throws Exception { @SuppressWarnings("unchecked") SkipListener skipListener = mock(SkipListener.class); @@ -79,7 +95,7 @@ public void testSkip() throws Exception { skipListener.onSkipInWrite("4", exception); factory.setListeners(new SkipListener[] { skipListener }); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); StepExecution stepExecution = new StepExecution(step.getName(), jobExecution); step.execute(stepExecution); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanRetryTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanRetryTests.java index 68c9f2acdc..7c20b5e0c6 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanRetryTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanRetryTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,9 +15,6 @@ */ package org.springframework.batch.core.step.item; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; - import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; @@ -30,6 +27,7 @@ import org.apache.commons.logging.LogFactory; import org.junit.Before; import org.junit.Test; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.JobExecution; @@ -55,11 +53,15 @@ import org.springframework.batch.item.support.ListItemReader; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; import org.springframework.batch.support.transaction.TransactionAwareProxyFactory; +import org.springframework.lang.Nullable; import org.springframework.retry.policy.MapRetryContextCache; import org.springframework.retry.policy.SimpleRetryPolicy; import org.springframework.transaction.support.TransactionSynchronizationManager; import org.springframework.util.StringUtils; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + /** * @author Dave Syer * @@ -70,11 +72,11 @@ public class FaultTolerantStepFactoryBeanRetryTests { private FaultTolerantStepFactoryBean factory; - private List recovered = new ArrayList(); + private List recovered = new ArrayList<>(); - private List processed = new ArrayList(); + private List processed = new ArrayList<>(); - private List provided = new ArrayList(); + private List provided = new ArrayList<>(); private List written = TransactionAwareProxyFactory .createTransactionalList(); @@ -100,11 +102,11 @@ public void write(List data) throws Exception { @Before public void setUp() throws Exception { - factory = new FaultTolerantStepFactoryBean(); + factory = new FaultTolerantStepFactoryBean<>(); factory.setBeanName("step"); - factory.setItemReader(new ListItemReader( - new ArrayList())); + factory.setItemReader(new ListItemReader<>( + new ArrayList<>())); factory.setItemWriter(writer); factory.setJobRepository(repository); factory.setTransactionManager(new ResourcelessTransactionManager()); @@ -125,6 +127,7 @@ public void testType() throws Exception { assertTrue(Step.class.isAssignableFrom(factory.getObjectType())); } + @SuppressWarnings("cast") @Test public void testDefaultValue() throws Exception { assertTrue(factory.getObject() instanceof Step); @@ -135,7 +138,7 @@ public void testProcessAllItemsWhenErrorInWriterTransformationWhenReaderTransact throws Exception { final int RETRY_LIMIT = 3; final List ITEM_LIST = TransactionAwareProxyFactory.createTransactionalList(Arrays.asList("1", "2", "3")); - FaultTolerantStepFactoryBean factory = new FaultTolerantStepFactoryBean(); + FaultTolerantStepFactoryBean factory = new FaultTolerantStepFactoryBean<>(); factory.setBeanName("step"); factory.setJobRepository(repository); @@ -154,13 +157,14 @@ public void write(List data) throws Exception { }; ItemProcessor processor = new ItemProcessor() { + @Nullable @Override public Integer process(String item) throws Exception { processed.add(item); return Integer.parseInt(item); } }; - ItemReader reader = new ListItemReader(TransactionAwareProxyFactory.createTransactionalList(ITEM_LIST)); + ItemReader reader = new ListItemReader<>(TransactionAwareProxyFactory.createTransactionalList(ITEM_LIST)); factory.setCommitInterval(3); factory.setRetryLimit(RETRY_LIMIT); factory.setSkipLimit(1); @@ -172,7 +176,7 @@ public Integer process(String item) throws Exception { factory.setItemReader(reader); factory.setItemProcessor(processor); factory.setItemWriter(failingWriter); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); StepExecution stepExecution = new StepExecution(step.getName(), jobExecution); @@ -180,7 +184,7 @@ public Integer process(String item) throws Exception { step.execute(stepExecution); /* * Each chunk tried up to RETRY_LIMIT, then the scan processes each item - * once, identfiying the skip as it goes + * once, identifying the skip as it goes */ assertEquals((RETRY_LIMIT +1) * ITEM_LIST.size(), processed.size()); } @@ -203,13 +207,14 @@ public void write(List data) throws Exception { }; ItemProcessor processor = new ItemProcessor() { + @Nullable @Override public String process(String item) throws Exception { processed.add(item); return item; } }; - ItemReader reader = new ListItemReader(ITEM_LIST); + ItemReader reader = new ListItemReader<>(ITEM_LIST); factory.setCommitInterval(3); factory.setRetryLimit(RETRY_LIMIT); factory.setSkipLimit(1); @@ -219,7 +224,7 @@ public String process(String item) throws Exception { factory.setItemReader(reader); factory.setItemProcessor(processor); factory.setItemWriter(failingWriter); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); StepExecution stepExecution = new StepExecution(step.getName(), jobExecution); @@ -229,7 +234,7 @@ public String process(String item) throws Exception { .getExitStatus().getExitCode()); /* * Each chunk tried up to RETRY_LIMIT, then the scan processes each item - * once, identfiying the skip as it goes + * once, identifying the skip as it goes */ assertEquals((RETRY_LIMIT +1) * ITEM_LIST.size(), processed.size()); } @@ -251,22 +256,23 @@ public void write(List data) throws Exception { }; ItemProcessor processor = new ItemProcessor() { + @Nullable @Override public String process(String item) throws Exception { processed.add(item); return item; } }; - ItemReader reader = new ListItemReader(Arrays.asList( + ItemReader reader = new ListItemReader<>(Arrays.asList( "a", "b", "c")); factory.setProcessorTransactional(false); factory.setCommitInterval(3); factory.setRetryLimit(3); - factory.setSkippableExceptionClasses(new HashMap, Boolean>()); + factory.setSkippableExceptionClasses(new HashMap<>()); factory.setItemReader(reader); factory.setItemProcessor(processor); factory.setItemWriter(failingWriter); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); StepExecution stepExecution = new StepExecution(step.getName(), jobExecution); @@ -288,6 +294,7 @@ public String process(String item) throws Exception { public void testSuccessfulRetryWithReadFailure() throws Exception { ItemReader provider = new ListItemReader(Arrays.asList( "a", "b", "c")) { + @Nullable @Override public String read() { String item = super.read(); @@ -303,7 +310,7 @@ public String read() { factory.setItemReader(provider); factory.setRetryLimit(10); factory.setSkippableExceptionClasses(getExceptionMap()); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); StepExecution stepExecution = new StepExecution(step.getName(), jobExecution); @@ -338,10 +345,11 @@ protected void doClose() throws Exception { @Override protected void doOpen() throws Exception { - reader = new ListItemReader(Arrays.asList("a", "b", + reader = new ListItemReader<>(Arrays.asList("a", "b", "c", "d", "e", "f")); } + @Nullable @Override protected String doRead() throws Exception { return reader.read(); @@ -362,7 +370,7 @@ public void write(List items) throws Exception { } }); factory.setRetryLimit(0); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); fail = true; StepExecution stepExecution = new StepExecution(step.getName(), @@ -392,6 +400,7 @@ public void testSkipAndRetry() throws Exception { factory.setSkipLimit(2); ItemReader provider = new ListItemReader(Arrays.asList( "a", "b", "c", "d", "e", "f")) { + @Nullable @Override public String read() { String item = super.read(); @@ -405,7 +414,7 @@ public String read() { }; factory.setItemReader(provider); factory.setRetryLimit(10); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); StepExecution stepExecution = new StepExecution(step.getName(), jobExecution); @@ -433,6 +442,7 @@ public void onSkipInWrite(String item, Throwable t) { factory.setSkipLimit(2); ItemReader provider = new ListItemReader(Arrays.asList( "a", "b", "c", "d", "e", "f")) { + @Nullable @Override public String read() { String item = super.read(); @@ -497,6 +507,7 @@ public void onSkipInWrite(String item, Throwable t) { factory.setSkipLimit(2); ItemReader provider = new ListItemReader(Arrays.asList( "a", "b", "c", "d", "e", "f")) { + @Nullable @Override public String read() { String item = super.read(); @@ -555,6 +566,7 @@ public void testRetryWithNoSkip() throws Exception { factory.setSkipLimit(0); ItemReader provider = new ListItemReader( Arrays.asList("b")) { + @Nullable @Override public String read() { String item = super.read(); @@ -575,7 +587,7 @@ public void write(List item) throws Exception { }; factory.setItemReader(provider); factory.setItemWriter(itemWriter); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); StepExecution stepExecution = new StepExecution(step.getName(), jobExecution); @@ -610,6 +622,7 @@ public void testNonSkippableException() throws Exception { factory.setSkipLimit(1); ItemReader provider = new ListItemReader( Arrays.asList("b")) { + @Nullable @Override public String read() { String item = super.read(); @@ -630,7 +643,7 @@ public void write(List item) throws Exception { }; factory.setItemReader(provider); factory.setItemWriter(itemWriter); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); StepExecution stepExecution = new StepExecution(step.getName(), jobExecution); @@ -663,6 +676,7 @@ public void testRetryPolicy() throws Exception { factory.setSkipLimit(0); ItemReader provider = new ListItemReader( Arrays.asList("b")) { + @Nullable @Override public String read() { String item = super.read(); @@ -713,6 +727,7 @@ public void testCacheLimitWithRetry() throws Exception { // set the cache limit stupidly low factory.setRetryContextCache(new MapRetryContextCache(0)); ItemReader provider = new ItemReader() { + @Nullable @Override public String read() { String item = "" + count; @@ -755,9 +770,10 @@ public void write(List item) throws Exception { assertEquals(0, recovered.size()); } + @SuppressWarnings("unchecked") private Map, Boolean> getExceptionMap( Class... args) { - Map, Boolean> map = new HashMap, Boolean>(); + Map, Boolean> map = new HashMap<>(); for (Class arg : args) { map.put(arg, true); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanRollbackTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanRollbackTests.java index 3c72c745a6..40e12e0409 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanRollbackTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanRollbackTests.java @@ -1,12 +1,20 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.item; -import static org.hamcrest.CoreMatchers.instanceOf; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertThat; -import static org.junit.Assert.assertTrue; -import static org.springframework.batch.core.BatchStatus.FAILED; - import java.util.ArrayList; import java.util.Arrays; import java.util.Collection; @@ -20,6 +28,7 @@ import org.junit.After; import org.junit.Before; import org.junit.Test; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.ChunkListener; import org.springframework.batch.core.JobExecution; @@ -43,6 +52,13 @@ import org.springframework.transaction.interceptor.TransactionAttributeEditor; import org.springframework.util.StringUtils; +import static org.hamcrest.CoreMatchers.instanceOf; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertThat; +import static org.junit.Assert.assertTrue; +import static org.springframework.batch.core.BatchStatus.FAILED; + /** * Tests for {@link FaultTolerantStepFactoryBean}. */ @@ -67,11 +83,11 @@ public class FaultTolerantStepFactoryBeanRollbackTests { @SuppressWarnings("unchecked") @Before public void setUp() throws Exception { - reader = new SkipReaderStub(); - processor = new SkipProcessorStub(); - writer = new SkipWriterStub(); + reader = new SkipReaderStub<>(); + processor = new SkipProcessorStub<>(); + writer = new SkipWriterStub<>(); - factory = new FaultTolerantStepFactoryBean(); + factory = new FaultTolerantStepFactoryBean<>(); factory.setBeanName("stepName"); ResourcelessTransactionManager transactionManager = new ResourcelessTransactionManager(); @@ -93,7 +109,7 @@ public void setUp() throws Exception { MapJobRepositoryFactoryBean repositoryFactory = new MapJobRepositoryFactoryBean(); repositoryFactory.setTransactionManager(transactionManager); repositoryFactory.afterPropertiesSet(); - repository = (JobRepository) repositoryFactory.getObject(); + repository = repositoryFactory.getObject(); factory.setJobRepository(repository); jobExecution = repository.createJobExecution("skipJob", new JobParameters()); @@ -112,7 +128,7 @@ public void tearDown() throws Exception { @Test public void testBeforeChunkListenerException() throws Exception{ factory.setListeners(new StepListener []{new ExceptionThrowingChunkListener(1)}); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(FAILED, stepExecution.getStatus()); assertEquals(FAILED.toString(), stepExecution.getExitStatus().getExitCode()); @@ -125,7 +141,7 @@ public void testBeforeChunkListenerException() throws Exception{ @Test public void testAfterChunkListenerException() throws Exception{ factory.setListeners(new StepListener []{new ExceptionThrowingChunkListener(2)}); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(FAILED, stepExecution.getStatus()); assertEquals(FAILED.toString(), stepExecution.getExitStatus().getExitCode()); @@ -173,7 +189,7 @@ public void testReaderDefaultNoRollbackOnCheckedException() throws Exception { reader.setFailures("2", "3"); reader.setExceptionType(SkippableException.class); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -196,7 +212,7 @@ public void testReaderAttributesOverrideSkippableNoRollback() throws Exception { // But this one is explicit in the tx-attrs so it should be skipped factory.setNoRollbackExceptionClasses(getExceptionList(SkippableException.class)); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -215,7 +231,7 @@ public void testProcessorDefaultRollbackOnCheckedException() throws Exception { processor.setFailures("1", "3"); processor.setExceptionType(SkippableException.class); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -233,7 +249,7 @@ public void testProcessorDefaultRollbackOnRuntimeException() throws Exception { processor.setFailures("1", "3"); processor.setExceptionType(SkippableRuntimeException.class); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -257,7 +273,7 @@ public void testNoRollbackInProcessorWhenSkipExceeded() throws Throwable { factory.setItemProcessor(processor); @SuppressWarnings("unchecked") - List> exceptions = Arrays.>asList(Exception.class); + List> exceptions = Arrays.asList(Exception.class); factory.setNoRollbackExceptionClasses(exceptions); @SuppressWarnings("unchecked") Map, Boolean> skippable = getExceptionMap(Exception.class); @@ -265,7 +281,7 @@ public void testNoRollbackInProcessorWhenSkipExceeded() throws Throwable { processor.setFailures("2"); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); stepExecution = jobExecution.createStepExecution(factory.getName()); repository.add(stepExecution); @@ -275,7 +291,7 @@ public void testNoRollbackInProcessorWhenSkipExceeded() throws Throwable { assertEquals("[1, 3, 4, 5]", writer.getCommitted().toString()); // No rollback on 2 so processor has side effect assertEquals("[1, 2, 3, 4, 5]", processor.getCommitted().toString()); - List processed = new ArrayList(processor.getProcessed()); + List processed = new ArrayList<>(processor.getProcessed()); Collections.sort(processed); assertEquals("[1, 2, 3, 4, 5]", processed.toString()); assertEquals(0, stepExecution.getSkipCount()); @@ -289,7 +305,7 @@ public void testProcessSkipWithNoRollbackForCheckedException() throws Exception factory.setNoRollbackExceptionClasses(getExceptionList(SkippableException.class)); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -317,7 +333,7 @@ public void testWriterDefaultRollbackOnCheckedException() throws Exception { writer.setFailures("2", "3"); writer.setExceptionType(SkippableException.class); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -333,7 +349,7 @@ public void testWriterDefaultRollbackOnError() throws Exception { writer.setFailures("2", "3"); writer.setExceptionType(AssertionError.class); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); @@ -349,7 +365,7 @@ public void testWriterDefaultRollbackOnRuntimeException() throws Exception { writer.setFailures("2", "3"); writer.setExceptionType(SkippableRuntimeException.class); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -369,7 +385,7 @@ public void testWriterNoRollbackOnRuntimeException() throws Exception { factory.setNoRollbackExceptionClasses(getExceptionList(SkippableRuntimeException.class)); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -390,7 +406,7 @@ public void testWriterNoRollbackOnCheckedException() throws Exception { factory.setNoRollbackExceptionClasses(getExceptionList(SkippableException.class)); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -405,7 +421,7 @@ public void testSkipInProcessor() throws Exception { processor.setFailures("4"); factory.setCommitInterval(30); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -421,7 +437,7 @@ public void testMultipleSkipsInProcessor() throws Exception { processor.setFailures("2", "4"); factory.setCommitInterval(30); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -438,7 +454,7 @@ public void testMultipleSkipsInNonTransactionalProcessor() throws Exception { factory.setCommitInterval(30); factory.setProcessorTransactional(false); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -455,7 +471,7 @@ public void testFilterInProcessor() throws Exception { processor.setFilter(true); factory.setCommitInterval(30); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -471,7 +487,7 @@ public void testSkipInWriter() throws Exception { writer.setFailures("4"); factory.setCommitInterval(30); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -493,7 +509,7 @@ public void testSkipInWriterNonTransactionalProcessor() throws Exception { factory.setCommitInterval(30); factory.setProcessorTransactional(false); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -506,13 +522,13 @@ public void testSkipInWriterNonTransactionalProcessor() throws Exception { @Test public void testSkipInWriterTransactionalReader() throws Exception { writer.setFailures("4"); - ItemReader reader = new ListItemReader(TransactionAwareProxyFactory.createTransactionalList(Arrays.asList("1", "2", "3", "4", "5"))); + ItemReader reader = new ListItemReader<>(TransactionAwareProxyFactory.createTransactionalList(Arrays.asList("1", "2", "3", "4", "5"))); factory.setItemReader(reader); factory.setCommitInterval(30); factory.setSkipLimit(10); factory.setIsReaderTransactionalQueue(true); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -529,7 +545,7 @@ public void testMultithreadedSkipInWriter() throws Exception { factory.setSkipLimit(10); factory.setTaskExecutor(new SimpleAsyncTaskExecutor()); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -544,7 +560,7 @@ public void testMultipleSkipsInWriter() throws Exception { writer.setFailures("2", "4"); factory.setCommitInterval(30); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -566,7 +582,7 @@ public void testMultipleSkipsInWriterNonTransactionalProcessor() throws Exceptio factory.setCommitInterval(30); factory.setProcessorTransactional(false); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -581,8 +597,9 @@ private Collection> getExceptionList(Class> asList(arg); } + @SuppressWarnings("unchecked") private Map, Boolean> getExceptionMap(Class... args) { - Map, Boolean> map = new HashMap, Boolean>(); + Map, Boolean> map = new HashMap<>(); for (Class arg : args) { map.put(arg, true); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanTests.java index 93b5db3644..b64fd274f8 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanTests.java @@ -1,9 +1,20 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.item; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; @@ -16,7 +27,10 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.junit.Before; +import org.junit.Rule; import org.junit.Test; + +import org.junit.rules.ExpectedException; import org.springframework.aop.framework.ProxyFactory; import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.ChunkListener; @@ -52,15 +66,23 @@ import org.springframework.batch.item.support.AbstractItemStreamItemReader; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; import org.springframework.beans.factory.FactoryBean; +import org.springframework.lang.Nullable; import org.springframework.scheduling.concurrent.ConcurrentTaskExecutor; import org.springframework.test.util.ReflectionTestUtils; import org.springframework.util.StringUtils; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + /** * Tests for {@link FaultTolerantStepFactoryBean}. */ public class FaultTolerantStepFactoryBeanTests { + @Rule + public ExpectedException expectedException = ExpectedException.none(); + protected final Log logger = LogFactory.getLog(getClass()); private FaultTolerantStepFactoryBean factory; @@ -82,15 +104,15 @@ public class FaultTolerantStepFactoryBeanTests { private boolean closed = false; public FaultTolerantStepFactoryBeanTests() throws Exception { - reader = new SkipReaderStub(); - processor = new SkipProcessorStub(); - writer = new SkipWriterStub(); + reader = new SkipReaderStub<>(); + processor = new SkipProcessorStub<>(); + writer = new SkipWriterStub<>(); } @SuppressWarnings("unchecked") @Before public void setUp() throws Exception { - factory = new FaultTolerantStepFactoryBean(); + factory = new FaultTolerantStepFactoryBean<>(); factory.setBeanName("stepName"); factory.setTransactionManager(new ResourcelessTransactionManager()); @@ -111,7 +133,7 @@ public void setUp() throws Exception { MapJobRepositoryFactoryBean repositoryFactory = new MapJobRepositoryFactoryBean(); repositoryFactory.afterPropertiesSet(); - repository = (JobRepository) repositoryFactory.getObject(); + repository = repositoryFactory.getObject(); factory.setJobRepository(repository); jobExecution = repository.createJobExecution("skipJob", new JobParameters()); @@ -119,6 +141,28 @@ public void setUp() throws Exception { repository.add(stepExecution); } + @Test + public void testMandatoryReader() throws Exception { + factory = new FaultTolerantStepFactoryBean<>(); + factory.setItemWriter(writer); + + expectedException.expect(IllegalStateException.class); + expectedException.expectMessage("ItemReader must be provided"); + + factory.getObject(); + } + + @Test + public void testMandatoryWriter() throws Exception { + factory = new FaultTolerantStepFactoryBean<>(); + factory.setItemReader(reader); + + expectedException.expect(IllegalStateException.class); + expectedException.expectMessage("ItemWriter must be provided"); + + factory.getObject(); + } + /** * Non-skippable (and non-fatal) exception causes failure immediately. * @@ -132,7 +176,7 @@ public void testNonSkippableExceptionOnRead() throws Exception { // nothing is skippable factory.setSkippableExceptionClasses(getExceptionMap(NonExistentException.class)); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); @@ -154,7 +198,7 @@ public void testNonSkippableException() throws Exception { reader.setItems("1", "2", "3", "4", "5"); writer.setFailures("1"); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); @@ -172,7 +216,7 @@ public void testNonSkippableException() throws Exception { public void testReadSkip() throws Exception { reader.setFailures("2"); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -224,7 +268,7 @@ public boolean shouldSkip(Throwable t, int skipCount) throws SkipLimitExceededEx reader.setFailures("2"); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -252,7 +296,7 @@ public boolean shouldSkip(Throwable t, int skipCount) throws SkipLimitExceededEx writer.setFailures("2"); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -271,11 +315,11 @@ public void testReadSkipItemStreamException() throws Exception { reader.setFailures("2"); reader.setExceptionType(ItemStreamException.class); - Map, Boolean> map = new HashMap, Boolean>(); + Map, Boolean> map = new HashMap<>(); map.put(ItemStreamException.class, true); factory.setSkippableExceptionClasses(map); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -305,7 +349,7 @@ public void testProcessSkip() throws Exception { processor.setFailures("4"); writer.setFailures("4"); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -331,9 +375,9 @@ public void testProcessSkip() throws Exception { public void testProcessFilter() throws Exception { processor.setFailures("4"); processor.setFilter(true); - ItemProcessListenerStub listenerStub = new ItemProcessListenerStub(); + ItemProcessListenerStub listenerStub = new ItemProcessListenerStub<>(); factory.setListeners(new StepListener[] { listenerStub }); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -356,30 +400,6 @@ public void testProcessFilter() throws Exception { .getName())); } - @Test - public void testNullWriter() throws Exception { - - factory.setItemWriter(null); - Step step = (Step) factory.getObject(); - - step.execute(stepExecution); - - assertEquals(0, stepExecution.getSkipCount()); - assertEquals(0, stepExecution.getReadSkipCount()); - assertEquals(5, stepExecution.getReadCount()); - // Write count is incremented even if nothing happens - assertEquals(5, stepExecution.getWriteCount()); - assertEquals(0, stepExecution.getFilterCount()); - assertEquals(0, stepExecution.getRollbackCount()); - - // writer skips "4" - assertTrue(reader.getRead().contains("4")); - - assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); - assertStepExecutionsAreEqual(stepExecution, repository.getLastStepExecution(jobExecution.getJobInstance(), step - .getName())); - } - /** * Check items causing errors are skipped as expected. */ @@ -387,7 +407,7 @@ public void testNullWriter() throws Exception { public void testWriteSkip() throws Exception { writer.setFailures("4"); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -418,7 +438,7 @@ public void testWriteSkip() throws Exception { public void testFatalException() throws Exception { reader.setFailures("2"); - Map, Boolean> map = new HashMap, Boolean>(); + Map, Boolean> map = new HashMap<>(); map.put(SkippableException.class, true); map.put(SkippableRuntimeException.class, true); map.put(FatalRuntimeException.class, false); @@ -430,7 +450,7 @@ public void write(List items) { } }); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); String message = stepExecution.getFailureExceptions().get(0).getCause().getMessage(); @@ -449,7 +469,7 @@ public void testSkipOverLimit() throws Exception { factory.setSkipLimit(1); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -480,7 +500,7 @@ public void testSkipOverLimitOnRead() throws Exception { factory.setSkipLimit(3); factory.setSkippableExceptionClasses(getExceptionMap(Exception.class)); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); @@ -509,7 +529,7 @@ public void testSkipOverLimitOnReadWithListener() throws Exception { reader.setFailures("1", "3", "5"); writer.setFailures(); - final List listenerCalls = new ArrayList(); + final List listenerCalls = new ArrayList<>(); factory.setListeners(new StepListener[] { new SkipListenerSupport() { @Override @@ -520,7 +540,7 @@ public void onSkipInRead(Throwable t) { factory.setCommitInterval(2); factory.setSkipLimit(2); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -554,7 +574,7 @@ public void onSkipInRead(Throwable t) { } }); factory.setSkippableExceptionClasses(getExceptionMap(Exception.class)); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); @@ -589,7 +609,7 @@ public void onSkipInWrite(String item, Throwable t) { } }); factory.setSkippableExceptionClasses(getExceptionMap(Exception.class)); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); @@ -613,7 +633,7 @@ public void testSkipOnReadNotDoubleCounted() throws Exception { factory.setSkipLimit(4); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(4, stepExecution.getSkipCount()); @@ -644,7 +664,7 @@ public void testSkipOnWriteNotDoubleCounted() throws Exception { factory.setSkipLimit(4); factory.setCommitInterval(3); // includes all expected skips - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(4, stepExecution.getSkipCount()); @@ -667,7 +687,7 @@ public void testDefaultSkipPolicy() throws Exception { factory.setSkippableExceptionClasses(getExceptionMap(Exception.class)); factory.setSkipLimit(1); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -692,7 +712,7 @@ public void testSkipOverLimitOnReadWithAllSkipsAtEnd() throws Exception { factory.setSkipLimit(3); factory.setSkippableExceptionClasses(getExceptionMap(Exception.class)); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); @@ -717,7 +737,7 @@ public void testReprocessingAfterWriterRollback() throws Exception { writer.setFailures("4"); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(1, stepExecution.getSkipCount()); @@ -734,7 +754,7 @@ public void testReprocessingAfterWriterRollback() throws Exception { public void testAutoRegisterItemListeners() throws Exception { reader.setFailures("2"); - final List listenerCalls = new ArrayList(); + final List listenerCalls = new ArrayList<>(); class TestItemListenerWriter implements ItemWriter, ItemReadListener, ItemWriteListener, ItemProcessListener, SkipListener, @@ -773,7 +793,7 @@ public void onWriteError(Exception exception, List items) { } @Override - public void afterProcess(String item, String result) { + public void afterProcess(String item, @Nullable String result) { listenerCalls.add(3); } @@ -815,7 +835,7 @@ public void afterChunkError(ChunkContext context) { factory.setItemWriter(new TestItemListenerWriter()); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); @@ -844,6 +864,7 @@ public void open(ExecutionContext executionContext) { opened = true; } + @Nullable @Override public String read() { return null; @@ -853,7 +874,7 @@ public String read() { factory.setItemReader(reader); factory.setTaskExecutor(new ConcurrentTaskExecutor()); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -882,6 +903,7 @@ public void open(ExecutionContext executionContext) throws ItemStreamException { public void update(ExecutionContext executionContext) throws ItemStreamException { } + @Nullable @Override public String read() throws Exception, UnexpectedInputException, ParseException { return null; @@ -903,6 +925,7 @@ public void open(ExecutionContext executionContext) throws ItemStreamException { public void update(ExecutionContext executionContext) throws ItemStreamException { } + @Nullable @Override public String read() throws Exception, UnexpectedInputException, ParseException { return null; @@ -912,7 +935,7 @@ public String read() throws Exception, UnexpectedInputException, ParseException factory.setItemReader(reader); factory.setStreams(new ItemStream[] { stream, reader }); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -944,6 +967,7 @@ public void open(ExecutionContext executionContext) throws ItemStreamException { public void update(ExecutionContext executionContext) throws ItemStreamException { } + @Nullable @Override public String read() throws Exception, UnexpectedInputException, ParseException { return null; @@ -964,7 +988,7 @@ public Object invoke(MethodInvocation invocation) throws Throwable { factory.setItemReader((ItemReader) advised); factory.setStreams(new ItemStream[] { (ItemStream) advised }); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); @@ -978,7 +1002,7 @@ private static class ItemProcessListenerStub implements ItemProcessListene private boolean filterEncountered = false; @Override - public void afterProcess(T item, S result) { + public void afterProcess(T item, @Nullable S result) { if (result == null) { filterEncountered = true; } @@ -1079,7 +1103,7 @@ public void testFatalSubsetFatal() throws Exception { } private SkipPolicy getSkippableSubsetSkipPolicy() throws Exception { - Map, Boolean> skippableExceptions = new HashMap, Boolean>(); + Map, Boolean> skippableExceptions = new HashMap<>(); skippableExceptions.put(WriteFailedException.class, true); skippableExceptions.put(ItemWriterException.class, false); factory.setSkippableExceptionClasses(skippableExceptions); @@ -1087,23 +1111,23 @@ private SkipPolicy getSkippableSubsetSkipPolicy() throws Exception { } private SkipPolicy getFatalSubsetSkipPolicy() throws Exception { - Map, Boolean> skippableExceptions = new HashMap, Boolean>(); + Map, Boolean> skippableExceptions = new HashMap<>(); skippableExceptions.put(ItemWriterException.class, true); skippableExceptions.put(WriteFailedException.class, false); factory.setSkippableExceptionClasses(skippableExceptions); return getSkipPolicy(factory); } - @SuppressWarnings("rawtypes") - private SkipPolicy getSkipPolicy(FactoryBean factory) throws Exception { + private SkipPolicy getSkipPolicy(FactoryBean factory) throws Exception { Object step = factory.getObject(); Object tasklet = ReflectionTestUtils.getField(step, "tasklet"); Object chunkProvider = ReflectionTestUtils.getField(tasklet, "chunkProvider"); return (SkipPolicy) ReflectionTestUtils.getField(chunkProvider, "skipPolicy"); } + @SuppressWarnings("unchecked") private Map, Boolean> getExceptionMap(Class... args) { - Map, Boolean> map = new HashMap, Boolean>(); + Map, Boolean> map = new HashMap<>(); for (Class arg : args) { map.put(arg, true); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanUnexpectedRollbackTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanUnexpectedRollbackTests.java index 7aed2fe90e..dc2e034a9c 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanUnexpectedRollbackTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/FaultTolerantStepFactoryBeanUnexpectedRollbackTests.java @@ -1,13 +1,23 @@ +/* + * Copyright 2010-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.item; -import static org.junit.Assert.assertEquals; - -import java.util.Arrays; - -import javax.sql.DataSource; - import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; +import org.junit.Ignore; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.batch.core.BatchStatus; @@ -28,6 +38,11 @@ import org.springframework.transaction.UnexpectedRollbackException; import org.springframework.transaction.support.DefaultTransactionStatus; +import javax.sql.DataSource; +import java.util.Arrays; + +import static org.junit.Assert.assertEquals; + /** * Tests for {@link FaultTolerantStepFactoryBean} with unexpected rollback. */ @@ -41,10 +56,11 @@ public class FaultTolerantStepFactoryBeanUnexpectedRollbackTests { private DataSource dataSource; @Test + @Ignore //FIXME public void testTransactionException() throws Exception { - final SkipWriterStub writer = new SkipWriterStub(); - FaultTolerantStepFactoryBean factory = new FaultTolerantStepFactoryBean(); + final SkipWriterStub writer = new SkipWriterStub<>(); + FaultTolerantStepFactoryBean factory = new FaultTolerantStepFactoryBean<>(); factory.setItemWriter(writer); @SuppressWarnings("serial") @@ -67,21 +83,21 @@ protected void doCommit(DefaultTransactionStatus status) throws TransactionExcep factory.setTransactionManager(transactionManager); factory.setCommitInterval(2); - ItemReader reader = new ListItemReader(Arrays.asList("1", "2")); + ItemReader reader = new ListItemReader<>(Arrays.asList("1", "2")); factory.setItemReader(reader); JobRepositoryFactoryBean repositoryFactory = new JobRepositoryFactoryBean(); repositoryFactory.setDataSource(dataSource); repositoryFactory.setTransactionManager(transactionManager); repositoryFactory.afterPropertiesSet(); - JobRepository repository = (JobRepository) repositoryFactory.getObject(); + JobRepository repository = repositoryFactory.getObject(); factory.setJobRepository(repository); JobExecution jobExecution = repository.createJobExecution("job", new JobParameters()); StepExecution stepExecution = jobExecution.createStepExecution(factory.getName()); repository.add(stepExecution); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(stepExecution); assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ForceRollbackForWriteSkipExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ForceRollbackForWriteSkipExceptionTests.java index f8cc0e9f66..84e15d544f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ForceRollbackForWriteSkipExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ForceRollbackForWriteSkipExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/RepeatOperationsStepFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/RepeatOperationsStepFactoryBeanTests.java index 904621e1ff..e3bffbad45 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/RepeatOperationsStepFactoryBeanTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/RepeatOperationsStepFactoryBeanTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,7 +40,7 @@ */ public class RepeatOperationsStepFactoryBeanTests extends TestCase { - private SimpleStepFactoryBean factory = new SimpleStepFactoryBean(); + private SimpleStepFactoryBean factory = new SimpleStepFactoryBean<>(); private List list; @@ -49,8 +49,8 @@ public class RepeatOperationsStepFactoryBeanTests extends TestCase { @Override protected void setUp() throws Exception { factory.setBeanName("RepeatOperationsStep"); - factory.setItemReader(new ListItemReader(new ArrayList())); - factory.setItemWriter(new EmptyItemWriter()); + factory.setItemReader(new ListItemReader<>(new ArrayList<>())); + factory.setItemWriter(new EmptyItemWriter<>()); factory.setJobRepository(new JobRepositorySupport()); factory.setTransactionManager(new ResourcelessTransactionManager()); } @@ -59,14 +59,15 @@ public void testType() throws Exception { assertTrue(Step.class.isAssignableFrom(factory.getObjectType())); } + @SuppressWarnings("cast") public void testDefaultValue() throws Exception { assertTrue(factory.getObject() instanceof Step); } public void testStepOperationsWithoutChunkListener() throws Exception { - factory.setItemReader(new ListItemReader(new ArrayList())); - factory.setItemWriter(new EmptyItemWriter()); + factory.setItemReader(new ListItemReader<>(new ArrayList<>())); + factory.setItemWriter(new EmptyItemWriter<>()); factory.setJobRepository(new JobRepositorySupport()); factory.setTransactionManager(new ResourcelessTransactionManager()); @@ -74,13 +75,13 @@ public void testStepOperationsWithoutChunkListener() throws Exception { @Override public RepeatStatus iterate(RepeatCallback callback) { - list = new ArrayList(); + list = new ArrayList<>(); list.add("foo"); return RepeatStatus.FINISHED; } }); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); step.execute(new StepExecution(step.getName(), jobExecution)); assertEquals(1, list.size()); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ScriptItemProcessorTest.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ScriptItemProcessorTest.java new file mode 100644 index 0000000000..768cd2dfc9 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/ScriptItemProcessorTest.java @@ -0,0 +1,64 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.step.item; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.item.ItemWriter; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.util.Assert; + +import java.util.List; + +/** + *

      + * Test job utilizing a {@link org.springframework.batch.item.support.ScriptItemProcessor}. + *

      + * + * @author Chris Schaefer + * @since 3.1 + */ +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class ScriptItemProcessorTest { + @Autowired + private Job job; + + @Autowired + private JobLauncher jobLauncher; + + @Test + public void testScriptProcessorJob() throws Exception { + jobLauncher.run(job, new JobParameters()); + } + + public static class TestItemWriter implements ItemWriter { + @Override + public void write(List items) throws Exception { + Assert.notNull(items, "Items cannot be null"); + Assert.isTrue(!items.isEmpty(), "Items cannot be empty"); + Assert.isTrue(items.size() == 1, "Items should only contain one entry"); + + String item = items.get(0); + Assert.isTrue("BLAH".equals(item), "Transformed item to write should have been: BLAH but got: " + item); + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleChunkProcessorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleChunkProcessorTests.java index 1155d9a741..4f9a1936bc 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleChunkProcessorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleChunkProcessorTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.item; import static org.junit.Assert.assertEquals; @@ -15,11 +30,13 @@ import org.springframework.batch.core.StepExecution; import org.springframework.batch.item.ItemProcessor; import org.springframework.batch.item.ItemWriter; +import org.springframework.lang.Nullable; public class SimpleChunkProcessorTests { - private SimpleChunkProcessor processor = new SimpleChunkProcessor( + private SimpleChunkProcessor processor = new SimpleChunkProcessor<>( new ItemProcessor() { + @Nullable @Override public String process(String item) throws Exception { if (item.equals("err")) { @@ -40,7 +57,7 @@ public void write(List items) throws Exception { private StepContribution contribution = new StepContribution(new StepExecution("foo", new JobExecution( new JobInstance(123L, "job"), new JobParameters()))); - private List list = new ArrayList(); + private List list = new ArrayList<>(); @Before public void setUp() { @@ -49,7 +66,7 @@ public void setUp() { @Test public void testProcess() throws Exception { - Chunk chunk = new Chunk(); + Chunk chunk = new Chunk<>(); chunk.add("foo"); chunk.add("err"); chunk.add("bar"); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleChunkProviderTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleChunkProviderTests.java index 2080514de0..d09c6b7515 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleChunkProviderTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleChunkProviderTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.item; import static org.junit.Assert.assertEquals; @@ -23,7 +38,7 @@ public class SimpleChunkProviderTests { @Test public void testProvide() throws Exception { - provider = new SimpleChunkProvider(new ListItemReader(Arrays.asList("foo", "bar")), + provider = new SimpleChunkProvider<>(new ListItemReader<>(Arrays.asList("foo", "bar")), new RepeatTemplate()); Chunk chunk = provider.provide(contribution); assertNotNull(chunk); @@ -32,7 +47,7 @@ public void testProvide() throws Exception { @Test public void testProvideWithOverflow() throws Exception { - provider = new SimpleChunkProvider(new ListItemReader(Arrays.asList("foo", "bar")), + provider = new SimpleChunkProvider(new ListItemReader<>(Arrays.asList("foo", "bar")), new RepeatTemplate()) { @Override protected String read(StepContribution contribution, Chunk chunk) throws SkipOverflowException, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleRetryExceptionHandlerTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleRetryExceptionHandlerTests.java index b4e3df0ece..a43cf6e770 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleRetryExceptionHandlerTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleRetryExceptionHandlerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleStepFactoryBeanTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleStepFactoryBeanTests.java index b78d6b48b1..3e3c3250bc 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleStepFactoryBeanTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SimpleStepFactoryBeanTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,7 +27,9 @@ import java.util.List; import org.junit.Before; +import org.junit.Rule; import org.junit.Test; +import org.junit.rules.ExpectedException; import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.ChunkListener; import org.springframework.batch.core.ItemProcessListener; @@ -57,18 +59,22 @@ import org.springframework.batch.repeat.policy.SimpleCompletionPolicy; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.lang.Nullable; /** * Tests for {@link SimpleStepFactoryBean}. */ public class SimpleStepFactoryBeanTests { - private List listened = new ArrayList(); + @Rule + public ExpectedException expectedException = ExpectedException.none(); + + private List listened = new ArrayList<>(); private SimpleJobRepository repository = new SimpleJobRepository(new MapJobInstanceDao(), new MapJobExecutionDao(), new MapStepExecutionDao(), new MapExecutionContextDao()); - private List written = new ArrayList(); + private List written = new ArrayList<>(); private ItemWriter writer = new ItemWriter() { @Override @@ -77,7 +83,7 @@ public void write(List data) throws Exception { } }; - private ItemReader reader; + private ItemReader reader = new ListItemReader<>(Arrays.asList("a", "b", "c")); private SimpleJob job = new SimpleJob(); @@ -92,10 +98,32 @@ public void testMandatoryProperties() throws Exception { new SimpleStepFactoryBean().getObject(); } + @Test + public void testMandatoryReader() throws Exception { + SimpleStepFactoryBean factory = new SimpleStepFactoryBean<>(); + factory.setItemWriter(writer); + + expectedException.expect(IllegalStateException.class); + expectedException.expectMessage("ItemReader must be provided"); + + factory.getObject(); + } + + @Test + public void testMandatoryWriter() throws Exception { + SimpleStepFactoryBean factory = new SimpleStepFactoryBean<>(); + factory.setItemReader(reader); + + expectedException.expect(IllegalStateException.class); + expectedException.expectMessage("ItemWriter must be provided"); + + factory.getObject(); + } + @Test public void testSimpleJob() throws Exception { - job.setSteps(new ArrayList()); + job.setSteps(new ArrayList<>()); AbstractStep step = (AbstractStep) getStepFactory("foo", "bar").getObject(); step.setName("step1"); job.addStep(step); @@ -153,7 +181,7 @@ public void onWriteError(Exception ex, List item) { } } }); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); job.setSteps(Collections.singletonList(step)); @@ -346,7 +374,7 @@ public void testAutoRegisterItemListeners() throws Exception { SimpleStepFactoryBean factory = getStepFactory(new String[] { "foo", "bar", "spam" }); - final List listenerCalls = new ArrayList(); + final List listenerCalls = new ArrayList<>(); class TestItemListenerWriter implements ItemWriter, ItemProcessor, ItemReadListener, ItemWriteListener, ItemProcessListener, ChunkListener { @@ -354,6 +382,7 @@ class TestItemListenerWriter implements ItemWriter, ItemProcessor items) throws Exception { } + @Nullable @Override public String process(String item) throws Exception { return item; @@ -386,7 +415,7 @@ public void onWriteError(Exception exception, List items) { } @Override - public void afterProcess(String item, String result) { + public void afterProcess(String item, @Nullable String result) { listenerCalls.add("process"); } @@ -417,7 +446,7 @@ public void afterChunkError(ChunkContext context) { factory.setItemWriter(itemWriter); factory.setItemProcessor(itemWriter); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); job.setSteps(Collections.singletonList(step)); @@ -436,7 +465,7 @@ public void testAutoRegisterItemListenersNoDoubleCounting() throws Exception { SimpleStepFactoryBean factory = getStepFactory(new String[] { "foo", "bar", "spam" }); - final List listenerCalls = new ArrayList(); + final List listenerCalls = new ArrayList<>(); class TestItemListenerWriter implements ItemWriter, ItemWriteListener { @Override @@ -462,7 +491,7 @@ public void onWriteError(Exception exception, List items) { factory.setListeners(new StepListener[] { itemWriter }); factory.setItemWriter(itemWriter); - Step step = (Step) factory.getObject(); + Step step = factory.getObject(); job.setSteps(Collections.singletonList(step)); @@ -475,38 +504,12 @@ public void onWriteError(Exception exception, List items) { } - @Test - public void testNullWriter() throws Exception { - - SimpleStepFactoryBean factory = getStepFactory(new String[] { "foo", "bar", "spam" }); - factory.setItemWriter(null); - factory.setItemProcessor(new ItemProcessor() { - @Override - public String process(String item) throws Exception { - written.add(item); - return null; - } - }); - - Step step = (Step) factory.getObject(); - - job.setSteps(Collections.singletonList(step)); - - JobExecution jobExecution = repository.createJobExecution(job.getName(), new JobParameters()); - - job.execute(jobExecution); - - assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); - assertEquals("[foo, bar, spam]", written.toString()); - - } - private SimpleStepFactoryBean getStepFactory(String... args) throws Exception { - SimpleStepFactoryBean factory = new SimpleStepFactoryBean(); + SimpleStepFactoryBean factory = new SimpleStepFactoryBean<>(); - List items = new ArrayList(); + List items = new ArrayList<>(); items.addAll(Arrays.asList(args)); - reader = new ListItemReader(items); + reader = new ListItemReader<>(items); factory.setItemReader(reader); factory.setItemWriter(writer); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipProcessorStub.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipProcessorStub.java index 195d4de096..dc83aea493 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipProcessorStub.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipProcessorStub.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ import org.springframework.batch.item.ItemProcessor; import org.springframework.batch.support.transaction.TransactionAwareProxyFactory; +import org.springframework.lang.Nullable; /** * @author Dan Garrette @@ -27,7 +28,7 @@ */ public class SkipProcessorStub extends AbstractExceptionThrowingItemHandlerStub implements ItemProcessor { - private List processed = new ArrayList(); + private List processed = new ArrayList<>(); private List committed = TransactionAwareProxyFactory.createTransactionalList(); @@ -55,6 +56,7 @@ public void clear() { filter = false; } + @Nullable @Override public T process(T item) throws Exception { processed.add(item); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipReaderStub.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipReaderStub.java index 3e510bb756..9aedbf1ad5 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipReaderStub.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipReaderStub.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,7 @@ import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ParseException; import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -31,7 +32,7 @@ public class SkipReaderStub extends AbstractExceptionThrowingItemHandlerStub< private T[] items; - private List read = new ArrayList(); + private List read = new ArrayList<>(); private int counter = -1; @@ -39,11 +40,13 @@ public SkipReaderStub() throws Exception { super(); } + @SuppressWarnings("unchecked") public SkipReaderStub(T... items) throws Exception { super(); this.items = items; } + @SuppressWarnings("unchecked") public void setItems(T... items) { Assert.isTrue(counter < 0, "Items cannot be set once reading has started"); this.items = items; @@ -58,6 +61,7 @@ public void clear() { read.clear(); } + @Nullable @Override public T read() throws Exception, UnexpectedInputException, ParseException { counter++; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipWrapperTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipWrapperTests.java index ec393d3157..b50d397987 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipWrapperTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipWrapperTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -33,7 +33,7 @@ public class SkipWrapperTests { */ @Test public void testItemWrapperT() { - SkipWrapper wrapper = new SkipWrapper("foo"); + SkipWrapper wrapper = new SkipWrapper<>("foo"); assertEquals("foo", wrapper.getItem()); assertEquals(null, wrapper.getException()); } @@ -43,7 +43,7 @@ public void testItemWrapperT() { */ @Test public void testItemWrapperTException() { - SkipWrapper wrapper = new SkipWrapper("foo",exception); + SkipWrapper wrapper = new SkipWrapper<>("foo", exception); assertEquals("foo", wrapper.getItem()); assertEquals(exception, wrapper.getException()); } @@ -53,7 +53,7 @@ public void testItemWrapperTException() { */ @Test public void testToString() { - SkipWrapper wrapper = new SkipWrapper("foo"); + SkipWrapper wrapper = new SkipWrapper<>("foo"); assertTrue("foo", wrapper.toString().contains("foo")); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipWriterStub.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipWriterStub.java index 048f85fbbd..b153343746 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipWriterStub.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkipWriterStub.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,7 +27,7 @@ */ public class SkipWriterStub extends AbstractExceptionThrowingItemHandlerStub implements ItemWriter { - private List written = new ArrayList(); + private List written = new ArrayList<>(); private List committed = TransactionAwareProxyFactory.createTransactionalList(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkippableException.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkippableException.java index fb4e1290d5..51321b4f2c 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkippableException.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkippableException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,6 +19,7 @@ * @author Dan Garrette * @since 2.0.1 */ +@SuppressWarnings("serial") public class SkippableException extends Exception { public SkippableException(String message) { super(message); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkippableRuntimeException.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkippableRuntimeException.java index 2ece29799f..dccc675795 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkippableRuntimeException.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/SkippableRuntimeException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,6 +19,7 @@ * @author Dan Garrette * @since 2.0.1 */ +@SuppressWarnings("serial") public class SkippableRuntimeException extends RuntimeException { public SkippableRuntimeException(String message) { super(message); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/TaskletStepExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/TaskletStepExceptionTests.java index 922d18f8cd..96f0e29cee 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/TaskletStepExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/item/TaskletStepExceptionTests.java @@ -1,15 +1,20 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.item; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; -import static org.springframework.batch.core.BatchStatus.COMPLETED; -import static org.springframework.batch.core.BatchStatus.FAILED; -import static org.springframework.batch.core.BatchStatus.STOPPED; -import static org.springframework.batch.core.BatchStatus.UNKNOWN; - -import java.util.Collection; - import org.junit.Before; import org.junit.Test; import org.springframework.batch.core.ExitStatus; @@ -17,6 +22,7 @@ import org.springframework.batch.core.JobInstance; import org.springframework.batch.core.JobInterruptedException; import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.Step; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.StepExecutionListener; @@ -34,11 +40,22 @@ import org.springframework.batch.item.ItemStreamSupport; import org.springframework.batch.repeat.RepeatStatus; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; +import org.springframework.lang.Nullable; import org.springframework.transaction.TransactionException; import org.springframework.transaction.UnexpectedRollbackException; import org.springframework.transaction.support.DefaultTransactionStatus; import org.springframework.transaction.support.TransactionSynchronizationManager; +import java.util.Collection; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; +import static org.springframework.batch.core.BatchStatus.COMPLETED; +import static org.springframework.batch.core.BatchStatus.FAILED; +import static org.springframework.batch.core.BatchStatus.STOPPED; +import static org.springframework.batch.core.BatchStatus.UNKNOWN; + /** * Tests for the behavior of TaskletStep in a failure scenario. * @@ -90,6 +107,7 @@ public void testInterrupted() throws Exception { @Test public void testInterruptedWithCustomStatus() throws Exception { taskletStep.setTasklet(new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { contribution.setExitStatus(new ExitStatus("FUNNY")); @@ -138,6 +156,7 @@ public void testAfterStepFailureWhenTaskletSucceeds() throws Exception { final RuntimeException exception = new RuntimeException(); taskletStep.setStepExecutionListeners(new StepExecutionListenerSupport[] { new StepExecutionListenerSupport() { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { throw exception; @@ -145,6 +164,7 @@ public ExitStatus afterStep(StepExecution stepExecution) { } }); taskletStep.setTasklet(new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext attributes) throws Exception { return RepeatStatus.FINISHED; @@ -165,6 +185,7 @@ public void testAfterStepFailureWhenTaskletFails() throws Exception { final RuntimeException exception = new RuntimeException(); taskletStep.setStepExecutionListeners(new StepExecutionListenerSupport[] { new StepExecutionListenerSupport() { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { throw exception; @@ -214,6 +235,7 @@ protected void doRollback(DefaultTransactionStatus status) throws TransactionExc taskletStep.setTasklet(new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext attributes) throws Exception { attributes.getStepContext().getStepExecution().getExecutionContext().putString("foo", "bar"); @@ -230,7 +252,9 @@ public RepeatStatus execute(StepContribution contribution, ChunkContext attribut assertEquals(1, stepExecution.getRollbackCount()); // Failed transaction // counts as // rollback - assertEquals(0, stepExecution.getExecutionContext().size()); + assertEquals(2, stepExecution.getExecutionContext().size()); + assertTrue(stepExecution.getExecutionContext().containsKey(Step.STEP_TYPE_KEY)); + assertTrue(stepExecution.getExecutionContext().containsKey(TaskletStep.TASKLET_TYPE_KEY)); } @SuppressWarnings("serial") @@ -247,6 +271,7 @@ protected void doCommit(DefaultTransactionStatus status) throws TransactionExcep taskletStep.setTasklet(new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext attributes) throws Exception { attributes.getStepContext().getStepExecution().getExecutionContext().putString("foo", "bar"); @@ -263,7 +288,9 @@ public RepeatStatus execute(StepContribution contribution, ChunkContext attribut assertEquals(1, stepExecution.getRollbackCount()); // Failed transaction // counts as // rollback - assertEquals(0, stepExecution.getExecutionContext().size()); + assertEquals(2, stepExecution.getExecutionContext().size()); + assertTrue(stepExecution.getExecutionContext().containsKey(Step.STEP_TYPE_KEY)); + assertTrue(stepExecution.getExecutionContext().containsKey(TaskletStep.TASKLET_TYPE_KEY)); } @Test @@ -271,6 +298,7 @@ public void testRepositoryErrorOnExecutionContext() throws Exception { taskletStep.setTasklet(new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext attributes) throws Exception { return RepeatStatus.FINISHED; @@ -291,6 +319,7 @@ public void testRepositoryErrorOnExecutionContextInTransaction() throws Exceptio taskletStep.setTasklet(new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext attributes) throws Exception { return RepeatStatus.FINISHED; @@ -298,12 +327,36 @@ public RepeatStatus execute(StepContribution contribution, ChunkContext attribut }); + jobRepository.setFailOnUpdateExecutionContext(true); + jobRepository.setFailInTransaction(true); + taskletStep.execute(stepExecution); + assertEquals(FAILED, stepExecution.getStatus()); + Throwable e = stepExecution.getFailureExceptions().get(0); + assertEquals("JobRepository failure forcing rollback", e.getMessage()); + + } + + @Test + public void testRepositoryErrorOnExecutionContextInTransactionRollbackFailed() throws Exception { + + taskletStep.setTasklet(new Tasklet() { + + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext attributes) throws Exception { + return RepeatStatus.FINISHED; + } + + }); + + taskletStep.setTransactionManager(new FailingRollbackTransactionManager()); + jobRepository.setFailOnUpdateExecutionContext(true); jobRepository.setFailInTransaction(true); taskletStep.execute(stepExecution); assertEquals(UNKNOWN, stepExecution.getStatus()); Throwable e = stepExecution.getFailureExceptions().get(0); - assertEquals("JobRepository failure forcing exit with unknown status", e.getMessage()); + assertEquals("Expected exception in rollback", e.getMessage()); } @@ -312,6 +365,28 @@ public void testRepositoryErrorOnUpdateStepExecution() throws Exception { taskletStep.setTasklet(new Tasklet() { + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext attributes) throws Exception { + return RepeatStatus.FINISHED; + } + + }); + + jobRepository.setFailOnUpdateStepExecution(2); + taskletStep.execute(stepExecution); + assertEquals(UNKNOWN, stepExecution.getStatus()); + Throwable e = stepExecution.getFailureExceptions().get(0); + assertEquals("Expected exception in step execution persistence", e.getMessage()); + + } + + @Test + public void testRepositoryErrorOnUpdateStepExecutionInTransaction() throws Exception { + + taskletStep.setTasklet(new Tasklet() { + + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext attributes) throws Exception { return RepeatStatus.FINISHED; @@ -320,10 +395,35 @@ public RepeatStatus execute(StepContribution contribution, ChunkContext attribut }); jobRepository.setFailOnUpdateStepExecution(1); + jobRepository.setFailInTransaction(true); + taskletStep.execute(stepExecution); + assertEquals(FAILED, stepExecution.getStatus()); + Throwable e = stepExecution.getFailureExceptions().get(0); + assertEquals("JobRepository failure forcing rollback", e.getMessage()); + + } + + @Test + public void testRepositoryErrorOnUpdateStepExecutionInTransactionRollbackFailed() throws Exception { + + taskletStep.setTasklet(new Tasklet() { + + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext attributes) throws Exception { + return RepeatStatus.FINISHED; + } + + }); + + taskletStep.setTransactionManager(new FailingRollbackTransactionManager()); + + jobRepository.setFailOnUpdateStepExecution(1); + jobRepository.setFailInTransaction(true); taskletStep.execute(stepExecution); assertEquals(UNKNOWN, stepExecution.getStatus()); Throwable e = stepExecution.getFailureExceptions().get(0); - assertEquals("JobRepository failure forcing exit with unknown status", e.getMessage()); + assertEquals("Expected exception in rollback", e.getMessage()); } @@ -332,6 +432,7 @@ public void testRepositoryErrorOnFailure() throws Exception { taskletStep.setTasklet(new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext attributes) throws Exception { throw new RuntimeException("Tasklet exception"); @@ -372,6 +473,7 @@ public void update(StepExecution arg0) { private static class ExceptionTasklet implements Tasklet { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { throw taskletException; @@ -418,6 +520,7 @@ public JobExecution createJobExecution(String jobName, JobParameters jobParamete return null; } + @Nullable @Override public StepExecution getLastStepExecution(JobInstance jobInstance, String stepName) { return null; @@ -439,10 +542,12 @@ public void update(JobExecution jobExecution) { @Override public void update(StepExecution stepExecution) { - if (updateCount == failOnUpdateExecution) { - throw new RuntimeException("Expected exception in step execution persistence"); + if (updateCount++ == failOnUpdateExecution) { + if (!failInTransaction + || (failInTransaction && TransactionSynchronizationManager.isActualTransactionActive())) { + throw new RuntimeException("Expected exception in step execution persistence"); + } } - updateCount++; } @Override @@ -459,6 +564,7 @@ public int getUpdateCount() { return updateCount; } + @Nullable @Override public JobExecution getLastJobExecution(String jobName, JobParameters jobParameters) { return null; @@ -471,6 +577,28 @@ public void updateExecutionContext(JobExecution jobExecution) { @Override public void addAll(Collection stepExecutions) { } + + @Override + public JobInstance createJobInstance(String jobName, + JobParameters jobParameters) { + return null; + } + + @Override + public JobExecution createJobExecution(JobInstance jobInstance, + JobParameters jobParameters, String jobConfigurationLocation) { + return null; + } + } + + @SuppressWarnings("serial") + private static class FailingRollbackTransactionManager extends ResourcelessTransactionManager { + + @Override + protected void doRollback(DefaultTransactionStatus status) throws TransactionException { + super.doRollback(status); + throw new RuntimeException("Expected exception in rollback"); + } } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/job/DefaultJobParametersExtractorJobParametersTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/job/DefaultJobParametersExtractorJobParametersTests.java index fde0762109..b1279dde51 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/job/DefaultJobParametersExtractorJobParametersTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/job/DefaultJobParametersExtractorJobParametersTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -49,7 +49,8 @@ public void testGetAllJobParameters() throws Exception { StepExecution stepExecution = getStepExecution("foo=bar,spam=bucket"); extractor.setKeys(new String[] {"foo", "bar"}); JobParameters jobParameters = extractor.getJobParameters(null, stepExecution); - assertEquals("{spam=bucket, foo=bar}", jobParameters.toString()); + assertEquals("bar", jobParameters.getString("foo")); + assertEquals("bucket", jobParameters.getString("spam")); } @Test diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/job/DefaultJobParametersExtractorTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/job/DefaultJobParametersExtractorTests.java index 6eadff04c9..188a28a59f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/job/DefaultJobParametersExtractorTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/job/DefaultJobParametersExtractorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ import org.junit.Test; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; import org.springframework.batch.core.StepExecution; /** @@ -88,4 +89,39 @@ public void testGetNamedDateJobParameters() throws Exception { assertEquals("{foo="+date.getTime()+"}", jobParameters.toString()); } + @Test + public void testUseParentParameters() throws Exception { + JobExecution jobExecution = new JobExecution(0L, new JobParametersBuilder() + .addString("parentParam", "val") + .toJobParameters()); + + StepExecution stepExecution = new StepExecution("step", jobExecution); + + stepExecution.getExecutionContext().putDouble("foo", 11.1); + extractor.setKeys(new String[] {"foo(double)"}); + JobParameters jobParameters = extractor.getJobParameters(null, stepExecution); + + String jobParams = jobParameters.toString(); + + assertTrue("Job parameters must contain parentParam=val", jobParams.contains("parentParam=val")); + assertTrue("Job parameters must contain foo=11.1", jobParams.contains("foo=11.1")); + } + + @Test + public void testDontUseParentParameters() throws Exception { + DefaultJobParametersExtractor extractor = new DefaultJobParametersExtractor(); + extractor.setUseAllParentParameters(false); + + JobExecution jobExecution = new JobExecution(0L, new JobParametersBuilder() + .addString("parentParam", "val") + .toJobParameters()); + + StepExecution stepExecution = new StepExecution("step", jobExecution); + + stepExecution.getExecutionContext().putDouble("foo", 11.1); + extractor.setKeys(new String[] {"foo(double)"}); + JobParameters jobParameters = extractor.getJobParameters(null, stepExecution); + + assertEquals("{foo=11.1}", jobParameters.toString()); + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/job/JobStepTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/job/JobStepTests.java index 5914fb57a0..c467cb3337 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/job/JobStepTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/job/JobStepTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2010 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,14 +15,13 @@ */ package org.springframework.batch.core.step.job; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; - import java.util.Date; import org.junit.Before; import org.junit.Test; + import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.StepExecution; @@ -33,6 +32,9 @@ import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean; import org.springframework.batch.item.ExecutionContext; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + /** * @author Dave Syer * @@ -49,7 +51,7 @@ public class JobStepTests { public void setUp() throws Exception { step.setName("step"); MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(); - jobRepository = (JobRepository) factory.getObject(); + jobRepository = factory.getObject(); step.setJobRepository(jobRepository); JobExecution jobExecution = jobRepository.createJobExecution("job", new JobParameters()); stepExecution = jobExecution.createStepExecution("step"); @@ -171,4 +173,50 @@ public boolean isRestartable() { } + @Test + public void testStoppedChild() throws Exception { + + DefaultJobParametersExtractor jobParametersExtractor = new DefaultJobParametersExtractor(); + jobParametersExtractor.setKeys(new String[] {"foo"}); + ExecutionContext executionContext = stepExecution.getExecutionContext(); + executionContext.put("foo", "bar"); + step.setJobParametersExtractor(jobParametersExtractor); + + step.setJob(new JobSupport("child") { + @Override + public void execute(JobExecution execution) { + assertEquals(1, execution.getJobParameters().getParameters().size()); + execution.setStatus(BatchStatus.STOPPED); + execution.setEndTime(new Date()); + jobRepository.update(execution); + } + @Override + public boolean isRestartable() { + return true; + } + }); + + step.afterPropertiesSet(); + step.execute(stepExecution); + JobExecution jobExecution = stepExecution.getJobExecution(); + jobExecution.setEndTime(new Date()); + jobRepository.update(jobExecution); + + assertEquals(BatchStatus.STOPPED, stepExecution.getStatus()); + } + + @Test + public void testStepExecutionExitStatus() throws Exception { + step.setJob(new JobSupport("child") { + @Override + public void execute(JobExecution execution) throws UnexpectedJobExecutionException { + execution.setStatus(BatchStatus.COMPLETED); + execution.setExitStatus(new ExitStatus("CUSTOM")); + execution.setEndTime(new Date()); + } + }); + step.afterPropertiesSet(); + step.execute(stepExecution); + assertEquals("CUSTOM", stepExecution.getExitStatus().getExitCode()); + } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/LimitCheckingItemSkipPolicyTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/LimitCheckingItemSkipPolicyTests.java index 18fb214504..d89bb315eb 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/LimitCheckingItemSkipPolicyTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/LimitCheckingItemSkipPolicyTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -41,7 +41,7 @@ public class LimitCheckingItemSkipPolicyTests { @Before public void setUp() throws Exception { - Map, Boolean> skippableExceptions = new HashMap, Boolean>(); + Map, Boolean> skippableExceptions = new HashMap<>(); skippableExceptions.put(FlatFileParseException.class, true); failurePolicy = new LimitCheckingItemSkipPolicy(1, skippableExceptions); } @@ -67,7 +67,7 @@ public void testSkip() { } private LimitCheckingItemSkipPolicy getSkippableSubsetSkipPolicy() { - Map, Boolean> skippableExceptions = new HashMap, Boolean>(); + Map, Boolean> skippableExceptions = new HashMap<>(); skippableExceptions.put(WriteFailedException.class, true); skippableExceptions.put(ItemWriterException.class, false); return new LimitCheckingItemSkipPolicy(1, skippableExceptions); @@ -104,7 +104,7 @@ public void testSkippableSubset_fatal() { } private LimitCheckingItemSkipPolicy getFatalSubsetSkipPolicy() { - Map, Boolean> skippableExceptions = new HashMap, Boolean>(); + Map, Boolean> skippableExceptions = new HashMap<>(); skippableExceptions.put(WriteFailedException.class, false); skippableExceptions.put(ItemWriterException.class, true); return new LimitCheckingItemSkipPolicy(1, skippableExceptions); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/NonSkippableReadExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/NonSkippableReadExceptionTests.java index 586bd31e75..6f0dd531d9 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/NonSkippableReadExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/NonSkippableReadExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/NonSkippableWriteExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/NonSkippableWriteExceptionTests.java index 3a20283506..1be5d29a0e 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/NonSkippableWriteExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/NonSkippableWriteExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/ReprocessExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/ReprocessExceptionTests.java new file mode 100644 index 0000000000..074ac5a1e2 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/ReprocessExceptionTests.java @@ -0,0 +1,125 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.step.skip; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemWriter; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.lang.Nullable; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +import java.util.List; + +import static org.junit.Assert.assertEquals; + +/** + * @author mminella + */ +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class ReprocessExceptionTests { + + @Autowired + public Job job; + + @Autowired + public JobLauncher jobLauncher; + + @Test + public void testReprocessException() throws Exception { + JobExecution execution = jobLauncher.run(job, new JobParametersBuilder().toJobParameters()); + + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + } + + public static class PersonProcessor implements ItemProcessor { + + private String mostRecentFirstName; + + @Nullable + @Override + public Person process(final Person person) throws Exception { + if (person.getFirstName().equals(mostRecentFirstName)) { + throw new RuntimeException("throwing a exception during process after a rollback"); + } + mostRecentFirstName = person.getFirstName(); + + final String firstName = person.getFirstName().toUpperCase(); + final String lastName = person.getLastName().toUpperCase(); + + final Person transformedPerson = new Person(firstName, lastName); + + System.out.println("Converting (" + person + ") into (" + transformedPerson + ")"); + + return transformedPerson; + } + } + + public static class PersonItemWriter implements ItemWriter { + @Override + public void write(List persons) throws Exception { + for (Person person : persons) { + System.out.println(person.getFirstName() + " " + person.getLastName()); + if (person.getFirstName().equals("JANE")) { + throw new RuntimeException("jane doe write exception causing rollback"); + } + } + } + } + + public static class Person { + private String lastName; + private String firstName; + + public Person() { + + } + + public Person(String firstName, String lastName) { + this.firstName = firstName; + this.lastName = lastName; + } + + public void setFirstName(String firstName) { + this.firstName = firstName; + } + + public String getFirstName() { + return firstName; + } + + public String getLastName() { + return lastName; + } + + public void setLastName(String lastName) { + this.lastName = lastName; + } + + @Override + public String toString() { + return "firstName: " + firstName + ", lastName: " + lastName; + } + } +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/SkipListenerFailedExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/SkipListenerFailedExceptionTests.java index f2e660a2fa..a8f86f9f3d 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/SkipListenerFailedExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/SkipListenerFailedExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/SkipPolicyFailedExceptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/SkipPolicyFailedExceptionTests.java index 85dbf1f0ff..3b7545a011 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/SkipPolicyFailedExceptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/skip/SkipPolicyFailedExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/AsyncChunkOrientedStepIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/AsyncChunkOrientedStepIntegrationTests.java index 8db0ffd12c..1ec37dadb3 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/AsyncChunkOrientedStepIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/AsyncChunkOrientedStepIntegrationTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,19 +15,18 @@ */ package org.springframework.batch.core.step.tasklet; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; - import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.List; -import org.apache.commons.dbcp.BasicDataSource; +import org.apache.commons.dbcp2.BasicDataSource; import org.junit.After; import org.junit.Before; +import org.junit.Ignore; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; @@ -51,9 +50,12 @@ import org.springframework.transaction.support.TransactionCallback; import org.springframework.transaction.support.TransactionTemplate; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; + /** * @author Dave Syer - * + * */ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "/org/springframework/batch/core/repository/dao/sql-dao-test.xml") @@ -63,7 +65,7 @@ public class AsyncChunkOrientedStepIntegrationTests { private Job job; - private List written = new ArrayList(); + private List written = new ArrayList<>(); @Autowired private PlatformTransactionManager transactionManager; @@ -81,24 +83,24 @@ public class AsyncChunkOrientedStepIntegrationTests { private int maxIdle; private ItemReader getReader(String[] args) { - return new ListItemReader(Arrays.asList(args)); + return new ListItemReader<>(Arrays.asList(args)); } @After public void reset() { // Reset concurrency settings to something reasonable - dataSource.setMaxActive(maxActive); + dataSource.setMaxTotal(maxActive); dataSource.setMaxIdle(maxIdle); } @Before public void init() throws Exception { - maxActive = dataSource.getMaxActive(); + maxActive = dataSource.getMaxTotal(); maxIdle = dataSource.getMaxIdle(); // Force deadlock with batch waiting for DB pool and vice versa - dataSource.setMaxActive(1); + dataSource.setMaxTotal(1); dataSource.setMaxIdle(1); step = new TaskletStep("stepName"); @@ -119,9 +121,10 @@ public void init() throws Exception { } @Test + @Ignore public void testStatus() throws Exception { - step.setTasklet(new TestingChunkOrientedTasklet(getReader(new String[] { "a", "b", "c", "a", "b", "c", + step.setTasklet(new TestingChunkOrientedTasklet<>(getReader(new String[] { "a", "b", "c", "a", "b", "c", "a", "b", "c", "a", "b", "c" }), new ItemWriter() { @Override public void write(List data) throws Exception { @@ -138,12 +141,12 @@ public void write(List data) throws Exception { assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); // Need a transaction so one connection is enough to get job execution and its parameters StepExecution lastStepExecution = new TransactionTemplate(transactionManager) - .execute(new TransactionCallback() { - @Override - public StepExecution doInTransaction(TransactionStatus status) { - return jobRepository.getLastStepExecution(jobExecution.getJobInstance(), step.getName()); - } - }); + .execute(new TransactionCallback() { + @Override + public StepExecution doInTransaction(TransactionStatus status) { + return jobRepository.getLastStepExecution(jobExecution.getJobInstance(), step.getName()); + } + }); assertEquals(lastStepExecution, stepExecution); assertFalse(lastStepExecution == stepExecution); } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/AsyncTaskletStepTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/AsyncTaskletStepTests.java index 3be1e951a1..a422a7199d 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/AsyncTaskletStepTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/AsyncTaskletStepTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -44,13 +44,14 @@ import org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.lang.Nullable; import org.springframework.util.StringUtils; public class AsyncTaskletStepTests { private static Log logger = LogFactory.getLog(AsyncTaskletStepTests.class); - private List processed = new CopyOnWriteArrayList(); + private List processed = new CopyOnWriteArrayList<>(); private TaskletStep step; @@ -74,7 +75,7 @@ public void write(List data) throws Exception { private int concurrencyLimit = 300; - private ItemProcessor itemProcessor = new PassThroughItemProcessor(); + private ItemProcessor itemProcessor = new PassThroughItemProcessor<>(); private void setUp() throws Exception { @@ -85,7 +86,7 @@ private void setUp() throws Exception { RepeatTemplate chunkTemplate = new RepeatTemplate(); chunkTemplate.setCompletionPolicy(new SimpleCompletionPolicy(2)); - step.setTasklet(new TestingChunkOrientedTasklet(new ListItemReader(items), itemProcessor, itemWriter, + step.setTasklet(new TestingChunkOrientedTasklet<>(new ListItemReader<>(items), itemProcessor, itemWriter, chunkTemplate)); jobRepository = new JobRepositorySupport(); @@ -116,7 +117,7 @@ public void update(ExecutionContext executionContext) { @Test public void testStepExecutionUpdates() throws Exception { - items = new ArrayList(Arrays.asList(StringUtils + items = new ArrayList<>(Arrays.asList(StringUtils .commaDelimitedListToStringArray("1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25"))); setUp(); @@ -175,6 +176,7 @@ public void testStepExecutionFailsWithProcessor() throws Exception { concurrencyLimit = 1; items = Arrays.asList("one", "barf", "three", "four"); itemProcessor = new ItemProcessor() { + @Nullable @Override public String process(String item) throws Exception { logger.info("Item: "+item); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/CallableTaskletAdapterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/CallableTaskletAdapterTests.java index 5254de2fe4..8164de2e70 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/CallableTaskletAdapterTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/CallableTaskletAdapterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/ChunkOrientedStepIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/ChunkOrientedStepIntegrationTests.java index 864ebc288d..a4282f20b2 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/ChunkOrientedStepIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/ChunkOrientedStepIntegrationTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,14 +15,8 @@ */ package org.springframework.batch.core.step.tasklet; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; - -import java.util.Arrays; -import java.util.Collections; -import java.util.List; - import org.junit.Before; +import org.junit.Ignore; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.batch.core.BatchStatus; @@ -46,6 +40,13 @@ import org.springframework.transaction.support.TransactionSynchronizationAdapter; import org.springframework.transaction.support.TransactionSynchronizationManager; +import java.util.Arrays; +import java.util.Collections; +import java.util.List; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; + /** * @author Dave Syer * @@ -67,7 +68,7 @@ public class ChunkOrientedStepIntegrationTests { private RepeatTemplate chunkOperations; private ItemReader getReader(String[] args) { - return new ListItemReader(Arrays.asList(args)); + return new ListItemReader<>(Arrays.asList(args)); } @Before @@ -89,9 +90,10 @@ public void onSetUp() throws Exception { @SuppressWarnings("serial") @Test + @Ignore public void testStatusForCommitFailedException() throws Exception { - step.setTasklet(new TestingChunkOrientedTasklet(getReader(new String[] { "a", "b", "c" }), + step.setTasklet(new TestingChunkOrientedTasklet<>(getReader(new String[] { "a", "b", "c" }), new ItemWriter() { @Override public void write(List data) throws Exception { diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/ConfigurableSystemProcessExitCodeMapperTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/ConfigurableSystemProcessExitCodeMapperTests.java index 52d212dee3..55d3cf420f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/ConfigurableSystemProcessExitCodeMapperTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/ConfigurableSystemProcessExitCodeMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.tasklet; import static org.junit.Assert.assertSame; @@ -23,6 +38,7 @@ public class ConfigurableSystemProcessExitCodeMapperTests { */ @Test public void testMapping() { + @SuppressWarnings("serial") Map mappings = new HashMap() { { put(0, ExitStatus.COMPLETED); @@ -54,7 +70,7 @@ public void testMapping() { */ @Test public void testSetMappingsMissingElseClause() { - Map missingElse = new HashMap(); + Map missingElse = new HashMap<>(); try { mapper.setMappings(missingElse); fail(); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/MethodInvokingTaskletAdapterTest.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/MethodInvokingTaskletAdapterTest.java new file mode 100644 index 0000000000..fafef8e147 --- /dev/null +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/MethodInvokingTaskletAdapterTest.java @@ -0,0 +1,258 @@ +/* + * Copyright 2006-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.step.tasklet; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.repeat.RepeatStatus; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.mock; + +/** + * @author Mahmoud Ben Hassine + */ +public class MethodInvokingTaskletAdapterTest { + + private StepContribution stepContribution; + private ChunkContext chunkContext; + private TestTasklet tasklet; + private MethodInvokingTaskletAdapter adapter; + + @Before + public void setUp() throws Exception { + stepContribution = new StepContribution(mock(StepExecution.class)); + chunkContext = mock(ChunkContext.class); + tasklet = new TestTasklet(); + adapter = new MethodInvokingTaskletAdapter(); + adapter.setTargetObject(tasklet); + } + + @Test + public void testExactlySameSignature() throws Exception { + adapter.setTargetMethod("execute"); + RepeatStatus repeatStatus = adapter.execute(stepContribution, chunkContext); + assertEquals(RepeatStatus.FINISHED, repeatStatus); + assertEquals(tasklet.getStepContribution(), stepContribution); + assertEquals(tasklet.getChunkContext(), chunkContext); + } + + @Test + public void testSameSignatureWithDifferentMethodName() throws Exception { + adapter.setTargetMethod("execute1"); + RepeatStatus repeatStatus = adapter.execute(stepContribution, chunkContext); + assertEquals(RepeatStatus.FINISHED, repeatStatus); + assertEquals(tasklet.getStepContribution(), stepContribution); + assertEquals(tasklet.getChunkContext(), chunkContext); + } + + @Test + public void testDifferentParametersOrder() throws Exception { + adapter.setTargetMethod("execute2"); + RepeatStatus repeatStatus = adapter.execute(stepContribution, chunkContext); + assertEquals(RepeatStatus.FINISHED, repeatStatus); + assertEquals(tasklet.getStepContribution(), stepContribution); + assertEquals(tasklet.getChunkContext(), chunkContext); + } + + @Test + public void testArgumentSubsetWithOnlyChunkContext() throws Exception { + adapter.setTargetMethod("execute3"); + RepeatStatus repeatStatus = adapter.execute(stepContribution, chunkContext); + assertEquals(RepeatStatus.FINISHED, repeatStatus); + assertEquals(tasklet.getChunkContext(), chunkContext); + } + + @Test + public void testArgumentSubsetWithOnlyStepContribution() throws Exception { + adapter.setTargetMethod("execute4"); + RepeatStatus repeatStatus = adapter.execute(stepContribution, chunkContext); + assertEquals(RepeatStatus.FINISHED, repeatStatus); + assertEquals(tasklet.getStepContribution(), stepContribution); + } + + @Test + public void testArgumentSubsetWithoutArguments() throws Exception { + adapter.setTargetMethod("execute5"); + RepeatStatus repeatStatus = adapter.execute(stepContribution, chunkContext); + assertEquals(RepeatStatus.FINISHED, repeatStatus); + } + + @Test + public void testCompatibleReturnTypeWhenBoolean() throws Exception { + adapter.setTargetMethod("execute6"); + RepeatStatus repeatStatus = adapter.execute(stepContribution, chunkContext); + assertEquals(RepeatStatus.FINISHED, repeatStatus); + } + + @Test + public void testCompatibleReturnTypeWhenVoid() throws Exception { + adapter.setTargetMethod("execute7"); + RepeatStatus repeatStatus = adapter.execute(stepContribution, chunkContext); + assertEquals(RepeatStatus.FINISHED, repeatStatus); + } + + @Test + public void testArgumentSubsetWithOnlyStepContributionAndCompatibleReturnTypeBoolean() throws Exception { + adapter.setTargetMethod("execute8"); + RepeatStatus repeatStatus = adapter.execute(stepContribution, chunkContext); + assertEquals(RepeatStatus.FINISHED, repeatStatus); + assertEquals(tasklet.getStepContribution(), stepContribution); + } + + @Test + public void testArgumentSubsetWithOnlyChunkContextAndCompatibleReturnTypeVoid() throws Exception { + adapter.setTargetMethod("execute9"); + RepeatStatus repeatStatus = adapter.execute(stepContribution, chunkContext); + assertEquals(RepeatStatus.FINISHED, repeatStatus); + assertEquals(tasklet.getChunkContext(), chunkContext); + } + + @Test(expected = IllegalArgumentException.class) + public void testIncorrectSignatureWithExtraParameter() throws Exception { + adapter.setTargetMethod("execute10"); + adapter.execute(stepContribution, chunkContext); + } + + @Test + public void testExitStatusReturnType() throws Exception { + adapter.setTargetMethod("execute11"); + adapter.execute(stepContribution, chunkContext); + assertEquals(new ExitStatus("DONE"), stepContribution.getExitStatus()); + } + + @Test + public void testNonExitStatusReturnType() throws Exception { + adapter.setTargetMethod("execute12"); + RepeatStatus repeatStatus = adapter.execute(stepContribution, chunkContext); + assertEquals(RepeatStatus.FINISHED, repeatStatus); + assertEquals(ExitStatus.COMPLETED, stepContribution.getExitStatus()); + } + + /* + + + + If the tasklet is specified as a bean definition, then a method can be + specified and a POJO will be adapted to the Tasklet interface. + The method suggested should have the same arguments as Tasklet.execute + (or a subset), and have a compatible return type (boolean, void or RepeatStatus). + + + + */ + public static class TestTasklet { + + private StepContribution stepContribution; + private ChunkContext chunkContext; + + /* exactly same signature */ + public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception { + this.stepContribution = stepContribution; + this.chunkContext = chunkContext; + return RepeatStatus.FINISHED; + } + + /* same signature, different method name */ + public RepeatStatus execute1(StepContribution stepContribution, ChunkContext chunkContext) throws Exception { + this.stepContribution = stepContribution; + this.chunkContext = chunkContext; + return RepeatStatus.FINISHED; + } + + /* different parameters order */ + public RepeatStatus execute2(ChunkContext chunkContext, StepContribution stepContribution) throws Exception { + this.stepContribution = stepContribution; + this.chunkContext = chunkContext; + return RepeatStatus.FINISHED; + } + + /* subset of arguments: only chunk context */ + public RepeatStatus execute3(ChunkContext chunkContext) throws Exception { + this.chunkContext = chunkContext; + return RepeatStatus.FINISHED; + } + + /* subset of arguments: only step contribution */ + public RepeatStatus execute4(StepContribution stepContribution) throws Exception { + this.stepContribution = stepContribution; + return RepeatStatus.FINISHED; + } + + /* subset of arguments: no arguments */ + public RepeatStatus execute5() throws Exception { + return RepeatStatus.FINISHED; + } + + /* compatible return type: boolean */ + public boolean execute6(StepContribution stepContribution, ChunkContext chunkContext) throws Exception { + this.stepContribution = stepContribution; + this.chunkContext = chunkContext; + return true; + } + + /* compatible return type: void */ + public void execute7(StepContribution stepContribution, ChunkContext chunkContext) throws Exception { + this.stepContribution = stepContribution; + this.chunkContext = chunkContext; + } + + /* subset of arguments (only step contribution) and compatible return type (boolean) */ + public boolean execute8(StepContribution stepContribution) throws Exception { + this.stepContribution = stepContribution; + return true; + } + + /* subset of arguments (only chunk context) and compatible return type (void) */ + public void execute9(ChunkContext chunkContext) throws Exception { + this.chunkContext = chunkContext; + } + + /* Incorrect signature: extra parameter (ie a superset not a subset as specified) */ + public RepeatStatus execute10(StepContribution stepContribution, ChunkContext chunkContext, String string) throws Exception { + this.stepContribution = stepContribution; + this.chunkContext = chunkContext; + return RepeatStatus.FINISHED; + } + + /* ExitStatus return type : should be returned as is */ + public ExitStatus execute11(StepContribution stepContribution, ChunkContext chunkContext) throws Exception { + this.stepContribution = stepContribution; + this.chunkContext = chunkContext; + return new ExitStatus("DONE"); + } + + /* Non ExitStatus return type : should return ExitStatus.COMPLETED */ + public String execute12(StepContribution stepContribution, ChunkContext chunkContext) throws Exception { + this.stepContribution = stepContribution; + this.chunkContext = chunkContext; + return "DONE"; + } + + public StepContribution getStepContribution() { + return stepContribution; + } + + public ChunkContext getChunkContext() { + return chunkContext; + } + } +} + diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/SimpleSystemProcessExitCodeMapperTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/SimpleSystemProcessExitCodeMapperTests.java index e6562bedfa..3d8808002b 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/SimpleSystemProcessExitCodeMapperTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/SimpleSystemProcessExitCodeMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.core.step.tasklet; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/StepExecutorInterruptionTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/StepExecutorInterruptionTests.java index 0b20c1be04..8f1908c2e9 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/StepExecutorInterruptionTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/StepExecutorInterruptionTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -45,6 +45,7 @@ import org.springframework.batch.repeat.policy.SimpleCompletionPolicy; import org.springframework.batch.repeat.support.RepeatTemplate; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; +import org.springframework.lang.Nullable; public class StepExecutorInterruptionTests { @@ -92,7 +93,8 @@ public void testInterruptStep() throws Exception { RepeatTemplate template = new RepeatTemplate(); // N.B, If we don't set the completion policy it might run forever template.setCompletionPolicy(new SimpleCompletionPolicy(2)); - step.setTasklet(new TestingChunkOrientedTasklet(new ItemReader() { + step.setTasklet(new TestingChunkOrientedTasklet<>(new ItemReader() { + @Nullable @Override public Object read() throws Exception { // do something non-trivial (and not Thread.sleep()) @@ -154,7 +156,8 @@ public void release() { Thread processingThread = createThread(stepExecution); - step.setTasklet(new TestingChunkOrientedTasklet(new ItemReader() { + step.setTasklet(new TestingChunkOrientedTasklet<>(new ItemReader() { + @Nullable @Override public Object read() throws Exception { return null; @@ -199,7 +202,8 @@ public void release() { } }); - step.setTasklet(new TestingChunkOrientedTasklet(new ItemReader() { + step.setTasklet(new TestingChunkOrientedTasklet<>(new ItemReader() { + @Nullable @Override public Object read() throws Exception { throw new RuntimeException("Planned!"); diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/StepHandlerAdapterTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/StepHandlerAdapterTests.java index 1600c2c81c..6d32e01ba1 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/StepHandlerAdapterTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/StepHandlerAdapterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/SystemCommandTaskletIntegrationTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/SystemCommandTaskletIntegrationTests.java index dcb3e9aa19..c7a1dd9f08 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/SystemCommandTaskletIntegrationTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/SystemCommandTaskletIntegrationTests.java @@ -1,238 +1,308 @@ -package org.springframework.batch.core.step.tasklet; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - -import java.io.File; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.core.ExitStatus; -import org.springframework.batch.core.JobExecution; -import org.springframework.batch.core.JobInstance; -import org.springframework.batch.core.JobInterruptedException; -import org.springframework.batch.core.JobParameters; -import org.springframework.batch.core.StepContribution; -import org.springframework.batch.core.StepExecution; -import org.springframework.batch.repeat.RepeatStatus; -import org.springframework.core.task.SimpleAsyncTaskExecutor; -import org.springframework.util.Assert; - -/** - * Tests for {@link SystemCommandTasklet}. - */ -public class SystemCommandTaskletIntegrationTests { - - private static final Log log = LogFactory.getLog(SystemCommandTaskletIntegrationTests.class); - - private SystemCommandTasklet tasklet = new SystemCommandTasklet(); - - private StepExecution stepExecution = new StepExecution("systemCommandStep", new JobExecution(new JobInstance(1L, - "systemCommandJob"), new JobParameters())); - - @Before - public void setUp() throws Exception { - tasklet.setEnvironmentParams(null); // inherit from parent process - tasklet.setWorkingDirectory(null); // inherit from parent process - tasklet.setSystemProcessExitCodeMapper(new TestExitCodeMapper()); - tasklet.setTimeout(5000); // long enough timeout - tasklet.setTerminationCheckInterval(500); - tasklet.setCommand("invalid command, change value for successful execution"); - tasklet.setInterruptOnCancel(true); - tasklet.setTaskExecutor(new SimpleAsyncTaskExecutor()); - tasklet.afterPropertiesSet(); - - tasklet.beforeStep(stepExecution); - } - - /* - * Regular usage scenario - successful execution of system command. - */ - @Test - public void testExecute() throws Exception { - String command = "java -version"; - tasklet.setCommand(command); - tasklet.afterPropertiesSet(); - - log.info("Executing command: " + command); - RepeatStatus exitStatus = tasklet.execute(stepExecution.createStepContribution(), null); - - assertEquals(RepeatStatus.FINISHED, exitStatus); - } - - /* - * Failed execution scenario - error exit code returned by system command. - */ - @Test - public void testExecuteFailure() throws Exception { - String command = "java org.springframework.batch.sample.tasklet.UnknownClass"; - tasklet.setCommand(command); - tasklet.setTimeout(200L); - tasklet.afterPropertiesSet(); - - log.info("Executing command: " + command); - try { - StepContribution contribution = stepExecution.createStepContribution(); - RepeatStatus exitStatus = tasklet.execute(contribution, null); - assertEquals(RepeatStatus.FINISHED, exitStatus); - assertEquals(ExitStatus.FAILED, contribution.getExitStatus()); - } - catch (RuntimeException e) { - // on some platforms the system call does not return - assertEquals("Execution of system command did not finish within the timeout", e.getMessage()); - } - } - - /* - * The attempt to execute the system command results in exception - */ - @Test(expected = java.util.concurrent.ExecutionException.class) - public void testExecuteException() throws Exception { - String command = "non-sense-that-should-cause-exception-when-attempted-to-execute"; - tasklet.setCommand(command); - tasklet.afterPropertiesSet(); - - tasklet.execute(null, null); - } - - /* - * Failed execution scenario - execution time exceeds timeout. - */ - @Test - public void testExecuteTimeout() throws Exception { - String command = "sleep 3"; - tasklet.setCommand(command); - tasklet.setTimeout(10); - tasklet.afterPropertiesSet(); - - log.info("Executing command: " + command); - try { - tasklet.execute(null, null); - fail(); - } - catch (SystemCommandException e) { - assertTrue(e.getMessage().contains("did not finish within the timeout")); - } - } - - /* - * Job interrupted scenario. - */ - @Test - public void testInterruption() throws Exception { - String command = "sleep 5"; - tasklet.setCommand(command); - tasklet.setTerminationCheckInterval(10); - tasklet.afterPropertiesSet(); - - stepExecution.setTerminateOnly(); - try { - tasklet.execute(null, null); - fail(); - } - catch (JobInterruptedException e) { - System.out.println(e.getMessage()); - assertTrue(e.getMessage().contains("Job interrupted while executing system command")); - assertTrue(e.getMessage().contains(command)); - } - } - - /* - * Command property value is required to be set. - */ - @Test - public void testCommandNotSet() throws Exception { - tasklet.setCommand(null); - try { - tasklet.afterPropertiesSet(); - fail(); - } - catch (IllegalArgumentException e) { - // expected - } - - tasklet.setCommand(""); - try { - tasklet.afterPropertiesSet(); - fail(); - } - catch (IllegalArgumentException e) { - // expected - } - } - - /* - * Timeout must be set to non-zero value. - */ - @Test - public void testTimeoutNotSet() throws Exception { - tasklet.setCommand("not-empty placeholder"); - tasklet.setTimeout(0); - try { - tasklet.afterPropertiesSet(); - fail(); - } - catch (IllegalArgumentException e) { - // expected - } - } - - /* - * Working directory property must point to an existing location and it must - * be a directory - */ - @Test - public void testWorkingDirectory() throws Exception { - File notExistingFile = new File("not-existing-path"); - Assert.state(!notExistingFile.exists()); - - try { - tasklet.setWorkingDirectory(notExistingFile.getCanonicalPath()); - fail(); - } - catch (IllegalArgumentException e) { - // expected - } - - File notDirectory = File.createTempFile(this.getClass().getName(), null); - Assert.state(notDirectory.exists()); - Assert.state(!notDirectory.isDirectory()); - - try { - tasklet.setWorkingDirectory(notDirectory.getCanonicalPath()); - fail(); - } - catch (IllegalArgumentException e) { - // expected - } - - File directory = notDirectory.getParentFile(); - Assert.state(directory.exists()); - Assert.state(directory.isDirectory()); - - // no error expected now - tasklet.setWorkingDirectory(directory.getCanonicalPath()); - } - - /** - * Exit code mapper containing mapping logic expected by the tests. 0 means - * finished successfully, other value means failure. - */ - private static class TestExitCodeMapper implements SystemProcessExitCodeMapper { - - @Override - public ExitStatus getExitStatus(int exitCode) { - if (exitCode == 0) { - return ExitStatus.COMPLETED; - } - else { - return ExitStatus.FAILED; - } - } - - } - -} +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.core.step.tasklet; + +import java.io.File; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobInterruptedException; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.scope.context.StepContext; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.util.Assert; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.when; + +/** + * Tests for {@link SystemCommandTasklet}. + */ +public class SystemCommandTaskletIntegrationTests { + + private static final Log log = LogFactory.getLog(SystemCommandTaskletIntegrationTests.class); + + private SystemCommandTasklet tasklet; + + private StepExecution stepExecution = new StepExecution("systemCommandStep", new JobExecution(new JobInstance(1L, + "systemCommandJob"), 1L, new JobParameters(), "configurationName")); + + @Mock + private JobExplorer jobExplorer; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + + initializeTasklet(); + tasklet.afterPropertiesSet(); + + tasklet.beforeStep(stepExecution); + } + + private void initializeTasklet() { + tasklet = new SystemCommandTasklet(); + tasklet.setEnvironmentParams(null); // inherit from parent process + tasklet.setWorkingDirectory(null); // inherit from parent process + tasklet.setSystemProcessExitCodeMapper(new SimpleSystemProcessExitCodeMapper()); + tasklet.setTimeout(5000); // long enough timeout + tasklet.setTerminationCheckInterval(500); + tasklet.setCommand("invalid command, change value for successful execution"); + tasklet.setInterruptOnCancel(true); + tasklet.setTaskExecutor(new SimpleAsyncTaskExecutor()); + } + + /* + * Regular usage scenario - successful execution of system command. + */ + @Test + public void testExecute() throws Exception { + String command = getJavaCommand() + " --version"; + tasklet.setCommand(command); + tasklet.afterPropertiesSet(); + + log.info("Executing command: " + command); + RepeatStatus exitStatus = tasklet.execute(stepExecution.createStepContribution(), null); + + assertEquals(RepeatStatus.FINISHED, exitStatus); + } + + /* + * Failed execution scenario - error exit code returned by system command. + */ + @Test + public void testExecuteFailure() throws Exception { + String command = getJavaCommand() + " org.springframework.batch.sample.tasklet.UnknownClass"; + tasklet.setCommand(command); + tasklet.setTimeout(200L); + tasklet.afterPropertiesSet(); + + log.info("Executing command: " + command); + try { + StepContribution contribution = stepExecution.createStepContribution(); + RepeatStatus exitStatus = tasklet.execute(contribution, null); + assertEquals(RepeatStatus.FINISHED, exitStatus); + assertEquals(ExitStatus.FAILED, contribution.getExitStatus()); + } + catch (RuntimeException e) { + // on some platforms the system call does not return + assertEquals("Execution of system command did not finish within the timeout", e.getMessage()); + } + } + + /* + * The attempt to execute the system command results in exception + */ + @Test(expected = java.util.concurrent.ExecutionException.class) + public void testExecuteException() throws Exception { + String command = "non-sense-that-should-cause-exception-when-attempted-to-execute"; + tasklet.setCommand(command); + tasklet.afterPropertiesSet(); + + tasklet.execute(null, null); + } + + /* + * Failed execution scenario - execution time exceeds timeout. + */ + @Test + public void testExecuteTimeout() throws Exception { + String command = isRunningOnWindows() ? + "ping 127.0.0.1" : + "sleep 3"; + tasklet.setCommand(command); + tasklet.setTimeout(10); + tasklet.afterPropertiesSet(); + + log.info("Executing command: " + command); + try { + tasklet.execute(null, null); + fail(); + } + catch (SystemCommandException e) { + assertTrue(e.getMessage().contains("did not finish within the timeout")); + } + } + + /* + * Job interrupted scenario. + */ + @Test + public void testInterruption() throws Exception { + String command = isRunningOnWindows() ? + "ping 127.0.0.1" : + "sleep 5"; + tasklet.setCommand(command); + tasklet.setTerminationCheckInterval(10); + tasklet.afterPropertiesSet(); + + stepExecution.setTerminateOnly(); + try { + tasklet.execute(null, null); + fail(); + } + catch (JobInterruptedException e) { + System.out.println(e.getMessage()); + assertTrue(e.getMessage().contains("Job interrupted while executing system command")); + assertTrue(e.getMessage().contains(command)); + } + } + + /* + * Command property value is required to be set. + */ + @Test + public void testCommandNotSet() throws Exception { + tasklet.setCommand(null); + try { + tasklet.afterPropertiesSet(); + fail(); + } + catch (IllegalArgumentException e) { + // expected + } + + tasklet.setCommand(""); + try { + tasklet.afterPropertiesSet(); + fail(); + } + catch (IllegalArgumentException e) { + // expected + } + } + + /* + * Timeout must be set to non-zero value. + */ + @Test + public void testTimeoutNotSet() throws Exception { + tasklet.setCommand("not-empty placeholder"); + tasklet.setTimeout(0); + try { + tasklet.afterPropertiesSet(); + fail(); + } + catch (IllegalArgumentException e) { + // expected + } + } + + /* + * Working directory property must point to an existing location and it must + * be a directory + */ + @Test + public void testWorkingDirectory() throws Exception { + File notExistingFile = new File("not-existing-path"); + Assert.state(!notExistingFile.exists(), "not-existing-path does actually exist"); + + try { + tasklet.setWorkingDirectory(notExistingFile.getCanonicalPath()); + fail(); + } + catch (IllegalArgumentException e) { + // expected + } + + File notDirectory = File.createTempFile(this.getClass().getName(), null); + Assert.state(notDirectory.exists(), "The file does not exist"); + Assert.state(!notDirectory.isDirectory(), "The file is actually a directory"); + + try { + tasklet.setWorkingDirectory(notDirectory.getCanonicalPath()); + fail(); + } + catch (IllegalArgumentException e) { + // expected + } + + File directory = notDirectory.getParentFile(); + Assert.state(directory.exists(), "The directory does not exist"); + Assert.state(directory.isDirectory(), "The directory is not a directory"); + + // no error expected now + tasklet.setWorkingDirectory(directory.getCanonicalPath()); + } + + /* + * test stopping a tasklet + */ + @Test + public void testStopped() throws Exception { + initializeTasklet(); + tasklet.setJobExplorer(jobExplorer); + tasklet.afterPropertiesSet(); + tasklet.beforeStep(stepExecution); + + JobExecution stoppedJobExecution = new JobExecution(stepExecution.getJobExecution()); + stoppedJobExecution.setStatus(BatchStatus.STOPPING); + + when(jobExplorer.getJobExecution(1L)).thenReturn(stepExecution.getJobExecution(), stepExecution.getJobExecution(), stoppedJobExecution); + + String command = isRunningOnWindows() ? + "ping 127.0.0.1 -n 5" : + "sleep 15"; + tasklet.setCommand(command); + tasklet.setTerminationCheckInterval(10); + tasklet.afterPropertiesSet(); + + StepContribution contribution = stepExecution.createStepContribution(); + StepContext stepContext = new StepContext(stepExecution); + ChunkContext chunkContext = new ChunkContext(stepContext); + tasklet.execute(contribution, chunkContext); + + assertEquals(ExitStatus.STOPPED.getExitCode(), contribution.getExitStatus().getExitCode()); + } + + private String getJavaCommand() { + String javaHome = System.getProperty("java.home"); + String fileSeparator = System.getProperty("file.separator"); + StringBuilder command = new StringBuilder(); + command.append(javaHome); + command.append(fileSeparator); + command.append("bin"); + command.append(fileSeparator); + command.append("java"); + + if(isRunningOnWindows()) { + command.append(".exe"); + } + + return command.toString(); + } + + private boolean isRunningOnWindows() { + return System.getProperty("os.name").toLowerCase().contains("win"); + } + +} diff --git a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/TaskletStepTests.java b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/TaskletStepTests.java index 70bbe524dc..8fca070f5f 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/TaskletStepTests.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/core/step/tasklet/TaskletStepTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -63,15 +63,16 @@ import org.springframework.batch.repeat.support.RepeatTemplate; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; import org.springframework.dao.DataAccessResourceFailureException; +import org.springframework.lang.Nullable; import org.springframework.transaction.TransactionException; import org.springframework.transaction.interceptor.DefaultTransactionAttribute; import org.springframework.transaction.support.DefaultTransactionStatus; public class TaskletStepTests { - List processed = new ArrayList(); + List processed = new ArrayList<>(); - private List list = new ArrayList(); + private List list = new ArrayList<>(); ItemWriter itemWriter = new ItemWriter() { @Override @@ -98,7 +99,7 @@ public void write(List data) throws Exception { }; private ItemReader getReader(String[] args) { - return new ListItemReader(Arrays.asList(args)); + return new ListItemReader<>(Arrays.asList(args)); } private TaskletStep getStep(String[] strings) throws Exception { @@ -110,7 +111,7 @@ private TaskletStep getStep(String[] strings, int commitInterval) throws Excepti // Only process one item: RepeatTemplate template = new RepeatTemplate(); template.setCompletionPolicy(new SimpleCompletionPolicy(commitInterval)); - step.setTasklet(new TestingChunkOrientedTasklet(getReader(strings), itemWriter, template)); + step.setTasklet(new TestingChunkOrientedTasklet<>(getReader(strings), itemWriter, template)); step.setJobRepository(new JobRepositorySupport()); step.setTransactionManager(transactionManager); return step; @@ -176,7 +177,7 @@ public void testEmptyReader() throws Exception { JobExecution jobExecutionContext = new JobExecution(jobInstance, jobParameters); StepExecution stepExecution = new StepExecution(step.getName(), jobExecutionContext); step = getStep(new String[0]); - step.setTasklet(new TestingChunkOrientedTasklet(getReader(new String[0]), itemWriter, + step.setTasklet(new TestingChunkOrientedTasklet<>(getReader(new String[0]), itemWriter, new RepeatTemplate())); step.setStepOperations(new RepeatTemplate()); step.execute(stepExecution); @@ -246,6 +247,7 @@ public void testIncrementRollbackCount() { ItemReader itemReader = new ItemReader() { + @Nullable @Override public String read() throws Exception { throw new RuntimeException(); @@ -253,7 +255,7 @@ public String read() throws Exception { }; - step.setTasklet(new TestingChunkOrientedTasklet(itemReader, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(itemReader, itemWriter)); JobExecution jobExecutionContext = new JobExecution(jobInstance, jobParameters); StepExecution stepExecution = new StepExecution(step.getName(), jobExecutionContext); @@ -271,6 +273,7 @@ public void testExitCodeDefaultClassification() throws Exception { ItemReader itemReader = new ItemReader() { + @Nullable @Override public String read() throws Exception { throw new RuntimeException(); @@ -279,7 +282,7 @@ public String read() throws Exception { }; - step.setTasklet(new TestingChunkOrientedTasklet(itemReader, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(itemReader, itemWriter)); JobExecution jobExecutionContext = new JobExecution(jobInstance, jobParameters); StepExecution stepExecution = new StepExecution(step.getName(), jobExecutionContext); @@ -297,6 +300,7 @@ public void testExitCodeCustomClassification() throws Exception { ItemReader itemReader = new ItemReader() { + @Nullable @Override public String read() throws Exception { throw new RuntimeException(); @@ -305,8 +309,9 @@ public String read() throws Exception { }; - step.setTasklet(new TestingChunkOrientedTasklet(itemReader, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(itemReader, itemWriter)); step.registerStepExecutionListener(new StepExecutionListenerSupport() { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { return ExitStatus.FAILED.addExitDescription("FOO"); @@ -333,7 +338,7 @@ public ExitStatus afterStep(StepExecution stepExecution) { @Test public void testNonRestartedJob() throws Exception { MockRestartableItemReader tasklet = new MockRestartableItemReader(); - step.setTasklet(new TestingChunkOrientedTasklet(tasklet, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(tasklet, itemWriter)); step.registerStream(tasklet); JobExecution jobExecutionContext = new JobExecution(jobInstance, jobParameters); StepExecution stepExecution = new StepExecution(step.getName(), jobExecutionContext); @@ -392,7 +397,7 @@ public void updateExecutionContext(StepExecution stepExecution) { @Test public void testNoSaveExecutionAttributesRestartableJob() { MockRestartableItemReader tasklet = new MockRestartableItemReader(); - step.setTasklet(new TestingChunkOrientedTasklet(tasklet, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(tasklet, itemWriter)); JobExecution jobExecutionContext = new JobExecution(jobInstance, jobParameters); StepExecution stepExecution = new StepExecution(step.getName(), jobExecutionContext); @@ -413,7 +418,8 @@ public void testNoSaveExecutionAttributesRestartableJob() { */ @Test public void testRestartJobOnNonRestartableTasklet() throws Exception { - step.setTasklet(new TestingChunkOrientedTasklet(new ItemReader() { + step.setTasklet(new TestingChunkOrientedTasklet<>(new ItemReader() { + @Nullable @Override public String read() throws Exception { return "foo"; @@ -428,6 +434,7 @@ public String read() throws Exception { @Test public void testStreamManager() throws Exception { MockRestartableItemReader reader = new MockRestartableItemReader() { + @Nullable @Override public String read() { return "foo"; @@ -439,7 +446,7 @@ public void update(ExecutionContext executionContext) { executionContext.putString("foo", "bar"); } }; - step.setTasklet(new TestingChunkOrientedTasklet(reader, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(reader, itemWriter)); step.registerStream(reader); JobExecution jobExecution = new JobExecution(jobInstance, jobParameters); StepExecution stepExecution = new StepExecution(step.getName(), jobExecution); @@ -480,6 +487,7 @@ public void beforeStep(StepExecution stepExecution) { list.add("foo"); } + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { list.add("bar"); @@ -519,6 +527,7 @@ public void testAfterStep() throws Exception { final ExitStatus customStatus = new ExitStatus("COMPLETED_CUSTOM"); step.setStepExecutionListeners(new StepExecutionListener[] { new StepExecutionListenerSupport() { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { list.add("afterStepCalled"); @@ -542,13 +551,15 @@ public ExitStatus afterStep(StepExecution stepExecution) { @Test public void testDirectlyInjectedListenerOnError() throws Exception { step.registerStepExecutionListener(new StepExecutionListenerSupport() { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { list.add("exception"); return null; } }); - step.setTasklet(new TestingChunkOrientedTasklet(new MockRestartableItemReader() { + step.setTasklet(new TestingChunkOrientedTasklet<>(new MockRestartableItemReader() { + @Nullable @Override public String read() throws RuntimeException { throw new RuntimeException("FOO"); @@ -564,6 +575,7 @@ public String read() throws RuntimeException { @Test public void testDirectlyInjectedStreamWhichIsAlsoReader() throws Exception { MockRestartableItemReader reader = new MockRestartableItemReader() { + @Nullable @Override public String read() { return "foo"; @@ -575,7 +587,7 @@ public void update(ExecutionContext executionContext) { executionContext.putString("foo", "bar"); } }; - step.setTasklet(new TestingChunkOrientedTasklet(reader, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(reader, itemWriter)); step.setStreams(new ItemStream[] { reader }); JobExecution jobExecution = new JobExecution(jobInstance, jobParameters); StepExecution stepExecution = new StepExecution(step.getName(), jobExecution); @@ -604,6 +616,7 @@ public void checkInterrupted(StepExecution stepExecution) throws JobInterruptedE ItemReader itemReader = new ItemReader() { + @Nullable @Override public String read() throws Exception { throw new RuntimeException(); @@ -612,7 +625,7 @@ public String read() throws Exception { }; - step.setTasklet(new TestingChunkOrientedTasklet(itemReader, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(itemReader, itemWriter)); JobExecution jobExecutionContext = new JobExecution(jobInstance, jobParameters); StepExecution stepExecution = new StepExecution(step.getName(), jobExecutionContext); @@ -630,13 +643,14 @@ public String read() throws Exception { public void testStatusForNormalFailure() throws Exception { ItemReader itemReader = new ItemReader() { + @Nullable @Override public String read() throws Exception { // Trigger a rollback throw new RuntimeException("Foo"); } }; - step.setTasklet(new TestingChunkOrientedTasklet(itemReader, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(itemReader, itemWriter)); JobExecution jobExecutionContext = new JobExecution(jobInstance, jobParameters); StepExecution stepExecution = new StepExecution(step.getName(), jobExecutionContext); @@ -654,13 +668,14 @@ public String read() throws Exception { public void testStatusForErrorFailure() throws Exception { ItemReader itemReader = new ItemReader() { + @Nullable @Override public String read() throws Exception { // Trigger a rollback throw new Error("Foo"); } }; - step.setTasklet(new TestingChunkOrientedTasklet(itemReader, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(itemReader, itemWriter)); JobExecution jobExecutionContext = new JobExecution(jobInstance, jobParameters); StepExecution stepExecution = new StepExecution(step.getName(), jobExecutionContext); @@ -679,13 +694,14 @@ public String read() throws Exception { public void testStatusForResetFailedException() throws Exception { ItemReader itemReader = new ItemReader() { + @Nullable @Override public String read() throws Exception { // Trigger a rollback throw new RuntimeException("Foo"); } }; - step.setTasklet(new TestingChunkOrientedTasklet(itemReader, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(itemReader, itemWriter)); step.setTransactionManager(new ResourcelessTransactionManager() { @Override protected void doRollback(DefaultTransactionStatus status) throws TransactionException { @@ -774,7 +790,7 @@ public void close() throws ItemStreamException { throw new RuntimeException("Bar"); } }; - step.setTasklet(new TestingChunkOrientedTasklet(itemReader, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(itemReader, itemWriter)); step.registerStream(itemReader); JobExecution jobExecutionContext = new JobExecution(jobInstance, jobParameters); @@ -795,19 +811,20 @@ public void close() throws ItemStreamException { /** * Execution context must not be left empty even if job failed before - * commiting first chunk - otherwise ItemStreams won't recognize it is + * committing first chunk - otherwise ItemStreams won't recognize it is * restart scenario on next run. */ @Test public void testRestartAfterFailureInFirstChunk() throws Exception { MockRestartableItemReader reader = new MockRestartableItemReader() { + @Nullable @Override public String read() throws RuntimeException { // fail on the very first item throw new RuntimeException("CRASH!"); } }; - step.setTasklet(new TestingChunkOrientedTasklet(reader, itemWriter)); + step.setTasklet(new TestingChunkOrientedTasklet<>(reader, itemWriter)); step.registerStream(reader); StepExecution stepExecution = new StepExecution(step.getName(), new JobExecution(jobInstance, jobParameters)); @@ -845,6 +862,7 @@ public void testStepToCompletion() throws Exception { @Test public void testStepFailureInAfterStepCallback() throws JobInterruptedException { StepExecutionListener listener = new StepExecutionListenerSupport() { + @Nullable @Override public ExitStatus afterStep(StepExecution stepExecution) { throw new RuntimeException("exception thrown in afterStep to signal failure"); @@ -862,6 +880,7 @@ public ExitStatus afterStep(StepExecution stepExecution) { public void testNoRollbackFor() throws Exception { step.setTasklet(new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { throw new RuntimeException("Bar"); @@ -887,6 +906,7 @@ public boolean rollbackOn(Throwable ex) { @Test public void testTaskletExecuteReturnNull() throws Exception { step.setTasklet(new Tasklet() { + @Nullable @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { return null; @@ -931,6 +951,7 @@ private class MockRestartableItemReader extends AbstractItemStreamItemReader extends ChunkOrientedTasklet { - /** - * - */ private static final RepeatTemplate repeatTemplate = new RepeatTemplate(); static { @@ -59,7 +56,7 @@ public TestingChunkOrientedTasklet(ItemReader itemReader, ItemWriter itemW */ public TestingChunkOrientedTasklet(ItemReader itemReader, ItemProcessor itemProcessor, ItemWriter itemWriter, RepeatOperations repeatOperations) { - super(new SimpleChunkProvider(itemReader, repeatOperations), new SimpleChunkProcessor( + super(new SimpleChunkProvider<>(itemReader, repeatOperations), new SimpleChunkProcessor<>( itemProcessor, itemWriter)); } @@ -69,7 +66,7 @@ public TestingChunkOrientedTasklet(ItemReader itemReader, ItemProcessor */ public TestingChunkOrientedTasklet(ItemReader itemReader, ItemWriter itemWriter, RepeatOperations repeatOperations) { - this(itemReader, new PassThroughItemProcessor(), itemWriter, repeatOperations); + this(itemReader, new PassThroughItemProcessor<>(), itemWriter, repeatOperations); } } diff --git a/spring-batch-core/src/test/java/org/springframework/batch/test/namespace/config/DummyNamespaceHandler.java b/spring-batch-core/src/test/java/org/springframework/batch/test/namespace/config/DummyNamespaceHandler.java index b82c45f4b3..09572eefb7 100644 --- a/spring-batch-core/src/test/java/org/springframework/batch/test/namespace/config/DummyNamespaceHandler.java +++ b/spring-batch-core/src/test/java/org/springframework/batch/test/namespace/config/DummyNamespaceHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/test/java/test/jdbc/datasource/DataSourceInitializer.java b/spring-batch-core/src/test/java/test/jdbc/datasource/DataSourceInitializer.java index c7d04860b5..c01bcb281e 100644 --- a/spring-batch-core/src/test/java/test/jdbc/datasource/DataSourceInitializer.java +++ b/spring-batch-core/src/test/java/test/jdbc/datasource/DataSourceInitializer.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,12 +18,12 @@ import java.io.IOException; import java.util.List; - import javax.sql.DataSource; import org.apache.commons.io.IOUtils; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.beans.factory.BeanInitializationException; import org.springframework.beans.factory.InitializingBean; import org.springframework.context.support.ClassPathXmlApplicationContext; @@ -40,7 +40,7 @@ /** * Wrapper for a {@link DataSource} that can run scripts on start up and shut - * down. Us as a bean definition

      + * down. Us as a bean definition

      * * Run this class to initialize a database in a running server process. * Make sure the server is running first by launching the "hsql-server" from the @@ -76,7 +76,7 @@ public static void main(String... args) { @Override public void afterPropertiesSet() throws Exception { - Assert.notNull(dataSource); + Assert.notNull(dataSource, "A DataSource is required"); initialize(); } @@ -92,22 +92,22 @@ private void initialize() { } } - @SuppressWarnings({ "unchecked", "rawtypes" }) + @SuppressWarnings({ "unchecked" }) private void doExecuteScript(final Resource scriptResource) { if (scriptResource == null || !scriptResource.exists()) { throw new IllegalArgumentException("Script resource is null or does not exist"); } TransactionTemplate transactionTemplate = new TransactionTemplate(new DataSourceTransactionManager(dataSource)); - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); String[] scripts; try { scripts = StringUtils.delimitedListToStringArray(stripComments(IOUtils.readLines(scriptResource - .getInputStream())), ";"); + .getInputStream(), "UTF-8")), ";"); } catch (IOException e) { throw new BeanInitializationException("Cannot load script from [" + scriptResource + "]", e); @@ -136,10 +136,10 @@ public Object doInTransaction(TransactionStatus status) { } private String stripComments(List list) { - StringBuffer buffer = new StringBuffer(); + StringBuilder buffer = new StringBuilder(); for (String line : list) { if (!line.startsWith("//") && !line.startsWith("--")) { - buffer.append(line + "\n"); + buffer.append(line).append("\n"); } } return buffer.toString(); diff --git a/spring-batch-core/src/test/resources/META-INF/alternativeJsrBaseContext.xml b/spring-batch-core/src/test/resources/META-INF/alternativeJsrBaseContext.xml new file mode 100644 index 0000000000..298ec1347f --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/alternativeJsrBaseContext.xml @@ -0,0 +1,9 @@ + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionAfterFlow-context.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionAfterFlow-context.xml new file mode 100644 index 0000000000..59e39409a7 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionAfterFlow-context.xml @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionAfterSplit-context.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionAfterSplit-context.xml new file mode 100644 index 0000000000..27c79a25fa --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionAfterSplit-context.xml @@ -0,0 +1,32 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionAsFirstStep-context.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionAsFirstStep-context.xml new file mode 100644 index 0000000000..f1e2cb9b20 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionAsFirstStep-context.xml @@ -0,0 +1,14 @@ + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionCustomExitStatus-context.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionCustomExitStatus-context.xml new file mode 100644 index 0000000000..51333699af --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionCustomExitStatus-context.xml @@ -0,0 +1,18 @@ + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionInvalidExitStatus-context.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionInvalidExitStatus-context.xml new file mode 100644 index 0000000000..64f8874ca0 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionInvalidExitStatus-context.xml @@ -0,0 +1,22 @@ + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionThrowsException-context.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionThrowsException-context.xml new file mode 100644 index 0000000000..ecbf2f9703 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionThrowsException-context.xml @@ -0,0 +1,22 @@ + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionValidExitStatus-context.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionValidExitStatus-context.xml new file mode 100644 index 0000000000..8b06afcd4e --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-decisionValidExitStatus-context.xml @@ -0,0 +1,23 @@ + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-restart-context.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-restart-context.xml new file mode 100644 index 0000000000..1742e2d77b --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/DecisionStepTests-restart-context.xml @@ -0,0 +1,20 @@ + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/ExceptionHandlingParsingTests-context.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/ExceptionHandlingParsingTests-context.xml new file mode 100644 index 0000000000..57b584f9c6 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/ExceptionHandlingParsingTests-context.xml @@ -0,0 +1,89 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + One + Two + + + + + + + + Three + Four + + + + + + + + Five + Six + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/FlowParserTests-context.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/FlowParserTests-context.xml new file mode 100644 index 0000000000..cce7a773f4 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/FlowParserTests-context.xml @@ -0,0 +1,39 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/FlowParserTestsStepGetsFailedTransitionWhenNextAttributePresent.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/FlowParserTestsStepGetsFailedTransitionWhenNextAttributePresent.xml new file mode 100644 index 0000000000..aac5d92bfb --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/FlowParserTestsStepGetsFailedTransitionWhenNextAttributePresent.xml @@ -0,0 +1,18 @@ + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/FlowParserTestsStepNoOverrideWhenNextAndFailedTransitionElementExists.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/FlowParserTestsStepNoOverrideWhenNextAndFailedTransitionElementExists.xml new file mode 100644 index 0000000000..2f1cde9e43 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/FlowParserTestsStepNoOverrideWhenNextAndFailedTransitionElementExists.xml @@ -0,0 +1,18 @@ + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/FlowParserTestsWildcardAndNextAttrJob.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/FlowParserTestsWildcardAndNextAttrJob.xml new file mode 100644 index 0000000000..959e5e450c --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/FlowParserTestsWildcardAndNextAttrJob.xml @@ -0,0 +1,17 @@ + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/ItemSkipParsingTests-context.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/ItemSkipParsingTests-context.xml new file mode 100644 index 0000000000..5a4a5416ff --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/ItemSkipParsingTests-context.xml @@ -0,0 +1,30 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/JsrSplitParsingTests-context.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/JsrSplitParsingTests-context.xml new file mode 100644 index 0000000000..f43b1597c4 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/JsrSplitParsingTests-context.xml @@ -0,0 +1,58 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + One + Two + Three + Four + Five + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/contextClosingTests.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/contextClosingTests.xml new file mode 100644 index 0000000000..c1b6a6e20b --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/contextClosingTests.xml @@ -0,0 +1,15 @@ + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/fullPartitionParserTests.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/fullPartitionParserTests.xml new file mode 100644 index 0000000000..f2c71af802 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/fullPartitionParserTests.xml @@ -0,0 +1,12 @@ + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/fullPartitionParserWithHardcodedPropertiesTests.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/fullPartitionParserWithHardcodedPropertiesTests.xml new file mode 100644 index 0000000000..302125f506 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/fullPartitionParserWithHardcodedPropertiesTests.xml @@ -0,0 +1,46 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/fullPartitionParserWithMapperPropertiesTests.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/fullPartitionParserWithMapperPropertiesTests.xml new file mode 100644 index 0000000000..b41e971261 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/fullPartitionParserWithMapperPropertiesTests.xml @@ -0,0 +1,28 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/fullPartitionParserWithPropertiesTests.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/fullPartitionParserWithPropertiesTests.xml new file mode 100644 index 0000000000..b1b15de4ba --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/fullPartitionParserWithPropertiesTests.xml @@ -0,0 +1,32 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/jobWithEndTransition.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jobWithEndTransition.xml new file mode 100644 index 0000000000..d31dc66403 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jobWithEndTransition.xml @@ -0,0 +1,12 @@ + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestBeanCreationException.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestBeanCreationException.xml new file mode 100644 index 0000000000..806f9ef99e --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestBeanCreationException.xml @@ -0,0 +1,7 @@ + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestJob.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestJob.xml new file mode 100644 index 0000000000..730f521c8a --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestJob.xml @@ -0,0 +1,6 @@ + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestNonRestartableJob.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestNonRestartableJob.xml new file mode 100644 index 0000000000..3ce3351235 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestNonRestartableJob.xml @@ -0,0 +1,7 @@ + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestRestartAbandonJob.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestRestartAbandonJob.xml new file mode 100644 index 0000000000..c732646250 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestRestartAbandonJob.xml @@ -0,0 +1,7 @@ + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestRestartJob.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestRestartJob.xml new file mode 100644 index 0000000000..120dfdcd9f --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobOperatorTestRestartJob.xml @@ -0,0 +1,6 @@ + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobPropertyTests.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobPropertyTests.xml new file mode 100644 index 0000000000..31a5cf2e27 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobPropertyTests.xml @@ -0,0 +1,109 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobPropertyTestsContext.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobPropertyTestsContext.xml new file mode 100644 index 0000000000..fff9d7978f --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrJobPropertyTestsContext.xml @@ -0,0 +1,11 @@ + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrPartitionHandlerRestartWithOverrideJob.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrPartitionHandlerRestartWithOverrideJob.xml new file mode 100644 index 0000000000..f83c508d47 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrPartitionHandlerRestartWithOverrideJob.xml @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrPropertyPreparseTestJob.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrPropertyPreparseTestJob.xml new file mode 100644 index 0000000000..caa838a550 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrPropertyPreparseTestJob.xml @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrSpringInstanceTests.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrSpringInstanceTests.xml new file mode 100644 index 0000000000..ebe602acfc --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrSpringInstanceTests.xml @@ -0,0 +1,45 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrUniqueInstanceTests.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrUniqueInstanceTests.xml new file mode 100644 index 0000000000..aab71ad341 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/jsrUniqueInstanceTests.xml @@ -0,0 +1,53 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/longRunningJob.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/longRunningJob.xml new file mode 100644 index 0000000000..fc3e208e58 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/longRunningJob.xml @@ -0,0 +1,6 @@ + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/partitionParserTestsBatchlet.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/partitionParserTestsBatchlet.xml new file mode 100644 index 0000000000..6aef4f36a2 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/partitionParserTestsBatchlet.xml @@ -0,0 +1,9 @@ + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/partitionParserTestsChunk.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/partitionParserTestsChunk.xml new file mode 100644 index 0000000000..6a9cf53a8a --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/partitionParserTestsChunk.xml @@ -0,0 +1,12 @@ + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch-jobs/threadLocalClassloaderBeanPostProcessorTestsJob.xml b/spring-batch-core/src/test/resources/META-INF/batch-jobs/threadLocalClassloaderBeanPostProcessorTestsJob.xml new file mode 100644 index 0000000000..e542a5d7a8 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch-jobs/threadLocalClassloaderBeanPostProcessorTestsJob.xml @@ -0,0 +1,10 @@ + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/META-INF/batch.xml b/spring-batch-core/src/test/resources/META-INF/batch.xml new file mode 100644 index 0000000000..347468b4c3 --- /dev/null +++ b/spring-batch-core/src/test/resources/META-INF/batch.xml @@ -0,0 +1,6 @@ + + + + + + diff --git a/archetypes/simple-cli/src/main/resources/support/sample-data.csv b/spring-batch-core/src/test/resources/data/person.csv similarity index 100% rename from archetypes/simple-cli/src/main/resources/support/sample-data.csv rename to spring-batch-core/src/test/resources/data/person.csv diff --git a/spring-batch-core/src/test/resources/log4j.properties b/spring-batch-core/src/test/resources/log4j.properties index 44dca7c68f..f54d89e450 100644 --- a/spring-batch-core/src/test/resources/log4j.properties +++ b/spring-batch-core/src/test/resources/log4j.properties @@ -1,7 +1,7 @@ log4j.rootCategory=INFO, stdout -log4j.appender.stdout=org.apache.log4j.ConsoleAppender -log4j.appender.stdout.layout=org.apache.log4j.PatternLayout +log4j.appender.stdout=org.apache.logging.log4j.core.appender.ConsoleAppender +log4j.appender.stdout.layout=org.apache.logging.log4j.core.layout.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %t %c{2}:%L - %m%n log4j.category.org.apache.activemq=ERROR diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsInheritance-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsInheritance-context.xml new file mode 100644 index 0000000000..edef5d31be --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsInheritance-context.xml @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsInterface-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsInterface-context.xml new file mode 100644 index 0000000000..2af2b3a504 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsInterface-context.xml @@ -0,0 +1,12 @@ + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsProxyTargetClass-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsProxyTargetClass-context.xml new file mode 100644 index 0000000000..14c563f558 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsProxyTargetClass-context.xml @@ -0,0 +1,13 @@ + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsXmlImportUsingNamespace-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsXmlImportUsingNamespace-context.xml new file mode 100644 index 0000000000..f49e1b9b36 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/JobScopeConfigurationTestsXmlImportUsingNamespace-context.xml @@ -0,0 +1,20 @@ + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInheritance-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInheritance-context.xml new file mode 100644 index 0000000000..093d7d0441 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInheritance-context.xml @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInheritence-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInheritence-context.xml deleted file mode 100644 index 4b8506ff6a..0000000000 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInheritence-context.xml +++ /dev/null @@ -1,15 +0,0 @@ - - - - - - - - - - - - - - \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInterface-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInterface-context.xml index 35a08a7e24..4d2af6023f 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInterface-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsInterface-context.xml @@ -1,6 +1,6 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsProxyTargetClass-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsProxyTargetClass-context.xml index 4d615c5fef..6067ec64a2 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsProxyTargetClass-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsProxyTargetClass-context.xml @@ -1,6 +1,6 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsXmlImportUsingNamespace-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsXmlImportUsingNamespace-context.xml new file mode 100644 index 0000000000..1975c46d6c --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/annotation/StepScopeConfigurationTestsXmlImportUsingNamespace-context.xml @@ -0,0 +1,18 @@ + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarContextTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarContextTests-context.xml index e124e6e837..9569329fd1 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarContextTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/AutomaticJobRegistrarContextTests-context.xml @@ -1,6 +1,6 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/JobRegistryIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/JobRegistryIntegrationTests-context.xml index adecf4e78b..f17e2c8c7b 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/JobRegistryIntegrationTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/JobRegistryIntegrationTests-context.xml @@ -2,8 +2,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/abstract-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/abstract-context.xml new file mode 100644 index 0000000000..87d8345e96 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/abstract-context.xml @@ -0,0 +1,37 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/child-context-with-abstract-job.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/child-context-with-abstract-job.xml new file mode 100644 index 0000000000..8caa34f0f4 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/child-context-with-abstract-job.xml @@ -0,0 +1,8 @@ + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/child-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/child-context.xml index bd2e5e00df..11173d14d9 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/child-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/child-context.xml @@ -2,9 +2,9 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/job-context-with-separate-steps.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/job-context-with-separate-steps.xml index 5d2c1f1b37..e25add6c05 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/job-context-with-separate-steps.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/job-context-with-separate-steps.xml @@ -2,7 +2,7 @@ + https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> Declares two jobs with a set of steps. Also declares two steps that are not attached to any job diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/job-context-with-steps.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/job-context-with-steps.xml index fd4d00de31..4334edb04e 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/job-context-with-steps.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/job-context-with-steps.xml @@ -2,7 +2,7 @@ + https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/parent-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/parent-context.xml index 3bd32cbc36..6c70c64d71 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/parent-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/parent-context.xml @@ -2,9 +2,9 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/placeholder-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/placeholder-context.xml index 09c8115a7b..a28ddcc628 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/placeholder-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/placeholder-context.xml @@ -2,9 +2,9 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/profiles.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/profiles.xml index aae2348689..f72470d784 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/profiles.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/profiles.xml @@ -7,11 +7,11 @@ xmlns:context="http://www.springframework.org/schema/context" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd - http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd - http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd + http://www.springframework.org/schema/context https://www.springframework.org/schema/context/spring-context-3.1.xsd + http://www.springframework.org/schema/util https://www.springframework.org/schema/util/spring-util-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/test-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/test-context.xml index 617b89cfe4..479f748ec4 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/test-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/test-context.xml @@ -5,9 +5,9 @@ xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/trivial-context-autoregister.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/trivial-context-autoregister.xml index c721c2a7a6..892c23ab0c 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/trivial-context-autoregister.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/trivial-context-autoregister.xml @@ -2,7 +2,7 @@ + https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/trivial-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/trivial-context.xml index cfd59c8e7a..c43b129aa5 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/trivial-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/support/trivial-context.xml @@ -2,7 +2,7 @@ + https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeForJobElementTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeForJobElementTests-context.xml new file mode 100644 index 0000000000..0fbe635e87 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeForJobElementTests-context.xml @@ -0,0 +1,17 @@ + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeForStepElementTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeForStepElementTests-context.xml new file mode 100644 index 0000000000..8eab2e7062 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringJobScopeForStepElementTests-context.xml @@ -0,0 +1,15 @@ + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForJobElementTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForJobElementTests-context.xml index 16da142812..4c2b6e4ceb 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForJobElementTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForJobElementTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForStepElementTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForStepElementTests-context.xml index 62846d1382..e70ab64678 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForStepElementTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/AutoRegisteringStepScopeForStepElementTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/BeanDefinitionOverrideTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/BeanDefinitionOverrideTests-context.xml new file mode 100644 index 0000000000..1e001f6b46 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/BeanDefinitionOverrideTests-context.xml @@ -0,0 +1,24 @@ + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/BranchStepJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/BranchStepJobParserTests-context.xml index 39b85ece62..6a64967c81 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/BranchStepJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/BranchStepJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalAttributeParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalAttributeParserTests-context.xml index 57e4505c6b..d4aa700729 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalAttributeParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalAttributeParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalSkipAndRetryAttributeParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalSkipAndRetryAttributeParserTests-context.xml index 5b5b69d20a..5d9e1e6d55 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalSkipAndRetryAttributeParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalSkipAndRetryAttributeParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalTransactionalAttributeParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalTransactionalAttributeParserTests-context.xml index 8bcfee7bf6..9a209d8f8a 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalTransactionalAttributeParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementIllegalTransactionalAttributeParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementLateBindingParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementLateBindingParserTests-context.xml index 0833bbaf8c..7f94818656 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementLateBindingParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementLateBindingParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> @@ -21,12 +21,12 @@ - - + + - + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementParentAttributeParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementParentAttributeParserTests-context.xml index c598b50b15..d00909478e 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementParentAttributeParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementParentAttributeParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementRetryPolicyParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementRetryPolicyParserTests-context.xml index ac3ee03826..fdf58766d9 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementRetryPolicyParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementRetryPolicyParserTests-context.xml @@ -1,7 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSimpleAttributeParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSimpleAttributeParserTests-context.xml index 89c458dcfb..633222f8c0 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSimpleAttributeParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSimpleAttributeParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSkipAndRetryAttributeParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSkipAndRetryAttributeParserTests-context.xml index 84cd805d42..0b324732b2 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSkipAndRetryAttributeParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSkipAndRetryAttributeParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSkipPolicyParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSkipPolicyParserTests-context.xml index 60e6cad4c6..f48589d9bf 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSkipPolicyParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementSkipPolicyParserTests-context.xml @@ -1,7 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementTransactionalAttributeParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementTransactionalAttributeParserTests-context.xml index 6d8016c429..1f3cbdbeea 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementTransactionalAttributeParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ChunkElementTransactionalAttributeParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DecisionJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DecisionJobParserTests-context.xml index f196ab3608..a79aabeea8 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DecisionJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DecisionJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DefaultFailureJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DefaultFailureJobParserTests-context.xml index 47feb7a51b..e1b14f97ee 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DefaultFailureJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DefaultFailureJobParserTests-context.xml @@ -3,8 +3,8 @@ xmlns="http://www.springframework.org/schema/batch" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DefaultSuccessJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DefaultSuccessJobParserTests-context.xml index 0db8b0baa4..d858d75652 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DefaultSuccessJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DefaultSuccessJobParserTests-context.xml @@ -3,8 +3,8 @@ xmlns="http://www.springframework.org/schema/batch" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DefaultUnknownJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DefaultUnknownJobParserTests-context.xml index c35abcb7b4..be20e56cf8 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DefaultUnknownJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DefaultUnknownJobParserTests-context.xml @@ -3,8 +3,8 @@ xmlns="http://www.springframework.org/schema/batch" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DuplicateTransitionJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DuplicateTransitionJobParserTests-context.xml index 57726e9bb9..69c067a8a3 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DuplicateTransitionJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/DuplicateTransitionJobParserTests-context.xml @@ -3,8 +3,8 @@ xmlns="http://www.springframework.org/schema/batch" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/EndTransitionDefaultStatusJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/EndTransitionDefaultStatusJobParserTests-context.xml index 663c00d179..6577de290a 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/EndTransitionDefaultStatusJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/EndTransitionDefaultStatusJobParserTests-context.xml @@ -3,8 +3,8 @@ xmlns="http://www.springframework.org/schema/batch" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/EndTransitionJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/EndTransitionJobParserTests-context.xml index ba0eeb5f79..ab9561b3d8 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/EndTransitionJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/EndTransitionJobParserTests-context.xml @@ -3,8 +3,8 @@ xmlns="http://www.springframework.org/schema/batch" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FailTransitionDefaultStatusJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FailTransitionDefaultStatusJobParserTests-context.xml index 1e78fd10d7..81bfe66494 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FailTransitionDefaultStatusJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FailTransitionDefaultStatusJobParserTests-context.xml @@ -3,8 +3,8 @@ xmlns="http://www.springframework.org/schema/batch" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FailTransitionJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FailTransitionJobParserTests-context.xml index 2ed18f56ec..a8a689379e 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FailTransitionJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FailTransitionJobParserTests-context.xml @@ -3,8 +3,8 @@ xmlns="http://www.springframework.org/schema/batch" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FlowJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FlowJobParserTests-context.xml index 5ec4319d2a..87f8f280cc 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FlowJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FlowJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FlowStepParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FlowStepParserTests-context.xml index 6dff40063f..4b0035bc30 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FlowStepParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/FlowStepParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/InlineItemHandlerParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/InlineItemHandlerParserTests-context.xml index 28f0f9b7b1..0b268b7a91 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/InlineItemHandlerParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/InlineItemHandlerParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/InlineItemHandlerWithStepScopeParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/InlineItemHandlerWithStepScopeParserTests-context.xml index 94cf251764..36d9519145 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/InlineItemHandlerWithStepScopeParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/InlineItemHandlerWithStepScopeParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobExecutionListenerMethodAttributeParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobExecutionListenerMethodAttributeParserTests-context.xml index 12286e5f98..5e567531d9 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobExecutionListenerMethodAttributeParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobExecutionListenerMethodAttributeParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobExecutionListenerParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobExecutionListenerParserTests-context.xml index aa650bf035..3c41780b50 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobExecutionListenerParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobExecutionListenerParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserNextOutOfScopeTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserNextOutOfScopeTests-context.xml index e3fb1a898b..47f6c114d0 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserNextOutOfScopeTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserNextOutOfScopeTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserParentAttributeTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserParentAttributeTests-context.xml index ba3af0d547..76628d3e05 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserParentAttributeTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserParentAttributeTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserUnreachableStepInFlowTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserUnreachableStepInFlowTests-context.xml index 874a7c811f..c789559202 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserUnreachableStepInFlowTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserUnreachableStepInFlowTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserUnreachableStepTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserUnreachableStepTests-context.xml index 8cafe36d94..beec0b68f0 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserUnreachableStepTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserUnreachableStepTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserValidatorTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserValidatorTests-context.xml index 83d0373af1..7f2cb4fabf 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserValidatorTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserValidatorTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserWrongSchemaInRootTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserWrongSchemaInRootTests-context.xml index 0cce4711f4..a88cc5d163 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserWrongSchemaInRootTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobParserWrongSchemaInRootTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.0.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRegistryJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRegistryJobParserTests-context.xml index 7fa98fbf51..90e1746c26 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRegistryJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRegistryJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryDefaultParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryDefaultParserTests-context.xml index 260cc48c82..09da1d8cad 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryDefaultParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryDefaultParserTests-context.xml @@ -1,11 +1,11 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> - + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryParserReferenceTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryParserReferenceTests-context.xml index 64ab56f551..1f8a495092 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryParserReferenceTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryParserReferenceTests-context.xml @@ -1,12 +1,12 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> - + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryParserTests-context.xml index 77cbaef4cf..357bc7ec4e 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobRepositoryParserTests-context.xml @@ -1,12 +1,15 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-3.0.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> - + + + + @@ -14,6 +17,6 @@ - + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobStepParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobStepParserTests-context.xml index ddd2256cec..74d96655b6 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobStepParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/JobStepParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NamespacePrefixedJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NamespacePrefixedJobParserTests-context.xml index 12adde5413..12c0684b3d 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NamespacePrefixedJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NamespacePrefixedJobParserTests-context.xml @@ -2,8 +2,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeJobParserTests-context.xml index a8bddc7482..778e72df10 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeJobParserTests-context.xml @@ -3,8 +3,8 @@ xmlns="http://www.springframework.org/schema/batch" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeMultipleFinalJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeMultipleFinalJobParserTests-context.xml index 32d8d8602d..f263a4d65a 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeMultipleFinalJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeMultipleFinalJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeUnknownJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeUnknownJobParserTests-context.xml index aae9486628..cb950550d9 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeUnknownJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/NextAttributeUnknownJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/OneStepJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/OneStepJobParserTests-context.xml index 20ce83eb3b..c707cffab3 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/OneStepJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/OneStepJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentRetryableLateBindingStepFactoryBeanParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentRetryableLateBindingStepFactoryBeanParserTests-context.xml index 56a235ebf5..80b99e8384 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentRetryableLateBindingStepFactoryBeanParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentRetryableLateBindingStepFactoryBeanParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> @@ -23,12 +23,12 @@ - - + + - + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentRetryableStepFactoryBeanParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentRetryableStepFactoryBeanParserTests-context.xml index 38c193579e..a898377dc5 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentRetryableStepFactoryBeanParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentRetryableStepFactoryBeanParserTests-context.xml @@ -1,7 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentSkippableLateBindingStepFactoryBeanParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentSkippableLateBindingStepFactoryBeanParserTests-context.xml index 1ae9edd9e7..d44df402c7 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentSkippableLateBindingStepFactoryBeanParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentSkippableLateBindingStepFactoryBeanParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentSkippableStepFactoryBeanParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentSkippableStepFactoryBeanParserTests-context.xml index ca26193256..bd5015800b 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentSkippableStepFactoryBeanParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentSkippableStepFactoryBeanParserTests-context.xml @@ -1,7 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentStepFactoryBeanParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentStepFactoryBeanParserTests-context.xml index de7f503af0..2dd85bf188 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentStepFactoryBeanParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/ParentStepFactoryBeanParserTests-context.xml @@ -1,11 +1,11 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> - + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepParserTests-context.xml index c656491d3c..99ea12e528 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithFlowParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithFlowParserTests-context.xml index 305bf78368..a8aa7183c5 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithFlowParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithFlowParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithLateBindingParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithLateBindingParserTests-context.xml index d0180cfac3..4fcc380660 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithLateBindingParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithLateBindingParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithNonDefaultTransactionManagerParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithNonDefaultTransactionManagerParserTests-context.xml index 6831284dc2..abc7f1ba57 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithNonDefaultTransactionManagerParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/PartitionStepWithNonDefaultTransactionManagerParserTests-context.xml @@ -1,7 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/RepositoryJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/RepositoryJobParserTests-context.xml index 205be17540..aa5e5a4896 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/RepositoryJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/RepositoryJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailFirstJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailFirstJobParserTests-context.xml index 51916ac1b8..42c8be74c7 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailFirstJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailFirstJobParserTests-context.xml @@ -3,8 +3,8 @@ xmlns="http://www.springframework.org/schema/batch" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailSecondJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailSecondJobParserTests-context.xml index c13fc110e5..feefcd677d 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailSecondJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitDifferentResultsFailSecondJobParserTests-context.xml @@ -3,8 +3,8 @@ xmlns="http://www.springframework.org/schema/batch" xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitInterruptedJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitInterruptedJobParserTests-context.xml index ef6eb602dd..1359cb6fbe 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitInterruptedJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitInterruptedJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitJobParserTests-context.xml index a243bbe402..ffe95cd603 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitNestedJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitNestedJobParserTests-context.xml index ffff2cb488..ae81ff80cd 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitNestedJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/SplitNestedJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+%20%20%20%20%20%20%20%20http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerInStepParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerInStepParserTests-context.xml index 576524e710..a2fc515ad2 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerInStepParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerInStepParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerMethodAttributeParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerMethodAttributeParserTests-context.xml index 3dff04b2e9..57a612fd8b 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerMethodAttributeParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerMethodAttributeParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerParserTests-context.xml index 0c7b79f40c..77db360ff8 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepListenerParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBadRetryListenerTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBadRetryListenerTests-context.xml index 239969eb11..8ba31531e4 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBadRetryListenerTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBadRetryListenerTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBadStepListenerTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBadStepListenerTests-context.xml index 026366833a..911da9576c 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBadStepListenerTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBadStepListenerTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBeanNameTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBeanNameTests-context.xml index 716cf9064d..accd143106 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBeanNameTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserBeanNameTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCommitIntervalCompletionPolicyTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCommitIntervalCompletionPolicyTests-context.xml index 6197e9fca9..d3b57c4727 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCommitIntervalCompletionPolicyTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCommitIntervalCompletionPolicyTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCommitIntervalTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCommitIntervalTests-context.xml index c9a5ed4aa1..6d910a4057 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCommitIntervalTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCommitIntervalTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCompletionPolicyTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCompletionPolicyTests-context.xml index e5136a7bd2..da3eff3a11 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCompletionPolicyTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserCompletionPolicyTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserNoCommitIntervalOrCompletionPolicyTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserNoCommitIntervalOrCompletionPolicyTests-context.xml index 2fb0a4b1d9..d0017c6826 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserNoCommitIntervalOrCompletionPolicyTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserNoCommitIntervalOrCompletionPolicyTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserParentAttributeTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserParentAttributeTests-context.xml index 5012e37ac8..029e55514b 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserParentAttributeTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserParentAttributeTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserTaskletAttributesTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserTaskletAttributesTests-context.xml index 643694b7cf..6fbfecbc9a 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserTaskletAttributesTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepParserTaskletAttributesTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithBasicProcessTaskJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithBasicProcessTaskJobParserTests-context.xml index dd0d5db04d..e04419f01b 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithBasicProcessTaskJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithBasicProcessTaskJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithFaultTolerantProcessTaskJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithFaultTolerantProcessTaskJobParserTests-context.xml index 63188ca191..e588563f59 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithFaultTolerantProcessTaskJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithFaultTolerantProcessTaskJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithPojoListenerJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithPojoListenerJobParserTests-context.xml index c25f2b74f8..af5f64ce9e 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithPojoListenerJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithPojoListenerJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithSimpleTaskJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithSimpleTaskJobParserTests-context.xml index 1bf6052ecd..e02f11c757 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithSimpleTaskJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StepWithSimpleTaskJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopAndRestartFailedJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopAndRestartFailedJobParserTests-context.xml index 82e1d9b36d..6d4710a894 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopAndRestartFailedJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopAndRestartFailedJobParserTests-context.xml @@ -2,8 +2,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopAndRestartJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopAndRestartJobParserTests-context.xml index 91ad6e831a..c330f74283 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopAndRestartJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopAndRestartJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopCustomStatusJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopCustomStatusJobParserTests-context.xml index b016a49cc0..4a0daf14a3 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopCustomStatusJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopCustomStatusJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopIncompleteJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopIncompleteJobParserTests-context.xml index ee221c654c..bf3c565c45 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopIncompleteJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopIncompleteJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopJobParserTests-context.xml index d60a6e78c7..5d43cc22ff 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopRestartOnCompletedStepJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopRestartOnCompletedStepJobParserTests-context.xml index 8489d2edfc..fefde71bf4 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopRestartOnCompletedStepJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopRestartOnCompletedStepJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopRestartOnFailedStepJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopRestartOnFailedStepJobParserTests-context.xml index 526b874996..b3e3ff72ff 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopRestartOnFailedStepJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/StopRestartOnFailedStepJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TaskletParserAdapterTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TaskletParserAdapterTests-context.xml index bf4f6019de..f5ed35abd6 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TaskletParserAdapterTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TaskletParserAdapterTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TaskletParserBeanPropertiesTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TaskletParserBeanPropertiesTests-context.xml index a8a54591ed..955ed17553 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TaskletParserBeanPropertiesTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TaskletParserBeanPropertiesTests-context.xml @@ -4,8 +4,8 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p" xmlns:test="http://www.springframework.org/schema/batch/test" - xsi:schemaLocation="http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd-http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd+http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TaskletStepAllowStartIfCompleteTest-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TaskletStepAllowStartIfCompleteTest-context.xml index 390350ad8d..68eac85938 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TaskletStepAllowStartIfCompleteTest-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TaskletStepAllowStartIfCompleteTest-context.xml @@ -2,8 +2,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd+http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TwoStepJobParserTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TwoStepJobParserTests-context.xml index a54a2a867a..d21e3f0823 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TwoStepJobParserTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/TwoStepJobParserTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/common-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/common-context.xml index 9bdb99c579..7264a16f85 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/common-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/configuration/xml/common-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/ChunkListenerParsingTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/ChunkListenerParsingTests-context.xml new file mode 100644 index 0000000000..8392254870 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/ChunkListenerParsingTests-context.xml @@ -0,0 +1,95 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + One + Two + + + + + + + + Three + Four + + + + + + + + Five + Six + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/CustomWiredJsrJobOperatorTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/CustomWiredJsrJobOperatorTests-context.xml new file mode 100644 index 0000000000..4e36f93c08 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/CustomWiredJsrJobOperatorTests-context.xml @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/ItemListenerParsingTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/ItemListenerParsingTests-context.xml new file mode 100644 index 0000000000..b1a5ee0a4f --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/ItemListenerParsingTests-context.xml @@ -0,0 +1,92 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + One + Two + + + + + + + + Three + Four + + + + + + + + Five + Six + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/JobListenerParsingTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/JobListenerParsingTests-context.xml new file mode 100644 index 0000000000..8d4fb38985 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/JobListenerParsingTests-context.xml @@ -0,0 +1,37 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/JobPropertySubstitutionTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/JobPropertySubstitutionTests-context.xml new file mode 100644 index 0000000000..f75bcefd0e --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/JobPropertySubstitutionTests-context.xml @@ -0,0 +1,58 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/JsrDecisionParsingTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/JsrDecisionParsingTests-context.xml new file mode 100644 index 0000000000..f71902293e --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/JsrDecisionParsingTests-context.xml @@ -0,0 +1,35 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryListenerTestBase-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryListenerTestBase-context.xml new file mode 100644 index 0000000000..803f12b0f6 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryListenerTestBase-context.xml @@ -0,0 +1,36 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerExhausted.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerExhausted.xml new file mode 100644 index 0000000000..37763e1699 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerExhausted.xml @@ -0,0 +1,15 @@ + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerListenerException.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerListenerException.xml new file mode 100644 index 0000000000..0fadd800fb --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerListenerException.xml @@ -0,0 +1,15 @@ + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerRetryOnce.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerRetryOnce.xml new file mode 100644 index 0000000000..c5b6f9d639 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/RetryReadListenerRetryOnce.xml @@ -0,0 +1,15 @@ + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/SimpleItemBasedJobParsingTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/SimpleItemBasedJobParsingTests-context.xml new file mode 100644 index 0000000000..493574feac --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/SimpleItemBasedJobParsingTests-context.xml @@ -0,0 +1,111 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + One + Two + Three + Four + Five + + + + + + + + One + Two + Three + Four + Five + + + + + + + + One + Two + + + + + + + + One + Two + Three + Four + Five + Six + Seven + Eight + Nine + Ten + Eleven + Twelve + Thirteen + Fourteen + Fifteen + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/SimpleJobParsingTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/SimpleJobParsingTests-context.xml new file mode 100644 index 0000000000..ef6ac6547a --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/SimpleJobParsingTests-context.xml @@ -0,0 +1,35 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/StepListenerParsingTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/StepListenerParsingTests-context.xml new file mode 100644 index 0000000000..23a6c891af --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/StepListenerParsingTests-context.xml @@ -0,0 +1,46 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/batch.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/batch.xml new file mode 100644 index 0000000000..6c798fdbd0 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/batch.xml @@ -0,0 +1,3 @@ + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/default-split-task-executor-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/default-split-task-executor-context.xml new file mode 100644 index 0000000000..f712b4ff27 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/default-split-task-executor-context.xml @@ -0,0 +1,69 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + One + Two + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/invalid-split-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/invalid-split-context.xml new file mode 100644 index 0000000000..02422db4bb --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/invalid-split-context.xml @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/override_batch.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/override_batch.xml new file mode 100644 index 0000000000..3eb0dfe608 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/override_batch.xml @@ -0,0 +1,8 @@ + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/user-specified-split-task-executor-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/user-specified-split-task-executor-context.xml new file mode 100644 index 0000000000..4902f1c4bb --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/jsr/configuration/xml/user-specified-split-task-executor-context.xml @@ -0,0 +1,71 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + One + Two + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/JobLauncherIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/JobLauncherIntegrationTests-context.xml index b59f9d2ef9..fe3b13ea0d 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/JobLauncherIntegrationTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/JobLauncherIntegrationTests-context.xml @@ -1,9 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -29,7 +28,7 @@ - + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/2jobs.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/2jobs.xml index 298091faed..c1b351e917 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/2jobs.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/2jobs.xml @@ -5,9 +5,9 @@ xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/error.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/error.xml index 260dcfa02f..5bd39d1d18 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/error.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/error.xml @@ -2,7 +2,7 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/job.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/job.xml index a60094b698..441c388770 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/job.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/job.xml @@ -3,9 +3,9 @@ xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/job2.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/job2.xml index cbcb6729e2..937956590e 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/job2.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/job2.xml @@ -5,9 +5,9 @@ xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/launcher-with-environment.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/launcher-with-environment.xml index 329410a4ef..156954cda3 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/launcher-with-environment.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/launcher-with-environment.xml @@ -5,9 +5,9 @@ xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/launcher-with-locator.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/launcher-with-locator.xml index 7470f061cf..2c9a736b3e 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/launcher-with-locator.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/launcher-with-locator.xml @@ -5,9 +5,9 @@ xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-loader.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-loader.xml index abfea3d663..92f5373522 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-loader.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-loader.xml @@ -3,9 +3,9 @@ xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-registry-and-auto-register.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-registry-and-auto-register.xml index 05140799ca..bf74cd76e9 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-registry-and-auto-register.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-registry-and-auto-register.xml @@ -3,9 +3,9 @@ xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-registry.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-registry.xml index fc36208c7e..685af35a55 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-registry.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment-with-registry.xml @@ -3,9 +3,9 @@ xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment.xml index b49cf76b40..e569c6b641 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/launch/support/test-environment.xml @@ -3,9 +3,9 @@ xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/OptimisticLockingFailureTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/OptimisticLockingFailureTests-context.xml new file mode 100644 index 0000000000..cbe1d992d8 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/OptimisticLockingFailureTests-context.xml @@ -0,0 +1,80 @@ + + + + + + + + + + + + + + + + + + + + + + + 1 + 2 + 3 + 4 + 5 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + org/springframework/batch/core/schema-drop-hsqldb.sql + org/springframework/batch/core/schema-hsqldb.sql + + + + + + + + + + + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/TablePrefixTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/TablePrefixTests-context.xml index 4ec9ac6462..8ef19eacc3 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/TablePrefixTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/TablePrefixTests-context.xml @@ -1,6 +1,6 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> @@ -28,9 +28,9 @@ - + - + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/data-source-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/data-source-context.xml index 02785c96da..1bd00d117c 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/data-source-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/data-source-context.xml @@ -1,6 +1,6 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -11,9 +11,9 @@ - + - + @@ -37,4 +37,4 @@ - \ No newline at end of file + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/schema-prefix-hsqldb.sql b/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/schema-prefix-hsqldb.sql index ba2a358217..b2c93d6bf3 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/schema-prefix-hsqldb.sql +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/schema-prefix-hsqldb.sql @@ -28,6 +28,7 @@ CREATE TABLE PREFIX_JOB_EXECUTION ( EXIT_CODE VARCHAR(20) , EXIT_MESSAGE VARCHAR(2500) , LAST_UPDATED TIMESTAMP, + JOB_CONFIGURATION_LOCATION VARCHAR(2500) NULL, constraint PREFIX_JOB_INST_EXEC_FK foreign key (JOB_INSTANCE_ID) references PREFIX_JOB_INSTANCE(JOB_INSTANCE_ID) ) ; diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/sql-dao-test.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/sql-dao-test.xml index af59d264d1..b95d87b838 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/sql-dao-test.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/sql-dao-test.xml @@ -1,6 +1,6 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -36,4 +36,4 @@ - \ No newline at end of file + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/support/SimpleJobRepositoryProxyTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/support/SimpleJobRepositoryProxyTests-context.xml index 69ab7a7ca7..1e3c766407 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/support/SimpleJobRepositoryProxyTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/support/SimpleJobRepositoryProxyTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/aop%20https://www.springframework.org/schema/aop/spring-aop.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/resource/ListPreparedStatementSetterTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/resource/ListPreparedStatementSetterTests-context.xml index fde6bc6b25..81c40ed8c7 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/resource/ListPreparedStatementSetterTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/resource/ListPreparedStatementSetterTests-context.xml @@ -3,9 +3,9 @@ xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans-%20%20%20%20%20%20%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd+%20%20%20%20%20%20%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd%20%20%20%20%20%20%20%20http://www.springframework.org/schema/batch-%20%20%20%20%20%20%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd"> + https://www.springframework.org/schema/batch/spring-batch-2.2.xsd"> @@ -30,7 +30,7 @@ - + #{jobParameters['min.id']} diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/AsyncJobScopeIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/AsyncJobScopeIntegrationTests-context.xml new file mode 100644 index 0000000000..4ed1be354e --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/AsyncJobScopeIntegrationTests-context.xml @@ -0,0 +1,13 @@ + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/AsyncStepScopeIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/AsyncStepScopeIntegrationTests-context.xml index 723774353e..6344c9c6a5 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/AsyncStepScopeIntegrationTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/AsyncStepScopeIntegrationTests-context.xml @@ -3,9 +3,9 @@ xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeDestructionCallbackIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeDestructionCallbackIntegrationTests-context.xml new file mode 100644 index 0000000000..0d41fa9bcb --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeDestructionCallbackIntegrationTests-context.xml @@ -0,0 +1,65 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeIntegrationTests-context.xml new file mode 100644 index 0000000000..2c88b7e59e --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeIntegrationTests-context.xml @@ -0,0 +1,54 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeNestedIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeNestedIntegrationTests-context.xml new file mode 100644 index 0000000000..080ba6a879 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeNestedIntegrationTests-context.xml @@ -0,0 +1,33 @@ + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopePlaceholderIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopePlaceholderIntegrationTests-context.xml new file mode 100644 index 0000000000..dd7122628e --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopePlaceholderIntegrationTests-context.xml @@ -0,0 +1,60 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #{jobExecutionContext[foo]} + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeProxyTargetClassIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeProxyTargetClassIntegrationTests-context.xml new file mode 100644 index 0000000000..855c6e1eca --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeProxyTargetClassIntegrationTests-context.xml @@ -0,0 +1,18 @@ + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeStartupIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeStartupIntegrationTests-context.xml new file mode 100644 index 0000000000..f5b790d14d --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/JobScopeStartupIntegrationTests-context.xml @@ -0,0 +1,31 @@ + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeClassIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeClassIntegrationTests-context.xml index 50bd104138..c974136c71 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeClassIntegrationTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeClassIntegrationTests-context.xml @@ -2,9 +2,9 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeDestructionCallbackIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeDestructionCallbackIntegrationTests-context.xml index 4cd6e7ae72..1fcdd23a8f 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeDestructionCallbackIntegrationTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeDestructionCallbackIntegrationTests-context.xml @@ -4,10 +4,10 @@ xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd - http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/context https://www.springframework.org/schema/context/spring-context.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeIntegrationTests-context.xml index dc92521e10..e232984540 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeIntegrationTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeIntegrationTests-context.xml @@ -2,9 +2,9 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeNestedIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeNestedIntegrationTests-context.xml index b34c64ad01..7a7860a32e 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeNestedIntegrationTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeNestedIntegrationTests-context.xml @@ -4,10 +4,10 @@ xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd - http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/context https://www.springframework.org/schema/context/spring-context.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopePerformanceTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopePerformanceTests-context.xml index a7692b249c..94dce4ce80 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopePerformanceTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopePerformanceTests-context.xml @@ -3,9 +3,9 @@ xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopePlaceholderIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopePlaceholderIntegrationTests-context.xml index c8bcf9d8aa..4927cc4020 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopePlaceholderIntegrationTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopePlaceholderIntegrationTests-context.xml @@ -2,9 +2,9 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeProxyTargetClassIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeProxyTargetClassIntegrationTests-context.xml index 56b52816c5..3fbceff861 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeProxyTargetClassIntegrationTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeProxyTargetClassIntegrationTests-context.xml @@ -3,9 +3,9 @@ xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeProxyTargetClassOverrideIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeProxyTargetClassOverrideIntegrationTests-context.xml new file mode 100644 index 0000000000..737cc4ba56 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeProxyTargetClassOverrideIntegrationTests-context.xml @@ -0,0 +1,69 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeStartupIntegrationTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeStartupIntegrationTests-context.xml index 3162bba5b2..7dce57c955 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeStartupIntegrationTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/StepScopeStartupIntegrationTests-context.xml @@ -4,14 +4,14 @@ xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd - http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/context https://www.springframework.org/schema/context/spring-context.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx.xsd"> - + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/context/CommitIntervalJobParameter-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/context/CommitIntervalJobParameter-context.xml new file mode 100644 index 0000000000..d1b492e213 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/context/CommitIntervalJobParameter-context.xml @@ -0,0 +1,42 @@ + + + + + + + + + + + + + + + + + + + + + + + + foo + bar + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/AsyncPlaceholderTargetSourceTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/AsyncPlaceholderTargetSourceTests-context.xml deleted file mode 100644 index 939c63553a..0000000000 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/AsyncPlaceholderTargetSourceTests-context.xml +++ /dev/null @@ -1,28 +0,0 @@ - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/MultipleContextPlaceholderTargetSourceTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/MultipleContextPlaceholderTargetSourceTests-context.xml deleted file mode 100644 index 27c79296ca..0000000000 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/MultipleContextPlaceholderTargetSourceTests-context.xml +++ /dev/null @@ -1,104 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %{attributes[foo]} - - - - - - - - - - %{attributes[foo]} - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/PlaceholderTargetSourceCustomEditorTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/PlaceholderTargetSourceCustomEditorTests-context.xml deleted file mode 100644 index d32ab9bf27..0000000000 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/PlaceholderTargetSourceCustomEditorTests-context.xml +++ /dev/null @@ -1,48 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/PlaceholderTargetSourceErrorTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/PlaceholderTargetSourceErrorTests-context.xml deleted file mode 100644 index f3e809972f..0000000000 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/PlaceholderTargetSourceErrorTests-context.xml +++ /dev/null @@ -1,48 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/PlaceholderTargetSourceTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/PlaceholderTargetSourceTests-context.xml deleted file mode 100644 index fc2094af50..0000000000 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/PlaceholderTargetSourceTests-context.xml +++ /dev/null @@ -1,192 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %{foo} - foo-%{integer} - bar-%{integer} - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/SimplePlaceholderTargetSourceTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/SimplePlaceholderTargetSourceTests-context.xml deleted file mode 100644 index 593bb3c303..0000000000 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/scope/util/SimplePlaceholderTargetSourceTests-context.xml +++ /dev/null @@ -1,30 +0,0 @@ - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/step/RestartInPriorStepTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/step/RestartInPriorStepTests-context.xml index 96f4aa9fa7..ef158c1216 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/step/RestartInPriorStepTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/step/RestartInPriorStepTests-context.xml @@ -1,14 +1,12 @@ - - + https://www.springframework.org/schema/batch/spring-batch-2.2.xsd"> + @@ -16,9 +14,9 @@ - + - + @@ -27,9 +25,9 @@ - + - + @@ -39,13 +37,13 @@ - + - + - + - + @@ -54,7 +52,9 @@ + + @@ -62,25 +62,21 @@ - + - + - + - - - - - + - + diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/step/RestartLoopTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/step/RestartLoopTests-context.xml new file mode 100644 index 0000000000..5fc49141e4 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/step/RestartLoopTests-context.xml @@ -0,0 +1,34 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/step/item/FaultTolerantExceptionClassesTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/step/item/FaultTolerantExceptionClassesTests-context.xml index ec8b4a1ecc..9f84a00a63 100644 --- a/spring-batch-core/src/test/resources/org/springframework/batch/core/step/item/FaultTolerantExceptionClassesTests-context.xml +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/step/item/FaultTolerantExceptionClassesTests-context.xml @@ -2,8 +2,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/step/item/ScriptItemProcessorTest-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/step/item/ScriptItemProcessorTest-context.xml new file mode 100644 index 0000000000..bac1b48321 --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/step/item/ScriptItemProcessorTest-context.xml @@ -0,0 +1,39 @@ + + + + + + + + + + + + + + + + + + + + + blah + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/step/item/processor-test-simple.js b/spring-batch-core/src/test/resources/org/springframework/batch/core/step/item/processor-test-simple.js new file mode 100644 index 0000000000..3a9b9db1dd --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/step/item/processor-test-simple.js @@ -0,0 +1 @@ +item.toUpperCase(); diff --git a/spring-batch-core/src/test/resources/org/springframework/batch/core/step/skip/ReprocessExceptionTests-context.xml b/spring-batch-core/src/test/resources/org/springframework/batch/core/step/skip/ReprocessExceptionTests-context.xml new file mode 100644 index 0000000000..e91bb0e17d --- /dev/null +++ b/spring-batch-core/src/test/resources/org/springframework/batch/core/step/skip/ReprocessExceptionTests-context.xml @@ -0,0 +1,47 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-core/template.mf b/spring-batch-core/template.mf deleted file mode 100644 index 729519bb92..0000000000 --- a/spring-batch-core/template.mf +++ /dev/null @@ -1,32 +0,0 @@ -Manifest-Version: 1.0 -Bundle-SymbolicName: org.springframework.batch.core -Bundle-Name: Spring Batch Core -Bundle-Vendor: Spring -Bundle-Version: ${version} -Bundle-ManifestVersion: 2 -Ignored-Existing-Headers: - Import-Package, - Export-Package -Import-Template: - com.thoughtworks.xstream.*;version="[1.3,1.4)";resolution:=optional, - org.codehaus.jettison.*;version="[1.0,1.1)";resolution:=optional, - org.aopalliance.*;version="[1.0.0, 2.0.0)", - org.aspectj.*;version="[1.5.2,1.7.0)";resolution:=optional, - org.apache.commons.logging.*;version="[1.0.4, 2.0.0)", - org.springframework.batch.*;version="[2.2.0, 3.0.0)";resolution:=optional, - org.springframework.aop.*;version="[3.1.2, 4.0.0)", - org.springframework.beans.*;version="[3.1.2, 4.0.0)", - org.springframework.context.*;version="[3.1.2, 4.0.0)", - org.springframework.core.*;version="[3.1.2, 4.0.0)", - org.springframework.dao.*;version="[3.1.2, 4.0.0)";resolution:=optional, - org.springframework.jdbc.*;version="[3.1.2, 4.0.0)";resolution:=optional, - org.springframework.transaction.*;version="[3.1.2, 4.0.0)";resolution:=optional, - org.springframework.util.*;version="[3.1.2, 4.0.0)";resolution:=optional, - org.springframework.stereotype.*;version="[3.1.2, 4.0.0)";resolution:=optional, - org.springframework.retry.*;version="[1.0.0,2.0.0)";resolution:=optional, - org.springframework.classify.*;version="[1.0.0,2.0.0)";resolution:=optional, - org.springframework.osgi.*;version="[1.1.0, 2.0.0)";resolution:=optional, - org.osgi.framework;version="0";resolution:=optional, - org.w3c.dom;version="0";resolution:=optional, - javax.annotation.*;version="0";resolution:=optional, - javax.sql.*;version="0";resolution:=optional diff --git a/spring-batch-docs/asciidoc/appendix.adoc b/spring-batch-docs/asciidoc/appendix.adoc new file mode 100644 index 0000000000..efd883b007 --- /dev/null +++ b/spring-batch-docs/asciidoc/appendix.adoc @@ -0,0 +1,132 @@ +:batch-asciidoc: ./ +:toc: left +:toclevels: 4 + +[[listOfReadersAndWriters]] + +[appendix] +== List of ItemReaders and ItemWriters + +[[itemReadersAppendix]] + +=== Item Readers + +.Available Item Readers +[options="header"] +|=============== +|Item Reader|Description +|AbstractItemCountingItemStreamItemReader|Abstract base class that provides basic + restart capabilities by counting the number of items returned from + an `ItemReader`. +|AggregateItemReader|An `ItemReader` that delivers a list as its + item, storing up objects from the injected `ItemReader` until they + are ready to be packed out as a collection. This `ItemReader` should + mark the beginning and end of records with the constant values in + `FieldSetMapper AggregateItemReader#__$$BEGIN_RECORD$$__` and + `AggregateItemReader#__$$END_RECORD$$__`. +|AmqpItemReader|Given a Spring `AmqpTemplate`, it provides + synchronous receive methods. The `receiveAndConvert()` method + lets you receive POJO objects. +|KafkaItemReader|An `ItemReader` that reads messages from an Apache Kafka topic. +It can be configured to read messages from multiple partitions of the same topic. +This reader stores message offsets in the execution context to support restart capabilities. +|FlatFileItemReader|Reads from a flat file. Includes `ItemStream` + and `Skippable` functionality. See link:readersAndWriters.html#flatFileItemReader[`FlatFileItemReader`]. +|HibernateCursorItemReader|Reads from a cursor based on an HQL query. See + link:readersAndWriters.html#cursorBasedItemReaders[`Cursor-based ItemReaders`]. +|HibernatePagingItemReader|Reads from a paginated HQL query +|ItemReaderAdapter|Adapts any class to the + `ItemReader` interface. +|JdbcCursorItemReader|Reads from a database cursor via JDBC. See + link:readersAndWriters.html#cursorBasedItemReaders[`Cursor-based ItemReaders`]. +|JdbcPagingItemReader|Given an SQL statement, pages through the rows, + such that large datasets can be read without running out of + memory. +|JmsItemReader|Given a Spring `JmsOperations` object and a JMS + Destination or destination name to which to send errors, provides items + received through the injected `JmsOperations#receive()` + method. +|JpaPagingItemReader|Given a JPQL statement, pages through the + rows, such that large datasets can be read without running out of + memory. +|ListItemReader|Provides the items from a list, one at a + time. +|MongoItemReader|Given a `MongoOperations` object and a JSON-based MongoDB + query, provides items received from the `MongoOperations#find()` method. +|Neo4jItemReader|Given a `Neo4jOperations` object and the components of a + Cyhper query, items are returned as the result of the Neo4jOperations.query + method. +|RepositoryItemReader|Given a Spring Data `PagingAndSortingRepository` object, + a `Sort`, and the name of method to execute, returns items provided by the + Spring Data repository implementation. +|StoredProcedureItemReader|Reads from a database cursor resulting from the + execution of a database stored procedure. See link:readersAndWriters.html#StoredProcedureItemReader[`StoredProcedureItemReader`] +|StaxEventItemReader|Reads via StAX. see link:readersAndWriters.html#StaxEventItemReader[`StaxEventItemReader`]. +|JsonItemReader|Reads items from a Json document. see link:readersAndWriters.html#JsonItemReader[`JsonItemReader`]. + +|=============== + + +[[itemWritersAppendix]] + + +=== Item Writers + +.Available Item Writers +[options="header"] +|=============== +|Item Writer|Description +|AbstractItemStreamItemWriter|Abstract base class that combines the + `ItemStream` and + `ItemWriter` interfaces. +|AmqpItemWriter|Given a Spring `AmqpTemplate`, it provides + for a synchronous `send` method. The `convertAndSend(Object)` + method lets you send POJO objects. +|CompositeItemWriter|Passes an item to the `write` method of each + in an injected `List` of `ItemWriter` objects. +|FlatFileItemWriter|Writes to a flat file. Includes `ItemStream` and + Skippable functionality. See link:readersAndWriters.html#flatFileItemWriter[`FlatFileItemWriter`]. +|GemfireItemWriter|Using a `GemfireOperations` object, items are either written + or removed from the Gemfire instance based on the configuration of the delete + flag. +|HibernateItemWriter|This item writer is Hibernate-session aware + and handles some transaction-related work that a non-"hibernate-aware" + item writer would not need to know about and then delegates + to another item writer to do the actual writing. +|ItemWriterAdapter|Adapts any class to the + `ItemWriter` interface. +|JdbcBatchItemWriter|Uses batching features from a + `PreparedStatement`, if available, and can + take rudimentary steps to locate a failure during a + `flush`. +|JmsItemWriter|Using a `JmsOperations` object, items are written + to the default queue through the `JmsOperations#convertAndSend()` method. +|JpaItemWriter|This item writer is JPA EntityManager-aware + and handles some transaction-related work that a non-"JPA-aware" + `ItemWriter` would not need to know about and + then delegates to another writer to do the actual writing. +|KafkaItemWriter|Using a `KafkaTemplate` object, items are written to the default topic through the + `KafkaTemplate#sendDefault(Object, Object)` method using a `Converter` to map the key from the item. + A delete flag can also be configured to send delete events to the topic. +|MimeMessageItemWriter|Using Spring's `JavaMailSender`, items of type `MimeMessage` + are sent as mail messages. +|MongoItemWriter|Given a `MongoOperations` object, items are written + through the `MongoOperations.save(Object)` method. The actual write is delayed + until the last possible moment before the transaction commits. +|Neo4jItemWriter|Given a `Neo4jOperations` object, items are persisted through the + `save(Object)` method or deleted through the `delete(Object)` per the + `ItemWriter's` configuration +|PropertyExtractingDelegatingItemWriter|Extends `AbstractMethodInvokingDelegator` + creating arguments on the fly. Arguments are created by retrieving + the values from the fields in the item to be processed (through a + `SpringBeanWrapper`), based on an injected array of field + names. +|RepositoryItemWriter|Given a Spring Data `CrudRepository` implementation, + items are saved through the method specified in the configuration. +|StaxEventItemWriter|Uses a `Marshaller` implementation to + convert each item to XML and then writes it to an XML file using + StAX. +|JsonFileItemWriter|Uses a `JsonObjectMarshaller` implementation to + convert each item to Json and then writes it to an Json file. + +|=============== diff --git a/spring-batch-docs/asciidoc/common-patterns.adoc b/spring-batch-docs/asciidoc/common-patterns.adoc new file mode 100644 index 0000000000..99bc39535d --- /dev/null +++ b/spring-batch-docs/asciidoc/common-patterns.adoc @@ -0,0 +1,727 @@ +:batch-asciidoc: ./ +:toc: left +:toclevels: 4 + +[[commonPatterns]] + +== Common Batch Patterns + +ifndef::onlyonetoggle[] +include::toggle.adoc[] +endif::onlyonetoggle[] + + +Some batch jobs can be assembled purely from off-the-shelf components in Spring Batch. +For instance, the `ItemReader` and `ItemWriter` implementations can be configured to +cover a wide range of scenarios. However, for the majority of cases, custom code must be +written. The main API entry points for application developers are the `Tasklet`, the +`ItemReader`, the `ItemWriter`, and the various listener interfaces. Most simple batch +jobs can use off-the-shelf input from a Spring Batch `ItemReader`, but it is often the +case that there are custom concerns in the processing and writing that require developers +to implement an `ItemWriter` or `ItemProcessor`. + +In this chapter, we provide a few examples of common patterns in custom business logic. +These examples primarily feature the listener interfaces. It should be noted that an +`ItemReader` or `ItemWriter` can implement a listener interface as well, if appropriate. + +[[loggingItemProcessingAndFailures]] +=== Logging Item Processing and Failures + +A common use case is the need for special handling of errors in a step, item by item, +perhaps logging to a special channel or inserting a record into a database. A +chunk-oriented `Step` (created from the step factory beans) lets users implement this use +case with a simple `ItemReadListener` for errors on `read` and an `ItemWriteListener` for +errors on `write`. The following code snippet illustrates a listener that logs both read +and write failures: + +[source, java] +---- +public class ItemFailureLoggerListener extends ItemListenerSupport { + + private static Log logger = LogFactory.getLog("item.error"); + + public void onReadError(Exception ex) { + logger.error("Encountered error on read", e); + } + + public void onWriteError(Exception ex, List items) { + logger.error("Encountered error on write", ex); + } +} +---- + +Having implemented this listener, it must be registered with a step, as shown in the +following example: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + +... + + + + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public Step simpleStep() { + return this.stepBuilderFactory.get("simpleStep") + ... + .listener(new ItemFailureLoggerListener()) + .build(); +} +---- + +IMPORTANT: if your listener does anything in an `onError()` method, it must be inside +a transaction that is going to be rolled back. If you need to use a transactional +resource, such as a database, inside an `onError()` method, consider adding a declarative +transaction to that method (see Spring Core Reference Guide for details), and giving its +propagation attribute a value of `REQUIRES_NEW`. + +[[stoppingAJobManuallyForBusinessReasons]] +=== Stopping a Job Manually for Business Reasons + +Spring Batch provides a `stop()` method through the `JobLauncher` interface, but this is +really for use by the operator rather than the application programmer. Sometimes, it is +more convenient or makes more sense to stop a job execution from within the business +logic. + +The simplest thing to do is to throw a `RuntimeException` (one that is neither retried +indefinitely nor skipped). For example, a custom exception type could be used, as shown +in the following example: + +[source, java] +---- +public class PoisonPillItemProcessor implements ItemProcessor { + + @Override + public T process(T item) throws Exception { + if (isPoisonPill(item)) { + throw new PoisonPillException("Poison pill detected: " + item); + } + return item; + } +} +---- + +Another simple way to stop a step from executing is to return `null` from the +`ItemReader`, as shown in the following example: + +[source, java] +---- +public class EarlyCompletionItemReader implements ItemReader { + + private ItemReader delegate; + + public void setDelegate(ItemReader delegate) { ... } + + public T read() throws Exception { + T item = delegate.read(); + if (isEndItem(item)) { + return null; // end the step here + } + return item; + } + +} +---- +The previous example actually relies on the fact that there is a default implementation +of the `CompletionPolicy` strategy that signals a complete batch when the item to be +processed is `null`. A more sophisticated completion policy could be implemented and +injected into the `Step` through the `SimpleStepFactoryBean`, as shown in the following +example: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + + + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public Step simpleStep() { + return this.stepBuilderFactory.get("simpleStep") + .chunk(new SpecialCompletionPolicy()) + .reader(reader()) + .writer(writer()) + .build(); +} +---- + +An alternative is to set a flag in the `StepExecution`, which is checked by the `Step` +implementations in the framework in between item processing. To implement this +alternative, we need access to the current `StepExecution`, and this can be achieved by +implementing a `StepListener` and registering it with the `Step`. The following example +shows a listener that sets the flag: + +[source, java] +---- +public class CustomItemWriter extends ItemListenerSupport implements StepListener { + + private StepExecution stepExecution; + + public void beforeStep(StepExecution stepExecution) { + this.stepExecution = stepExecution; + } + + public void afterRead(Object item) { + if (isPoisonPill(item)) { + stepExecution.setTerminateOnly(); + } + } + +} +---- + +When the flag is set, the default behavior is for the step to throw a +`JobInterruptedException`. This behavior can be controlled through the +`StepInterruptionPolicy`. However, the only choice is to throw or not throw an exception, +so this is always an abnormal ending to a job. + +[[addingAFooterRecord]] +=== Adding a Footer Record + +Often, when writing to flat files, a "footer" record must be appended to the end of the +file, after all processing has be completed. This can be achieved using the +`FlatFileFooterCallback` interface provided by Spring Batch. The `FlatFileFooterCallback` +(and its counterpart, the `FlatFileHeaderCallback`) are optional properties of the +`FlatFileItemWriter` and can be added to an item writer as shown in the following +example: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public FlatFileItemWriter itemWriter(Resource outputResource) { + return new FlatFileItemWriterBuilder() + .name("itemWriter") + .resource(outputResource) + .lineAggregator(lineAggregator()) + .headerCallback(headerCallback()) + .footerCallback(footerCallback()) + .build(); +} +---- + +The footer callback interface has just one method that is called when the footer must be +written, as shown in the following interface definition: + +[source, java] +---- +public interface FlatFileFooterCallback { + + void writeFooter(Writer writer) throws IOException; + +} +---- + +[[writingASummaryFooter]] +==== Writing a Summary Footer + +A common requirement involving footer records is to aggregate information during the +output process and to append this information to the end of the file. This footer often +serves as a summarization of the file or provides a checksum. + +For example, if a batch job is writing `Trade` records to a flat file, and there is a +requirement that the total amount from all the `Trades` is placed in a footer, then the +following `ItemWriter` implementation can be used: + +[source, java] +---- +public class TradeItemWriter implements ItemWriter, + FlatFileFooterCallback { + + private ItemWriter delegate; + + private BigDecimal totalAmount = BigDecimal.ZERO; + + public void write(List items) throws Exception { + BigDecimal chunkTotal = BigDecimal.ZERO; + for (Trade trade : items) { + chunkTotal = chunkTotal.add(trade.getAmount()); + } + + delegate.write(items); + + // After successfully writing all items + totalAmount = totalAmount.add(chunkTotal); + } + + public void writeFooter(Writer writer) throws IOException { + writer.write("Total Amount Processed: " + totalAmount); + } + + public void setDelegate(ItemWriter delegate) {...} +} +---- + +This `TradeItemWriter` stores a `totalAmount` value that is increased with the `amount` +from each `Trade` item written. After the last `Trade` is processed, the framework calls +`writeFooter`, which puts the `totalAmount` into the file. Note that the `write` method +makes use of a temporary variable, `chunkTotal`, that stores the total of the +`Trade` amounts in the chunk. This is done to ensure that, if a skip occurs in the +`write` method, the `totalAmount` is left unchanged. It is only at the end of the `write` +method, once we are guaranteed that no exceptions are thrown, that we update the +`totalAmount`. + +In order for the `writeFooter` method to be called, the `TradeItemWriter` (which +implements `FlatFileFooterCallback`) must be wired into the `FlatFileItemWriter` as the +`footerCallback`. The following example shows how to do so: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + + + + + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public TradeItemWriter tradeItemWriter() { + TradeItemWriter itemWriter = new TradeItemWriter(); + + itemWriter.setDelegate(flatFileItemWriter(null)); + + return itemWriter; +} + +@Bean +public FlatFileItemWriter flatFileItemWriter(Resource outputResource) { + return new FlatFileItemWriterBuilder() + .name("itemWriter") + .resource(outputResource) + .lineAggregator(lineAggregator()) + .footerCallback(tradeItemWriter()) + .build(); +} +---- + +The way that the `TradeItemWriter` has been written so far functions correctly only if +the `Step` is not restartable. This is because the class is stateful (since it stores the +`totalAmount`), but the `totalAmount` is not persisted to the database. Therefore, it +cannot be retrieved in the event of a restart. In order to make this class restartable, +the `ItemStream` interface should be implemented along with the methods `open` and +`update`, as shown in the following example: + +[source, java] +---- +public void open(ExecutionContext executionContext) { + if (executionContext.containsKey("total.amount") { + totalAmount = (BigDecimal) executionContext.get("total.amount"); + } +} + +public void update(ExecutionContext executionContext) { + executionContext.put("total.amount", totalAmount); +} +---- + +The update method stores the most current version of `totalAmount` to the +`ExecutionContext` just before that object is persisted to the database. The open method +retrieves any existing `totalAmount` from the `ExecutionContext` and uses it as the +starting point for processing, allowing the `TradeItemWriter` to pick up on restart where +it left off the previous time the `Step` was run. + +[[drivingQueryBasedItemReaders]] +=== Driving Query Based ItemReaders + +In the link:readersAndWriters.html[chapter on readers and writers], database input using +paging was discussed. Many database vendors, such as DB2, have extremely pessimistic +locking strategies that can cause issues if the table being read also needs to be used by +other portions of the online application. Furthermore, opening cursors over extremely +large datasets can cause issues on databases from certain vendors. Therefore, many +projects prefer to use a 'Driving Query' approach to reading in data. This approach works +by iterating over keys, rather than the entire object that needs to be returned, as the +following image illustrates: + +.Driving Query Job +image::{batch-asciidoc}images/drivingQueryExample.png[Driving Query Job, scaledwidth="60%"] + +As you can see, the example shown in the preceding image uses the same 'FOO' table as was +used in the cursor-based example. However, rather than selecting the entire row, only the +IDs were selected in the SQL statement. So, rather than a `FOO` object being returned +from `read`, an `Integer` is returned. This number can then be used to query for the +'details', which is a complete `Foo` object, as shown in the following image: + +.Driving Query Example +image::{batch-asciidoc}images/drivingQueryJob.png[Driving Query Example, scaledwidth="60%"] + +An `ItemProcessor` should be used to transform the key obtained from the driving query +into a full 'Foo' object. An existing DAO can be used to query for the full object based +on the key. + +[[multiLineRecords]] +=== Multi-Line Records + +While it is usually the case with flat files that each record is confined to a single +line, it is common that a file might have records spanning multiple lines with multiple +formats. The following excerpt from a file shows an example of such an arrangement: + +---- +HEA;0013100345;2007-02-15 +NCU;Smith;Peter;;T;20014539;F +BAD;;Oak Street 31/A;;Small Town;00235;IL;US +FOT;2;2;267.34 +---- +Everything between the line starting with 'HEA' and the line starting with 'FOT' is +considered one record. There are a few considerations that must be made in order to +handle this situation correctly: + +* Instead of reading one record at a time, the `ItemReader` must read every line of the +multi-line record as a group, so that it can be passed to the `ItemWriter` intact. +* Each line type may need to be tokenized differently. + +Because a single record spans multiple lines and because we may not know how many lines +there are, the `ItemReader` must be careful to always read an entire record. In order to +do this, a custom `ItemReader` should be implemented as a wrapper for the +`FlatFileItemReader`, as shown in the following example: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + + + + + + + + + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public MultiLineTradeItemReader itemReader() { + MultiLineTradeItemReader itemReader = new MultiLineTradeItemReader(); + + itemReader.setDelegate(flatFileItemReader()); + + return itemReader; +} + +@Bean +public FlatFileItemReader flatFileItemReader() { + FlatFileItemReader reader = new FlatFileItemReaderBuilder<>() + .name("flatFileItemReader") + .resource(new ClassPathResource("data/iosample/input/multiLine.txt")) + .lineTokenizer(orderFileTokenizer()) + .fieldSetMapper(orderFieldSetMapper()) + .build(); + return reader; +} +---- + +To ensure that each line is tokenized properly, which is especially important for +fixed-length input, the `PatternMatchingCompositeLineTokenizer` can be used on the +delegate `FlatFileItemReader`. See +link:readersAndWriters.html#flatFileItemReader[`FlatFileItemReader` in the Readers and +Writers chapter] for more details. The delegate reader then uses a +`PassThroughFieldSetMapper` to deliver a `FieldSet` for each line back to the wrapping +`ItemReader`, as shown in the following example: + +.XML Content +[source, xml, role="xmlContent"] +---- + + + + + + + + + + +---- + +.Java Content +[source, java, role="javaContent"] +---- +@Bean +public PatternMatchingCompositeLineTokenizer orderFileTokenizer() { + PatternMatchingCompositeLineTokenizer tokenizer = + new PatternMatchingCompositeLineTokenizer(); + + Map tokenizers = new HashMap<>(4); + + tokenizers.put("HEA*", headerRecordTokenizer()); + tokenizers.put("FOT*", footerRecordTokenizer()); + tokenizers.put("NCU*", customerLineTokenizer()); + tokenizers.put("BAD*", billingAddressLineTokenizer()); + + tokenizer.setTokenizers(tokenizers); + + return tokenizer; +} +---- + +This wrapper has to be able to recognize the end of a record so that it can continually +call `read()` on its delegate until the end is reached. For each line that is read, the +wrapper should build up the item to be returned. Once the footer is reached, the item can +be returned for delivery to the `ItemProcessor` and `ItemWriter`, as shown in the +following example: + +[source, java] +---- +private FlatFileItemReader
      delegate; + +public Trade read() throws Exception { + Trade t = null; + + for (FieldSet line = null; (line = this.delegate.read()) != null;) { + String prefix = line.readString(0); + if (prefix.equals("HEA")) { + t = new Trade(); // Record must start with header + } + else if (prefix.equals("NCU")) { + Assert.notNull(t, "No header was found."); + t.setLast(line.readString(1)); + t.setFirst(line.readString(2)); + ... + } + else if (prefix.equals("BAD")) { + Assert.notNull(t, "No header was found."); + t.setCity(line.readString(4)); + t.setState(line.readString(6)); + ... + } + else if (prefix.equals("FOT")) { + return t; // Record must end with footer + } + } + Assert.isNull(t, "No 'END' was found."); + return null; +} +---- + +[[executingSystemCommands]] +=== Executing System Commands + +Many batch jobs require that an external command be called from within the batch job. +Such a process could be kicked off separately by the scheduler, but the advantage of +common metadata about the run would be lost. Furthermore, a multi-step job would also +need to be split up into multiple jobs as well. + +Because the need is so common, Spring Batch provides a `Tasklet` implementation for +calling system commands, as shown in the following example: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public SystemCommandTasklet tasklet() { + SystemCommandTasklet tasklet = new SystemCommandTasklet(); + + tasklet.setCommand("echo hello"); + tasklet.setTimeout(5000); + + return tasklet; +} +---- + +[[handlingStepCompletionWhenNoInputIsFound]] +=== Handling Step Completion When No Input is Found + +In many batch scenarios, finding no rows in a database or file to process is not +exceptional. The `Step` is simply considered to have found no work and completes with 0 +items read. All of the `ItemReader` implementations provided out of the box in Spring +Batch default to this approach. This can lead to some confusion if nothing is written out +even when input is present (which usually happens if a file was misnamed or some similar +issue arises). For this reason, the metadata itself should be inspected to determine how +much work the framework found to be processed. However, what if finding no input is +considered exceptional? In this case, programmatically checking the metadata for no items +processed and causing failure is the best solution. Because this is a common use case, +Spring Batch provides a listener with exactly this functionality, as shown in +the class definition for `NoWorkFoundStepExecutionListener`: + +[source, java] +---- +public class NoWorkFoundStepExecutionListener extends StepExecutionListenerSupport { + + public ExitStatus afterStep(StepExecution stepExecution) { + if (stepExecution.getReadCount() == 0) { + return ExitStatus.FAILED; + } + return null; + } + +} +---- + +The preceding `StepExecutionListener` inspects the `readCount` property of the +`StepExecution` during the 'afterStep' phase to determine if no items were read. If that +is the case, an exit code of FAILED is returned, indicating that the `Step` should fail. +Otherwise, `null` is returned, which does not affect the status of the `Step`. + +[[passingDataToFutureSteps]] +=== Passing Data to Future Steps + +It is often useful to pass information from one step to another. This can be done through +the `ExecutionContext`. The catch is that there are two `ExecutionContexts`: one at the +`Step` level and one at the `Job` level. The `Step` `ExecutionContext` remains only as +long as the step, while the `Job` `ExecutionContext` remains through the whole `Job`. On +the other hand, the `Step` `ExecutionContext` is updated every time the `Step` commits a +chunk, while the `Job` `ExecutionContext` is updated only at the end of each `Step`. + +The consequence of this separation is that all data must be placed in the `Step` +`ExecutionContext` while the `Step` is executing. Doing so ensures that the data is +stored properly while the `Step` runs. If data is stored to the `Job` `ExecutionContext`, +then it is not persisted during `Step` execution. If the `Step` fails, that data is lost. + +[source, java] +---- +public class SavingItemWriter implements ItemWriter { + private StepExecution stepExecution; + + public void write(List items) throws Exception { + // ... + + ExecutionContext stepContext = this.stepExecution.getExecutionContext(); + stepContext.put("someKey", someObject); + } + + @BeforeStep + public void saveStepExecution(StepExecution stepExecution) { + this.stepExecution = stepExecution; + } +} +---- + +To make the data available to future `Steps`, it must be "promoted" to the `Job` +`ExecutionContext` after the step has finished. Spring Batch provides the +`ExecutionContextPromotionListener` for this purpose. The listener must be configured +with the keys related to the data in the `ExecutionContext` that must be promoted. It can +also, optionally, be configured with a list of exit code patterns for which the promotion +should occur (`COMPLETED` is the default). As with all listeners, it must be registered +on the `Step` as shown in the following example: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + + + + + + + + + + ... + + + + + + + someKey + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public Job job1() { + return this.jobBuilderFactory.get("job1") + .start(step1()) + .next(step1()) + .build(); +} + +@Bean +public Step step1() { + return this.stepBuilderFactory.get("step1") + .chunk(10) + .reader(reader()) + .writer(savingWriter()) + .listener(promotionListener()) + .build(); +} + +@Bean +public ExecutionContextPromotionListener promotionListener() { + ExecutionContextPromotionListener listener = new ExecutionContextPromotionListener(); + + listener.setKeys(new String[] {"someKey" }); + + return listener; +} +---- + +Finally, the saved values must be retrieved from the `Job` `ExecutionContext`, as shown +in the following example: + +[source, java] +---- +public class RetrievingItemWriter implements ItemWriter { + private Object someObject; + + public void write(List items) throws Exception { + // ... + } + + @BeforeStep + public void retrieveInterstepData(StepExecution stepExecution) { + JobExecution jobExecution = stepExecution.getJobExecution(); + ExecutionContext jobContext = jobExecution.getExecutionContext(); + this.someObject = jobContext.get("someKey"); + } +} +---- diff --git a/spring-batch-docs/asciidoc/domain.adoc b/spring-batch-docs/asciidoc/domain.adoc new file mode 100644 index 0000000000..0d69f56a53 --- /dev/null +++ b/spring-batch-docs/asciidoc/domain.adoc @@ -0,0 +1,657 @@ +:batch-asciidoc: ./ +:toc: left +:toclevels: 4 + +[[domainLanguageOfBatch]] + +== The Domain Language of Batch + +ifndef::onlyonetoggle[] +include::toggle.adoc[] +endif::onlyonetoggle[] + +To any experienced batch architect, the overall concepts of batch processing used in +Spring Batch should be familiar and comfortable. There are "Jobs" and "Steps" and +developer-supplied processing units called `ItemReader` and `ItemWriter`. However, +because of the Spring patterns, operations, templates, callbacks, and idioms, there are +opportunities for the following: + +* Significant improvement in adherence to a clear separation of concerns. +* Clearly delineated architectural layers and services provided as interfaces. +* Simple and default implementations that allow for quick adoption and ease of use +out-of-the-box. +* Significantly enhanced extensibility. + +The following diagram is a simplified version of the batch reference architecture that +has been used for decades. It provides an overview of the components that make up the +domain language of batch processing. This architecture framework is a blueprint that has +been proven through decades of implementations on the last several generations of +platforms (COBOL/Mainframe, C++/Unix, and now Java/anywhere). JCL and COBOL developers +are likely to be as comfortable with the concepts as C++, C#, and Java developers. Spring +Batch provides a physical implementation of the layers, components, and technical +services commonly found in the robust, maintainable systems that are used to address the +creation of simple to complex batch applications, with the infrastructure and extensions +to address very complex processing needs. + +.Batch Stereotypes +image::{batch-asciidoc}images/spring-batch-reference-model.png[Figure 2.1: Batch Stereotypes, scaledwidth="60%"] + +The preceding diagram highlights the key concepts that make up the domain language of +Spring Batch. A Job has one to many steps, each of which has exactly one `ItemReader`, +one `ItemProcessor`, and one `ItemWriter`. A job needs to be launched (with +`JobLauncher`), and metadata about the currently running process needs to be stored (in +`JobRepository`). + +=== Job + +This section describes stereotypes relating to the concept of a batch job. A `Job` is an +entity that encapsulates an entire batch process. As is common with other Spring +projects, a `Job` is wired together with either an XML configuration file or Java-based +configuration. This configuration may be referred to as the "job configuration". However, +`Job` is just the top of an overall hierarchy, as shown in the following diagram: + +.Job Hierarchy +image::{batch-asciidoc}images/job-heirarchy.png[Job Hierarchy, scaledwidth="60%"] + +In Spring Batch, a `Job` is simply a container for `Step` instances. It combines multiple +steps that belong logically together in a flow and allows for configuration of properties +global to all steps, such as restartability. The job configuration contains: + +* The simple name of the job. +* Definition and ordering of `Step` instances. +* Whether or not the job is restartable. + +ifdef::backend-html5[] +[role="javaContent"] +A default simple implementation of the Job interface is provided by Spring Batch in the +form of the `SimpleJob` class, which creates some standard functionality on top of `Job`. +When using java based configuration, a collection of builders is made available for the +instantiation of a `Job`, as shown in the following example: + +[source, java, role="javaContent"] +---- +@Bean +public Job footballJob() { + return this.jobBuilderFactory.get("footballJob") + .start(playerLoad()) + .next(gameLoad()) + .next(playerSummarization()) + .end() + .build(); +} +---- + +[role="xmlContent"] +A default simple implementation of the `Job` interface is provided by Spring Batch in the +form of the `SimpleJob` class, which creates some standard functionality on top of `Job`. +However, the batch namespace abstracts away the need to instantiate it directly. Instead, +the `` tag can be used as shown in the following example: + +[source, xml, role="xmlContent"] +---- + + + + + +---- +endif::backend-html5[] + +ifdef::backend-pdf[] +A default simple implementation of the Job interface is provided by Spring Batch in the +form of the `SimpleJob` class, which creates some standard functionality on top of `Job`. +When using java based configuration, a collection of builders are made available for the +instantiation of a `Job`, as shown in the following example: + +[source, java] +---- +@Bean +public Job footballJob() { + return this.jobBuilderFactory.get("footballJob") + .start(playerLoad()) + .next(gameLoad()) + .next(playerSummarization()) + .end() + .build(); +} +---- + +However, when using XML configuration, the batch namespace abstracts away the need to +instantiate it directly. Instead, the `` tag can be used as shown in the following +example: + +[source, xml] +---- + + + + + +---- +endif::backend-pdf[] + +==== JobInstance + +A `JobInstance` refers to the concept of a logical job run. Consider a batch job that +should be run once at the end of the day, such as the 'EndOfDay' `Job` from the preceding +diagram. There is one 'EndOfDay' job, but each individual run of the `Job` must be +tracked separately. In the case of this job, there is one logical `JobInstance` per day. +For example, there is a January 1st run, a January 2nd run, and so on. If the January 1st +run fails the first time and is run again the next day, it is still the January 1st run. +(Usually, this corresponds with the data it is processing as well, meaning the January +1st run processes data for January 1st). Therefore, each `JobInstance` can have multiple +executions (`JobExecution` is discussed in more detail later in this chapter), and only +one `JobInstance` corresponding to a particular `Job` and identifying `JobParameters` can +run at a given time. + +The definition of a `JobInstance` has absolutely no bearing on the data the to be loaded. +It is entirely up to the `ItemReader` implementation to determine how data is loaded. For +example, in the EndOfDay scenario, there may be a column on the data that indicates the +'effective date' or 'schedule date' to which the data belongs. So, the January 1st run +would load only data from the 1st, and the January 2nd run would use only data from the +2nd. Because this determination is likely to be a business decision, it is left up to the +`ItemReader` to decide. However, using the same `JobInstance` determines whether or not +the 'state' (that is, the `ExecutionContext`, which is discussed later in this chapter) +from previous executions is used. Using a new `JobInstance` means 'start from the +beginning', and using an existing instance generally means 'start from where you left +off'. + +==== JobParameters + +Having discussed `JobInstance` and how it differs from Job, the natural question to ask +is: "How is one `JobInstance` distinguished from another?" The answer is: +`JobParameters`. A `JobParameters` object holds a set of parameters used to start a batch +job. They can be used for identification or even as reference data during the run, as +shown in the following image: + +.Job Parameters +image::{batch-asciidoc}images/job-stereotypes-parameters.png[Job Parameters, scaledwidth="60%"] + +In the preceding example, where there are two instances, one for January 1st, and another +for January 2nd, there is really only one `Job`, but it has two `JobParameter` objects: +one that was started with a job parameter of 01-01-2017 and another that was started with +a parameter of 01-02-2017. Thus, the contract can be defined as: `JobInstance` = `Job` + + identifying `JobParameters`. This allows a developer to effectively control how a +`JobInstance` is defined, since they control what parameters are passed in. + +NOTE: Not all job parameters are required to contribute to the identification of a +`JobInstance`. By default, they do so. However, the framework also allows the submission +of a `Job` with parameters that do not contribute to the identity of a `JobInstance`. + +==== JobExecution + +A `JobExecution` refers to the technical concept of a single attempt to run a Job. An +execution may end in failure or success, but the `JobInstance` corresponding to a given +execution is not considered to be complete unless the execution completes successfully. +Using the EndOfDay `Job` described previously as an example, consider a `JobInstance` for +01-01-2017 that failed the first time it was run. If it is run again with the same +identifying job parameters as the first run (01-01-2017), a new `JobExecution` is +created. However, there is still only one `JobInstance`. + +A `Job` defines what a job is and how it is to be executed, and a `JobInstance` is a +purely organizational object to group executions together, primarily to enable correct +restart semantics. A `JobExecution`, however, is the primary storage mechanism for what +actually happened during a run and contains many more properties that must be controlled +and persisted, as shown in the following table: + +.JobExecution Properties + +|=== +|Property |Definition +|Status +|A `BatchStatus` object that indicates the status of the execution. While running, it is +`BatchStatus#STARTED`. If it fails, it is `BatchStatus#FAILED`. If it finishes +successfully, it is `BatchStatus#COMPLETED` + +|startTime +|A `java.util.Date` representing the current system time when the execution was started. +This field is empty if the job has yet to start. + +|endTime +|A `java.util.Date` representing the current system time when the execution finished, +regardless of whether or not it was successful. The field is empty if the job has yet to +finish. + +|exitStatus +|The `ExitStatus`, indicating the result of the run. It is most important, because it +contains an exit code that is returned to the caller. See chapter 5 for more details. The +field is empty if the job has yet to finish. + +|createTime +|A `java.util.Date` representing the current system time when the `JobExecution` was +first persisted. The job may not have been started yet (and thus has no start time), but +it always has a createTime, which is required by the framework for managing job level +`ExecutionContexts`. + +|lastUpdated +|A `java.util.Date` representing the last time a `JobExecution` was persisted. This field +is empty if the job has yet to start. + +|executionContext +|The "property bag" containing any user data that needs to be persisted between +executions. + +|failureExceptions +|The list of exceptions encountered during the execution of a `Job`. These can be useful +if more than one exception is encountered during the failure of a `Job`. +|=== + +These properties are important because they are persisted and can be used to completely +determine the status of an execution. For example, if the EndOfDay job for 01-01 is +executed at 9:00 PM and fails at 9:30, the following entries are made in the batch +metadata tables: + +.BATCH_JOB_INSTANCE + +|=== +|JOB_INST_ID |JOB_NAME +|1 +|EndOfDayJob +|=== + +.BATCH_JOB_EXECUTION_PARAMS +|=== +|JOB_EXECUTION_ID|TYPE_CD|KEY_NAME|DATE_VAL|IDENTIFYING +|1 +|DATE +|schedule.Date +|2017-01-01 +|TRUE +|=== + +.BATCH_JOB_EXECUTION +|=== +|JOB_EXEC_ID|JOB_INST_ID|START_TIME|END_TIME|STATUS +|1 +|1 +|2017-01-01 21:00 +|2017-01-01 21:30 +|FAILED +|=== + +NOTE: Column names may have been abbreviated or removed for the sake of clarity and +formatting. + +Now that the job has failed, assume that it took the entire night for the problem to be +determined, so that the 'batch window' is now closed. Further assuming that the window +starts at 9:00 PM, the job is kicked off again for 01-01, starting where it left off and +completing successfully at 9:30. Because it is now the next day, the 01-02 job must be +run as well, and it is kicked off just afterwards at 9:31 and completes in its normal one +hour time at 10:30. There is no requirement that one `JobInstance` be kicked off after +another, unless there is potential for the two jobs to attempt to access the same data, +causing issues with locking at the database level. It is entirely up to the scheduler to +determine when a `Job` should be run. Since they are separate `JobInstances`, Spring +Batch makes no attempt to stop them from being run concurrently. (Attempting to run the +same `JobInstance` while another is already running results in a +`JobExecutionAlreadyRunningException` being thrown). There should now be an extra entry +in both the `JobInstance` and `JobParameters` tables and two extra entries in the +`JobExecution` table, as shown in the following tables: + +.BATCH_JOB_INSTANCE +|=== +|JOB_INST_ID |JOB_NAME +|1 +|EndOfDayJob + +|2 +|EndOfDayJob +|=== + +.BATCH_JOB_EXECUTION_PARAMS +|=== +|JOB_EXECUTION_ID|TYPE_CD|KEY_NAME|DATE_VAL|IDENTIFYING +|1 +|DATE +|schedule.Date +|2017-01-01 00:00:00 +|TRUE + +|2 +|DATE +|schedule.Date +|2017-01-01 00:00:00 +|TRUE + +|3 +|DATE +|schedule.Date +|2017-01-02 00:00:00 +|TRUE +|=== + +.BATCH_JOB_EXECUTION +|=== +|JOB_EXEC_ID|JOB_INST_ID|START_TIME|END_TIME|STATUS +|1 +|1 +|2017-01-01 21:00 +|2017-01-01 21:30 +|FAILED + +|2 +|1 +|2017-01-02 21:00 +|2017-01-02 21:30 +|COMPLETED + +|3 +|2 +|2017-01-02 21:31 +|2017-01-02 22:29 +|COMPLETED +|=== + +NOTE: Column names may have been abbreviated or removed for the sake of clarity and +formatting. + +=== Step + +A `Step` is a domain object that encapsulates an independent, sequential phase of a batch +job. Therefore, every Job is composed entirely of one or more steps. A `Step` contains +all of the information necessary to define and control the actual batch processing. This +is a necessarily vague description because the contents of any given `Step` are at the +discretion of the developer writing a `Job`. A `Step` can be as simple or complex as the +developer desires. A simple `Step` might load data from a file into the database, +requiring little or no code (depending upon the implementations used). A more complex +`Step` may have complicated business rules that are applied as part of the processing. As +with a `Job`, a `Step` has an individual `StepExecution` that correlates with a unique +`JobExecution`, as shown in the following image: + +.Job Hierarchy With Steps +image::{batch-asciidoc}images/jobHeirarchyWithSteps.png[Figure 2.1: Job Hierarchy With Steps, scaledwidth="60%"] + +==== StepExecution + +A `StepExecution` represents a single attempt to execute a `Step`. A new `StepExecution` +is created each time a `Step` is run, similar to `JobExecution`. However, if a step fails +to execute because the step before it fails, no execution is persisted for it. A +`StepExecution` is created only when its `Step` is actually started. + +`Step` executions are represented by objects of the `StepExecution` class. Each execution +contains a reference to its corresponding step and `JobExecution` and transaction related +data, such as commit and rollback counts and start and end times. Additionally, each step +execution contains an `ExecutionContext`, which contains any data a developer needs to +have persisted across batch runs, such as statistics or state information needed to +restart. The following table lists the properties for `StepExecution`: + +.StepExecution Properties +|=== +|Property|Definition +|Status +|A `BatchStatus` object that indicates the status of the execution. While running, the +status is `BatchStatus.STARTED`. If it fails, the status is `BatchStatus.FAILED`. If it +finishes successfully, the status is `BatchStatus.COMPLETED`. + +|startTime +|A `java.util.Date` representing the current system time when the execution was started. +This field is empty if the step has yet to start. + +|endTime + +|A `java.util.Date` representing the current system time when the execution finished, +regardless of whether or not it was successful. This field is empty if the step has yet to +exit. + +|exitStatus +|The `ExitStatus` indicating the result of the execution. It is most important, because +it contains an exit code that is returned to the caller. See chapter 5 for more details. +This field is empty if the job has yet to exit. + +|executionContext +|The "property bag" containing any user data that needs to be persisted between +executions. + +|readCount +|The number of items that have been successfully read. + +|writeCount +|The number of items that have been successfully written. + +|commitCount +|The number of transactions that have been committed for this execution. + +|rollbackCount +|The number of times the business transaction controlled by the `Step` has been rolled +back. + +|readSkipCount +|The number of times `read` has failed, resulting in a skipped item. + +|processSkipCount +|The number of times `process` has failed, resulting in a skipped item. + +|filterCount +|The number of items that have been 'filtered' by the `ItemProcessor`. + +|writeSkipCount +|The number of times `write` has failed, resulting in a skipped item. +|=== + +=== ExecutionContext + +An `ExecutionContext` represents a collection of key/value pairs that are persisted and +controlled by the framework in order to allow developers a place to store persistent +state that is scoped to a `StepExecution` object or a `JobExecution` object. For those +familiar with Quartz, it is very similar to JobDataMap. The best usage example is to +facilitate restart. Using flat file input as an example, while processing individual +lines, the framework periodically persists the `ExecutionContext` at commit points. Doing +so allows the `ItemReader` to store its state in case a fatal error occurs during the run +or even if the power goes out. All that is needed is to put the current number of lines +read into the context, as shown in the following example, and the framework will do the +rest: + +[source, java] +---- +executionContext.putLong(getKey(LINES_READ_COUNT), reader.getPosition()); +---- + +Using the EndOfDay example from the `Job` Stereotypes section as an example, assume there +is one step, 'loadData', that loads a file into the database. After the first failed run, +the metadata tables would look like the following example: + +.BATCH_JOB_INSTANCE +|=== +|JOB_INST_ID|JOB_NAME +|1 +|EndOfDayJob +|=== + +.BATCH_JOB_EXECUTION_PARAMS +|=== +|JOB_INST_ID|TYPE_CD|KEY_NAME|DATE_VAL +|1 +|DATE +|schedule.Date +|2017-01-01 +|=== + +.BATCH_JOB_EXECUTION +|=== +|JOB_EXEC_ID|JOB_INST_ID|START_TIME|END_TIME|STATUS +|1 +|1 +|2017-01-01 21:00 +|2017-01-01 21:30 +|FAILED +|=== + +.BATCH_STEP_EXECUTION +|=== +|STEP_EXEC_ID|JOB_EXEC_ID|STEP_NAME|START_TIME|END_TIME|STATUS +|1 +|1 +|loadData +|2017-01-01 21:00 +|2017-01-01 21:30 +|FAILED +|=== + +.BATCH_STEP_EXECUTION_CONTEXT +|=== +|STEP_EXEC_ID|SHORT_CONTEXT +|1 +|{piece.count=40321} +|=== + +In the preceding case, the `Step` ran for 30 minutes and processed 40,321 'pieces', which +would represent lines in a file in this scenario. This value is updated just before each +commit by the framework and can contain multiple rows corresponding to entries within the +`ExecutionContext`. Being notified before a commit requires one of the various +`StepListener` implementations (or an `ItemStream`), which are discussed in more detail +later in this guide. As with the previous example, it is assumed that the `Job` is +restarted the next day. When it is restarted, the values from the `ExecutionContext` of +the last run are reconstituted from the database. When the `ItemReader` is opened, it can +check to see if it has any stored state in the context and initialize itself from there, +as shown in the following example: + +[source, java] +---- +if (executionContext.containsKey(getKey(LINES_READ_COUNT))) { + log.debug("Initializing for restart. Restart data is: " + executionContext); + + long lineCount = executionContext.getLong(getKey(LINES_READ_COUNT)); + + LineReader reader = getReader(); + + Object record = ""; + while (reader.getPosition() < lineCount && record != null) { + record = readLine(); + } +} +---- +In this case, after the above code runs, the current line is 40,322, allowing the `Step` +to start again from where it left off. The `ExecutionContext` can also be used for +statistics that need to be persisted about the run itself. For example, if a flat file +contains orders for processing that exist across multiple lines, it may be necessary to +store how many orders have been processed (which is much different from the number of +lines read), so that an email can be sent at the end of the `Step` with the total number +of orders processed in the body. The framework handles storing this for the developer, in +order to correctly scope it with an individual `JobInstance`. It can be very difficult to +know whether an existing `ExecutionContext` should be used or not. For example, using the +'EndOfDay' example from above, when the 01-01 run starts again for the second time, the +framework recognizes that it is the same `JobInstance` and on an individual `Step` basis, +pulls the `ExecutionContext` out of the database, and hands it (as part of the +`StepExecution`) to the `Step` itself. Conversely, for the 01-02 run, the framework +recognizes that it is a different instance, so an empty context must be handed to the +`Step`. There are many of these types of determinations that the framework makes for the +developer, to ensure the state is given to them at the correct time. It is also important +to note that exactly one `ExecutionContext` exists per `StepExecution` at any given time. +Clients of the `ExecutionContext` should be careful, because this creates a shared +keyspace. As a result, care should be taken when putting values in to ensure no data is +overwritten. However, the `Step` stores absolutely no data in the context, so there is no +way to adversely affect the framework. + +It is also important to note that there is at least one `ExecutionContext` per +`JobExecution` and one for every `StepExecution`. For example, consider the following +code snippet: + +[source, java] +---- +ExecutionContext ecStep = stepExecution.getExecutionContext(); +ExecutionContext ecJob = jobExecution.getExecutionContext(); +//ecStep does not equal ecJob +---- + +As noted in the comment, `ecStep` does not equal `ecJob`. They are two different +`ExecutionContexts`. The one scoped to the `Step` is saved at every commit point in the +`Step`, whereas the one scoped to the Job is saved in between every `Step` execution. + +=== JobRepository + +`JobRepository` is the persistence mechanism for all of the Stereotypes mentioned above. +It provides CRUD operations for `JobLauncher`, `Job`, and `Step` implementations. When a +`Job` is first launched, a `JobExecution` is obtained from the repository, and, during +the course of execution, `StepExecution` and `JobExecution` implementations are persisted +by passing them to the repository. + +[role="xmlContent"] +The batch namespace provides support for configuring a `JobRepository` instance with the +`` tag, as shown in the following example: + +[source, xml, role="xmlContent"] +---- + +---- + +[role="javaContent"] +When using java configuration, `@EnableBatchProcessing` annotation provides a +`JobRepository` as one of the components automatically configured out of the box. + +=== JobLauncher + +`JobLauncher` represents a simple interface for launching a `Job` with a given set of +`JobParameters`, as shown in the following example: + +[source, java] +---- +public interface JobLauncher { + +public JobExecution run(Job job, JobParameters jobParameters) + throws JobExecutionAlreadyRunningException, JobRestartException, + JobInstanceAlreadyCompleteException, JobParametersInvalidException; +} +---- +It is expected that implementations obtain a valid `JobExecution` from the +`JobRepository` and execute the `Job`. + +=== Item Reader + +`ItemReader` is an abstraction that represents the retrieval of input for a `Step`, one +item at a time. When the `ItemReader` has exhausted the items it can provide, it +indicates this by returning `null`. More details about the `ItemReader` interface and its +various implementations can be found in +<>. + +=== Item Writer + +`ItemWriter` is an abstraction that represents the output of a `Step`, one batch or chunk +of items at a time. Generally, an `ItemWriter` has no knowledge of the input it should +receive next and knows only the item that was passed in its current invocation. More +details about the `ItemWriter` interface and its various implementations can be found in +<>. + +=== Item Processor + +`ItemProcessor` is an abstraction that represents the business processing of an item. +While the `ItemReader` reads one item, and the `ItemWriter` writes them, the +`ItemProcessor` provides an access point to transform or apply other business processing. +If, while processing the item, it is determined that the item is not valid, returning +`null` indicates that the item should not be written out. More details about the +`ItemProcessor` interface can be found in +<>. + +[role="xmlContent"] +=== Batch Namespace + +Many of the domain concepts listed previously need to be configured in a Spring +`ApplicationContext`. While there are implementations of the interfaces above that can be +used in a standard bean definition, a namespace has been provided for ease of +configuration, as shown in the following example: + +[source, xml, role="xmlContent"] +---- + + + + + + + + + + + +---- + +[role="xmlContent"] +As long as the batch namespace has been declared, any of its elements can be used. More +information on configuring a Job can be found in <>. More information on configuring a `Step` can be found in +<>. diff --git a/spring-batch-docs/asciidoc/footer/index-footer.adoc b/spring-batch-docs/asciidoc/footer/index-footer.adoc new file mode 100644 index 0000000000..c7bed9e102 --- /dev/null +++ b/spring-batch-docs/asciidoc/footer/index-footer.adoc @@ -0,0 +1,11 @@ +''' +Lucas Ward, Dave Syer, Thomas Risberg, Robert Kasanicky, Dan Garrette, Wayne Lund, +Michael Minella, Chris Schaefer, Gunnar Hillert, Glenn Renfro, Jay Bryant, Mahmoud Ben Hassine + +Copyright © 2009 - 2019 Pivotal, Inc. All Rights +Reserved. + +Copies of this document may be made for your own use and for +distribution to others, provided that you do not charge any fee for such +copies and further provided that each copy contains this Copyright +Notice, whether distributed in print or electronically. diff --git a/spring-batch-docs/asciidoc/glossary.adoc b/spring-batch-docs/asciidoc/glossary.adoc new file mode 100644 index 0000000000..98b9fc25e5 --- /dev/null +++ b/spring-batch-docs/asciidoc/glossary.adoc @@ -0,0 +1,103 @@ +[[glossary]] +[appendix] +== Glossary +[glossary] +=== Spring Batch Glossary + +Batch:: + An accumulation of business transactions over time. + +Batch Application Style:: + Term used to designate batch as an application style in its own right, similar to + online, Web, or SOA. It has standard elements of input, validation, transformation of + information to business model, business processing, and output. In addition, it + requires monitoring at a macro level. + +Batch Processing:: + The handling of a batch of many business transactions that have accumulated over a + period of time (such as an hour, a day, a week, a month, or a year). It is the + application of a process or set of processes to many data entities or objects in a + repetitive and predictable fashion with either no manual element or a separate manual + element for error processing. + +Batch Window:: + The time frame within which a batch job must complete. This can be constrained by other + systems coming online, other dependent jobs needing to execute, or other factors + specific to the batch environment. + +Step:: + The main batch task or unit of work. It initializes the business logic and controls the + transaction environment, based on commit interval setting and other factors. + +Tasklet:: + A component created by an application developer to process the business logic for a + Step. + +Batch Job Type:: + Job types describe application of jobs for particular types of processing. Common areas + are interface processing (typically flat files), forms processing (either for online + PDF generation or print formats), and report processing. + +Driving Query:: + A driving query identifies the set of work for a job to do. The job then breaks that + work into individual units of work. For instance, a driving query might be to identify + all financial transactions that have a status of "pending transmission" and send them + to a partner system. The driving query returns a set of record IDs to process. Each + record ID then becomes a unit of work. A driving query may involve a join (if the + criteria for selection falls across two or more tables) or it may work with a single + table. + +Item:: + An item represents the smallest amount of complete data for processing. In the simplest + terms, this might be a line in a file, a row in a database table, or a particular + element in an XML file. + +Logicial Unit of Work (LUW):: + A batch job iterates through a driving query (or other input source, such as a file) to + perform the set of work that the job must accomplish. Each iteration of work performed + is a unit of work. + +Commit Interval:: + A set of LUWs processed within a single transaction. + +Partitioning:: + Splitting a job into multiple threads where each thread is responsible for a subset of + the overall data to be processed. The threads of execution may be within the same JVM + or they may span JVMs in a clustered environment that supports workload balancing. + +Staging Table:: + A table that holds temporary data while it is being processed. + +Restartable:: + A job that can be executed again and assumes the same identity as when run initially. + In other words, it is has the same job instance ID. + +Rerunnable:: + A job that is restartable and manages its own state in terms of the previous run's + record processing. An example of a rerunnable step is one based on a driving query. If + the driving query can be formed so that it limits the processed rows when the job is + restarted, then it is re-runnable. This is managed by the application logic. Often, a + condition is added to the `where` statement to limit the rows returned by the driving + query with logic resembling "and processedFlag!= true". + +Repeat:: + One of the most basic units of batch processing, it defines by repeatability calling a + portion of code until it is finished and while there is no error. Typically, a batch + process would be repeatable as long as there is input. + +Retry:: + Simplifies the execution of operations with retry semantics most frequently associated + with handling transactional output exceptions. Retry is slightly different from repeat, + rather than continually calling a block of code, retry is stateful and continually + calls the same block of code with the same input, until it either succeeds or some type + of retry limit has been exceeded. It is only generally useful when a subsequent + invocation of the operation might succeed because something in the environment has + improved. + +Recover:: + Recover operations handle an exception in such a way that a repeat process is able to + continue. + +Skip:: + Skip is a recovery strategy often used on file input sources as the strategy for + ignoring bad input records that failed validation. diff --git a/spring-batch-docs/asciidoc/header/index-header.adoc b/spring-batch-docs/asciidoc/header/index-header.adoc new file mode 100644 index 0000000000..82a07ca31a --- /dev/null +++ b/spring-batch-docs/asciidoc/header/index-header.adoc @@ -0,0 +1,3 @@ += Spring Batch - Reference Documentation + +:batch-asciidoc: https://docs.spring.io/spring-batch/reference/html/ diff --git a/src/site/docbook/reference/images/1-1-step.png b/spring-batch-docs/asciidoc/images/1-1-step.png similarity index 100% rename from src/site/docbook/reference/images/1-1-step.png rename to spring-batch-docs/asciidoc/images/1-1-step.png diff --git a/src/site/docbook/reference/images/BatchExecutionEnvironments.bmp b/spring-batch-docs/asciidoc/images/BatchExecutionEnvironments.bmp similarity index 100% rename from src/site/docbook/reference/images/BatchExecutionEnvironments.bmp rename to spring-batch-docs/asciidoc/images/BatchExecutionEnvironments.bmp diff --git a/src/site/docbook/reference/images/ExecutionEnvironment.png b/spring-batch-docs/asciidoc/images/ExecutionEnvironment.png similarity index 100% rename from src/site/docbook/reference/images/ExecutionEnvironment.png rename to spring-batch-docs/asciidoc/images/ExecutionEnvironment.png diff --git a/src/site/docbook/reference/images/PipeAndFilter.jpg b/spring-batch-docs/asciidoc/images/PipeAndFilter.jpg similarity index 100% rename from src/site/docbook/reference/images/PipeAndFilter.jpg rename to spring-batch-docs/asciidoc/images/PipeAndFilter.jpg diff --git a/src/site/docbook/reference/images/PipeAndFilter.png b/spring-batch-docs/asciidoc/images/PipeAndFilter.png similarity index 100% rename from src/site/docbook/reference/images/PipeAndFilter.png rename to spring-batch-docs/asciidoc/images/PipeAndFilter.png diff --git a/src/site/docbook/reference/images/RepeatTemplate.png b/spring-batch-docs/asciidoc/images/RepeatTemplate.png similarity index 100% rename from src/site/docbook/reference/images/RepeatTemplate.png rename to spring-batch-docs/asciidoc/images/RepeatTemplate.png diff --git a/src/site/docbook/reference/images/RuntimeDependencies.png b/spring-batch-docs/asciidoc/images/RuntimeDependencies.png similarity index 100% rename from src/site/docbook/reference/images/RuntimeDependencies.png rename to spring-batch-docs/asciidoc/images/RuntimeDependencies.png diff --git a/src/site/docbook/reference/images/application-tier.png b/spring-batch-docs/asciidoc/images/application-tier.png similarity index 100% rename from src/site/docbook/reference/images/application-tier.png rename to spring-batch-docs/asciidoc/images/application-tier.png diff --git a/src/site/docbook/reference/images/chunk-oriented-processing.png b/spring-batch-docs/asciidoc/images/chunk-oriented-processing.png similarity index 100% rename from src/site/docbook/reference/images/chunk-oriented-processing.png rename to spring-batch-docs/asciidoc/images/chunk-oriented-processing.png diff --git a/src/site/docbook/reference/images/composite-transformer.png b/spring-batch-docs/asciidoc/images/composite-transformer.png similarity index 100% rename from src/site/docbook/reference/images/composite-transformer.png rename to spring-batch-docs/asciidoc/images/composite-transformer.png diff --git a/src/site/docbook/reference/images/conditional-flow.png b/spring-batch-docs/asciidoc/images/conditional-flow.png similarity index 100% rename from src/site/docbook/reference/images/conditional-flow.png rename to spring-batch-docs/asciidoc/images/conditional-flow.png diff --git a/src/site/docbook/reference/images/cursorExample.png b/spring-batch-docs/asciidoc/images/cursorExample.png similarity index 100% rename from src/site/docbook/reference/images/cursorExample.png rename to spring-batch-docs/asciidoc/images/cursorExample.png diff --git a/src/site/docbook/reference/images/domain-classdiagram.jpg b/spring-batch-docs/asciidoc/images/domain-classdiagram.jpg similarity index 100% rename from src/site/docbook/reference/images/domain-classdiagram.jpg rename to spring-batch-docs/asciidoc/images/domain-classdiagram.jpg diff --git a/src/site/docbook/reference/images/drivingQueryExample.png b/spring-batch-docs/asciidoc/images/drivingQueryExample.png similarity index 100% rename from src/site/docbook/reference/images/drivingQueryExample.png rename to spring-batch-docs/asciidoc/images/drivingQueryExample.png diff --git a/src/site/docbook/reference/images/drivingQueryJob.png b/spring-batch-docs/asciidoc/images/drivingQueryJob.png similarity index 100% rename from src/site/docbook/reference/images/drivingQueryJob.png rename to spring-batch-docs/asciidoc/images/drivingQueryJob.png diff --git a/src/site/docbook/reference/images/errorOnFlush.png b/spring-batch-docs/asciidoc/images/errorOnFlush.png similarity index 100% rename from src/site/docbook/reference/images/errorOnFlush.png rename to spring-batch-docs/asciidoc/images/errorOnFlush.png diff --git a/src/site/docbook/reference/images/errorOnWrite.png b/spring-batch-docs/asciidoc/images/errorOnWrite.png similarity index 100% rename from src/site/docbook/reference/images/errorOnWrite.png rename to spring-batch-docs/asciidoc/images/errorOnWrite.png diff --git a/src/site/docbook/reference/images/execution-environment-config.jpg b/spring-batch-docs/asciidoc/images/execution-environment-config.jpg similarity index 100% rename from src/site/docbook/reference/images/execution-environment-config.jpg rename to spring-batch-docs/asciidoc/images/execution-environment-config.jpg diff --git a/src/site/docbook/reference/images/flat-file-input-source-design.gif b/spring-batch-docs/asciidoc/images/flat-file-input-source-design.gif similarity index 100% rename from src/site/docbook/reference/images/flat-file-input-source-design.gif rename to spring-batch-docs/asciidoc/images/flat-file-input-source-design.gif diff --git a/src/site/docbook/reference/images/flat-file-input-source-design.jpg b/spring-batch-docs/asciidoc/images/flat-file-input-source-design.jpg similarity index 100% rename from src/site/docbook/reference/images/flat-file-input-source-design.jpg rename to spring-batch-docs/asciidoc/images/flat-file-input-source-design.jpg diff --git a/src/site/docbook/reference/images/flatfile-input-source-diagram.jpg b/spring-batch-docs/asciidoc/images/flatfile-input-source-diagram.jpg similarity index 100% rename from src/site/docbook/reference/images/flatfile-input-source-diagram.jpg rename to spring-batch-docs/asciidoc/images/flatfile-input-source-diagram.jpg diff --git a/spring-batch-docs/asciidoc/images/handling-informational-messages.png b/spring-batch-docs/asciidoc/images/handling-informational-messages.png new file mode 100644 index 0000000000..8bb7697291 Binary files /dev/null and b/spring-batch-docs/asciidoc/images/handling-informational-messages.png differ diff --git a/src/site/docbook/reference/images/io-design.jpg b/spring-batch-docs/asciidoc/images/io-design.jpg similarity index 100% rename from src/site/docbook/reference/images/io-design.jpg rename to spring-batch-docs/asciidoc/images/io-design.jpg diff --git a/src/site/docbook/reference/images/item-oriented-processing.png b/spring-batch-docs/asciidoc/images/item-oriented-processing.png similarity index 100% rename from src/site/docbook/reference/images/item-oriented-processing.png rename to spring-batch-docs/asciidoc/images/item-oriented-processing.png diff --git a/src/site/docbook/reference/images/item-reader-design.jpg b/spring-batch-docs/asciidoc/images/item-reader-design.jpg similarity index 100% rename from src/site/docbook/reference/images/item-reader-design.jpg rename to spring-batch-docs/asciidoc/images/item-reader-design.jpg diff --git a/src/site/docbook/reference/images/item-stream-adapter.jpg b/spring-batch-docs/asciidoc/images/item-stream-adapter.jpg similarity index 100% rename from src/site/docbook/reference/images/item-stream-adapter.jpg rename to spring-batch-docs/asciidoc/images/item-stream-adapter.jpg diff --git a/src/site/docbook/reference/images/jmx-job.jpg b/spring-batch-docs/asciidoc/images/jmx-job.jpg similarity index 100% rename from src/site/docbook/reference/images/jmx-job.jpg rename to spring-batch-docs/asciidoc/images/jmx-job.jpg diff --git a/src/site/docbook/reference/images/jmx.jpg b/spring-batch-docs/asciidoc/images/jmx.jpg similarity index 100% rename from src/site/docbook/reference/images/jmx.jpg rename to spring-batch-docs/asciidoc/images/jmx.jpg diff --git a/src/site/docbook/reference/images/job-heirarchy.png b/spring-batch-docs/asciidoc/images/job-heirarchy.png similarity index 100% rename from src/site/docbook/reference/images/job-heirarchy.png rename to spring-batch-docs/asciidoc/images/job-heirarchy.png diff --git a/src/site/docbook/reference/images/job-launcher-sequence-async.png b/spring-batch-docs/asciidoc/images/job-launcher-sequence-async.png similarity index 100% rename from src/site/docbook/reference/images/job-launcher-sequence-async.png rename to spring-batch-docs/asciidoc/images/job-launcher-sequence-async.png diff --git a/src/site/docbook/reference/images/job-launcher-sequence-sync.png b/spring-batch-docs/asciidoc/images/job-launcher-sequence-sync.png similarity index 100% rename from src/site/docbook/reference/images/job-launcher-sequence-sync.png rename to spring-batch-docs/asciidoc/images/job-launcher-sequence-sync.png diff --git a/src/site/docbook/reference/images/job-repository-advanced.png b/spring-batch-docs/asciidoc/images/job-repository-advanced.png similarity index 100% rename from src/site/docbook/reference/images/job-repository-advanced.png rename to spring-batch-docs/asciidoc/images/job-repository-advanced.png diff --git a/src/site/docbook/reference/images/job-repository.png b/spring-batch-docs/asciidoc/images/job-repository.png similarity index 100% rename from src/site/docbook/reference/images/job-repository.png rename to spring-batch-docs/asciidoc/images/job-repository.png diff --git a/src/site/docbook/reference/images/job-stereotypes-parameters.png b/spring-batch-docs/asciidoc/images/job-stereotypes-parameters.png similarity index 100% rename from src/site/docbook/reference/images/job-stereotypes-parameters.png rename to spring-batch-docs/asciidoc/images/job-stereotypes-parameters.png diff --git a/src/site/docbook/reference/images/jobHeirarchyWithSteps.png b/spring-batch-docs/asciidoc/images/jobHeirarchyWithSteps.png similarity index 100% rename from src/site/docbook/reference/images/jobHeirarchyWithSteps.png rename to spring-batch-docs/asciidoc/images/jobHeirarchyWithSteps.png diff --git a/src/site/docbook/reference/images/jobTier.png b/spring-batch-docs/asciidoc/images/jobTier.png similarity index 100% rename from src/site/docbook/reference/images/jobTier.png rename to spring-batch-docs/asciidoc/images/jobTier.png diff --git a/spring-batch-docs/asciidoc/images/launch-batch-job-svg.svg b/spring-batch-docs/asciidoc/images/launch-batch-job-svg.svg new file mode 100644 index 0000000000..77488a4a41 --- /dev/null +++ b/spring-batch-docs/asciidoc/images/launch-batch-job-svg.svg @@ -0,0 +1,3 @@ + + +2012-10-16 03:37ZCanvas 1Layer 1DCFTPInbound Channel AdapterJobLauncherTransformerFileJobLaunchRequest diff --git a/spring-batch-docs/asciidoc/images/launch-batch-job.png b/spring-batch-docs/asciidoc/images/launch-batch-job.png new file mode 100644 index 0000000000..006eb60911 Binary files /dev/null and b/spring-batch-docs/asciidoc/images/launch-batch-job.png differ diff --git a/src/site/docbook/reference/images/launch-from-request.png b/spring-batch-docs/asciidoc/images/launch-from-request.png similarity index 100% rename from src/site/docbook/reference/images/launch-from-request.png rename to spring-batch-docs/asciidoc/images/launch-from-request.png diff --git a/src/site/docbook/reference/images/meta-data-erd.png b/spring-batch-docs/asciidoc/images/meta-data-erd.png similarity index 100% rename from src/site/docbook/reference/images/meta-data-erd.png rename to spring-batch-docs/asciidoc/images/meta-data-erd.png diff --git a/src/site/docbook/reference/images/nfljob-config.jpg b/spring-batch-docs/asciidoc/images/nfljob-config.jpg similarity index 100% rename from src/site/docbook/reference/images/nfljob-config.jpg rename to spring-batch-docs/asciidoc/images/nfljob-config.jpg diff --git a/src/site/docbook/reference/images/nfljob.jpg b/spring-batch-docs/asciidoc/images/nfljob.jpg similarity index 100% rename from src/site/docbook/reference/images/nfljob.jpg rename to spring-batch-docs/asciidoc/images/nfljob.jpg diff --git a/src/site/docbook/reference/images/oxm-fragments.png b/spring-batch-docs/asciidoc/images/oxm-fragments.png similarity index 100% rename from src/site/docbook/reference/images/oxm-fragments.png rename to spring-batch-docs/asciidoc/images/oxm-fragments.png diff --git a/src/site/resources/images/partitioned.png b/spring-batch-docs/asciidoc/images/partitioned.png similarity index 100% rename from src/site/resources/images/partitioned.png rename to spring-batch-docs/asciidoc/images/partitioned.png diff --git a/spring-batch-docs/asciidoc/images/partitioning-overview.png b/spring-batch-docs/asciidoc/images/partitioning-overview.png new file mode 100644 index 0000000000..9bb4b62857 Binary files /dev/null and b/spring-batch-docs/asciidoc/images/partitioning-overview.png differ diff --git a/src/site/docbook/reference/images/partitioning-spi.png b/spring-batch-docs/asciidoc/images/partitioning-spi.png similarity index 100% rename from src/site/docbook/reference/images/partitioning-spi.png rename to spring-batch-docs/asciidoc/images/partitioning-spi.png diff --git a/spring-batch-docs/asciidoc/images/remote-chunking-config.png b/spring-batch-docs/asciidoc/images/remote-chunking-config.png new file mode 100644 index 0000000000..53d3ac6a5d Binary files /dev/null and b/spring-batch-docs/asciidoc/images/remote-chunking-config.png differ diff --git a/spring-batch-docs/asciidoc/images/remote-chunking-sbi.png b/spring-batch-docs/asciidoc/images/remote-chunking-sbi.png new file mode 100644 index 0000000000..b1510778e5 Binary files /dev/null and b/spring-batch-docs/asciidoc/images/remote-chunking-sbi.png differ diff --git a/spring-batch-docs/asciidoc/images/remote-chunking.png b/spring-batch-docs/asciidoc/images/remote-chunking.png new file mode 100644 index 0000000000..6eb1279a94 Binary files /dev/null and b/spring-batch-docs/asciidoc/images/remote-chunking.png differ diff --git a/spring-batch-docs/asciidoc/images/remote-partitioning-aggregation-config.png b/spring-batch-docs/asciidoc/images/remote-partitioning-aggregation-config.png new file mode 100644 index 0000000000..c78447d0ea Binary files /dev/null and b/spring-batch-docs/asciidoc/images/remote-partitioning-aggregation-config.png differ diff --git a/spring-batch-docs/asciidoc/images/remote-partitioning-polling-config.png b/spring-batch-docs/asciidoc/images/remote-partitioning-polling-config.png new file mode 100644 index 0000000000..cc48e824f1 Binary files /dev/null and b/spring-batch-docs/asciidoc/images/remote-partitioning-polling-config.png differ diff --git a/spring-batch-docs/asciidoc/images/remote-partitioning.png b/spring-batch-docs/asciidoc/images/remote-partitioning.png new file mode 100644 index 0000000000..f675b15255 Binary files /dev/null and b/spring-batch-docs/asciidoc/images/remote-partitioning.png differ diff --git a/src/site/docbook/reference/images/repository-classdiagram.jpg b/spring-batch-docs/asciidoc/images/repository-classdiagram.jpg similarity index 100% rename from src/site/docbook/reference/images/repository-classdiagram.jpg rename to spring-batch-docs/asciidoc/images/repository-classdiagram.jpg diff --git a/src/site/docbook/reference/images/run-tier.png b/spring-batch-docs/asciidoc/images/run-tier.png similarity index 100% rename from src/site/docbook/reference/images/run-tier.png rename to spring-batch-docs/asciidoc/images/run-tier.png diff --git a/src/site/docbook/reference/images/s1-job-configuration.jpg b/spring-batch-docs/asciidoc/images/s1-job-configuration.jpg similarity index 100% rename from src/site/docbook/reference/images/s1-job-configuration.jpg rename to spring-batch-docs/asciidoc/images/s1-job-configuration.jpg diff --git a/src/site/docbook/reference/images/sequential-flow.png b/spring-batch-docs/asciidoc/images/sequential-flow.png similarity index 100% rename from src/site/docbook/reference/images/sequential-flow.png rename to spring-batch-docs/asciidoc/images/sequential-flow.png diff --git a/src/site/docbook/reference/images/simple-batch-execution-env.jpg b/spring-batch-docs/asciidoc/images/simple-batch-execution-env.jpg similarity index 100% rename from src/site/docbook/reference/images/simple-batch-execution-env.jpg rename to spring-batch-docs/asciidoc/images/simple-batch-execution-env.jpg diff --git a/src/site/docbook/reference/images/simple-tasklet-job-configuration.jpg b/spring-batch-docs/asciidoc/images/simple-tasklet-job-configuration.jpg similarity index 100% rename from src/site/docbook/reference/images/simple-tasklet-job-configuration.jpg rename to spring-batch-docs/asciidoc/images/simple-tasklet-job-configuration.jpg diff --git a/src/site/docbook/reference/images/simplified-chunk-oriented-processing.png b/spring-batch-docs/asciidoc/images/simplified-chunk-oriented-processing.png similarity index 100% rename from src/site/docbook/reference/images/simplified-chunk-oriented-processing.png rename to spring-batch-docs/asciidoc/images/simplified-chunk-oriented-processing.png diff --git a/src/site/docbook/reference/images/spring-batch-football-graph.jpg b/spring-batch-docs/asciidoc/images/spring-batch-football-graph.jpg similarity index 100% rename from src/site/docbook/reference/images/spring-batch-football-graph.jpg rename to spring-batch-docs/asciidoc/images/spring-batch-football-graph.jpg diff --git a/src/site/docbook/reference/images/spring-batch-layers.png b/spring-batch-docs/asciidoc/images/spring-batch-layers.png similarity index 100% rename from src/site/docbook/reference/images/spring-batch-layers.png rename to spring-batch-docs/asciidoc/images/spring-batch-layers.png diff --git a/src/site/docbook/reference/images/spring-batch-reference-model.png b/spring-batch-docs/asciidoc/images/spring-batch-reference-model.png similarity index 100% rename from src/site/docbook/reference/images/spring-batch-reference-model.png rename to spring-batch-docs/asciidoc/images/spring-batch-reference-model.png diff --git a/src/site/docbook/reference/images/step.png b/spring-batch-docs/asciidoc/images/step.png similarity index 100% rename from src/site/docbook/reference/images/step.png rename to spring-batch-docs/asciidoc/images/step.png diff --git a/src/site/docbook/reference/images/xmlinput.png b/spring-batch-docs/asciidoc/images/xmlinput.png similarity index 100% rename from src/site/docbook/reference/images/xmlinput.png rename to spring-batch-docs/asciidoc/images/xmlinput.png diff --git a/spring-batch-docs/asciidoc/index-single.adoc b/spring-batch-docs/asciidoc/index-single.adoc new file mode 100644 index 0000000000..081f1fe012 --- /dev/null +++ b/spring-batch-docs/asciidoc/index-single.adoc @@ -0,0 +1,45 @@ +:doctype: book +:toc: left +:toclevels: 4 +:sectnums: +:onlyonetoggle: true + +include::header/index-header.adoc[] + +include::toggle.adoc[] + +include::spring-batch-intro.adoc[] + +include::whatsnew.adoc[] + +include::domain.adoc[] + +include::job.adoc[] + +include::step.adoc[] + +include::readersAndWriters.adoc[] + +include::scalability.adoc[] + +include::repeat.adoc[] + +include::retry.adoc[] + +include::testing.adoc[] + +include::common-patterns.adoc[] + +include::jsr-352.adoc[] + +include::spring-batch-integration.adoc[] + +include::monitoring-and-metrics.adoc[] + +include::appendix.adoc[] + +include::schema-appendix.adoc[] + +include::transaction-appendix.adoc[] + +include::glossary.adoc[] diff --git a/spring-batch-docs/asciidoc/index.adoc b/spring-batch-docs/asciidoc/index.adoc new file mode 100644 index 0000000000..3fcde2b23b --- /dev/null +++ b/spring-batch-docs/asciidoc/index.adoc @@ -0,0 +1,48 @@ +include::header/index-header.adoc[] + +// ====================================================================================== + +Welcome to the Spring Batch reference documentation! This documentation is also available +as single link:index-single.html[html] and link:../pdf/spring-batch-reference.pdf[pdf] documents. + +The reference documentation is divided into several sections: + +[horizontal] +<> :: Background, usage + scenarios and general guidelines. +<> :: New features introduced in version 4.2. +<> :: Core concepts and abstractions +of the Batch domain language. +<> :: Job configuration, execution and +administration. +<> :: Step configuration, different types of steps, +controlling step flow. +<> :: Item readers +and writers interfaces and how to use them. +<> :: Multi-threaded steps, +parallel steps, remote chunking and partitioning. +<> :: Completion policies and exception handling of repetitive actions. +<> :: Retry and backoff policies of retryable operations. +<> :: Job and Step testing facilities and APIs. +<> :: Common batch processing patterns +and guidelines. +<> :: JSR-352 support, similarities and differences +with Spring Batch. +<> :: Integration +between Spring Batch and Spring Integration projects. +<> :: Batch jobs +monitoring and metrics + +The following appendices are available: + +[horizontal] +<> :: List of +all item readers and writers provided out-of-the box. +<> :: Core tables used by the Batch +domain model. +<> :: Transaction +boundaries, propagation and isolation levels used in Spring Batch. +<> :: Glossary of common terms, concepts and vocabulary of +the Batch domain. + +include::footer/index-footer.adoc[] diff --git a/spring-batch-docs/asciidoc/job.adoc b/spring-batch-docs/asciidoc/job.adoc new file mode 100644 index 0000000000..6f7c7260b9 --- /dev/null +++ b/spring-batch-docs/asciidoc/job.adoc @@ -0,0 +1,1780 @@ +:batch-asciidoc: ./ +:toc: left +:toclevels: 4 + +[[configureJob]] + +== Configuring and Running a Job + +ifndef::onlyonetoggle[] +include::toggle.adoc[] +endif::onlyonetoggle[] + +In the <> , the overall + architecture design was discussed, using the following diagram as a + guide: + +.Batch Stereotypes +image::{batch-asciidoc}images/spring-batch-reference-model.png[Figure 2.1: Batch Stereotypes, scaledwidth="60%"] + +While the `Job` object may seem like a simple +container for steps, there are many configuration options of which a +developer must be aware. Furthermore, there are many considerations for +how a `Job` will be run and how its meta-data will be +stored during that run. This chapter will explain the various configuration +options and runtime concerns of a `Job`. + +[[configuringAJob]] + +=== Configuring a Job + +ifdef::backend-html5[] +[role="javaContent"] +There are multiple implementations of the <> interface, however +builders abstract away the difference in configuration. + +[source, java, role="javaContent"] +---- +@Bean +public Job footballJob() { + return this.jobBuilderFactory.get("footballJob") + .start(playerLoad()) + .next(gameLoad()) + .next(playerSummarization()) + .end() + .build(); +} +---- + +[role="javaContent"] +A `Job` (and typically any `Step` within it) requires a `JobRepository`. The +configuration of the `JobRepository` is handled via the <>. + +[role="javaContent"] +The above example illustrates a `Job` that consists of three `Step` instances. The job related +builders can also contain other elements that help with parallelisation (`Split`), +declarative flow control (`Decision`) and externalization of flow definitions (`Flow`). + +[role="xmlContent"] +There are multiple implementations of the <> interface, however, the namespace +abstracts away the differences in configuration. It has only three +required dependencies: a name, a `JobRepository` , and +a list of `Step` instances. + +[source, xml, role="xmlContent"] +---- + + + + + +---- + +[role="xmlContent"] +The examples here use a parent bean definition to create the steps; +see the section on <> +for more options declaring specific step details inline. The XML namespace +defaults to referencing a repository with an id of 'jobRepository', which +is a sensible default. However, this can be overridden explicitly: + + +[source, xml, role="xmlContent"] +---- + + + + + +---- + +[role="xmlContent"] +In addition to steps a job configuration can contain other elements + that help with parallelisation (``), + declarative flow control (``) and + externalization of flow definitions + (``). +endif::backend-html5[] + +ifdef::backend-pdf[] +There are multiple implementations of the <> interface, however +this is abstracted behind either the builders provided for java configuration or the XML +namespace when using XML based configuration. + +.Java Configuration +[source, java] +---- +@Bean +public Job footballJob() { + return this.jobBuilderFactory.get("footballJob") + .start(playerLoad()) + .next(gameLoad()) + .next(playerSummarization()) + .end() + .build(); +} +---- + +.XML Configuration +[source, xml] +---- + + + + + +---- + +The examples here use a parent bean definition to create the steps; +see the section on <> +for more options declaring specific step details inline. The XML namespace +defaults to referencing a repository with an id of 'jobRepository', which +is a sensible default. However, this can be overridden explicitly: + + +[source, xml] +---- + + + + + +---- + +In addition to steps a job configuration can contain other elements + that help with parallelisation (``), + declarative flow control (``) and + externalization of flow definitions + (``). + + +endif::backend-pdf[] + +[[restartability]] + +==== Restartability + +One key issue when executing a batch job concerns the behavior of +a `Job` when it is restarted. The launching of a +`Job` is considered to be a 'restart' if a +`JobExecution` already exists for the particular +`JobInstance`. Ideally, all jobs should be able to +start up where they left off, but there are scenarios where this is not +possible. __It is entirely up to the developer to ensure that a new `JobInstance` is created in this scenario__. However, Spring Batch does provide some help. If a +`Job` should never be restarted, but should always +be run as part of a new `JobInstance`, then the +restartable property may be set to 'false': + + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + ... + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public Job footballJob() { + return this.jobBuilderFactory.get("footballJob") + .preventRestart() + ... + .build(); +} +---- + +To phrase it another way, setting restartable to false means "this +`Job` does not support being started again". Restarting a `Job` that is not +restartable will cause a `JobRestartException` to +be thrown: + +[source, java] +---- +Job job = new SimpleJob(); +job.setRestartable(false); + +JobParameters jobParameters = new JobParameters(); + +JobExecution firstExecution = jobRepository.createJobExecution(job, jobParameters); +jobRepository.saveOrUpdate(firstExecution); + +try { + jobRepository.createJobExecution(job, jobParameters); + fail(); +} +catch (JobRestartException e) { + // expected +} +---- + +This snippet of JUnit code shows how attempting to create a +`JobExecution` the first time for a non restartable +job will cause no issues. However, the second +attempt will throw a `JobRestartException`. + +[[interceptingJobExecution]] + +==== Intercepting Job Execution + +During the course of the execution of a +Job, it may be useful to be notified of various +events in its lifecycle so that custom code may be executed. The +`SimpleJob` allows for this by calling a +`JobListener` at the appropriate time: + +[source, java] +---- +public interface JobExecutionListener { + + void beforeJob(JobExecution jobExecution); + + void afterJob(JobExecution jobExecution); + +} +---- + +`JobListeners` can be added to a +`SimpleJob` via the listeners element on the +job: + + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + + + + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public Job footballJob() { + return this.jobBuilderFactory.get("footballJob") + .listener(sampleListener()) + ... + .build(); +} +---- + +It should be noted that `afterJob` will be + called regardless of the success or failure of the + Job. If success or failure needs to be determined + it can be obtained from the `JobExecution`: + + +[source, java] +---- +public void afterJob(JobExecution jobExecution){ + if( jobExecution.getStatus() == BatchStatus.COMPLETED ){ + //job success + } + else if(jobExecution.getStatus() == BatchStatus.FAILED){ + //job failure + } +} +---- + +The annotations corresponding to this interface are: + +* `@BeforeJob` +* `@AfterJob` + +[[inheritingFromAParentJob]] + + +[role="xmlContent"] +==== Inheriting from a Parent Job + +[role="xmlContent"] +If a group of Jobs share similar, but not + identical, configurations, then it may be helpful to define a "parent" + `Job` from which the concrete + Jobs may inherit properties. Similar to class + inheritance in Java, the "child" `Job` will combine + its elements and attributes with the parent's. + +[role="xmlContent"] +In the following example, "baseJob" is an abstract + `Job` definition that defines only a list of + listeners. The `Job` "job1" is a concrete + definition that inherits the list of listeners from "baseJob" and merges + it with its own list of listeners to produce a + `Job` with two listeners and one + `Step`, "step1". + + +[source, xml, role="xmlContent"] +---- + + + + + + + + + + + + + +---- + +[role="xmlContent"] +Please see the section on <> + for more detailed information. + +ifdef::backend-pdf[] +This section only applies to XML based configuration as java configuration provides better +reuse capabilities. + +endif::backend-pdf[] + +==== JobParametersValidator + +A job declared in the XML namespace or using any subclass of + `AbstractJob` can optionally declare a validator for the job parameters at + runtime. This is useful when for instance you need to assert that a job + is started with all its mandatory parameters. There is a + `DefaultJobParametersValidator` that can be used to constrain combinations + of simple mandatory and optional parameters, and for more complex + constraints you can implement the interface yourself. + +ifdef::backend-html5[] +[role="xmlContent"] +The configuration of a validator is supported through the XML namespace through a child + element of the job, e.g: + +[source, xml, role="xmlContent"] +---- + + + + +---- + +[role="xmlContent"] +The validator can be specified as a reference (as above) or as a + nested bean definition in the beans namespace. + +[role="javaContent"] +The configuration of a validator is supported through the java builders, e.g: + +[source, java, role="javaContent"] +---- +@Bean +public Job job1() { + return this.jobBuilderFactory.get("job1") + .validator(parametersValidator()) + ... + .build(); +} +---- + +endif::backend-html5[] + +ifdef::backend-pdf[] +The configuration of a validator is supported through the java builders, e.g: + +[source, java] +---- +@Bean +public Job job1() { + return this.jobBuilderFactory.get("job1") + .validator(parametersValidator()) + ... + .build(); +} +---- + +XML namespace support is also available for configuration of a `JobParametersValidator`: + +[source, xml] +---- + + + + +---- + +The validator can be specified as a reference (as above) or as a + nested bean definition in the beans namespace. + +endif::backend-pdf[] + +[[javaConfig]] + + +=== Java Config + +Spring 3 brought the ability to configure applications via java in addition to XML. + As of Spring Batch 2.2.0, batch jobs can be configured using the same + java config. There are two components for the java based configuration: + the `@EnableBatchProcessing` annotation and two builders. + +The `@EnableBatchProcessing` works similarly to the other + @Enable* annotations in the Spring family. In this case, + `@EnableBatchProcessing` provides a base configuration for + building batch jobs. Within this base configuration, an instance of + `StepScope` is created in addition to a number of beans made + available to be autowired: + + + +* `JobRepository` - bean name "jobRepository" + + +* `JobLauncher` - bean name "jobLauncher" + + +* `JobRegistry` - bean name "jobRegistry" + + +* `PlatformTransactionManager` - bean name "transactionManager" + + +* `JobBuilderFactory` - bean name "jobBuilders" + + +* `StepBuilderFactory` - bean name "stepBuilders" + +The core interface for this configuration is the `BatchConfigurer`. + The default implementation provides the beans mentioned above and requires a + `DataSource` as a bean within the context to be provided. This data + source will be used by the JobRepository. You can customize any of these beans + by creating a custom implementation of the `BatchConfigurer` interface. + Typically, extending the `DefaultBatchConfigurer` (which is provided if a + `BatchConfigurer` is not found) and overriding the required getter is sufficient. + However, implementing your own from scratch may be required. The following + example shows how to provide a custom transaction manager: + +[source, java] +---- +@Bean +public BatchConfigurer batchConfigurer() { + return new DefaultBatchConfigurer() { + @Override + public PlatformTransactionManager getTransactionManager() { + return new MyTransactionManager(); + } + }; +} +---- + + +[NOTE] +==== +Only one configuration class needs to have the + `@EnableBatchProcessing` annotation. Once you have a class + annotated with it, you will have all of the above available. + +==== + + +With the base configuration in place, a user can use the provided builder factories + to configure a job. Below is an example of a two step job configured via the + `JobBuilderFactory` and the `StepBuilderFactory`. + + +[source, java] +---- +@Configuration +@EnableBatchProcessing +@Import(DataSourceConfiguration.class) +public class AppConfig { + + @Autowired + private JobBuilderFactory jobs; + + @Autowired + private StepBuilderFactory steps; + + @Bean + public Job job(@Qualifier("step1") Step step1, @Qualifier("step2") Step step2) { + return jobs.get("myJob").start(step1).next(step2).build(); + } + + @Bean + protected Step step1(ItemReader reader, + ItemProcessor processor, + ItemWriter writer) { + return steps.get("step1") + . chunk(10) + .reader(reader) + .processor(processor) + .writer(writer) + .build(); + } + + @Bean + protected Step step2(Tasklet tasklet) { + return steps.get("step2") + .tasklet(tasklet) + .build(); + } +} +---- + +[[configuringJobRepository]] + +=== Configuring a JobRepository + +[role="javaContent"] +When using `@EnableBatchProcessing`, a `JobRepository` is provided out of the box for you. +This section addresses configuring your own. + + +As described in earlier, the <> is used for basic CRUD operations of the various persisted + domain objects within Spring Batch, such as + `JobExecution` and + `StepExecution`. It is required by many of the major + framework features, such as the `JobLauncher`, + `Job`, and `Step`. + +[role="xmlContent"] +The batch + namespace abstracts away many of the implementation details of the + `JobRepository` implementations and their + collaborators. However, there are still a few configuration options + available: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + +---- + +[role="xmlContent"] +None of the configuration options listed above are required except + the id. If they are not set, the defaults shown above will be used. They + are shown above for awareness purposes. The + `max-varchar-length` defaults to 2500, which is the + length of the long `VARCHAR` columns in the <> + +[role="javaContent"] +When using java configuration, a `JobRepository` is provided for you. A JDBC based one is +provided out of the box if a `DataSource` is provided, the `Map` based one if not. However +you can customize the configuration of the `JobRepository` via an implementation of the +`BatchConfigurer` interface. + +.Java Configuration +[source, java, role="javaContent"] +---- +... +// This would reside in your BatchConfigurer implementation +@Override +protected JobRepository createJobRepository() throws Exception { + JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean(); + factory.setDataSource(dataSource); + factory.setTransactionManager(transactionManager); + factory.setIsolationLevelForCreate("ISOLATION_SERIALIZABLE"); + factory.setTablePrefix("BATCH_"); + factory.setMaxVarCharLength(1000); + return factory.getObject(); +} +... +---- + +[role="javaContent"] +None of the configuration options listed above are required except + the dataSource and transactionManager. If they are not set, the defaults shown above + will be used. They are shown above for awareness purposes. The + max varchar length defaults to 2500, which is the + length of the long `VARCHAR` columns in the + <> + +[[txConfigForJobRepository]] + + +==== Transaction Configuration for the JobRepository + +If the namespace or the provided `FactoryBean` is used, transactional advice will be + automatically created around the repository. This is to ensure that the + batch meta data, including state that is necessary for restarts after a + failure, is persisted correctly. The behavior of the framework is not + well defined if the repository methods are not transactional. The + isolation level in the `create*` method attributes is + specified separately to ensure that when jobs are launched, if two + processes are trying to launch the same job at the same time, only one + will succeed. The default isolation level for that method is + SERIALIZABLE, which is quite aggressive: READ_COMMITTED would work just + as well; READ_UNCOMMITTED would be fine if two processes are not likely + to collide in this way. However, since a call to the + `create*` method is quite short, it is unlikely + that the SERIALIZED will cause problems, as long as the database + platform supports it. However, this can be overridden: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +// This would reside in your BatchConfigurer implementation +@Override +protected JobRepository createJobRepository() throws Exception { + JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean(); + factory.setDataSource(dataSource); + factory.setTransactionManager(transactionManager); + factory.setIsolationLevelForCreate("ISOLATION_REPEATABLE_READ"); + return factory.getObject(); +} +---- + +If the namespace or factory beans aren't used then it is also + essential to configure the transactional behavior of the repository + using AOP: + + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + + + + + + + + +---- + +[role="xmlContent"] +This fragment can be used as is, with almost no changes. Remember + also to include the appropriate namespace declarations and to make sure + spring-tx and spring-aop (or the whole of spring) are on the + classpath. + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public TransactionProxyFactoryBean baseProxy() { + TransactionProxyFactoryBean transactionProxyFactoryBean = new TransactionProxyFactoryBean(); + Properties transactionAttributes = new Properties(); + transactionAttributes.setProperty("*", "PROPAGATION_REQUIRED"); + transactionProxyFactoryBean.setTransactionAttributes(transactionAttributes); + transactionProxyFactoryBean.setTarget(jobRepository()); + transactionProxyFactoryBean.setTransactionManager(transactionManager()); + return transactionProxyFactoryBean; +} +---- + +[[repositoryTablePrefix]] + + +==== Changing the Table Prefix + +Another modifiable property of the + `JobRepository` is the table prefix of the + meta-data tables. By default they are all prefaced with BATCH_. + BATCH_JOB_EXECUTION and BATCH_STEP_EXECUTION are two examples. However, + there are potential reasons to modify this prefix. If the schema names + needs to be prepended to the table names, or if more than one set of + meta data tables is needed within the same schema, then the table prefix + will need to be changed: + + +.XML Configuration +[source, xml, role="xmlContent"] +---- + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +// This would reside in your BatchConfigurer implementation +@Override +protected JobRepository createJobRepository() throws Exception { + JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean(); + factory.setDataSource(dataSource); + factory.setTransactionManager(transactionManager); + factory.setTablePrefix("SYSTEM.TEST_"); + return factory.getObject(); +} +---- + +Given the above changes, every query to the meta data tables will + be prefixed with "SYSTEM.TEST_". BATCH_JOB_EXECUTION will be referred to + as SYSTEM.TEST_JOB_EXECUTION. + + +[NOTE] +==== +Only the table prefix is configurable. The table and column + names are not. + +==== + + +[[inMemoryRepository]] + + +==== In-Memory Repository + +There are scenarios in which you may not want to persist your + domain objects to the database. One reason may be speed; storing domain + objects at each commit point takes extra time. Another reason may be + that you just don't need to persist status for a particular job. For + this reason, Spring batch provides an in-memory Map version of the job + repository: + + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +// This would reside in your BatchConfigurer implementation +@Override +protected JobRepository createJobRepository() throws Exception { + MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(); + factory.setTransactionManager(transactionManager); + return factory.getObject(); +} + +---- + +Note that the in-memory repository is volatile and so does not + allow restart between JVM instances. It also cannot guarantee that two + job instances with the same parameters are launched simultaneously, and + is not suitable for use in a multi-threaded Job, or a locally + partitioned `Step`. So use the database version of the repository wherever + you need those features. + +However it does require a transaction manager to be defined + because there are rollback semantics within the repository, and because + the business logic might still be transactional (e.g. RDBMS access). For + testing purposes many people find the + `ResourcelessTransactionManager` useful. + +[[nonStandardDatabaseTypesInRepository]] + + +==== Non-standard Database Types in a Repository + +If you are using a database platform that is not in the list of + supported platforms, you may be able to use one of the supported types, + if the SQL variant is close enough. To do this you can use the raw + `JobRepositoryFactoryBean` instead of the namespace + shortcut and use it to set the database type to the closest + match: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +// This would reside in your BatchConfigurer implementation +@Override +protected JobRepository createJobRepository() throws Exception { + JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean(); + factory.setDataSource(dataSource); + factory.setDatabaseType("db2"); + factory.setTransactionManager(transactionManager); + return factory.getObject(); +} + +---- + +(The `JobRepositoryFactoryBean` tries to + auto-detect the database type from the `DataSource` + if it is not specified.) The major differences between platforms are + mainly accounted for by the strategy for incrementing primary keys, so + often it might be necessary to override the + `incrementerFactory` as well (using one of the standard + implementations from the Spring Framework). + +If even that doesn't work, or you are not using an RDBMS, then the + only option may be to implement the various `Dao` + interfaces that the `SimpleJobRepository` depends + on and wire one up manually in the normal Spring way. + +[[configuringJobLauncher]] + + +=== Configuring a JobLauncher + +[role="javaContent"] +When using `@EnableBatchProcessing`, a `JobRegistry` is provided out of the box for you. +This section addresses configuring your own. + +The most basic implementation of the + `JobLauncher` interface is the + `SimpleJobLauncher`. Its only required dependency is + a `JobRepository`, in order to obtain an + execution: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +... +// This would reside in your BatchConfigurer implementation +@Override +protected JobLauncher createJobLauncher() throws Exception { + SimpleJobLauncher jobLauncher = new SimpleJobLauncher(); + jobLauncher.setJobRepository(jobRepository); + jobLauncher.afterPropertiesSet(); + return jobLauncher; +} +... +---- + +Once a <> is + obtained, it is passed to the execute method of + Job, ultimately returning the + `JobExecution` to the caller: + +.Job Launcher Sequence +image::{batch-asciidoc}images/job-launcher-sequence-sync.png[Job Launcher Sequence, scaledwidth="60%"] + +The sequence is straightforward and works well when launched from a + scheduler. However, issues arise when trying to launch from an HTTP + request. In this scenario, the launching needs to be done asynchronously + so that the `SimpleJobLauncher` returns immediately + to its caller. This is because it is not good practice to keep an HTTP + request open for the amount of time needed by long running processes such + as batch. An example sequence is below: + +.Asynchronous Job Launcher Sequence +image::{batch-asciidoc}images/job-launcher-sequence-async.png[Async Job Launcher Sequence, scaledwidth="60%"] + + +The `SimpleJobLauncher` can easily be + configured to allow for this scenario by configuring a + `TaskExecutor`: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public JobLauncher jobLauncher() { + SimpleJobLauncher jobLauncher = new SimpleJobLauncher(); + jobLauncher.setJobRepository(jobRepository()); + jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor()); + jobLauncher.afterPropertiesSet(); + return jobLauncher; +} +---- + +Any implementation of the spring `TaskExecutor` + interface can be used to control how jobs are asynchronously + executed. + +[[runningAJob]] + + +=== Running a Job + +At a minimum, launching a batch job requires two things: the + `Job` to be launched and a + `JobLauncher`. Both can be contained within the same + context or different contexts. For example, if launching a job from the + command line, a new JVM will be instantiated for each Job, and thus every + job will have its own `JobLauncher`. However, if + running from within a web container within the scope of an + `HttpRequest`, there will usually be one + `JobLauncher`, configured for asynchronous job + launching, that multiple requests will invoke to launch their jobs. + +[[runningJobsFromCommandLine]] + + +==== Running Jobs from the Command Line + +For users that want to run their jobs from an enterprise + scheduler, the command line is the primary interface. This is because + most schedulers (with the exception of Quartz unless using the + NativeJob) work directly with operating system + processes, primarily kicked off with shell scripts. There are many ways + to launch a Java process besides a shell script, such as Perl, Ruby, or + even 'build tools' such as ant or maven. However, because most people + are familiar with shell scripts, this example will focus on them. + +[[commandLineJobRunner]] + + +===== The CommandLineJobRunner + +Because the script launching the job must kick off a Java + Virtual Machine, there needs to be a class with a main method to act + as the primary entry point. Spring Batch provides an implementation + that serves just this purpose: + `CommandLineJobRunner`. It's important to note + that this is just one way to bootstrap your application, but there are + many ways to launch a Java process, and this class should in no way be + viewed as definitive. The `CommandLineJobRunner` + performs four tasks: + + +* Load the appropriate + `ApplicationContext` + + +* Parse command line arguments into + `JobParameters` + + +* Locate the appropriate job based on arguments + + +* Use the `JobLauncher` provided in the + application context to launch the job. + +All of these tasks are accomplished using only the arguments + passed in. The following are required arguments: + +.CommandLineJobRunner arguments + +|=============== +|jobPath|The location of the XML file that will be used to + create an `ApplicationContext`. This file + should contain everything needed to run the complete + Job +|jobName|The name of the job to be run. + +|=============== + + +These arguments must be passed in with the path first and the + name second. All arguments after these are considered to be + `JobParameters` and must be in the format of 'name=value': + + +[source, role="xmlContent"] +---- +>. The first argument is + 'endOfDayJob.xml', which is the Spring + `ApplicationContext` containing the + Job. The second argument, 'endOfDay' represents + the job name. The final argument, 'schedule.date(date)=2007/05/05' + will be converted into `JobParameters`. An + example of the XML configuration is below: + + +[source, xml, role="xmlContent"] +---- + + + + + + +---- + +[role="javaContent"] +In most cases you would want to use a manifest to declare your + main class in a jar, but for simplicity, the class was used directly. + This example is using the same 'EndOfDay' example from the <>. The first argument is + 'io.spring.EndOfDayJobConfiguration', which is the fully qualified class name to + the configuration class containing the + Job. The second argument, 'endOfDay' represents + the job name. The final argument, 'schedule.date(date)=2007/05/05' + will be converted into JobParameters. An + example of the java configuration is below: + +[source, java, role="javaContent"] +---- +@Configuration +@EnableBatchProcessing +public class EndOfDayJobConfiguration { + + @Autowired + private JobBuilderFactory jobBuilderFactory; + + @Autowired + private StepBuilderFactory stepBuilderFactory; + + @Bean + public Job endOfDay() { + return this.jobBuilderFactory.get("endOfDay") + .start(step1()) + .build(); + } + + @Bean + public Step step1() { + return this.stepBuilderFactory.get("step1") + .tasklet((contribution, chunkContext) -> null) + .build(); + } +} +---- +endif::backend-html5[] + +ifdef::backend-pdf[] +In most cases you would want to use a manifest to declare your + main class in a jar, but for simplicity, the class was used directly. + This example is using the same 'EndOfDay' example from the <>. The first argument is + where your job is configured (either an XML file or a fully qualified class name). + The second argument, 'endOfDay' represents + the job name. The final argument, 'schedule.date(date)=2007/05/05' + will be converted into `JobParameters`. An + example of the configuration is below: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Configuration +@EnableBatchProcessing +public class EndOfDayJobConfiguration { + + @Autowired + private JobBuilderFactory jobBuilderFactory; + + @Autowired + private StepBuilderFactory stepBuilderFactory; + + @Bean + public Job endOfDay() { + return this.jobBuilderFactory.get("endOfDay") + .start(step1()) + .build(); + } + + @Bean + public Step step1() { + return this.stepBuilderFactory.get("step1") + .tasklet((contribution, chunkContext) -> null) + .build(); + } +} +---- + +endif::backend-pdf[] + +This example is overly simplistic, since there are many more + requirements to a run a batch job in Spring Batch in general, but it + serves to show the two main requirements of the + `CommandLineJobRunner`: + `Job` and + `JobLauncher` + +[[exitCodes]] + + +===== ExitCodes + +When launching a batch job from the command-line, an enterprise + scheduler is often used. Most schedulers are fairly dumb and work only + at the process level. This means that they only know about some + operating system process such as a shell script that they're invoking. + In this scenario, the only way to communicate back to the scheduler + about the success or failure of a job is through return codes. A + return code is a number that is returned to a scheduler by the process + that indicates the result of the run. In the simplest case: 0 is + success and 1 is failure. However, there may be more complex + scenarios: If job A returns 4 kick off job B, and if it returns 5 kick + off job C. This type of behavior is configured at the scheduler level, + but it is important that a processing framework such as Spring Batch + provide a way to return a numeric representation of the 'Exit Code' + for a particular batch job. In Spring Batch this is encapsulated + within an `ExitStatus`, which is covered in more + detail in Chapter 5. For the purposes of discussing exit codes, the + only important thing to know is that an + `ExitStatus` has an exit code property that is + set by the framework (or the developer) and is returned as part of the + `JobExecution` returned from the + `JobLauncher`. The + `CommandLineJobRunner` converts this string value + to a number using the `ExitCodeMapper` + interface: + + +[source, java] +---- +public interface ExitCodeMapper { + + public int intValue(String exitCode); + +} +---- + +The essential contract of an + `ExitCodeMapper` is that, given a string exit + code, a number representation will be returned. The default + implementation used by the job runner is the `SimpleJvmExitCodeMapper` + that returns 0 for completion, 1 for generic errors, and 2 for any job + runner errors such as not being able to find a + `Job` in the provided context. If anything more + complex than the 3 values above is needed, then a custom + implementation of the `ExitCodeMapper` interface + must be supplied. Because the + `CommandLineJobRunner` is the class that creates + an `ApplicationContext`, and thus cannot be + 'wired together', any values that need to be overwritten must be + autowired. This means that if an implementation of + `ExitCodeMapper` is found within the `BeanFactory`, + it will be injected into the runner after the context is created. All + that needs to be done to provide your own + `ExitCodeMapper` is to declare the implementation + as a root level bean and ensure that it is part of the + `ApplicationContext` that is loaded by the + runner. + +[[runningJobsFromWebContainer]] + + +==== Running Jobs from within a Web Container + +Historically, offline processing such as batch jobs have been + launched from the command-line, as described above. However, there are + many cases where launching from an `HttpRequest` is + a better option. Many such use cases include reporting, ad-hoc job + running, and web application support. Because a batch job by definition + is long running, the most important concern is ensuring to launch the + job asynchronously: + +.Asynchronous Job Launcher Sequence From Web Container +image::{batch-asciidoc}images/launch-from-request.png[Async Job Launcher Sequence from web container, scaledwidth="60%"] + + +The controller in this case is a Spring MVC controller. More + information on Spring MVC can be found here: link:$$https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#mvc$$[https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#mvc]. + The controller launches a `Job` using a + `JobLauncher` that has been configured to launch + <>, which + immediately returns a `JobExecution`. The + `Job` will likely still be running, however, this + nonblocking behaviour allows the controller to return immediately, which + is required when handling an `HttpRequest`. An + example is below: + + +[source, java] +---- +@Controller +public class JobLauncherController { + + @Autowired + JobLauncher jobLauncher; + + @Autowired + Job job; + + @RequestMapping("/jobLauncher.html") + public void handle() throws Exception{ + jobLauncher.run(job, new JobParameters()); + } +} +---- + +[[advancedMetaData]] + + +=== Advanced Meta-Data Usage + +So far, both the `JobLauncher` and `JobRepository` interfaces have been + discussed. Together, they represent simple launching of a job, and basic + CRUD operations of batch domain objects: + +.Job Repository +image::{batch-asciidoc}images/job-repository.png[Job Repository, scaledwidth="60%"] + +A `JobLauncher` uses the + `JobRepository` to create new + `JobExecution` objects and run them. + `Job` and `Step` implementations + later use the same `JobRepository` for basic updates + of the same executions during the running of a Job. + The basic operations suffice for simple scenarios, but in a large batch + environment with hundreds of batch jobs and complex scheduling + requirements, more advanced access of the meta data is required: + +.Advanced Job Repository Access +image::{batch-asciidoc}images/job-repository-advanced.png[Job Repository Advanced, scaledwidth="80%"] + +The `JobExplorer` and + `JobOperator` interfaces, which will be discussed + below, add additional functionality for querying and controlling the meta + data. + +[[queryingRepository]] + + +==== Querying the Repository + +The most basic need before any advanced features is the ability to + query the repository for existing executions. This functionality is + provided by the `JobExplorer` interface: + + +[source, java] +---- +public interface JobExplorer { + + List getJobInstances(String jobName, int start, int count); + + JobExecution getJobExecution(Long executionId); + + StepExecution getStepExecution(Long jobExecutionId, Long stepExecutionId); + + JobInstance getJobInstance(Long instanceId); + + List getJobExecutions(JobInstance jobInstance); + + Set findRunningJobExecutions(String jobName); +} +---- + +As is evident from the method signatures above, + `JobExplorer` is a read-only version of the + `JobRepository`, and like the + `JobRepository`, it can be easily configured via a + factory bean: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +... +// This would reside in your BatchConfigurer implementation +@Override +public JobExplorer getJobExplorer() throws Exception { + JobExplorerFactoryBean factoryBean = new JobExplorerFactoryBean(); + factoryBean.setDataSource(this.dataSource); + return factoryBean.getObject(); +} +... +---- + +<>, it was mentioned that the table prefix of the + `JobRepository` can be modified to allow for + different versions or schemas. Because the + `JobExplorer` is working with the same tables, it + too needs the ability to set a prefix: + +.XML Configuration +[source, xml, role="xmlContent"] +---- + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +... +// This would reside in your BatchConfigurer implementation +@Override +public JobExplorer getJobExplorer() throws Exception { + JobExplorerFactoryBean factoryBean = new JobExplorerFactoryBean(); + factoryBean.setDataSource(this.dataSource); + factoryBean.setTablePrefix("SYSTEM."); + return factoryBean.getObject(); +} +... +---- + + +==== JobRegistry + +A `JobRegistry` (and its parent interface `JobLocator`) is not + mandatory, but it can be useful if you want to keep track of which jobs + are available in the context. It is also useful for collecting jobs + centrally in an application context when they have been created + elsewhere (e.g. in child contexts). Custom `JobRegistry` implementations + can also be used to manipulate the names and other properties of the + jobs that are registered. There is only one implementation provided by + the framework and this is based on a simple map from job name to job + instance. + +[source, xml, role="xmlContent"] +---- + +---- + +[role="javaContent"] +When using `@EnableBatchProcessing`, a `JobRegistry` is provided out of the box for you. +If you want to configure your own: + +[source, java, role="javaContent"] +---- +... +// This is already provided via the @EnableBatchProcessing but can be customized via +// overriding the getter in the SimpleBatchConfiguration +@Override +@Bean +public JobRegistry jobRegistry() throws Exception { + return new MapJobRegistry(); +} +... +---- + +There are two ways to populate a `JobRegistry` automatically: using + a bean post processor and using a registrar lifecycle component. These + two mechanisms are described in the following sections. + +===== JobRegistryBeanPostProcessor + +This is a bean post-processor that can register all jobs as they + are created: + + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor() { + JobRegistryBeanPostProcessor postProcessor = new JobRegistryBeanPostProcessor(); + postProcessor.setJobRegistry(jobRegistry()); + return postProcessor; +} +---- + +Although it is not strictly necessary, the post-processor in the + example has been given an id so that it can be included in child + contexts (e.g. as a parent bean definition) and cause all jobs created + there to also be registered automatically. + + + +===== AutomaticJobRegistrar + +This is a lifecycle component that creates child contexts and + registers jobs from those contexts as they are created. One advantage + of doing this is that, while the job names in the child contexts still + have to be globally unique in the registry, their dependencies can + have "natural" names. So for example, you can create a set of XML + configuration files each having only one Job, + but all having different definitions of an + `ItemReader` with the same bean name, e.g. + "reader". If all those files were imported into the same context, the + reader definitions would clash and override one another, but with the + automatic registrar this is avoided. This makes it easier to + integrate jobs contributed from separate modules of an + application. + + +.XML Configuration +[source, xml, role="xmlContent"] +---- + + + + + + + + + + + + +---- + +.Java Configuration +[source, java, role="javaContent"] +---- +@Bean +public AutomaticJobRegistrar registrar() { + + AutomaticJobRegistrar registrar = new AutomaticJobRegistrar(); + registrar.setJobLoader(jobLoader()); + registrar.setApplicationContextFactories(applicationContextFactories()); + registrar.afterPropertiesSet(); + return registrar; + +} +---- + +The registrar has two mandatory properties, one is an array of + `ApplicationContextFactory` (here created from a + convenient factory bean), and the other is a + `JobLoader`. The `JobLoader` + is responsible for managing the lifecycle of the child contexts and + registering jobs in the `JobRegistry`. + +The `ApplicationContextFactory` is + responsible for creating the child context and the most common usage + would be as above using a + `ClassPathXmlApplicationContextFactory`. One of + the features of this factory is that by default it copies some of the + configuration down from the parent context to the child. So for + instance you don't have to re-define the + `PropertyPlaceholderConfigurer` or AOP + configuration in the child, if it should be the same as the + parent. + +The `AutomaticJobRegistrar` can be used in + conjunction with a `JobRegistryBeanPostProcessor` + if desired (as long as the `DefaultJobLoader` is + used as well). For instance this might be desirable if there are jobs + defined in the main parent context as well as in the child + locations. + +[[JobOperator]] + + +==== JobOperator + +As previously discussed, the `JobRepository` + provides CRUD operations on the meta-data, and the + `JobExplorer` provides read-only operations on the + meta-data. However, those operations are most useful when used together + to perform common monitoring tasks such as stopping, restarting, or + summarizing a Job, as is commonly done by batch operators. Spring Batch + provides these types of operations via the + `JobOperator` interface: + + +[source, java] +---- +public interface JobOperator { + + List getExecutions(long instanceId) throws NoSuchJobInstanceException; + + List getJobInstances(String jobName, int start, int count) + throws NoSuchJobException; + + Set getRunningExecutions(String jobName) throws NoSuchJobException; + + String getParameters(long executionId) throws NoSuchJobExecutionException; + + Long start(String jobName, String parameters) + throws NoSuchJobException, JobInstanceAlreadyExistsException; + + Long restart(long executionId) + throws JobInstanceAlreadyCompleteException, NoSuchJobExecutionException, + NoSuchJobException, JobRestartException; + + Long startNextInstance(String jobName) + throws NoSuchJobException, JobParametersNotFoundException, JobRestartException, + JobExecutionAlreadyRunningException, JobInstanceAlreadyCompleteException; + + boolean stop(long executionId) + throws NoSuchJobExecutionException, JobExecutionNotRunningException; + + String getSummary(long executionId) throws NoSuchJobExecutionException; + + Map getStepExecutionSummaries(long executionId) + throws NoSuchJobExecutionException; + + Set getJobNames(); + +} +---- + +The above operations represent methods from many different + interfaces, such as `JobLauncher`, + `JobRepository`, + `JobExplorer`, and + `JobRegistry`. For this reason, the provided + implementation of `JobOperator`, + `SimpleJobOperator`, has many dependencies: + + +[source, xml, role="xmlContent"] +---- + + + + + + + + + + +---- + +[source, java, role="javaContent"] +---- + /** + * All injected dependencies for this bean are provided by the @EnableBatchProcessing + * infrastructure out of the box. + */ + @Bean + public SimpleJobOperator jobOperator(JobExplorer jobExplorer, + JobRepository jobRepository, + JobRegistry jobRegistry) { + + SimpleJobOperator jobOperator = new SimpleJobOperator(); + + jobOperator.setJobExplorer(jobExplorer); + jobOperator.setJobRepository(jobRepository); + jobOperator.setJobRegistry(jobRegistry); + jobOperator.setJobLauncher(jobLauncher); + + return jobOperator; + } +---- + + +[NOTE] +==== + +If you set the table prefix on the job repository, don't forget to set it on the job explorer as well. +==== + + +[[JobParametersIncrementer]] + + +==== JobParametersIncrementer + +Most of the methods on `JobOperator` are + self-explanatory, and more detailed explanations can be found on the + link:$$https://docs.spring.io/spring-batch/apidocs/org/springframework/batch/core/launch/JobOperator.html$$[javadoc of the interface]. However, the + `startNextInstance` method is worth noting. This + method will always start a new instance of a Job. + This can be extremely useful if there are serious issues in a + `JobExecution` and the Job + needs to be started over again from the beginning. Unlike + `JobLauncher` though, which requires a new + `JobParameters` object that will trigger a new + `JobInstance` if the parameters are different from + any previous set of parameters, the + `startNextInstance` method will use the + `JobParametersIncrementer` tied to the + `Job` to force the `Job` to a + new instance: + + +[source, java] +---- +public interface JobParametersIncrementer { + + JobParameters getNext(JobParameters parameters); + +} +---- + +The contract of `JobParametersIncrementer` is + that, given a <> + object, it will return the 'next' JobParameters + object by incrementing any necessary values it may contain. This + strategy is useful because the framework has no way of knowing what + changes to the `JobParameters` make it the 'next' + instance. For example, if the only value in + `JobParameters` is a date, and the next instance + should be created, should that value be incremented by one day? Or one + week (if the job is weekly for instance)? The same can be said for any + numerical values that help to identify the Job, + as shown below: + + +[source, java] +---- +public class SampleIncrementer implements JobParametersIncrementer { + + public JobParameters getNext(JobParameters parameters) { + if (parameters==null || parameters.isEmpty()) { + return new JobParametersBuilder().addLong("run.id", 1L).toJobParameters(); + } + long id = parameters.getLong("run.id",1L) + 1; + return new JobParametersBuilder().addLong("run.id", id).toJobParameters(); + } +} +---- + +In this example, the value with a key of 'run.id' is used to + discriminate between `JobInstances`. If the + `JobParameters` passed in is null, it can be + assumed that the `Job` has never been run before + and thus its initial state can be returned. However, if not, the old + value is obtained, incremented by one, and returned. + +ifdef::backend-html5[] +[role="xmlContent"] +An incrementer can + be associated with `Job` via the 'incrementer' + attribute in the namespace: + + +[source, xml, role="xmlContent"] +---- + + ... + +---- + +[role="javaContent"] +An incrementer can be associated with a 'Job' via the `incrementer` method provided in the +builders: + +[source, java, role="javaContent"] +---- +@Bean +public Job footballJob() { + return this.jobBuilderFactory.get("footballJob") + .incrementer(sampleIncrementer()) + ... + .build(); +} +---- +endif::backend-html5[] + +ifdef::backend-pdf[] +An incrementer can + be associated with `Job` via the 'incrementer' + attribute in the namespace: + +[source, xml] +---- + + ... + +---- + +The java config builders also provide facilities for the configuration of an incrementer: + +[source, java] +---- +@Bean +public Job footballJob() { + return this.jobBuilderFactory.get("footballJob") + .incrementer(sampleIncrementer()) + ... + .build(); +} +---- +endif::backend-pdf[] + + +[[stoppingAJob]] + + +==== Stopping a Job + +One of the most common use cases of + `JobOperator` is gracefully stopping a + Job: + + +[source, java] +---- +Set executions = jobOperator.getRunningExecutions("sampleJob"); +jobOperator.stop(executions.iterator().next()); +---- + +The shutdown is not immediate, since there is no way to force + immediate shutdown, especially if the execution is currently in + developer code that the framework has no control over, such as a + business service. However, as soon as control is returned back to the + framework, it will set the status of the current + `StepExecution` to + `BatchStatus.STOPPED`, save it, then do the same + for the `JobExecution` before finishing. + + + +==== Aborting a Job + +A job execution which is `FAILED` can be + restarted (if the `Job` is restartable). A job execution whose status is + `ABANDONED` will not be restarted by the framework. + The `ABANDONED` status is also used in step + executions to mark them as skippable in a restarted job execution: if a + job is executing and encounters a step that has been marked + `ABANDONED` in the previous failed job execution, it + will move on to the next step (as determined by the job flow definition + and the step execution exit status). + +If the process died (`"kill -9"` or server + failure) the job is, of course, not running, but the `JobRepository` has + no way of knowing because no-one told it before the process died. You + have to tell it manually that you know that the execution either failed + or should be considered aborted (change its status to + `FAILED` or `ABANDONED`) - it's + a business decision and there is no way to automate it. Only change the + status to `FAILED` if it is not restartable, or if + you know the restart data is valid. There is a utility in Spring Batch + Admin `JobService` to abort a job execution. diff --git a/spring-batch-docs/asciidoc/js/DocumentToggle.js b/spring-batch-docs/asciidoc/js/DocumentToggle.js new file mode 100644 index 0000000000..8f5d76981e --- /dev/null +++ b/spring-batch-docs/asciidoc/js/DocumentToggle.js @@ -0,0 +1,64 @@ +$(document).ready(function(){ + + setJava(); + + // Initial cookie handler. This part remembers the reader's choice and sets the toggle + // accordingly. + var docToggleCookieString = Cookies.get("docToggle"); + if (docToggleCookieString != null) { + if (docToggleCookieString === "xml") { + $("#xmlButton").prop("checked", true); + setXml(); + } else if (docToggleCookieString === "java") { + $("#javaButton").prop("checked", true); + setJava(); + } else if (docToggleCookieString === "both") { + $("#bothButton").prop("checked", true); + setBoth(); + } + } + + // Click handlers + $("#xmlButton").on("click", function() { + setXml(); + }); + $("#javaButton").on("click", function() { + setJava(); + }); + $("#bothButton").on("click", function() { + setBoth(); + }); + + // Functions to do the work of handling the reader's choice, whether through a click + // or through a cookie. 3652 days is 10 years, give or take a leap day. + function setXml() { + $("*.xmlContent").show(); + $("*.javaContent").hide(); + $("*.javaContent > *").addClass("js-toc-ignore"); + $("*.xmlContent > *").removeClass("js-toc-ignore"); + window.dispatchEvent(new Event("tocRefresh")); + tocbot.refresh(); + Cookies.set('docToggle', 'xml', { expires: 3652 }); + }; + + function setJava() { + $("*.javaContent").show(); + $("*.xmlContent").hide(); + $("*.xmlContent > *").addClass("js-toc-ignore"); + $("*.javaContent > *").removeClass("js-toc-ignore"); + window.dispatchEvent(new Event("tocRefresh")); + tocbot.refresh(); + Cookies.set('docToggle', 'java', { expires: 3652 }); + }; + + function setBoth() { + $("*.javaContent").show(); + $("*.xmlContent").show(); + $("*.javaContent > *").removeClass("js-toc-ignore"); + $("*.xmlContent > *").removeClass("js-toc-ignore"); + window.dispatchEvent(new Event("tocRefresh")); + tocbot.refresh(); + Cookies.set('docToggle', 'both', { expires: 3652 }); + }; + +}); diff --git a/spring-batch-docs/asciidoc/js/jquery-3.2.1.min.js b/spring-batch-docs/asciidoc/js/jquery-3.2.1.min.js new file mode 100644 index 0000000000..644d35e274 --- /dev/null +++ b/spring-batch-docs/asciidoc/js/jquery-3.2.1.min.js @@ -0,0 +1,4 @@ +/*! jQuery v3.2.1 | (c) JS Foundation and other contributors | jquery.org/license */ +!function(a,b){"use strict";"object"==typeof module&&"object"==typeof module.exports?module.exports=a.document?b(a,!0):function(a){if(!a.document)throw new Error("jQuery requires a window with a document");return b(a)}:b(a)}("undefined"!=typeof window?window:this,function(a,b){"use strict";var c=[],d=a.document,e=Object.getPrototypeOf,f=c.slice,g=c.concat,h=c.push,i=c.indexOf,j={},k=j.toString,l=j.hasOwnProperty,m=l.toString,n=m.call(Object),o={};function p(a,b){b=b||d;var c=b.createElement("script");c.text=a,b.head.appendChild(c).parentNode.removeChild(c)}var q="3.2.1",r=function(a,b){return new r.fn.init(a,b)},s=/^[\s\uFEFF\xA0]+|[\s\uFEFF\xA0]+$/g,t=/^-ms-/,u=/-([a-z])/g,v=function(a,b){return b.toUpperCase()};r.fn=r.prototype={jquery:q,constructor:r,length:0,toArray:function(){return f.call(this)},get:function(a){return null==a?f.call(this):a<0?this[a+this.length]:this[a]},pushStack:function(a){var b=r.merge(this.constructor(),a);return b.prevObject=this,b},each:function(a){return r.each(this,a)},map:function(a){return this.pushStack(r.map(this,function(b,c){return a.call(b,c,b)}))},slice:function(){return this.pushStack(f.apply(this,arguments))},first:function(){return this.eq(0)},last:function(){return this.eq(-1)},eq:function(a){var b=this.length,c=+a+(a<0?b:0);return this.pushStack(c>=0&&c0&&b-1 in a)}var x=function(a){var b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u="sizzle"+1*new Date,v=a.document,w=0,x=0,y=ha(),z=ha(),A=ha(),B=function(a,b){return a===b&&(l=!0),0},C={}.hasOwnProperty,D=[],E=D.pop,F=D.push,G=D.push,H=D.slice,I=function(a,b){for(var c=0,d=a.length;c+~]|"+K+")"+K+"*"),S=new RegExp("="+K+"*([^\\]'\"]*?)"+K+"*\\]","g"),T=new RegExp(N),U=new RegExp("^"+L+"$"),V={ID:new RegExp("^#("+L+")"),CLASS:new RegExp("^\\.("+L+")"),TAG:new RegExp("^("+L+"|[*])"),ATTR:new RegExp("^"+M),PSEUDO:new RegExp("^"+N),CHILD:new RegExp("^:(only|first|last|nth|nth-last)-(child|of-type)(?:\\("+K+"*(even|odd|(([+-]|)(\\d*)n|)"+K+"*(?:([+-]|)"+K+"*(\\d+)|))"+K+"*\\)|)","i"),bool:new RegExp("^(?:"+J+")$","i"),needsContext:new RegExp("^"+K+"*[>+~]|:(even|odd|eq|gt|lt|nth|first|last)(?:\\("+K+"*((?:-\\d)?\\d*)"+K+"*\\)|)(?=[^-]|$)","i")},W=/^(?:input|select|textarea|button)$/i,X=/^h\d$/i,Y=/^[^{]+\{\s*\[native \w/,Z=/^(?:#([\w-]+)|(\w+)|\.([\w-]+))$/,$=/[+~]/,_=new RegExp("\\\\([\\da-f]{1,6}"+K+"?|("+K+")|.)","ig"),aa=function(a,b,c){var d="0x"+b-65536;return d!==d||c?b:d<0?String.fromCharCode(d+65536):String.fromCharCode(d>>10|55296,1023&d|56320)},ba=/([\0-\x1f\x7f]|^-?\d)|^-$|[^\0-\x1f\x7f-\uFFFF\w-]/g,ca=function(a,b){return b?"\0"===a?"\ufffd":a.slice(0,-1)+"\\"+a.charCodeAt(a.length-1).toString(16)+" ":"\\"+a},da=function(){m()},ea=ta(function(a){return a.disabled===!0&&("form"in a||"label"in a)},{dir:"parentNode",next:"legend"});try{G.apply(D=H.call(v.childNodes),v.childNodes),D[v.childNodes.length].nodeType}catch(fa){G={apply:D.length?function(a,b){F.apply(a,H.call(b))}:function(a,b){var c=a.length,d=0;while(a[c++]=b[d++]);a.length=c-1}}}function ga(a,b,d,e){var f,h,j,k,l,o,r,s=b&&b.ownerDocument,w=b?b.nodeType:9;if(d=d||[],"string"!=typeof a||!a||1!==w&&9!==w&&11!==w)return d;if(!e&&((b?b.ownerDocument||b:v)!==n&&m(b),b=b||n,p)){if(11!==w&&(l=Z.exec(a)))if(f=l[1]){if(9===w){if(!(j=b.getElementById(f)))return d;if(j.id===f)return d.push(j),d}else if(s&&(j=s.getElementById(f))&&t(b,j)&&j.id===f)return d.push(j),d}else{if(l[2])return G.apply(d,b.getElementsByTagName(a)),d;if((f=l[3])&&c.getElementsByClassName&&b.getElementsByClassName)return G.apply(d,b.getElementsByClassName(f)),d}if(c.qsa&&!A[a+" "]&&(!q||!q.test(a))){if(1!==w)s=b,r=a;else if("object"!==b.nodeName.toLowerCase()){(k=b.getAttribute("id"))?k=k.replace(ba,ca):b.setAttribute("id",k=u),o=g(a),h=o.length;while(h--)o[h]="#"+k+" "+sa(o[h]);r=o.join(","),s=$.test(a)&&qa(b.parentNode)||b}if(r)try{return G.apply(d,s.querySelectorAll(r)),d}catch(x){}finally{k===u&&b.removeAttribute("id")}}}return i(a.replace(P,"$1"),b,d,e)}function ha(){var a=[];function b(c,e){return a.push(c+" ")>d.cacheLength&&delete b[a.shift()],b[c+" "]=e}return b}function ia(a){return a[u]=!0,a}function ja(a){var b=n.createElement("fieldset");try{return!!a(b)}catch(c){return!1}finally{b.parentNode&&b.parentNode.removeChild(b),b=null}}function ka(a,b){var c=a.split("|"),e=c.length;while(e--)d.attrHandle[c[e]]=b}function la(a,b){var c=b&&a,d=c&&1===a.nodeType&&1===b.nodeType&&a.sourceIndex-b.sourceIndex;if(d)return d;if(c)while(c=c.nextSibling)if(c===b)return-1;return a?1:-1}function ma(a){return function(b){var c=b.nodeName.toLowerCase();return"input"===c&&b.type===a}}function na(a){return function(b){var c=b.nodeName.toLowerCase();return("input"===c||"button"===c)&&b.type===a}}function oa(a){return function(b){return"form"in b?b.parentNode&&b.disabled===!1?"label"in b?"label"in b.parentNode?b.parentNode.disabled===a:b.disabled===a:b.isDisabled===a||b.isDisabled!==!a&&ea(b)===a:b.disabled===a:"label"in b&&b.disabled===a}}function pa(a){return ia(function(b){return b=+b,ia(function(c,d){var e,f=a([],c.length,b),g=f.length;while(g--)c[e=f[g]]&&(c[e]=!(d[e]=c[e]))})})}function qa(a){return a&&"undefined"!=typeof a.getElementsByTagName&&a}c=ga.support={},f=ga.isXML=function(a){var b=a&&(a.ownerDocument||a).documentElement;return!!b&&"HTML"!==b.nodeName},m=ga.setDocument=function(a){var b,e,g=a?a.ownerDocument||a:v;return g!==n&&9===g.nodeType&&g.documentElement?(n=g,o=n.documentElement,p=!f(n),v!==n&&(e=n.defaultView)&&e.top!==e&&(e.addEventListener?e.addEventListener("unload",da,!1):e.attachEvent&&e.attachEvent("onunload",da)),c.attributes=ja(function(a){return a.className="i",!a.getAttribute("className")}),c.getElementsByTagName=ja(function(a){return a.appendChild(n.createComment("")),!a.getElementsByTagName("*").length}),c.getElementsByClassName=Y.test(n.getElementsByClassName),c.getById=ja(function(a){return o.appendChild(a).id=u,!n.getElementsByName||!n.getElementsByName(u).length}),c.getById?(d.filter.ID=function(a){var b=a.replace(_,aa);return function(a){return a.getAttribute("id")===b}},d.find.ID=function(a,b){if("undefined"!=typeof b.getElementById&&p){var c=b.getElementById(a);return c?[c]:[]}}):(d.filter.ID=function(a){var b=a.replace(_,aa);return function(a){var c="undefined"!=typeof a.getAttributeNode&&a.getAttributeNode("id");return c&&c.value===b}},d.find.ID=function(a,b){if("undefined"!=typeof b.getElementById&&p){var c,d,e,f=b.getElementById(a);if(f){if(c=f.getAttributeNode("id"),c&&c.value===a)return[f];e=b.getElementsByName(a),d=0;while(f=e[d++])if(c=f.getAttributeNode("id"),c&&c.value===a)return[f]}return[]}}),d.find.TAG=c.getElementsByTagName?function(a,b){return"undefined"!=typeof b.getElementsByTagName?b.getElementsByTagName(a):c.qsa?b.querySelectorAll(a):void 0}:function(a,b){var c,d=[],e=0,f=b.getElementsByTagName(a);if("*"===a){while(c=f[e++])1===c.nodeType&&d.push(c);return d}return f},d.find.CLASS=c.getElementsByClassName&&function(a,b){if("undefined"!=typeof b.getElementsByClassName&&p)return b.getElementsByClassName(a)},r=[],q=[],(c.qsa=Y.test(n.querySelectorAll))&&(ja(function(a){o.appendChild(a).innerHTML="",a.querySelectorAll("[msallowcapture^='']").length&&q.push("[*^$]="+K+"*(?:''|\"\")"),a.querySelectorAll("[selected]").length||q.push("\\["+K+"*(?:value|"+J+")"),a.querySelectorAll("[id~="+u+"-]").length||q.push("~="),a.querySelectorAll(":checked").length||q.push(":checked"),a.querySelectorAll("a#"+u+"+*").length||q.push(".#.+[+~]")}),ja(function(a){a.innerHTML="";var b=n.createElement("input");b.setAttribute("type","hidden"),a.appendChild(b).setAttribute("name","D"),a.querySelectorAll("[name=d]").length&&q.push("name"+K+"*[*^$|!~]?="),2!==a.querySelectorAll(":enabled").length&&q.push(":enabled",":disabled"),o.appendChild(a).disabled=!0,2!==a.querySelectorAll(":disabled").length&&q.push(":enabled",":disabled"),a.querySelectorAll("*,:x"),q.push(",.*:")})),(c.matchesSelector=Y.test(s=o.matches||o.webkitMatchesSelector||o.mozMatchesSelector||o.oMatchesSelector||o.msMatchesSelector))&&ja(function(a){c.disconnectedMatch=s.call(a,"*"),s.call(a,"[s!='']:x"),r.push("!=",N)}),q=q.length&&new RegExp(q.join("|")),r=r.length&&new RegExp(r.join("|")),b=Y.test(o.compareDocumentPosition),t=b||Y.test(o.contains)?function(a,b){var c=9===a.nodeType?a.documentElement:a,d=b&&b.parentNode;return a===d||!(!d||1!==d.nodeType||!(c.contains?c.contains(d):a.compareDocumentPosition&&16&a.compareDocumentPosition(d)))}:function(a,b){if(b)while(b=b.parentNode)if(b===a)return!0;return!1},B=b?function(a,b){if(a===b)return l=!0,0;var d=!a.compareDocumentPosition-!b.compareDocumentPosition;return d?d:(d=(a.ownerDocument||a)===(b.ownerDocument||b)?a.compareDocumentPosition(b):1,1&d||!c.sortDetached&&b.compareDocumentPosition(a)===d?a===n||a.ownerDocument===v&&t(v,a)?-1:b===n||b.ownerDocument===v&&t(v,b)?1:k?I(k,a)-I(k,b):0:4&d?-1:1)}:function(a,b){if(a===b)return l=!0,0;var c,d=0,e=a.parentNode,f=b.parentNode,g=[a],h=[b];if(!e||!f)return a===n?-1:b===n?1:e?-1:f?1:k?I(k,a)-I(k,b):0;if(e===f)return la(a,b);c=a;while(c=c.parentNode)g.unshift(c);c=b;while(c=c.parentNode)h.unshift(c);while(g[d]===h[d])d++;return d?la(g[d],h[d]):g[d]===v?-1:h[d]===v?1:0},n):n},ga.matches=function(a,b){return ga(a,null,null,b)},ga.matchesSelector=function(a,b){if((a.ownerDocument||a)!==n&&m(a),b=b.replace(S,"='$1']"),c.matchesSelector&&p&&!A[b+" "]&&(!r||!r.test(b))&&(!q||!q.test(b)))try{var d=s.call(a,b);if(d||c.disconnectedMatch||a.document&&11!==a.document.nodeType)return d}catch(e){}return ga(b,n,null,[a]).length>0},ga.contains=function(a,b){return(a.ownerDocument||a)!==n&&m(a),t(a,b)},ga.attr=function(a,b){(a.ownerDocument||a)!==n&&m(a);var e=d.attrHandle[b.toLowerCase()],f=e&&C.call(d.attrHandle,b.toLowerCase())?e(a,b,!p):void 0;return void 0!==f?f:c.attributes||!p?a.getAttribute(b):(f=a.getAttributeNode(b))&&f.specified?f.value:null},ga.escape=function(a){return(a+"").replace(ba,ca)},ga.error=function(a){throw new Error("Syntax error, unrecognized expression: "+a)},ga.uniqueSort=function(a){var b,d=[],e=0,f=0;if(l=!c.detectDuplicates,k=!c.sortStable&&a.slice(0),a.sort(B),l){while(b=a[f++])b===a[f]&&(e=d.push(f));while(e--)a.splice(d[e],1)}return k=null,a},e=ga.getText=function(a){var b,c="",d=0,f=a.nodeType;if(f){if(1===f||9===f||11===f){if("string"==typeof a.textContent)return a.textContent;for(a=a.firstChild;a;a=a.nextSibling)c+=e(a)}else if(3===f||4===f)return a.nodeValue}else while(b=a[d++])c+=e(b);return c},d=ga.selectors={cacheLength:50,createPseudo:ia,match:V,attrHandle:{},find:{},relative:{">":{dir:"parentNode",first:!0}," ":{dir:"parentNode"},"+":{dir:"previousSibling",first:!0},"~":{dir:"previousSibling"}},preFilter:{ATTR:function(a){return a[1]=a[1].replace(_,aa),a[3]=(a[3]||a[4]||a[5]||"").replace(_,aa),"~="===a[2]&&(a[3]=" "+a[3]+" "),a.slice(0,4)},CHILD:function(a){return a[1]=a[1].toLowerCase(),"nth"===a[1].slice(0,3)?(a[3]||ga.error(a[0]),a[4]=+(a[4]?a[5]+(a[6]||1):2*("even"===a[3]||"odd"===a[3])),a[5]=+(a[7]+a[8]||"odd"===a[3])):a[3]&&ga.error(a[0]),a},PSEUDO:function(a){var b,c=!a[6]&&a[2];return V.CHILD.test(a[0])?null:(a[3]?a[2]=a[4]||a[5]||"":c&&T.test(c)&&(b=g(c,!0))&&(b=c.indexOf(")",c.length-b)-c.length)&&(a[0]=a[0].slice(0,b),a[2]=c.slice(0,b)),a.slice(0,3))}},filter:{TAG:function(a){var b=a.replace(_,aa).toLowerCase();return"*"===a?function(){return!0}:function(a){return a.nodeName&&a.nodeName.toLowerCase()===b}},CLASS:function(a){var b=y[a+" "];return b||(b=new RegExp("(^|"+K+")"+a+"("+K+"|$)"))&&y(a,function(a){return b.test("string"==typeof a.className&&a.className||"undefined"!=typeof a.getAttribute&&a.getAttribute("class")||"")})},ATTR:function(a,b,c){return function(d){var e=ga.attr(d,a);return null==e?"!="===b:!b||(e+="","="===b?e===c:"!="===b?e!==c:"^="===b?c&&0===e.indexOf(c):"*="===b?c&&e.indexOf(c)>-1:"$="===b?c&&e.slice(-c.length)===c:"~="===b?(" "+e.replace(O," ")+" ").indexOf(c)>-1:"|="===b&&(e===c||e.slice(0,c.length+1)===c+"-"))}},CHILD:function(a,b,c,d,e){var f="nth"!==a.slice(0,3),g="last"!==a.slice(-4),h="of-type"===b;return 1===d&&0===e?function(a){return!!a.parentNode}:function(b,c,i){var j,k,l,m,n,o,p=f!==g?"nextSibling":"previousSibling",q=b.parentNode,r=h&&b.nodeName.toLowerCase(),s=!i&&!h,t=!1;if(q){if(f){while(p){m=b;while(m=m[p])if(h?m.nodeName.toLowerCase()===r:1===m.nodeType)return!1;o=p="only"===a&&!o&&"nextSibling"}return!0}if(o=[g?q.firstChild:q.lastChild],g&&s){m=q,l=m[u]||(m[u]={}),k=l[m.uniqueID]||(l[m.uniqueID]={}),j=k[a]||[],n=j[0]===w&&j[1],t=n&&j[2],m=n&&q.childNodes[n];while(m=++n&&m&&m[p]||(t=n=0)||o.pop())if(1===m.nodeType&&++t&&m===b){k[a]=[w,n,t];break}}else if(s&&(m=b,l=m[u]||(m[u]={}),k=l[m.uniqueID]||(l[m.uniqueID]={}),j=k[a]||[],n=j[0]===w&&j[1],t=n),t===!1)while(m=++n&&m&&m[p]||(t=n=0)||o.pop())if((h?m.nodeName.toLowerCase()===r:1===m.nodeType)&&++t&&(s&&(l=m[u]||(m[u]={}),k=l[m.uniqueID]||(l[m.uniqueID]={}),k[a]=[w,t]),m===b))break;return t-=e,t===d||t%d===0&&t/d>=0}}},PSEUDO:function(a,b){var c,e=d.pseudos[a]||d.setFilters[a.toLowerCase()]||ga.error("unsupported pseudo: "+a);return e[u]?e(b):e.length>1?(c=[a,a,"",b],d.setFilters.hasOwnProperty(a.toLowerCase())?ia(function(a,c){var d,f=e(a,b),g=f.length;while(g--)d=I(a,f[g]),a[d]=!(c[d]=f[g])}):function(a){return e(a,0,c)}):e}},pseudos:{not:ia(function(a){var b=[],c=[],d=h(a.replace(P,"$1"));return d[u]?ia(function(a,b,c,e){var f,g=d(a,null,e,[]),h=a.length;while(h--)(f=g[h])&&(a[h]=!(b[h]=f))}):function(a,e,f){return b[0]=a,d(b,null,f,c),b[0]=null,!c.pop()}}),has:ia(function(a){return function(b){return ga(a,b).length>0}}),contains:ia(function(a){return a=a.replace(_,aa),function(b){return(b.textContent||b.innerText||e(b)).indexOf(a)>-1}}),lang:ia(function(a){return U.test(a||"")||ga.error("unsupported lang: "+a),a=a.replace(_,aa).toLowerCase(),function(b){var c;do if(c=p?b.lang:b.getAttribute("xml:lang")||b.getAttribute("lang"))return c=c.toLowerCase(),c===a||0===c.indexOf(a+"-");while((b=b.parentNode)&&1===b.nodeType);return!1}}),target:function(b){var c=a.location&&a.location.hash;return c&&c.slice(1)===b.id},root:function(a){return a===o},focus:function(a){return a===n.activeElement&&(!n.hasFocus||n.hasFocus())&&!!(a.type||a.href||~a.tabIndex)},enabled:oa(!1),disabled:oa(!0),checked:function(a){var b=a.nodeName.toLowerCase();return"input"===b&&!!a.checked||"option"===b&&!!a.selected},selected:function(a){return a.parentNode&&a.parentNode.selectedIndex,a.selected===!0},empty:function(a){for(a=a.firstChild;a;a=a.nextSibling)if(a.nodeType<6)return!1;return!0},parent:function(a){return!d.pseudos.empty(a)},header:function(a){return X.test(a.nodeName)},input:function(a){return W.test(a.nodeName)},button:function(a){var b=a.nodeName.toLowerCase();return"input"===b&&"button"===a.type||"button"===b},text:function(a){var b;return"input"===a.nodeName.toLowerCase()&&"text"===a.type&&(null==(b=a.getAttribute("type"))||"text"===b.toLowerCase())},first:pa(function(){return[0]}),last:pa(function(a,b){return[b-1]}),eq:pa(function(a,b,c){return[c<0?c+b:c]}),even:pa(function(a,b){for(var c=0;c=0;)a.push(d);return a}),gt:pa(function(a,b,c){for(var d=c<0?c+b:c;++d1?function(b,c,d){var e=a.length;while(e--)if(!a[e](b,c,d))return!1;return!0}:a[0]}function va(a,b,c){for(var d=0,e=b.length;d-1&&(f[j]=!(g[j]=l))}}else r=wa(r===g?r.splice(o,r.length):r),e?e(null,g,r,i):G.apply(g,r)})}function ya(a){for(var b,c,e,f=a.length,g=d.relative[a[0].type],h=g||d.relative[" "],i=g?1:0,k=ta(function(a){return a===b},h,!0),l=ta(function(a){return I(b,a)>-1},h,!0),m=[function(a,c,d){var e=!g&&(d||c!==j)||((b=c).nodeType?k(a,c,d):l(a,c,d));return b=null,e}];i1&&ua(m),i>1&&sa(a.slice(0,i-1).concat({value:" "===a[i-2].type?"*":""})).replace(P,"$1"),c,i0,e=a.length>0,f=function(f,g,h,i,k){var l,o,q,r=0,s="0",t=f&&[],u=[],v=j,x=f||e&&d.find.TAG("*",k),y=w+=null==v?1:Math.random()||.1,z=x.length;for(k&&(j=g===n||g||k);s!==z&&null!=(l=x[s]);s++){if(e&&l){o=0,g||l.ownerDocument===n||(m(l),h=!p);while(q=a[o++])if(q(l,g||n,h)){i.push(l);break}k&&(w=y)}c&&((l=!q&&l)&&r--,f&&t.push(l))}if(r+=s,c&&s!==r){o=0;while(q=b[o++])q(t,u,g,h);if(f){if(r>0)while(s--)t[s]||u[s]||(u[s]=E.call(i));u=wa(u)}G.apply(i,u),k&&!f&&u.length>0&&r+b.length>1&&ga.uniqueSort(i)}return k&&(w=y,j=v),t};return c?ia(f):f}return h=ga.compile=function(a,b){var c,d=[],e=[],f=A[a+" "];if(!f){b||(b=g(a)),c=b.length;while(c--)f=ya(b[c]),f[u]?d.push(f):e.push(f);f=A(a,za(e,d)),f.selector=a}return f},i=ga.select=function(a,b,c,e){var f,i,j,k,l,m="function"==typeof a&&a,n=!e&&g(a=m.selector||a);if(c=c||[],1===n.length){if(i=n[0]=n[0].slice(0),i.length>2&&"ID"===(j=i[0]).type&&9===b.nodeType&&p&&d.relative[i[1].type]){if(b=(d.find.ID(j.matches[0].replace(_,aa),b)||[])[0],!b)return c;m&&(b=b.parentNode),a=a.slice(i.shift().value.length)}f=V.needsContext.test(a)?0:i.length;while(f--){if(j=i[f],d.relative[k=j.type])break;if((l=d.find[k])&&(e=l(j.matches[0].replace(_,aa),$.test(i[0].type)&&qa(b.parentNode)||b))){if(i.splice(f,1),a=e.length&&sa(i),!a)return G.apply(c,e),c;break}}}return(m||h(a,n))(e,b,!p,c,!b||$.test(a)&&qa(b.parentNode)||b),c},c.sortStable=u.split("").sort(B).join("")===u,c.detectDuplicates=!!l,m(),c.sortDetached=ja(function(a){return 1&a.compareDocumentPosition(n.createElement("fieldset"))}),ja(function(a){return a.innerHTML="","#"===a.firstChild.getAttribute("href")})||ka("type|href|height|width",function(a,b,c){if(!c)return a.getAttribute(b,"type"===b.toLowerCase()?1:2)}),c.attributes&&ja(function(a){return a.innerHTML="",a.firstChild.setAttribute("value",""),""===a.firstChild.getAttribute("value")})||ka("value",function(a,b,c){if(!c&&"input"===a.nodeName.toLowerCase())return a.defaultValue}),ja(function(a){return null==a.getAttribute("disabled")})||ka(J,function(a,b,c){var d;if(!c)return a[b]===!0?b.toLowerCase():(d=a.getAttributeNode(b))&&d.specified?d.value:null}),ga}(a);r.find=x,r.expr=x.selectors,r.expr[":"]=r.expr.pseudos,r.uniqueSort=r.unique=x.uniqueSort,r.text=x.getText,r.isXMLDoc=x.isXML,r.contains=x.contains,r.escapeSelector=x.escape;var y=function(a,b,c){var d=[],e=void 0!==c;while((a=a[b])&&9!==a.nodeType)if(1===a.nodeType){if(e&&r(a).is(c))break;d.push(a)}return d},z=function(a,b){for(var c=[];a;a=a.nextSibling)1===a.nodeType&&a!==b&&c.push(a);return c},A=r.expr.match.needsContext;function B(a,b){return a.nodeName&&a.nodeName.toLowerCase()===b.toLowerCase()}var C=/^<([a-z][^\/\0>:\x20\t\r\n\f]*)[\x20\t\r\n\f]*\/?>(?:<\/\1>|)$/i,D=/^.[^:#\[\.,]*$/;function E(a,b,c){return r.isFunction(b)?r.grep(a,function(a,d){return!!b.call(a,d,a)!==c}):b.nodeType?r.grep(a,function(a){return a===b!==c}):"string"!=typeof b?r.grep(a,function(a){return i.call(b,a)>-1!==c}):D.test(b)?r.filter(b,a,c):(b=r.filter(b,a),r.grep(a,function(a){return i.call(b,a)>-1!==c&&1===a.nodeType}))}r.filter=function(a,b,c){var d=b[0];return c&&(a=":not("+a+")"),1===b.length&&1===d.nodeType?r.find.matchesSelector(d,a)?[d]:[]:r.find.matches(a,r.grep(b,function(a){return 1===a.nodeType}))},r.fn.extend({find:function(a){var b,c,d=this.length,e=this;if("string"!=typeof a)return this.pushStack(r(a).filter(function(){for(b=0;b1?r.uniqueSort(c):c},filter:function(a){return this.pushStack(E(this,a||[],!1))},not:function(a){return this.pushStack(E(this,a||[],!0))},is:function(a){return!!E(this,"string"==typeof a&&A.test(a)?r(a):a||[],!1).length}});var F,G=/^(?:\s*(<[\w\W]+>)[^>]*|#([\w-]+))$/,H=r.fn.init=function(a,b,c){var e,f;if(!a)return this;if(c=c||F,"string"==typeof a){if(e="<"===a[0]&&">"===a[a.length-1]&&a.length>=3?[null,a,null]:G.exec(a),!e||!e[1]&&b)return!b||b.jquery?(b||c).find(a):this.constructor(b).find(a);if(e[1]){if(b=b instanceof r?b[0]:b,r.merge(this,r.parseHTML(e[1],b&&b.nodeType?b.ownerDocument||b:d,!0)),C.test(e[1])&&r.isPlainObject(b))for(e in b)r.isFunction(this[e])?this[e](b[e]):this.attr(e,b[e]);return this}return f=d.getElementById(e[2]),f&&(this[0]=f,this.length=1),this}return a.nodeType?(this[0]=a,this.length=1,this):r.isFunction(a)?void 0!==c.ready?c.ready(a):a(r):r.makeArray(a,this)};H.prototype=r.fn,F=r(d);var I=/^(?:parents|prev(?:Until|All))/,J={children:!0,contents:!0,next:!0,prev:!0};r.fn.extend({has:function(a){var b=r(a,this),c=b.length;return this.filter(function(){for(var a=0;a-1:1===c.nodeType&&r.find.matchesSelector(c,a))){f.push(c);break}return this.pushStack(f.length>1?r.uniqueSort(f):f)},index:function(a){return a?"string"==typeof a?i.call(r(a),this[0]):i.call(this,a.jquery?a[0]:a):this[0]&&this[0].parentNode?this.first().prevAll().length:-1},add:function(a,b){return this.pushStack(r.uniqueSort(r.merge(this.get(),r(a,b))))},addBack:function(a){return this.add(null==a?this.prevObject:this.prevObject.filter(a))}});function K(a,b){while((a=a[b])&&1!==a.nodeType);return a}r.each({parent:function(a){var b=a.parentNode;return b&&11!==b.nodeType?b:null},parents:function(a){return y(a,"parentNode")},parentsUntil:function(a,b,c){return y(a,"parentNode",c)},next:function(a){return K(a,"nextSibling")},prev:function(a){return K(a,"previousSibling")},nextAll:function(a){return y(a,"nextSibling")},prevAll:function(a){return y(a,"previousSibling")},nextUntil:function(a,b,c){return y(a,"nextSibling",c)},prevUntil:function(a,b,c){return y(a,"previousSibling",c)},siblings:function(a){return z((a.parentNode||{}).firstChild,a)},children:function(a){return z(a.firstChild)},contents:function(a){return B(a,"iframe")?a.contentDocument:(B(a,"template")&&(a=a.content||a),r.merge([],a.childNodes))}},function(a,b){r.fn[a]=function(c,d){var e=r.map(this,b,c);return"Until"!==a.slice(-5)&&(d=c),d&&"string"==typeof d&&(e=r.filter(d,e)),this.length>1&&(J[a]||r.uniqueSort(e),I.test(a)&&e.reverse()),this.pushStack(e)}});var L=/[^\x20\t\r\n\f]+/g;function M(a){var b={};return r.each(a.match(L)||[],function(a,c){b[c]=!0}),b}r.Callbacks=function(a){a="string"==typeof a?M(a):r.extend({},a);var b,c,d,e,f=[],g=[],h=-1,i=function(){for(e=e||a.once,d=b=!0;g.length;h=-1){c=g.shift();while(++h-1)f.splice(c,1),c<=h&&h--}),this},has:function(a){return a?r.inArray(a,f)>-1:f.length>0},empty:function(){return f&&(f=[]),this},disable:function(){return e=g=[],f=c="",this},disabled:function(){return!f},lock:function(){return e=g=[],c||b||(f=c=""),this},locked:function(){return!!e},fireWith:function(a,c){return e||(c=c||[],c=[a,c.slice?c.slice():c],g.push(c),b||i()),this},fire:function(){return j.fireWith(this,arguments),this},fired:function(){return!!d}};return j};function N(a){return a}function O(a){throw a}function P(a,b,c,d){var e;try{a&&r.isFunction(e=a.promise)?e.call(a).done(b).fail(c):a&&r.isFunction(e=a.then)?e.call(a,b,c):b.apply(void 0,[a].slice(d))}catch(a){c.apply(void 0,[a])}}r.extend({Deferred:function(b){var c=[["notify","progress",r.Callbacks("memory"),r.Callbacks("memory"),2],["resolve","done",r.Callbacks("once memory"),r.Callbacks("once memory"),0,"resolved"],["reject","fail",r.Callbacks("once memory"),r.Callbacks("once memory"),1,"rejected"]],d="pending",e={state:function(){return d},always:function(){return f.done(arguments).fail(arguments),this},"catch":function(a){return e.then(null,a)},pipe:function(){var a=arguments;return r.Deferred(function(b){r.each(c,function(c,d){var e=r.isFunction(a[d[4]])&&a[d[4]];f[d[1]](function(){var a=e&&e.apply(this,arguments);a&&r.isFunction(a.promise)?a.promise().progress(b.notify).done(b.resolve).fail(b.reject):b[d[0]+"With"](this,e?[a]:arguments)})}),a=null}).promise()},then:function(b,d,e){var f=0;function g(b,c,d,e){return function(){var h=this,i=arguments,j=function(){var a,j;if(!(b=f&&(d!==O&&(h=void 0,i=[a]),c.rejectWith(h,i))}};b?k():(r.Deferred.getStackHook&&(k.stackTrace=r.Deferred.getStackHook()),a.setTimeout(k))}}return r.Deferred(function(a){c[0][3].add(g(0,a,r.isFunction(e)?e:N,a.notifyWith)),c[1][3].add(g(0,a,r.isFunction(b)?b:N)),c[2][3].add(g(0,a,r.isFunction(d)?d:O))}).promise()},promise:function(a){return null!=a?r.extend(a,e):e}},f={};return r.each(c,function(a,b){var g=b[2],h=b[5];e[b[1]]=g.add,h&&g.add(function(){d=h},c[3-a][2].disable,c[0][2].lock),g.add(b[3].fire),f[b[0]]=function(){return f[b[0]+"With"](this===f?void 0:this,arguments),this},f[b[0]+"With"]=g.fireWith}),e.promise(f),b&&b.call(f,f),f},when:function(a){var b=arguments.length,c=b,d=Array(c),e=f.call(arguments),g=r.Deferred(),h=function(a){return function(c){d[a]=this,e[a]=arguments.length>1?f.call(arguments):c,--b||g.resolveWith(d,e)}};if(b<=1&&(P(a,g.done(h(c)).resolve,g.reject,!b),"pending"===g.state()||r.isFunction(e[c]&&e[c].then)))return g.then();while(c--)P(e[c],h(c),g.reject);return g.promise()}});var Q=/^(Eval|Internal|Range|Reference|Syntax|Type|URI)Error$/;r.Deferred.exceptionHook=function(b,c){a.console&&a.console.warn&&b&&Q.test(b.name)&&a.console.warn("jQuery.Deferred exception: "+b.message,b.stack,c)},r.readyException=function(b){a.setTimeout(function(){throw b})};var R=r.Deferred();r.fn.ready=function(a){return R.then(a)["catch"](function(a){r.readyException(a)}),this},r.extend({isReady:!1,readyWait:1,ready:function(a){(a===!0?--r.readyWait:r.isReady)||(r.isReady=!0,a!==!0&&--r.readyWait>0||R.resolveWith(d,[r]))}}),r.ready.then=R.then;function S(){d.removeEventListener("DOMContentLoaded",S), +a.removeEventListener("load",S),r.ready()}"complete"===d.readyState||"loading"!==d.readyState&&!d.documentElement.doScroll?a.setTimeout(r.ready):(d.addEventListener("DOMContentLoaded",S),a.addEventListener("load",S));var T=function(a,b,c,d,e,f,g){var h=0,i=a.length,j=null==c;if("object"===r.type(c)){e=!0;for(h in c)T(a,b,h,c[h],!0,f,g)}else if(void 0!==d&&(e=!0,r.isFunction(d)||(g=!0),j&&(g?(b.call(a,d),b=null):(j=b,b=function(a,b,c){return j.call(r(a),c)})),b))for(;h1,null,!0)},removeData:function(a){return this.each(function(){X.remove(this,a)})}}),r.extend({queue:function(a,b,c){var d;if(a)return b=(b||"fx")+"queue",d=W.get(a,b),c&&(!d||Array.isArray(c)?d=W.access(a,b,r.makeArray(c)):d.push(c)),d||[]},dequeue:function(a,b){b=b||"fx";var c=r.queue(a,b),d=c.length,e=c.shift(),f=r._queueHooks(a,b),g=function(){r.dequeue(a,b)};"inprogress"===e&&(e=c.shift(),d--),e&&("fx"===b&&c.unshift("inprogress"),delete f.stop,e.call(a,g,f)),!d&&f&&f.empty.fire()},_queueHooks:function(a,b){var c=b+"queueHooks";return W.get(a,c)||W.access(a,c,{empty:r.Callbacks("once memory").add(function(){W.remove(a,[b+"queue",c])})})}}),r.fn.extend({queue:function(a,b){var c=2;return"string"!=typeof a&&(b=a,a="fx",c--),arguments.length\x20\t\r\n\f]+)/i,la=/^$|\/(?:java|ecma)script/i,ma={option:[1,""],thead:[1,"","
      "],col:[2,"","
      "],tr:[2,"","
      "],td:[3,"","
      "],_default:[0,"",""]};ma.optgroup=ma.option,ma.tbody=ma.tfoot=ma.colgroup=ma.caption=ma.thead,ma.th=ma.td;function na(a,b){var c;return c="undefined"!=typeof a.getElementsByTagName?a.getElementsByTagName(b||"*"):"undefined"!=typeof a.querySelectorAll?a.querySelectorAll(b||"*"):[],void 0===b||b&&B(a,b)?r.merge([a],c):c}function oa(a,b){for(var c=0,d=a.length;c-1)e&&e.push(f);else if(j=r.contains(f.ownerDocument,f),g=na(l.appendChild(f),"script"),j&&oa(g),c){k=0;while(f=g[k++])la.test(f.type||"")&&c.push(f)}return l}!function(){var a=d.createDocumentFragment(),b=a.appendChild(d.createElement("div")),c=d.createElement("input");c.setAttribute("type","radio"),c.setAttribute("checked","checked"),c.setAttribute("name","t"),b.appendChild(c),o.checkClone=b.cloneNode(!0).cloneNode(!0).lastChild.checked,b.innerHTML="",o.noCloneChecked=!!b.cloneNode(!0).lastChild.defaultValue}();var ra=d.documentElement,sa=/^key/,ta=/^(?:mouse|pointer|contextmenu|drag|drop)|click/,ua=/^([^.]*)(?:\.(.+)|)/;function va(){return!0}function wa(){return!1}function xa(){try{return d.activeElement}catch(a){}}function ya(a,b,c,d,e,f){var g,h;if("object"==typeof b){"string"!=typeof c&&(d=d||c,c=void 0);for(h in b)ya(a,h,c,d,b[h],f);return a}if(null==d&&null==e?(e=c,d=c=void 0):null==e&&("string"==typeof c?(e=d,d=void 0):(e=d,d=c,c=void 0)),e===!1)e=wa;else if(!e)return a;return 1===f&&(g=e,e=function(a){return r().off(a),g.apply(this,arguments)},e.guid=g.guid||(g.guid=r.guid++)),a.each(function(){r.event.add(this,b,e,d,c)})}r.event={global:{},add:function(a,b,c,d,e){var f,g,h,i,j,k,l,m,n,o,p,q=W.get(a);if(q){c.handler&&(f=c,c=f.handler,e=f.selector),e&&r.find.matchesSelector(ra,e),c.guid||(c.guid=r.guid++),(i=q.events)||(i=q.events={}),(g=q.handle)||(g=q.handle=function(b){return"undefined"!=typeof r&&r.event.triggered!==b.type?r.event.dispatch.apply(a,arguments):void 0}),b=(b||"").match(L)||[""],j=b.length;while(j--)h=ua.exec(b[j])||[],n=p=h[1],o=(h[2]||"").split(".").sort(),n&&(l=r.event.special[n]||{},n=(e?l.delegateType:l.bindType)||n,l=r.event.special[n]||{},k=r.extend({type:n,origType:p,data:d,handler:c,guid:c.guid,selector:e,needsContext:e&&r.expr.match.needsContext.test(e),namespace:o.join(".")},f),(m=i[n])||(m=i[n]=[],m.delegateCount=0,l.setup&&l.setup.call(a,d,o,g)!==!1||a.addEventListener&&a.addEventListener(n,g)),l.add&&(l.add.call(a,k),k.handler.guid||(k.handler.guid=c.guid)),e?m.splice(m.delegateCount++,0,k):m.push(k),r.event.global[n]=!0)}},remove:function(a,b,c,d,e){var f,g,h,i,j,k,l,m,n,o,p,q=W.hasData(a)&&W.get(a);if(q&&(i=q.events)){b=(b||"").match(L)||[""],j=b.length;while(j--)if(h=ua.exec(b[j])||[],n=p=h[1],o=(h[2]||"").split(".").sort(),n){l=r.event.special[n]||{},n=(d?l.delegateType:l.bindType)||n,m=i[n]||[],h=h[2]&&new RegExp("(^|\\.)"+o.join("\\.(?:.*\\.|)")+"(\\.|$)"),g=f=m.length;while(f--)k=m[f],!e&&p!==k.origType||c&&c.guid!==k.guid||h&&!h.test(k.namespace)||d&&d!==k.selector&&("**"!==d||!k.selector)||(m.splice(f,1),k.selector&&m.delegateCount--,l.remove&&l.remove.call(a,k));g&&!m.length&&(l.teardown&&l.teardown.call(a,o,q.handle)!==!1||r.removeEvent(a,n,q.handle),delete i[n])}else for(n in i)r.event.remove(a,n+b[j],c,d,!0);r.isEmptyObject(i)&&W.remove(a,"handle events")}},dispatch:function(a){var b=r.event.fix(a),c,d,e,f,g,h,i=new Array(arguments.length),j=(W.get(this,"events")||{})[b.type]||[],k=r.event.special[b.type]||{};for(i[0]=b,c=1;c=1))for(;j!==this;j=j.parentNode||this)if(1===j.nodeType&&("click"!==a.type||j.disabled!==!0)){for(f=[],g={},c=0;c-1:r.find(e,this,null,[j]).length),g[e]&&f.push(d);f.length&&h.push({elem:j,handlers:f})}return j=this,i\x20\t\r\n\f]*)[^>]*)\/>/gi,Aa=/\s*$/g;function Ea(a,b){return B(a,"table")&&B(11!==b.nodeType?b:b.firstChild,"tr")?r(">tbody",a)[0]||a:a}function Fa(a){return a.type=(null!==a.getAttribute("type"))+"/"+a.type,a}function Ga(a){var b=Ca.exec(a.type);return b?a.type=b[1]:a.removeAttribute("type"),a}function Ha(a,b){var c,d,e,f,g,h,i,j;if(1===b.nodeType){if(W.hasData(a)&&(f=W.access(a),g=W.set(b,f),j=f.events)){delete g.handle,g.events={};for(e in j)for(c=0,d=j[e].length;c1&&"string"==typeof q&&!o.checkClone&&Ba.test(q))return a.each(function(e){var f=a.eq(e);s&&(b[0]=q.call(this,e,f.html())),Ja(f,b,c,d)});if(m&&(e=qa(b,a[0].ownerDocument,!1,a,d),f=e.firstChild,1===e.childNodes.length&&(e=f),f||d)){for(h=r.map(na(e,"script"),Fa),i=h.length;l")},clone:function(a,b,c){var d,e,f,g,h=a.cloneNode(!0),i=r.contains(a.ownerDocument,a);if(!(o.noCloneChecked||1!==a.nodeType&&11!==a.nodeType||r.isXMLDoc(a)))for(g=na(h),f=na(a),d=0,e=f.length;d0&&oa(g,!i&&na(a,"script")),h},cleanData:function(a){for(var b,c,d,e=r.event.special,f=0;void 0!==(c=a[f]);f++)if(U(c)){if(b=c[W.expando]){if(b.events)for(d in b.events)e[d]?r.event.remove(c,d):r.removeEvent(c,d,b.handle);c[W.expando]=void 0}c[X.expando]&&(c[X.expando]=void 0)}}}),r.fn.extend({detach:function(a){return Ka(this,a,!0)},remove:function(a){return Ka(this,a)},text:function(a){return T(this,function(a){return void 0===a?r.text(this):this.empty().each(function(){1!==this.nodeType&&11!==this.nodeType&&9!==this.nodeType||(this.textContent=a)})},null,a,arguments.length)},append:function(){return Ja(this,arguments,function(a){if(1===this.nodeType||11===this.nodeType||9===this.nodeType){var b=Ea(this,a);b.appendChild(a)}})},prepend:function(){return Ja(this,arguments,function(a){if(1===this.nodeType||11===this.nodeType||9===this.nodeType){var b=Ea(this,a);b.insertBefore(a,b.firstChild)}})},before:function(){return Ja(this,arguments,function(a){this.parentNode&&this.parentNode.insertBefore(a,this)})},after:function(){return Ja(this,arguments,function(a){this.parentNode&&this.parentNode.insertBefore(a,this.nextSibling)})},empty:function(){for(var a,b=0;null!=(a=this[b]);b++)1===a.nodeType&&(r.cleanData(na(a,!1)),a.textContent="");return this},clone:function(a,b){return a=null!=a&&a,b=null==b?a:b,this.map(function(){return r.clone(this,a,b)})},html:function(a){return T(this,function(a){var b=this[0]||{},c=0,d=this.length;if(void 0===a&&1===b.nodeType)return b.innerHTML;if("string"==typeof a&&!Aa.test(a)&&!ma[(ka.exec(a)||["",""])[1].toLowerCase()]){a=r.htmlPrefilter(a);try{for(;c1)}});function _a(a,b,c,d,e){return new _a.prototype.init(a,b,c,d,e)}r.Tween=_a,_a.prototype={constructor:_a,init:function(a,b,c,d,e,f){this.elem=a,this.prop=c,this.easing=e||r.easing._default,this.options=b,this.start=this.now=this.cur(),this.end=d,this.unit=f||(r.cssNumber[c]?"":"px")},cur:function(){var a=_a.propHooks[this.prop];return a&&a.get?a.get(this):_a.propHooks._default.get(this)},run:function(a){var b,c=_a.propHooks[this.prop];return this.options.duration?this.pos=b=r.easing[this.easing](a,this.options.duration*a,0,1,this.options.duration):this.pos=b=a,this.now=(this.end-this.start)*b+this.start,this.options.step&&this.options.step.call(this.elem,this.now,this),c&&c.set?c.set(this):_a.propHooks._default.set(this),this}},_a.prototype.init.prototype=_a.prototype,_a.propHooks={_default:{get:function(a){var b;return 1!==a.elem.nodeType||null!=a.elem[a.prop]&&null==a.elem.style[a.prop]?a.elem[a.prop]:(b=r.css(a.elem,a.prop,""),b&&"auto"!==b?b:0)},set:function(a){r.fx.step[a.prop]?r.fx.step[a.prop](a):1!==a.elem.nodeType||null==a.elem.style[r.cssProps[a.prop]]&&!r.cssHooks[a.prop]?a.elem[a.prop]=a.now:r.style(a.elem,a.prop,a.now+a.unit)}}},_a.propHooks.scrollTop=_a.propHooks.scrollLeft={set:function(a){a.elem.nodeType&&a.elem.parentNode&&(a.elem[a.prop]=a.now)}},r.easing={linear:function(a){return a},swing:function(a){return.5-Math.cos(a*Math.PI)/2},_default:"swing"},r.fx=_a.prototype.init,r.fx.step={};var ab,bb,cb=/^(?:toggle|show|hide)$/,db=/queueHooks$/;function eb(){bb&&(d.hidden===!1&&a.requestAnimationFrame?a.requestAnimationFrame(eb):a.setTimeout(eb,r.fx.interval),r.fx.tick())}function fb(){return a.setTimeout(function(){ab=void 0}),ab=r.now()}function gb(a,b){var c,d=0,e={height:a};for(b=b?1:0;d<4;d+=2-b)c=ca[d],e["margin"+c]=e["padding"+c]=a;return b&&(e.opacity=e.width=a),e}function hb(a,b,c){for(var d,e=(kb.tweeners[b]||[]).concat(kb.tweeners["*"]),f=0,g=e.length;f1)},removeAttr:function(a){return this.each(function(){r.removeAttr(this,a)})}}),r.extend({attr:function(a,b,c){var d,e,f=a.nodeType;if(3!==f&&8!==f&&2!==f)return"undefined"==typeof a.getAttribute?r.prop(a,b,c):(1===f&&r.isXMLDoc(a)||(e=r.attrHooks[b.toLowerCase()]||(r.expr.match.bool.test(b)?lb:void 0)),void 0!==c?null===c?void r.removeAttr(a,b):e&&"set"in e&&void 0!==(d=e.set(a,c,b))?d:(a.setAttribute(b,c+""),c):e&&"get"in e&&null!==(d=e.get(a,b))?d:(d=r.find.attr(a,b), +null==d?void 0:d))},attrHooks:{type:{set:function(a,b){if(!o.radioValue&&"radio"===b&&B(a,"input")){var c=a.value;return a.setAttribute("type",b),c&&(a.value=c),b}}}},removeAttr:function(a,b){var c,d=0,e=b&&b.match(L);if(e&&1===a.nodeType)while(c=e[d++])a.removeAttribute(c)}}),lb={set:function(a,b,c){return b===!1?r.removeAttr(a,c):a.setAttribute(c,c),c}},r.each(r.expr.match.bool.source.match(/\w+/g),function(a,b){var c=mb[b]||r.find.attr;mb[b]=function(a,b,d){var e,f,g=b.toLowerCase();return d||(f=mb[g],mb[g]=e,e=null!=c(a,b,d)?g:null,mb[g]=f),e}});var nb=/^(?:input|select|textarea|button)$/i,ob=/^(?:a|area)$/i;r.fn.extend({prop:function(a,b){return T(this,r.prop,a,b,arguments.length>1)},removeProp:function(a){return this.each(function(){delete this[r.propFix[a]||a]})}}),r.extend({prop:function(a,b,c){var d,e,f=a.nodeType;if(3!==f&&8!==f&&2!==f)return 1===f&&r.isXMLDoc(a)||(b=r.propFix[b]||b,e=r.propHooks[b]),void 0!==c?e&&"set"in e&&void 0!==(d=e.set(a,c,b))?d:a[b]=c:e&&"get"in e&&null!==(d=e.get(a,b))?d:a[b]},propHooks:{tabIndex:{get:function(a){var b=r.find.attr(a,"tabindex");return b?parseInt(b,10):nb.test(a.nodeName)||ob.test(a.nodeName)&&a.href?0:-1}}},propFix:{"for":"htmlFor","class":"className"}}),o.optSelected||(r.propHooks.selected={get:function(a){var b=a.parentNode;return b&&b.parentNode&&b.parentNode.selectedIndex,null},set:function(a){var b=a.parentNode;b&&(b.selectedIndex,b.parentNode&&b.parentNode.selectedIndex)}}),r.each(["tabIndex","readOnly","maxLength","cellSpacing","cellPadding","rowSpan","colSpan","useMap","frameBorder","contentEditable"],function(){r.propFix[this.toLowerCase()]=this});function pb(a){var b=a.match(L)||[];return b.join(" ")}function qb(a){return a.getAttribute&&a.getAttribute("class")||""}r.fn.extend({addClass:function(a){var b,c,d,e,f,g,h,i=0;if(r.isFunction(a))return this.each(function(b){r(this).addClass(a.call(this,b,qb(this)))});if("string"==typeof a&&a){b=a.match(L)||[];while(c=this[i++])if(e=qb(c),d=1===c.nodeType&&" "+pb(e)+" "){g=0;while(f=b[g++])d.indexOf(" "+f+" ")<0&&(d+=f+" ");h=pb(d),e!==h&&c.setAttribute("class",h)}}return this},removeClass:function(a){var b,c,d,e,f,g,h,i=0;if(r.isFunction(a))return this.each(function(b){r(this).removeClass(a.call(this,b,qb(this)))});if(!arguments.length)return this.attr("class","");if("string"==typeof a&&a){b=a.match(L)||[];while(c=this[i++])if(e=qb(c),d=1===c.nodeType&&" "+pb(e)+" "){g=0;while(f=b[g++])while(d.indexOf(" "+f+" ")>-1)d=d.replace(" "+f+" "," ");h=pb(d),e!==h&&c.setAttribute("class",h)}}return this},toggleClass:function(a,b){var c=typeof a;return"boolean"==typeof b&&"string"===c?b?this.addClass(a):this.removeClass(a):r.isFunction(a)?this.each(function(c){r(this).toggleClass(a.call(this,c,qb(this),b),b)}):this.each(function(){var b,d,e,f;if("string"===c){d=0,e=r(this),f=a.match(L)||[];while(b=f[d++])e.hasClass(b)?e.removeClass(b):e.addClass(b)}else void 0!==a&&"boolean"!==c||(b=qb(this),b&&W.set(this,"__className__",b),this.setAttribute&&this.setAttribute("class",b||a===!1?"":W.get(this,"__className__")||""))})},hasClass:function(a){var b,c,d=0;b=" "+a+" ";while(c=this[d++])if(1===c.nodeType&&(" "+pb(qb(c))+" ").indexOf(b)>-1)return!0;return!1}});var rb=/\r/g;r.fn.extend({val:function(a){var b,c,d,e=this[0];{if(arguments.length)return d=r.isFunction(a),this.each(function(c){var e;1===this.nodeType&&(e=d?a.call(this,c,r(this).val()):a,null==e?e="":"number"==typeof e?e+="":Array.isArray(e)&&(e=r.map(e,function(a){return null==a?"":a+""})),b=r.valHooks[this.type]||r.valHooks[this.nodeName.toLowerCase()],b&&"set"in b&&void 0!==b.set(this,e,"value")||(this.value=e))});if(e)return b=r.valHooks[e.type]||r.valHooks[e.nodeName.toLowerCase()],b&&"get"in b&&void 0!==(c=b.get(e,"value"))?c:(c=e.value,"string"==typeof c?c.replace(rb,""):null==c?"":c)}}}),r.extend({valHooks:{option:{get:function(a){var b=r.find.attr(a,"value");return null!=b?b:pb(r.text(a))}},select:{get:function(a){var b,c,d,e=a.options,f=a.selectedIndex,g="select-one"===a.type,h=g?null:[],i=g?f+1:e.length;for(d=f<0?i:g?f:0;d-1)&&(c=!0);return c||(a.selectedIndex=-1),f}}}}),r.each(["radio","checkbox"],function(){r.valHooks[this]={set:function(a,b){if(Array.isArray(b))return a.checked=r.inArray(r(a).val(),b)>-1}},o.checkOn||(r.valHooks[this].get=function(a){return null===a.getAttribute("value")?"on":a.value})});var sb=/^(?:focusinfocus|focusoutblur)$/;r.extend(r.event,{trigger:function(b,c,e,f){var g,h,i,j,k,m,n,o=[e||d],p=l.call(b,"type")?b.type:b,q=l.call(b,"namespace")?b.namespace.split("."):[];if(h=i=e=e||d,3!==e.nodeType&&8!==e.nodeType&&!sb.test(p+r.event.triggered)&&(p.indexOf(".")>-1&&(q=p.split("."),p=q.shift(),q.sort()),k=p.indexOf(":")<0&&"on"+p,b=b[r.expando]?b:new r.Event(p,"object"==typeof b&&b),b.isTrigger=f?2:3,b.namespace=q.join("."),b.rnamespace=b.namespace?new RegExp("(^|\\.)"+q.join("\\.(?:.*\\.|)")+"(\\.|$)"):null,b.result=void 0,b.target||(b.target=e),c=null==c?[b]:r.makeArray(c,[b]),n=r.event.special[p]||{},f||!n.trigger||n.trigger.apply(e,c)!==!1)){if(!f&&!n.noBubble&&!r.isWindow(e)){for(j=n.delegateType||p,sb.test(j+p)||(h=h.parentNode);h;h=h.parentNode)o.push(h),i=h;i===(e.ownerDocument||d)&&o.push(i.defaultView||i.parentWindow||a)}g=0;while((h=o[g++])&&!b.isPropagationStopped())b.type=g>1?j:n.bindType||p,m=(W.get(h,"events")||{})[b.type]&&W.get(h,"handle"),m&&m.apply(h,c),m=k&&h[k],m&&m.apply&&U(h)&&(b.result=m.apply(h,c),b.result===!1&&b.preventDefault());return b.type=p,f||b.isDefaultPrevented()||n._default&&n._default.apply(o.pop(),c)!==!1||!U(e)||k&&r.isFunction(e[p])&&!r.isWindow(e)&&(i=e[k],i&&(e[k]=null),r.event.triggered=p,e[p](),r.event.triggered=void 0,i&&(e[k]=i)),b.result}},simulate:function(a,b,c){var d=r.extend(new r.Event,c,{type:a,isSimulated:!0});r.event.trigger(d,null,b)}}),r.fn.extend({trigger:function(a,b){return this.each(function(){r.event.trigger(a,b,this)})},triggerHandler:function(a,b){var c=this[0];if(c)return r.event.trigger(a,b,c,!0)}}),r.each("blur focus focusin focusout resize scroll click dblclick mousedown mouseup mousemove mouseover mouseout mouseenter mouseleave change select submit keydown keypress keyup contextmenu".split(" "),function(a,b){r.fn[b]=function(a,c){return arguments.length>0?this.on(b,null,a,c):this.trigger(b)}}),r.fn.extend({hover:function(a,b){return this.mouseenter(a).mouseleave(b||a)}}),o.focusin="onfocusin"in a,o.focusin||r.each({focus:"focusin",blur:"focusout"},function(a,b){var c=function(a){r.event.simulate(b,a.target,r.event.fix(a))};r.event.special[b]={setup:function(){var d=this.ownerDocument||this,e=W.access(d,b);e||d.addEventListener(a,c,!0),W.access(d,b,(e||0)+1)},teardown:function(){var d=this.ownerDocument||this,e=W.access(d,b)-1;e?W.access(d,b,e):(d.removeEventListener(a,c,!0),W.remove(d,b))}}});var tb=a.location,ub=r.now(),vb=/\?/;r.parseXML=function(b){var c;if(!b||"string"!=typeof b)return null;try{c=(new a.DOMParser).parseFromString(b,"text/xml")}catch(d){c=void 0}return c&&!c.getElementsByTagName("parsererror").length||r.error("Invalid XML: "+b),c};var wb=/\[\]$/,xb=/\r?\n/g,yb=/^(?:submit|button|image|reset|file)$/i,zb=/^(?:input|select|textarea|keygen)/i;function Ab(a,b,c,d){var e;if(Array.isArray(b))r.each(b,function(b,e){c||wb.test(a)?d(a,e):Ab(a+"["+("object"==typeof e&&null!=e?b:"")+"]",e,c,d)});else if(c||"object"!==r.type(b))d(a,b);else for(e in b)Ab(a+"["+e+"]",b[e],c,d)}r.param=function(a,b){var c,d=[],e=function(a,b){var c=r.isFunction(b)?b():b;d[d.length]=encodeURIComponent(a)+"="+encodeURIComponent(null==c?"":c)};if(Array.isArray(a)||a.jquery&&!r.isPlainObject(a))r.each(a,function(){e(this.name,this.value)});else for(c in a)Ab(c,a[c],b,e);return d.join("&")},r.fn.extend({serialize:function(){return r.param(this.serializeArray())},serializeArray:function(){return this.map(function(){var a=r.prop(this,"elements");return a?r.makeArray(a):this}).filter(function(){var a=this.type;return this.name&&!r(this).is(":disabled")&&zb.test(this.nodeName)&&!yb.test(a)&&(this.checked||!ja.test(a))}).map(function(a,b){var c=r(this).val();return null==c?null:Array.isArray(c)?r.map(c,function(a){return{name:b.name,value:a.replace(xb,"\r\n")}}):{name:b.name,value:c.replace(xb,"\r\n")}}).get()}});var Bb=/%20/g,Cb=/#.*$/,Db=/([?&])_=[^&]*/,Eb=/^(.*?):[ \t]*([^\r\n]*)$/gm,Fb=/^(?:about|app|app-storage|.+-extension|file|res|widget):$/,Gb=/^(?:GET|HEAD)$/,Hb=/^\/\//,Ib={},Jb={},Kb="*/".concat("*"),Lb=d.createElement("a");Lb.href=tb.href;function Mb(a){return function(b,c){"string"!=typeof b&&(c=b,b="*");var d,e=0,f=b.toLowerCase().match(L)||[];if(r.isFunction(c))while(d=f[e++])"+"===d[0]?(d=d.slice(1)||"*",(a[d]=a[d]||[]).unshift(c)):(a[d]=a[d]||[]).push(c)}}function Nb(a,b,c,d){var e={},f=a===Jb;function g(h){var i;return e[h]=!0,r.each(a[h]||[],function(a,h){var j=h(b,c,d);return"string"!=typeof j||f||e[j]?f?!(i=j):void 0:(b.dataTypes.unshift(j),g(j),!1)}),i}return g(b.dataTypes[0])||!e["*"]&&g("*")}function Ob(a,b){var c,d,e=r.ajaxSettings.flatOptions||{};for(c in b)void 0!==b[c]&&((e[c]?a:d||(d={}))[c]=b[c]);return d&&r.extend(!0,a,d),a}function Pb(a,b,c){var d,e,f,g,h=a.contents,i=a.dataTypes;while("*"===i[0])i.shift(),void 0===d&&(d=a.mimeType||b.getResponseHeader("Content-Type"));if(d)for(e in h)if(h[e]&&h[e].test(d)){i.unshift(e);break}if(i[0]in c)f=i[0];else{for(e in c){if(!i[0]||a.converters[e+" "+i[0]]){f=e;break}g||(g=e)}f=f||g}if(f)return f!==i[0]&&i.unshift(f),c[f]}function Qb(a,b,c,d){var e,f,g,h,i,j={},k=a.dataTypes.slice();if(k[1])for(g in a.converters)j[g.toLowerCase()]=a.converters[g];f=k.shift();while(f)if(a.responseFields[f]&&(c[a.responseFields[f]]=b),!i&&d&&a.dataFilter&&(b=a.dataFilter(b,a.dataType)),i=f,f=k.shift())if("*"===f)f=i;else if("*"!==i&&i!==f){if(g=j[i+" "+f]||j["* "+f],!g)for(e in j)if(h=e.split(" "),h[1]===f&&(g=j[i+" "+h[0]]||j["* "+h[0]])){g===!0?g=j[e]:j[e]!==!0&&(f=h[0],k.unshift(h[1]));break}if(g!==!0)if(g&&a["throws"])b=g(b);else try{b=g(b)}catch(l){return{state:"parsererror",error:g?l:"No conversion from "+i+" to "+f}}}return{state:"success",data:b}}r.extend({active:0,lastModified:{},etag:{},ajaxSettings:{url:tb.href,type:"GET",isLocal:Fb.test(tb.protocol),global:!0,processData:!0,async:!0,contentType:"application/x-www-form-urlencoded; charset=UTF-8",accepts:{"*":Kb,text:"text/plain",html:"text/html",xml:"application/xml, text/xml",json:"application/json, text/javascript"},contents:{xml:/\bxml\b/,html:/\bhtml/,json:/\bjson\b/},responseFields:{xml:"responseXML",text:"responseText",json:"responseJSON"},converters:{"* text":String,"text html":!0,"text json":JSON.parse,"text xml":r.parseXML},flatOptions:{url:!0,context:!0}},ajaxSetup:function(a,b){return b?Ob(Ob(a,r.ajaxSettings),b):Ob(r.ajaxSettings,a)},ajaxPrefilter:Mb(Ib),ajaxTransport:Mb(Jb),ajax:function(b,c){"object"==typeof b&&(c=b,b=void 0),c=c||{};var e,f,g,h,i,j,k,l,m,n,o=r.ajaxSetup({},c),p=o.context||o,q=o.context&&(p.nodeType||p.jquery)?r(p):r.event,s=r.Deferred(),t=r.Callbacks("once memory"),u=o.statusCode||{},v={},w={},x="canceled",y={readyState:0,getResponseHeader:function(a){var b;if(k){if(!h){h={};while(b=Eb.exec(g))h[b[1].toLowerCase()]=b[2]}b=h[a.toLowerCase()]}return null==b?null:b},getAllResponseHeaders:function(){return k?g:null},setRequestHeader:function(a,b){return null==k&&(a=w[a.toLowerCase()]=w[a.toLowerCase()]||a,v[a]=b),this},overrideMimeType:function(a){return null==k&&(o.mimeType=a),this},statusCode:function(a){var b;if(a)if(k)y.always(a[y.status]);else for(b in a)u[b]=[u[b],a[b]];return this},abort:function(a){var b=a||x;return e&&e.abort(b),A(0,b),this}};if(s.promise(y),o.url=((b||o.url||tb.href)+"").replace(Hb,tb.protocol+"//"),o.type=c.method||c.type||o.method||o.type,o.dataTypes=(o.dataType||"*").toLowerCase().match(L)||[""],null==o.crossDomain){j=d.createElement("a");try{j.href=o.url,j.href=j.href,o.crossDomain=Lb.protocol+"//"+Lb.host!=j.protocol+"//"+j.host}catch(z){o.crossDomain=!0}}if(o.data&&o.processData&&"string"!=typeof o.data&&(o.data=r.param(o.data,o.traditional)),Nb(Ib,o,c,y),k)return y;l=r.event&&o.global,l&&0===r.active++&&r.event.trigger("ajaxStart"),o.type=o.type.toUpperCase(),o.hasContent=!Gb.test(o.type),f=o.url.replace(Cb,""),o.hasContent?o.data&&o.processData&&0===(o.contentType||"").indexOf("application/x-www-form-urlencoded")&&(o.data=o.data.replace(Bb,"+")):(n=o.url.slice(f.length),o.data&&(f+=(vb.test(f)?"&":"?")+o.data,delete o.data),o.cache===!1&&(f=f.replace(Db,"$1"),n=(vb.test(f)?"&":"?")+"_="+ub++ +n),o.url=f+n),o.ifModified&&(r.lastModified[f]&&y.setRequestHeader("If-Modified-Since",r.lastModified[f]),r.etag[f]&&y.setRequestHeader("If-None-Match",r.etag[f])),(o.data&&o.hasContent&&o.contentType!==!1||c.contentType)&&y.setRequestHeader("Content-Type",o.contentType),y.setRequestHeader("Accept",o.dataTypes[0]&&o.accepts[o.dataTypes[0]]?o.accepts[o.dataTypes[0]]+("*"!==o.dataTypes[0]?", "+Kb+"; q=0.01":""):o.accepts["*"]);for(m in o.headers)y.setRequestHeader(m,o.headers[m]);if(o.beforeSend&&(o.beforeSend.call(p,y,o)===!1||k))return y.abort();if(x="abort",t.add(o.complete),y.done(o.success),y.fail(o.error),e=Nb(Jb,o,c,y)){if(y.readyState=1,l&&q.trigger("ajaxSend",[y,o]),k)return y;o.async&&o.timeout>0&&(i=a.setTimeout(function(){y.abort("timeout")},o.timeout));try{k=!1,e.send(v,A)}catch(z){if(k)throw z;A(-1,z)}}else A(-1,"No Transport");function A(b,c,d,h){var j,m,n,v,w,x=c;k||(k=!0,i&&a.clearTimeout(i),e=void 0,g=h||"",y.readyState=b>0?4:0,j=b>=200&&b<300||304===b,d&&(v=Pb(o,y,d)),v=Qb(o,v,y,j),j?(o.ifModified&&(w=y.getResponseHeader("Last-Modified"),w&&(r.lastModified[f]=w),w=y.getResponseHeader("etag"),w&&(r.etag[f]=w)),204===b||"HEAD"===o.type?x="nocontent":304===b?x="notmodified":(x=v.state,m=v.data,n=v.error,j=!n)):(n=x,!b&&x||(x="error",b<0&&(b=0))),y.status=b,y.statusText=(c||x)+"",j?s.resolveWith(p,[m,x,y]):s.rejectWith(p,[y,x,n]),y.statusCode(u),u=void 0,l&&q.trigger(j?"ajaxSuccess":"ajaxError",[y,o,j?m:n]),t.fireWith(p,[y,x]),l&&(q.trigger("ajaxComplete",[y,o]),--r.active||r.event.trigger("ajaxStop")))}return y},getJSON:function(a,b,c){return r.get(a,b,c,"json")},getScript:function(a,b){return r.get(a,void 0,b,"script")}}),r.each(["get","post"],function(a,b){r[b]=function(a,c,d,e){return r.isFunction(c)&&(e=e||d,d=c,c=void 0),r.ajax(r.extend({url:a,type:b,dataType:e,data:c,success:d},r.isPlainObject(a)&&a))}}),r._evalUrl=function(a){return r.ajax({url:a,type:"GET",dataType:"script",cache:!0,async:!1,global:!1,"throws":!0})},r.fn.extend({wrapAll:function(a){var b;return this[0]&&(r.isFunction(a)&&(a=a.call(this[0])),b=r(a,this[0].ownerDocument).eq(0).clone(!0),this[0].parentNode&&b.insertBefore(this[0]),b.map(function(){var a=this;while(a.firstElementChild)a=a.firstElementChild;return a}).append(this)),this},wrapInner:function(a){return r.isFunction(a)?this.each(function(b){r(this).wrapInner(a.call(this,b))}):this.each(function(){var b=r(this),c=b.contents();c.length?c.wrapAll(a):b.append(a)})},wrap:function(a){var b=r.isFunction(a);return this.each(function(c){r(this).wrapAll(b?a.call(this,c):a)})},unwrap:function(a){return this.parent(a).not("body").each(function(){r(this).replaceWith(this.childNodes)}),this}}),r.expr.pseudos.hidden=function(a){return!r.expr.pseudos.visible(a)},r.expr.pseudos.visible=function(a){return!!(a.offsetWidth||a.offsetHeight||a.getClientRects().length)},r.ajaxSettings.xhr=function(){try{return new a.XMLHttpRequest}catch(b){}};var Rb={0:200,1223:204},Sb=r.ajaxSettings.xhr();o.cors=!!Sb&&"withCredentials"in Sb,o.ajax=Sb=!!Sb,r.ajaxTransport(function(b){var c,d;if(o.cors||Sb&&!b.crossDomain)return{send:function(e,f){var g,h=b.xhr();if(h.open(b.type,b.url,b.async,b.username,b.password),b.xhrFields)for(g in b.xhrFields)h[g]=b.xhrFields[g];b.mimeType&&h.overrideMimeType&&h.overrideMimeType(b.mimeType),b.crossDomain||e["X-Requested-With"]||(e["X-Requested-With"]="XMLHttpRequest");for(g in e)h.setRequestHeader(g,e[g]);c=function(a){return function(){c&&(c=d=h.onload=h.onerror=h.onabort=h.onreadystatechange=null,"abort"===a?h.abort():"error"===a?"number"!=typeof h.status?f(0,"error"):f(h.status,h.statusText):f(Rb[h.status]||h.status,h.statusText,"text"!==(h.responseType||"text")||"string"!=typeof h.responseText?{binary:h.response}:{text:h.responseText},h.getAllResponseHeaders()))}},h.onload=c(),d=h.onerror=c("error"),void 0!==h.onabort?h.onabort=d:h.onreadystatechange=function(){4===h.readyState&&a.setTimeout(function(){c&&d()})},c=c("abort");try{h.send(b.hasContent&&b.data||null)}catch(i){if(c)throw i}},abort:function(){c&&c()}}}),r.ajaxPrefilter(function(a){a.crossDomain&&(a.contents.script=!1)}),r.ajaxSetup({accepts:{script:"text/javascript, application/javascript, application/ecmascript, application/x-ecmascript"},contents:{script:/\b(?:java|ecma)script\b/},converters:{"text script":function(a){return r.globalEval(a),a}}}),r.ajaxPrefilter("script",function(a){void 0===a.cache&&(a.cache=!1),a.crossDomain&&(a.type="GET")}),r.ajaxTransport("script",function(a){if(a.crossDomain){var b,c;return{send:function(e,f){b=r(" + + +
      + + + +
      + ++++ +endif::backend-html5[] diff --git a/spring-batch-docs/asciidoc/transaction-appendix.adoc b/spring-batch-docs/asciidoc/transaction-appendix.adoc new file mode 100644 index 0000000000..db61583bd1 --- /dev/null +++ b/spring-batch-docs/asciidoc/transaction-appendix.adoc @@ -0,0 +1,345 @@ +:batch-asciidoc: ./ +:toc: left +:toclevels: 4 + +[[transactions]] + +[appendix] +== Batch Processing and Transactions + +[[transactionsNoRetry]] +=== Simple Batching with No Retry + +Consider the following simple example of a nested batch with no retries. It shows a +common scenario for batch processing: An input source is processed until exhausted, and +we commit periodically at the end of a "chunk" of processing. + +---- + +1 | REPEAT(until=exhausted) { +| +2 | TX { +3 | REPEAT(size=5) { +3.1 | input; +3.2 | output; +| } +| } +| +| } + +---- + +The input operation (3.1) could be a message-based receive (such as from JMS), or a +file-based read, but to recover and continue processing with a chance of completing the +whole job, it must be transactional. The same applies to the operation at 3.2. It must +be either transactional or idempotent. + +If the chunk at `REPEAT` (3) fails because of a database exception at 3.2, then `TX` (2) +must roll back the whole chunk. + +[[transactionStatelessRetry]] +=== Simple Stateless Retry + +It is also useful to use a retry for an operation which is not transactional, such as a +call to a web-service or other remote resource, as shown in the following example: + +---- + +0 | TX { +1 | input; +1.1 | output; +2 | RETRY { +2.1 | remote access; +| } +| } + +---- + +This is actually one of the most useful applications of a retry, since a remote call is +much more likely to fail and be retryable than a database update. As long as the remote +access (2.1) eventually succeeds, the transaction, `TX` (0), commits. If the remote +access (2.1) eventually fails, then the transaction, `TX` (0), is guaranteed to roll +back. + +[[repeatRetry]] +=== Typical Repeat-Retry Pattern + +The most typical batch processing pattern is to add a retry to the inner block of the +chunk, as shown in the following example: + +---- + +1 | REPEAT(until=exhausted, exception=not critical) { +| +2 | TX { +3 | REPEAT(size=5) { +| +4 | RETRY(stateful, exception=deadlock loser) { +4.1 | input; +5 | } PROCESS { +5.1 | output; +6 | } SKIP and RECOVER { +| notify; +| } +| +| } +| } +| +| } + +---- + +The inner `RETRY` (4) block is marked as "stateful". See <> for a description of a stateful retry. This means that if the +retry `PROCESS` (5) block fails, the behavior of the `RETRY` (4) is as follows: + +. Throw an exception, rolling back the transaction, `TX` (2), at the chunk level, and +allowing the item to be re-presented to the input queue. +. When the item re-appears, it might be retried depending on the retry policy in place, +executing `PROCESS` (5) again. The second and subsequent attempts might fail again and +re-throw the exception. +. Eventually, the item reappears for the final time. The retry policy disallows another +attempt, so `PROCESS` (5) is never executed. In this case, we follow the `RECOVER` (6) +path, effectively "skipping" the item that was received and is being processed. + +Note that the notation used for the `RETRY` (4) in the plan above explicitly shows that +the input step (4.1) is part of the retry. It also makes clear that there are two +alternate paths for processing: the normal case, as denoted by `PROCESS` (5), and the +recovery path, as denoted in a separate block by `RECOVER` (6). The two alternate paths +are completely distinct. Only one is ever taken in normal circumstances. + +In special cases (such as a special `TranscationValidException` type), the retry policy +might be able to determine that the `RECOVER` (6) path can be taken on the last attempt +after `PROCESS` (5) has just failed, instead of waiting for the item to be re-presented. +This is not the default behavior, because it requires detailed knowledge of what has +happened inside the `PROCESS` (5) block, which is not usually available. For example, if +the output included write access before the failure, then the exception should be +re-thrown to ensure transactional integrity. + +The completion policy in the outer `REPEAT` (1) is crucial to the success of the above +plan. If the output (5.1) fails, it may throw an exception (it usually does, as +described), in which case the transaction, `TX` (2), fails, and the exception could +propagate up through the outer batch `REPEAT` (1). We do not want the whole batch to +stop, because the `RETRY` (4) might still be successful if we try again, so we add +`exception=not critical` to the outer `REPEAT` (1). + +Note, however, that if the `TX` (2) fails and we __do__ try again, by virtue of the outer +completion policy, the item that is next processed in the inner `REPEAT` (3) is not +guaranteed to be the one that just failed. It might be, but it depends on the +implementation of the input (4.1). Thus, the output (5.1) might fail again on either a +new item or the old one. The client of the batch should not assume that each `RETRY` (4) +attempt is going to process the same items as the last one that failed. For example, if +the termination policy for `REPEAT` (1) is to fail after 10 attempts, it fails after 10 +consecutive attempts but not necessarily at the same item. This is consistent with the +overall retry strategy. The inner `RETRY` (4) is aware of the history of each item and +can decide whether or not to have another attempt at it. + +[[asyncChunkProcessing]] +=== Asynchronous Chunk Processing + +The inner batches or chunks in the <> can be executed +concurrently by configuring the outer batch to use an `AsyncTaskExecutor`. The outer +batch waits for all the chunks to complete before completing. The following example shows +asynchronous chunk processing: + +---- + +1 | REPEAT(until=exhausted, concurrent, exception=not critical) { +| +2 | TX { +3 | REPEAT(size=5) { +| +4 | RETRY(stateful, exception=deadlock loser) { +4.1 | input; +5 | } PROCESS { +| output; +6 | } RECOVER { +| recover; +| } +| +| } +| } +| +| } + +---- + +[[asyncItemProcessing]] +=== Asynchronous Item Processing + +The individual items in chunks in the <> can also, in +principle, be processed concurrently. In this case, the transaction boundary has to move +to the level of the individual item, so that each transaction is on a single thread, as +shown in the following example: + +---- + +1 | REPEAT(until=exhausted, exception=not critical) { +| +2 | REPEAT(size=5, concurrent) { +| +3 | TX { +4 | RETRY(stateful, exception=deadlock loser) { +4.1 | input; +5 | } PROCESS { +| output; +6 | } RECOVER { +| recover; +| } +| } +| +| } +| +| } + +---- + +This plan sacrifices the optimization benefit, which the simple plan had, of having all +the transactional resources chunked together. It is only useful if the cost of the +processing (5) is much higher than the cost of transaction management (3). + +[[transactionPropagation]] +=== Interactions Between Batching and Transaction Propagation + +There is a tighter coupling between batch-retry and transaction management than we would +ideally like. In particular, a stateless retry cannot be used to retry database +operations with a transaction manager that does not support NESTED propagation. + +The following example uses retry without repeat: + +---- + +1 | TX { +| +1.1 | input; +2.2 | database access; +2 | RETRY { +3 | TX { +3.1 | database access; +| } +| } +| +| } + +---- + +Again, and for the same reason, the inner transaction, `TX` (3), can cause the outer +transaction, `TX` (1), to fail, even if the `RETRY` (2) is eventually successful. + +Unfortunately, the same effect percolates from the retry block up to the surrounding +repeat batch if there is one, as shown in the following example: + +---- + +1 | TX { +| +2 | REPEAT(size=5) { +2.1 | input; +2.2 | database access; +3 | RETRY { +4 | TX { +4.1 | database access; +| } +| } +| } +| +| } + +---- + +Now, if TX (3) rolls back, it can pollute the whole batch at TX (1) and force it to roll +back at the end. + +What about non-default propagation? + +* In the preceding example, `PROPAGATION_REQUIRES_NEW` at `TX` (3) prevents the outer +`TX` (1) from being polluted if both transactions are eventually successful. But if `TX` +(3) commits and `TX` (1) rolls back, then `TX` (3) stays committed, so we violate the +transaction contract for `TX` (1). If `TX` (3) rolls back, `TX` (1) does not necessarily +(but it probably does in practice, because the retry throws a roll back exception). + +* `PROPAGATION_NESTED` at `TX` (3) works as we require in the retry case (and for a +batch with skips): `TX` (3) can commit but subsequently be rolled back by the outer +transaction, `TX` (1). If `TX` (3) rolls back, `TX` (1) rolls back in practice. This +option is only available on some platforms, not including Hibernate or +JTA, but it is the only one that consistently works. + +Consequently, the `NESTED` pattern is best if the retry block contains any database +access. + +[[specialTransactionOrthonogonal]] +=== Special Case: Transactions with Orthogonal Resources + +Default propagation is always OK for simple cases where there are no nested database +transactions. Consider the following example, where the `SESSION` and `TX` are not +global `XA` resources, so their resources are orthogonal: + +---- + +0 | SESSION { +1 | input; +2 | RETRY { +3 | TX { +3.1 | database access; +| } +| } +| } + +---- + +Here there is a transactional message `SESSION` (0), but it does not participate in other +transactions with `PlatformTransactionManager`, so it does not propagate when `TX` (3) +starts. There is no database access outside the `RETRY` (2) block. If `TX` (3) fails and +then eventually succeeds on a retry, `SESSION` (0) can commit (independently of a `TX` +block). This is similar to the vanilla "best-efforts-one-phase-commit" scenario. The +worst that can happen is a duplicate message when the `RETRY` (2) succeeds and the +`SESSION` (0) cannot commit (for example, because the message system is unavailable). + +[[statelessRetryCannotRecover]] +=== Stateless Retry Cannot Recover + +The distinction between a stateless and a stateful retry in the typical example above is +important. It is actually ultimately a transactional constraint that forces the +distinction, and this constraint also makes it obvious why the distinction exists. + +We start with the observation that there is no way to skip an item that failed and +successfully commit the rest of the chunk unless we wrap the item processing in a +transaction. Consequently, we simplify the typical batch execution plan to be as +follows: + +---- + +0 | REPEAT(until=exhausted) { +| +1 | TX { +2 | REPEAT(size=5) { +| +3 | RETRY(stateless) { +4 | TX { +4.1 | input; +4.2 | database access; +| } +5 | } RECOVER { +5.1 | skip; +| } +| +| } +| } +| +| } + +---- + +The preceding example shows a stateless `RETRY` (3) with a `RECOVER` (5) path that kicks +in after the final attempt fails. The `stateless` label means that the block is repeated +without re-throwing any exception up to some limit. This only works if the transaction, +`TX` (4), has propagation NESTED. + +If the inner `TX` (4) has default propagation properties and rolls back, it pollutes the +outer `TX` (1). The inner transaction is assumed by the transaction manager to have +corrupted the transactional resource, so it cannot be used again. + +Support for NESTED propagation is sufficiently rare that we choose not to support +recovery with stateless retries in the current versions of Spring Batch. The same effect +can always be achieved (at the expense of repeating more processing) by using the +typical pattern above. diff --git a/spring-batch-docs/asciidoc/whatsnew.adoc b/spring-batch-docs/asciidoc/whatsnew.adoc new file mode 100644 index 0000000000..c0beda4a16 --- /dev/null +++ b/spring-batch-docs/asciidoc/whatsnew.adoc @@ -0,0 +1,47 @@ +:batch-asciidoc: ./ +:toc: left +:toclevels: 4 + +[[whatsNew]] + +== What's New in Spring Batch 4.2 + +Spring Batch 4.2 adds the following features: + +* Support for batch metrics with https://micrometer.io[Micrometer] +* Support for reading/writing data from/to https://kafka.apache.org[Apache Kafka] topics +* Support for reading/writing data from/to https://avro.apache.org[Apache Avro] resources +* Improved documentation + +[[whatsNewMetrics]] +=== Batch metrics with Micrometer + +This release introduces a new feature that lets you monitor your batch jobs +by using Micrometer. By default, Spring Batch collects metrics (such as job duration, +step duration, item read and write throughput, and others) and registers them in Micrometer's +global metrics registry under the `spring.batch` prefix. +These metrics can be sent to any https://micrometer.io/docs/concepts#_supported_monitoring_systems[monitoring system] +supported by Micrometer. + +For more details about this feature, please refer to the +<> chapter. + +[[whatsNewKafka]] +=== Apache Kafka item reader/writer + +This release adds a new `KafkaItemReader` and `KafkaItemWriter` to read data from and +write it to Kafka topics. For more details about these new components, please refer +to the https://docs.spring.io/spring-batch/4.2.x/api/index.html[Javadoc]. + +[[whatsNewAvro]] +=== Apache Avro item reader/writer + +This release adds a new `AvroItemReader` and `AvroItemWriter` to read data from and +write it to Avro resources. For more details about these new components, please refer +to the https://docs.spring.io/spring-batch/4.2.x/api/index.html[Javadoc]. + +[[whatsNewDocs]] +=== Documentation updates + +The reference documentation has been updated to match the same style as other +Spring projects. diff --git a/spring-batch-infrastructure-tests/pom.xml b/spring-batch-infrastructure-tests/pom.xml deleted file mode 100644 index 3c36e5747c..0000000000 --- a/spring-batch-infrastructure-tests/pom.xml +++ /dev/null @@ -1,253 +0,0 @@ - - - 4.0.0 - spring-batch-infrastructure-tests - Infrastructure Tests - Integration tests for the Spring Batch Infrastructure - - org.springframework.batch - spring-batch-parent - 2.2.2.BUILD-SNAPSHOT - ../spring-batch-parent - - - - hsql - - - - default - - true - - - - - org.apache.maven.plugins - maven-surefire-plugin - - true - - - - - - - test - - - - org.apache.maven.plugins - maven-surefire-plugin - - false - - - ENVIRONMENT - ${environment} - - - - - - - - - - - org.springframework.batch - spring-batch-infrastructure - 2.2.2.BUILD-SNAPSHOT - - - org.hsqldb - hsqldb - - - commons-io - commons-io - test - - - org.apache.derby - derby - - - org.apache.activemq - activemq-core - 5.1.0 - test - - - camel-core - org.apache.camel - - - commons-logging-api - commons-logging - - - - - junit - junit - - - org.apache.geronimo.specs - geronimo-jms_1.1_spec - - - org.apache.geronimo.specs - geronimo-j2ee-management_1.1_spec - 1.0.1 - test - - - xmlunit - xmlunit - 1.2 - test - - - org.codehaus.castor - castor-xml - 1.3.2 - test - - - stax - stax - - - commons-lang - commons-lang - - - - - org.slf4j - slf4j-log4j12 - true - - - log4j - log4j - test - - - xerces - xercesImpl - 2.8.1 - test - - - com.thoughtworks.xstream - xstream - test - - - org.codehaus.woodstox - woodstox-core-asl - test - - - org.apache.ibatis - ibatis-sqlmap - true - - - commons-dbcp - commons-dbcp - - - commons-lang - commons-lang - 2.5 - test - - - org.hibernate - hibernate-core - true - - - org.hibernate - hibernate-entitymanager - true - - - org.hibernate - hibernate-annotations - true - - - org.apache.geronimo.specs - geronimo-jta_1.1_spec - true - - - org.springframework - spring-oxm - test - - - org.springframework - spring-jdbc - test - - - org.springframework - spring-orm - true - - - org.springframework - spring-jms - true - - - org.springframework - spring-test - test - - - org.springframework - spring-tx - - - org.springframework - spring-aop - - - mysql - mysql-connector-java - 5.1.6 - runtime - - - postgresql - postgresql - 8.3-603.jdbc3 - true - runtime - - - org.mockito - mockito-all - test - - - - - - - org.apache.maven.plugins - maven-surefire-plugin - - junit:junit - - - - - - diff --git a/spring-batch-infrastructure-tests/src/main/java/org/springframework/batch/container/jms/BatchMessageListenerContainer.java b/spring-batch-infrastructure-tests/src/main/java/org/springframework/batch/container/jms/BatchMessageListenerContainer.java index a41b14f343..a1e4459f30 100644 --- a/spring-batch-infrastructure-tests/src/main/java/org/springframework/batch/container/jms/BatchMessageListenerContainer.java +++ b/spring-batch-infrastructure-tests/src/main/java/org/springframework/batch/container/jms/BatchMessageListenerContainer.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,10 +16,6 @@ package org.springframework.batch.container.jms; -import javax.jms.JMSException; -import javax.jms.MessageConsumer; -import javax.jms.Session; - import org.aopalliance.aop.Advice; import org.springframework.aop.framework.ProxyFactory; import org.springframework.aop.support.DefaultPointcutAdvisor; @@ -30,9 +26,13 @@ import org.springframework.jms.listener.DefaultMessageListenerContainer; import org.springframework.transaction.interceptor.TransactionInterceptor; +import javax.jms.JMSException; +import javax.jms.MessageConsumer; +import javax.jms.Session; + /** * Message listener container adapted for intercepting the message reception - * with advice provided through configuration.
      + * with advice provided through configuration.
      * * To enable batching of messages in a single transaction, use the * {@link TransactionInterceptor} and the {@link RepeatOperationsInterceptor} in @@ -64,6 +64,7 @@ public static interface ContainerDelegate { private Advice[] advices = new Advice[0]; private ContainerDelegate delegate = new ContainerDelegate() { + @Override public boolean receiveAndExecute(Object invoker, Session session, MessageConsumer consumer) throws JMSException { return BatchMessageListenerContainer.super.receiveAndExecute(invoker, session, consumer); } @@ -85,6 +86,7 @@ public void setAdviceChain(Advice[] advices) { * * @see org.springframework.jms.listener.AbstractJmsListeningContainer#afterPropertiesSet() */ + @Override public void afterPropertiesSet() { super.afterPropertiesSet(); initializeProxy(); @@ -96,6 +98,7 @@ public void afterPropertiesSet() { * * @see org.springframework.jms.listener.AbstractMessageListenerContainer#handleListenerException(java.lang.Throwable) */ + @Override protected void handleListenerException(Throwable ex) { if (!isSessionTransacted()) { // Log the exceptions in base class if not transactional anyway @@ -120,6 +123,7 @@ else if (ex instanceof Error) { * @see org.springframework.jms.listener.AbstractPollingMessageListenerContainer#receiveAndExecute(Object, * javax.jms.Session, javax.jms.MessageConsumer) */ + @Override protected boolean receiveAndExecute(final Object invoker, final Session session, final MessageConsumer consumer) throws JMSException { return proxy.receiveAndExecute(invoker, session, consumer); diff --git a/spring-batch-infrastructure-tests/src/site/apt/index.apt b/spring-batch-infrastructure-tests/src/site/apt/index.apt deleted file mode 100644 index 45d636ad3c..0000000000 --- a/spring-batch-infrastructure-tests/src/site/apt/index.apt +++ /dev/null @@ -1,15 +0,0 @@ - ------ - Spring Batch Integration Tests - ------ - Dave Syer - ------ - March 2008 - -Overview of the Spring Batch Integration Tests - - This module contains integration tests for the Spring Batch Infrastructure module. In the tests we exercise some extended and mostly realistic scenarios using the infrastructure components. - - * Message-pipeline tests. There are a bunch of tests using JMS and RDBMS in the same transaction, exercising various failure and retry scenarios. The basic premise is that throughput in such systems is greatly increased by widening the transactio nboundaries to process more than one message in each transaction. If there is a failure we may want to retry the operation in the next transaction. - - * OXM integration tests. There are also some tests of the XML reader and writers in Spring Batch with more extensive configuration than in the unit tests. - diff --git a/spring-batch-infrastructure-tests/src/site/site.xml b/spring-batch-infrastructure-tests/src/site/site.xml deleted file mode 100644 index 29c2887830..0000000000 --- a/spring-batch-infrastructure-tests/src/site/site.xml +++ /dev/null @@ -1,17 +0,0 @@ - - - - - - - - - - - - - - - - - diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/config/DatasourceTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/config/DatasourceTests.java index 3dfe865c35..b65e9e7c60 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/config/DatasourceTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/config/DatasourceTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -55,7 +55,7 @@ public static void cleanup() { public void testTemplate() throws Exception { System.err.println(System.getProperty("java.class.path")); jdbcTemplate.execute("delete from T_BARS"); - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(0, count); jdbcTemplate.update("INSERT into T_BARS (id,name,foo_date) values (?,?,null)", 0, "foo"); diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/config/MessagingTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/config/MessagingTests.java index 4d4d75b216..c41f6d65f4 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/config/MessagingTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/config/MessagingTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -55,7 +55,7 @@ public void testMessaging() throws Exception { private List getMessages() { String next = ""; - List msgs = new ArrayList(); + List msgs = new ArrayList<>(); while (next != null) { next = (String) jmsTemplate.receiveAndConvert("queue"); if (next != null) diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/container/jms/BatchMessageListenerContainerIntegrationTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/container/jms/BatchMessageListenerContainerIntegrationTests.java index 1c9daf5f26..2afac42d97 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/container/jms/BatchMessageListenerContainerIntegrationTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/container/jms/BatchMessageListenerContainerIntegrationTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,14 +34,16 @@ import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.jms.core.JmsTemplate; import org.springframework.retry.RecoveryCallback; import org.springframework.retry.RetryCallback; import org.springframework.retry.RetryContext; import org.springframework.retry.policy.NeverRetryPolicy; import org.springframework.retry.support.DefaultRetryState; import org.springframework.retry.support.RetryTemplate; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.jms.core.JmsTemplate; +import org.springframework.test.annotation.DirtiesContext; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @@ -51,6 +53,7 @@ */ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "/org/springframework/batch/jms/jms-context.xml") +@DirtiesContext public class BatchMessageListenerContainerIntegrationTests { @Autowired @@ -59,9 +62,9 @@ public class BatchMessageListenerContainerIntegrationTests { @Autowired private BatchMessageListenerContainer container; - private volatile BlockingQueue recovered = new LinkedBlockingQueue(); + private volatile BlockingQueue recovered = new LinkedBlockingQueue<>(); - private volatile BlockingQueue processed = new LinkedBlockingQueue(); + private volatile BlockingQueue processed = new LinkedBlockingQueue<>(); @After @Before @@ -86,6 +89,7 @@ public void testConfiguration() throws Exception { @Test public void testSendAndReceive() throws Exception { container.setMessageListener(new MessageListener() { + @Override public void onMessage(Message msg) { try { processed.add(((TextMessage) msg).getText()); @@ -99,7 +103,7 @@ public void onMessage(Message msg) { container.start(); jmsTemplate.convertAndSend("queue", "foo"); jmsTemplate.convertAndSend("queue", "bar"); - SortedSet result = new TreeSet(); + SortedSet result = new TreeSet<>(); for (int i = 0; i < 2; i++) { result.add(processed.poll(5, TimeUnit.SECONDS)); } @@ -109,6 +113,7 @@ public void onMessage(Message msg) { @Test public void testFailureAndRepresent() throws Exception { container.setMessageListener(new MessageListener() { + @Override public void onMessage(Message msg) { try { processed.add(((TextMessage) msg).getText()); @@ -132,9 +137,11 @@ public void testFailureAndRecovery() throws Exception { final RetryTemplate retryTemplate = new RetryTemplate(); retryTemplate.setRetryPolicy(new NeverRetryPolicy()); container.setMessageListener(new MessageListener() { + @Override public void onMessage(final Message msg) { try { - RetryCallback callback = new RetryCallback() { + RetryCallback callback = new RetryCallback() { + @Override public Message doWithRetry(RetryContext context) throws Exception { try { processed.add(((TextMessage) msg).getText()); @@ -146,6 +153,7 @@ public Message doWithRetry(RetryContext context) throws Exception { } }; RecoveryCallback recoveryCallback = new RecoveryCallback() { + @Override public Message recover(RetryContext context) { try { recovered.add(((TextMessage) msg).getText()); diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/container/jms/BatchMessageListenerContainerTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/container/jms/BatchMessageListenerContainerTests.java index fca1abde81..c281987870 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/container/jms/BatchMessageListenerContainerTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/container/jms/BatchMessageListenerContainerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,16 +16,8 @@ package org.springframework.batch.container.jms; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - import java.lang.reflect.InvocationTargetException; import java.lang.reflect.Method; - import javax.jms.ConnectionFactory; import javax.jms.JMSException; import javax.jms.Message; @@ -35,11 +27,19 @@ import org.aopalliance.aop.Advice; import org.junit.Test; + import org.springframework.batch.repeat.interceptor.RepeatOperationsInterceptor; import org.springframework.batch.repeat.policy.SimpleCompletionPolicy; import org.springframework.batch.repeat.support.RepeatTemplate; import org.springframework.util.ReflectionUtils; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + public class BatchMessageListenerContainerTests { BatchMessageListenerContainer container; @@ -51,6 +51,7 @@ public void testReceiveAndExecuteWithCallback() throws Exception { container = getContainer(template); container.setMessageListener(new MessageListener() { + @Override public void onMessage(Message arg0) { } }); @@ -135,8 +136,10 @@ private BatchMessageListenerContainer getContainer(RepeatTemplate template) { // Yuck: we need to turn these method in base class to no-ops because the invoker is a private class // we can't create for test purposes... BatchMessageListenerContainer container = new BatchMessageListenerContainer() { + @Override protected void messageReceived(Object invoker, Session session) { } + @Override protected void noMessageReceived(Object invoker, Session session) { } }; @@ -153,6 +156,7 @@ private boolean doTestWithException(final Throwable t, boolean expectRollback, i throws JMSException, IllegalAccessException { container.setAcceptMessagesWhileStopping(true); container.setMessageListener(new MessageListener() { + @Override public void onMessage(Message arg0) { if (t instanceof RuntimeException) throw (RuntimeException) t; @@ -182,13 +186,12 @@ public void onMessage(Message arg0) { } private boolean doExecute(Session session, MessageConsumer consumer) throws IllegalAccessException { - Method method = ReflectionUtils.findMethod(container.getClass(), "receiveAndExecute", new Class[] { - Object.class, Session.class, MessageConsumer.class }); + Method method = ReflectionUtils.findMethod(container.getClass(), "receiveAndExecute", Object.class, Session.class, MessageConsumer.class); method.setAccessible(true); boolean received; try { // A null invoker is not normal, but we don't care about the invoker for a unit test - received = ((Boolean) method.invoke(container, new Object[] { null, session, consumer })).booleanValue(); + received = (Boolean) method.invoke(container, null, session, consumer); } catch (InvocationTargetException e) { if (e.getCause() instanceof RuntimeException) { diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/IbatisPagingItemReaderAsyncTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/IbatisPagingItemReaderAsyncTests.java deleted file mode 100644 index 646daaf2db..0000000000 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/IbatisPagingItemReaderAsyncTests.java +++ /dev/null @@ -1,164 +0,0 @@ -package org.springframework.batch.item.database; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; - -import java.util.ArrayList; -import java.util.HashSet; -import java.util.List; -import java.util.Set; -import java.util.concurrent.Callable; -import java.util.concurrent.CompletionService; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.ExecutorCompletionService; -import java.util.concurrent.Executors; - -import javax.sql.DataSource; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.junit.After; -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.springframework.batch.item.sample.Foo; -import org.springframework.test.jdbc.JdbcTestUtils; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.core.io.ClassPathResource; -import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.orm.ibatis.SqlMapClientFactoryBean; -import org.springframework.test.context.ContextConfiguration; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; - -import com.ibatis.sqlmap.client.SqlMapClient; - -@RunWith(SpringJUnit4ClassRunner.class) -@ContextConfiguration(locations = "JdbcPagingItemReaderCommonTests-context.xml") -public class IbatisPagingItemReaderAsyncTests { - - /** - * The number of items to read - */ - private static final int ITEM_COUNT = 1000; - - /** - * The number of threads to create - */ - private static final int THREAD_COUNT = 10; - - private static Log logger = LogFactory.getLog(IbatisPagingItemReaderAsyncTests.class); - - @Autowired - private DataSource dataSource; - - private int maxId; - - @Before - public void init() { - JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); - maxId = jdbcTemplate.queryForInt("SELECT MAX(ID) from T_FOOS"); - for (int i = ITEM_COUNT; i > maxId; i--) { - jdbcTemplate.update("INSERT into T_FOOS (ID,NAME,VALUE) values (?, ?, ?)", i, "foo" + i, i); - } - assertEquals(ITEM_COUNT, JdbcTestUtils.countRowsInTable(jdbcTemplate, "T_FOOS")); - } - - @After - public void destroy() { - JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); - jdbcTemplate.update("DELETE from T_FOOS where ID>?", maxId); - } - - @Test - public void testAsyncReader() throws Throwable { - List throwables = new ArrayList(); - int max = 10; - for (int i = 0; i < max; i++) { - try { - logger.info("Testing asynch reader, iteration="+i); - doTest(); - } - catch (Throwable e) { - throwables.add(e); - } - } - if (!throwables.isEmpty()) { - throw new IllegalStateException(String.format("Failed %d out of %d", throwables.size(), max), throwables - .get(0)); - } - } - - /** - * @throws Exception - * @throws InterruptedException - * @throws ExecutionException - */ - private void doTest() throws Exception, InterruptedException, ExecutionException { - final IbatisPagingItemReader reader = getItemReader(); - CompletionService> completionService = new ExecutorCompletionService>(Executors - .newFixedThreadPool(THREAD_COUNT)); - for (int i = 0; i < THREAD_COUNT; i++) { - completionService.submit(new Callable>() { - public List call() throws Exception { - List list = new ArrayList(); - Foo next = null; - do { - next = reader.read(); - Thread.sleep(10L); // try to make it fairer - logger.debug("Reading item: " + next); - if (next != null) { - list.add(next); - } - } while (next != null); - return list; - } - }); - } - int count = 0; - Set results = new HashSet(); - for (int i = 0; i < THREAD_COUNT; i++) { - List items = completionService.take().get(); - count += items.size(); - logger.debug("Finished items count: " + items.size()); - logger.debug("Finished items: " + items); - assertNotNull(items); - results.addAll(items); - } - assertEquals(ITEM_COUNT, count); - assertEquals(ITEM_COUNT, results.size()); - reader.close(); - } - - private IbatisPagingItemReader getItemReader() throws Exception { - SqlMapClientFactoryBean factory = new SqlMapClientFactoryBean(); - factory.setConfigLocation(new ClassPathResource("ibatis-config.xml", getClass())); - factory.setDataSource(dataSource); - factory.afterPropertiesSet(); - SqlMapClient sqlMapClient = createSqlMapClient(); - - IbatisPagingItemReader reader = new IbatisPagingItemReader(); - if ("postgres".equals(System.getProperty("ENVIRONMENT"))) { - reader.setQueryId("getPagedFoosPostgres"); - } else if ("oracle".equals(System.getProperty("ENVIRONMENT"))) { - reader.setQueryId("getPagedFoosOracle"); - } else { - reader.setQueryId("getPagedFoos"); - } - reader.setPageSize(2); - reader.setSqlMapClient(sqlMapClient); - reader.setSaveState(true); - - reader.afterPropertiesSet(); - - return reader; - } - - private SqlMapClient createSqlMapClient() throws Exception { - SqlMapClientFactoryBean factory = new SqlMapClientFactoryBean(); - factory.setConfigLocation(new ClassPathResource("ibatis-config.xml", getClass())); - factory.setDataSource(dataSource); - factory.afterPropertiesSet(); - return (SqlMapClient) factory.getObject(); - } - -} diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderAsyncTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderAsyncTests.java index 2371a21f2a..53f3311b1e 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderAsyncTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderAsyncTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,15 +40,16 @@ import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean; import org.springframework.batch.item.sample.Foo; -import org.springframework.test.jdbc.JdbcTestUtils; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.test.jdbc.JdbcTestUtils; /** * @author Dave Syer @@ -83,8 +84,9 @@ public class JdbcPagingItemReaderAsyncTests { @Before public void init() { - JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); - maxId = jdbcTemplate.queryForInt("SELECT MAX(ID) from T_FOOS"); + JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); + Integer tempMaxId = jdbcTemplate.queryForObject("SELECT MAX(ID) from T_FOOS", Integer.class); + maxId = tempMaxId != null? tempMaxId : 0; for (int i = ITEM_COUNT; i > maxId; i--) { jdbcTemplate.update("INSERT into T_FOOS (ID,NAME,VALUE) values (?, ?, ?)", i, "foo" + i, i); } @@ -99,7 +101,7 @@ public void destroy() { @Test public void testAsyncReader() throws Throwable { - List throwables = new ArrayList(); + List throwables = new ArrayList<>(); int max = 10; for (int i = 0; i < max; i++) { try { @@ -122,12 +124,13 @@ public void testAsyncReader() throws Throwable { */ private void doTest() throws Exception, InterruptedException, ExecutionException { final ItemReader reader = getItemReader(); - CompletionService> completionService = new ExecutorCompletionService>(Executors + CompletionService> completionService = new ExecutorCompletionService<>(Executors .newFixedThreadPool(THREAD_COUNT)); for (int i = 0; i < THREAD_COUNT; i++) { completionService.submit(new Callable>() { + @Override public List call() throws Exception { - List list = new ArrayList(); + List list = new ArrayList<>(); Foo next = null; do { next = reader.read(); @@ -142,7 +145,7 @@ public List call() throws Exception { }); } int count = 0; - Set results = new HashSet(); + Set results = new HashSet<>(); for (int i = 0; i < THREAD_COUNT; i++) { List items = completionService.take().get(); count += items.size(); @@ -157,17 +160,18 @@ public List call() throws Exception { protected ItemReader getItemReader() throws Exception { - JdbcPagingItemReader reader = new JdbcPagingItemReader(); + JdbcPagingItemReader reader = new JdbcPagingItemReader<>(); reader.setDataSource(dataSource); SqlPagingQueryProviderFactoryBean factory = new SqlPagingQueryProviderFactoryBean(); factory.setDataSource(dataSource); factory.setSelectClause("select ID, NAME, VALUE"); factory.setFromClause("from T_FOOS"); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("VALUE", Order.ASCENDING); factory.setSortKeys(sortKeys); - reader.setQueryProvider((PagingQueryProvider) factory.getObject()); - reader.setRowMapper(new ParameterizedRowMapper() { + reader.setQueryProvider(factory.getObject()); + reader.setRowMapper(new RowMapper() { + @Override public Foo mapRow(ResultSet rs, int i) throws SQLException { Foo foo = new Foo(); foo.setId(rs.getInt(1)); diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JdbcPagingQueryIntegrationTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JdbcPagingQueryIntegrationTests.java index b4ad625904..fd8596d32a 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JdbcPagingQueryIntegrationTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JdbcPagingQueryIntegrationTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,17 +16,12 @@ package org.springframework.batch.item.database; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotSame; -import static org.junit.Assert.assertTrue; - import java.util.ArrayList; import java.util.LinkedHashMap; import java.util.List; import java.util.Map; import java.util.SortedMap; import java.util.TreeMap; - import javax.sql.DataSource; import org.apache.commons.logging.Log; @@ -36,13 +31,19 @@ import org.junit.Ignore; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.item.database.support.AbstractSqlPagingQueryProvider; import org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean; -import org.springframework.test.jdbc.JdbcTestUtils; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.annotation.DirtiesContext; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.test.jdbc.JdbcTestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotSame; +import static org.junit.Assert.assertTrue; /** * @author Dave Syer @@ -51,6 +52,7 @@ */ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "JdbcPagingItemReaderCommonTests-context.xml") +@DirtiesContext(classMode = DirtiesContext.ClassMode.BEFORE_CLASS) public class JdbcPagingQueryIntegrationTests { private static Log logger = LogFactory.getLog(JdbcPagingQueryIntegrationTests.class); @@ -71,7 +73,8 @@ public void testInit() { jdbcTemplate = new JdbcTemplate(dataSource); String[] names = {"Foo", "Bar", "Baz", "Foo", "Bar", "Baz", "Foo", "Bar", "Baz"}; String[] codes = {"A", "B", "A", "B", "B", "B", "A", "B", "A"}; - jdbcTemplate.update("DELETE from T_FOOS"); + JdbcTestUtils.deleteFromTables(jdbcTemplate, "T_FOOS"); +// jdbcTemplate.update("DELETE from T_FOOS"); for(int i = 0; i < names.length; i++) { jdbcTemplate.update("INSERT into T_FOOS (ID,NAME, CODE, VALUE) values (?, ?, ?, ?)", maxId, names[i], codes[i], i); maxId++; @@ -81,7 +84,8 @@ public void testInit() { @After public void destroy() { - jdbcTemplate.update("DELETE from T_FOOS"); + JdbcTestUtils.deleteFromTables(jdbcTemplate, "T_FOOS"); +// jdbcTemplate.update("DELETE from T_FOOS"); } @Test @@ -125,14 +129,13 @@ public void testQueryFromStart() throws Exception { @Test public void testQueryFromStartWithGroupBy() throws Exception { AbstractSqlPagingQueryProvider queryProvider = (AbstractSqlPagingQueryProvider) getPagingQueryProvider(); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("NAME", Order.ASCENDING); sortKeys.put("CODE", Order.DESCENDING); queryProvider.setSortKeys(sortKeys); queryProvider.setSelectClause("select NAME, CODE, sum(VALUE)"); queryProvider.setGroupClause("NAME, CODE"); - int pages = 3; int count = 0; int total = 5; @@ -163,7 +166,7 @@ public void testQueryFromStartWithGroupBy() throws Exception { private Map getStartAfterValues( PagingQueryProvider queryProvider, List> list) { - Map startAfterValues = new LinkedHashMap(); + Map startAfterValues = new LinkedHashMap<>(); for (Map.Entry sortKey : queryProvider.getSortKeys().entrySet()) { startAfterValues.put(sortKey.getKey(), list.get(list.size() - 1).get(sortKey.getKey())); } @@ -176,7 +179,7 @@ public void testJumpToItem() throws Exception { PagingQueryProvider queryProvider = getPagingQueryProvider(); - int minId = jdbcTemplate.queryForInt("SELECT MIN(VALUE) FROM T_FOOS"); + int minId = jdbcTemplate.queryForObject("SELECT MIN(VALUE) FROM T_FOOS", Integer.class); String query = queryProvider.generateJumpToItemQuery(pageSize, pageSize); List> list = jdbcTemplate.queryForList(query); @@ -188,7 +191,6 @@ public void testJumpToItem() throws Exception { Object startAfterValue = list.get(0).entrySet().iterator().next().getValue(); list = jdbcTemplate.queryForList(queryProvider.generateRemainingPagesQuery(pageSize), startAfterValue); assertEquals(pageSize, list.size()); - expected = "[{id=" + (minId + pageSize); } protected PagingQueryProvider getPagingQueryProvider() throws Exception { @@ -197,22 +199,22 @@ protected PagingQueryProvider getPagingQueryProvider() throws Exception { factory.setDataSource(dataSource); factory.setSelectClause("select ID, NAME, VALUE"); factory.setFromClause("from T_FOOS"); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("VALUE", Order.ASCENDING); factory.setSortKeys(sortKeys); - return (PagingQueryProvider) factory.getObject(); + return factory.getObject(); } private List getParameterList(Map values, Map sortKeyValue) { - SortedMap sm = new TreeMap(); + SortedMap sm = new TreeMap<>(); if (values != null) { sm.putAll(values); } - List parameterList = new ArrayList(); + List parameterList = new ArrayList<>(); parameterList.addAll(sm.values()); if (sortKeyValue != null && sortKeyValue.size() > 0) { - List> keys = new ArrayList>(sortKeyValue.entrySet()); + List> keys = new ArrayList<>(sortKeyValue.entrySet()); for(int i = 0; i < keys.size(); i++) { for(int j = 0; j < i; j++) { diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JdbcPagingRestartIntegrationTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JdbcPagingRestartIntegrationTests.java index 83c95a6b58..185549ce12 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JdbcPagingRestartIntegrationTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JdbcPagingRestartIntegrationTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,19 +32,21 @@ import org.apache.commons.logging.LogFactory; import org.junit.After; import org.junit.Before; +import org.junit.Ignore; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ItemStream; import org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean; import org.springframework.batch.item.sample.Foo; -import org.springframework.test.jdbc.JdbcTestUtils; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.test.jdbc.JdbcTestUtils; /** * @author Dave Syer @@ -71,11 +73,11 @@ public class JdbcPagingRestartIntegrationTests { @Before public void init() { jdbcTemplate = new JdbcTemplate(dataSource); - maxId = jdbcTemplate.queryForInt("SELECT MAX(ID) from T_FOOS"); + maxId = jdbcTemplate.queryForObject("SELECT MAX(ID) from T_FOOS", Integer.class); for (int i = itemCount; i > maxId; i--) { jdbcTemplate.update("INSERT into T_FOOS (ID,NAME,VALUE) values (?, ?, ?)", i, "foo" + i, i); } - + assertEquals(itemCount, JdbcTestUtils.countRowsInTable(jdbcTemplate, "T_FOOS")); } @@ -85,6 +87,7 @@ public void destroy() { } @Test + @Ignore //FIXME public void testReaderFromStart() throws Exception { ItemReader reader = getItemReader(); @@ -107,6 +110,7 @@ public void testReaderFromStart() throws Exception { } @Test + @Ignore //FIXME public void testReaderOnRestart() throws Exception { ItemReader reader = getItemReader(); @@ -127,7 +131,7 @@ public void testReaderOnRestart() throws Exception { logger.debug("Ids: "+ids); int startAfterValue = (new Long(ids.get(count - 1).get("ID").toString())).intValue(); logger.debug("Start after: " + startAfterValue); - Map startAfterValues = new LinkedHashMap(); + Map startAfterValues = new LinkedHashMap<>(); startAfterValues.put("ID", startAfterValue); executionContext.put("JdbcPagingItemReader.start.after", startAfterValues); ((ItemStream) reader).open(executionContext); @@ -146,17 +150,18 @@ public void testReaderOnRestart() throws Exception { protected ItemReader getItemReader() throws Exception { - JdbcPagingItemReader reader = new JdbcPagingItemReader(); + JdbcPagingItemReader reader = new JdbcPagingItemReader<>(); reader.setDataSource(dataSource); SqlPagingQueryProviderFactoryBean factory = new SqlPagingQueryProviderFactoryBean(); factory.setDataSource(dataSource); factory.setSelectClause("select ID, NAME, VALUE"); factory.setFromClause("from T_FOOS"); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("VALUE", Order.ASCENDING); factory.setSortKeys(sortKeys); - reader.setQueryProvider((PagingQueryProvider) factory.getObject()); - reader.setRowMapper(new ParameterizedRowMapper() { + reader.setQueryProvider(factory.getObject()); + reader.setRowMapper(new RowMapper() { + @Override public Foo mapRow(ResultSet rs, int i) throws SQLException { Foo foo = new Foo(); foo.setId(rs.getInt(1)); diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JpaItemWriterIntegrationTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JpaItemWriterIntegrationTests.java new file mode 100644 index 0000000000..cd214500f0 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JpaItemWriterIntegrationTests.java @@ -0,0 +1,146 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database; + +import java.util.Arrays; +import java.util.List; +import javax.persistence.EntityManagerFactory; +import javax.sql.DataSource; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.batch.item.sample.Person; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType; +import org.springframework.orm.jpa.JpaTransactionManager; +import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean; +import org.springframework.orm.jpa.persistenceunit.DefaultPersistenceUnitManager; +import org.springframework.orm.jpa.persistenceunit.PersistenceUnitManager; +import org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringRunner; +import org.springframework.test.jdbc.JdbcTestUtils; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.transaction.annotation.Transactional; + +import static org.junit.Assert.assertEquals; + +@RunWith(SpringRunner.class) +@ContextConfiguration(classes = JpaItemWriterIntegrationTests.JpaConfiguration.class) +@Transactional +@DirtiesContext +public class JpaItemWriterIntegrationTests { + + @Autowired + private EntityManagerFactory entityManagerFactory; + + @Autowired + private JdbcTemplate jdbcTemplate; + + @Before + public void init() { + this.jdbcTemplate.update("create table person (id int not null primary key, name varchar(32))"); + } + + @After + public void destroy() { + JdbcTestUtils.dropTables(this.jdbcTemplate, "person"); + } + + @Test + public void testMerge() throws Exception { + // given + JpaItemWriter writer = new JpaItemWriter<>(); + writer.setEntityManagerFactory(this.entityManagerFactory); + writer.afterPropertiesSet(); + List items = Arrays.asList( + new Person(1, "foo"), + new Person(2, "bar")); + + // when + writer.write(items); + + // then + assertEquals(2, JdbcTestUtils.countRowsInTable(this.jdbcTemplate, "person")); + } + + @Test + public void testPersist() throws Exception { + // given + JpaItemWriter writer = new JpaItemWriter<>(); + writer.setEntityManagerFactory(this.entityManagerFactory); + writer.setUsePersist(true); + writer.afterPropertiesSet(); + List items = Arrays.asList( + new Person(1, "foo"), + new Person(2, "bar")); + + // when + writer.write(items); + + // then + assertEquals(2, JdbcTestUtils.countRowsInTable(this.jdbcTemplate, "person")); + } + + @Configuration + public static class JpaConfiguration { + + @Bean + public DataSource dataSource() { + return new EmbeddedDatabaseBuilder() + .setType(EmbeddedDatabaseType.HSQL) + .build(); + } + + @Bean + public JdbcTemplate jdbcTemplate(DataSource dataSource) { + return new JdbcTemplate(dataSource); + } + + @Bean + public PersistenceUnitManager persistenceUnitManager() { + DefaultPersistenceUnitManager persistenceUnitManager = new DefaultPersistenceUnitManager(); + persistenceUnitManager.setDefaultDataSource(dataSource()); + persistenceUnitManager.setPackagesToScan("org.springframework.batch.item.sample"); + persistenceUnitManager.afterPropertiesSet(); + return persistenceUnitManager; + } + + @Bean + public EntityManagerFactory entityManagerFactory() { + LocalContainerEntityManagerFactoryBean factoryBean = new LocalContainerEntityManagerFactoryBean(); + factoryBean.setDataSource(dataSource()); + factoryBean.setPersistenceUnitManager(persistenceUnitManager()); + factoryBean.setJpaVendorAdapter(new HibernateJpaVendorAdapter()); + factoryBean.afterPropertiesSet(); + return factoryBean.getObject(); + } + + @Bean + public PlatformTransactionManager transactionManager() { + return new JpaTransactionManager(entityManagerFactory()); + } + } + +} diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderAsyncTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderAsyncTests.java index d8afb943b8..c2f60958d2 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderAsyncTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderAsyncTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import static org.junit.Assert.assertEquals; @@ -59,7 +74,7 @@ public class JpaPagingItemReaderAsyncTests { @Before public void init() { JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); - maxId = jdbcTemplate.queryForInt("SELECT MAX(ID) from T_FOOS"); + maxId = jdbcTemplate.queryForObject("SELECT MAX(ID) from T_FOOS", Integer.class); for (int i = ITEM_COUNT; i > maxId; i--) { jdbcTemplate.update("INSERT into T_FOOS (ID,NAME,VALUE) values (?, ?, ?)", i, "foo" + i, i); } @@ -74,7 +89,7 @@ public void destroy() { @Test public void testAsyncReader() throws Throwable { - List throwables = new ArrayList(); + List throwables = new ArrayList<>(); int max = 10; for (int i = 0; i < max; i++) { try { @@ -97,12 +112,13 @@ public void testAsyncReader() throws Throwable { */ private void doTest() throws Exception, InterruptedException, ExecutionException { final JpaPagingItemReader reader = getItemReader(); - CompletionService> completionService = new ExecutorCompletionService>(Executors + CompletionService> completionService = new ExecutorCompletionService<>(Executors .newFixedThreadPool(THREAD_COUNT)); for (int i = 0; i < THREAD_COUNT; i++) { completionService.submit(new Callable>() { + @Override public List call() throws Exception { - List list = new ArrayList(); + List list = new ArrayList<>(); Foo next = null; do { next = reader.read(); @@ -117,7 +133,7 @@ public List call() throws Exception { }); } int count = 0; - Set results = new HashSet(); + Set results = new HashSet<>(); for (int i = 0; i < THREAD_COUNT; i++) { List items = completionService.take().get(); count += items.size(); @@ -135,7 +151,7 @@ private JpaPagingItemReader getItemReader() throws Exception { String jpqlQuery = "select f from Foo f"; - JpaPagingItemReader reader = new JpaPagingItemReader(); + JpaPagingItemReader reader = new JpaPagingItemReader<>(); reader.setQueryString(jpqlQuery); reader.setEntityManagerFactory(entityManagerFactory); reader.setPageSize(PAGE_SIZE); diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/GsonJsonFileItemWriterFunctionalTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/GsonJsonFileItemWriterFunctionalTests.java new file mode 100644 index 0000000000..bcd6b4e7a0 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/GsonJsonFileItemWriterFunctionalTests.java @@ -0,0 +1,47 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import com.google.gson.Gson; +import com.google.gson.GsonBuilder; + +import org.springframework.batch.item.json.domain.Trade; + +/** + * @author Mahmoud Ben Hassine + */ +public class GsonJsonFileItemWriterFunctionalTests extends JsonFileItemWriterFunctionalTests { + + @Override + protected JsonObjectMarshaller getJsonObjectMarshaller() { + return new GsonJsonObjectMarshaller<>(); + } + + @Override + protected JsonObjectMarshaller getJsonObjectMarshallerWithPrettyPrint() { + Gson gson = new GsonBuilder().setPrettyPrinting().create(); + GsonJsonObjectMarshaller jsonObjectMarshaller = new GsonJsonObjectMarshaller<>(); + jsonObjectMarshaller.setGson(gson); + return jsonObjectMarshaller; + } + + @Override + protected String getExpectedPrettyPrintedFile() { + return "expected-trades-gson-pretty-print.json"; + } + +} diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/GsonJsonItemReaderFunctionalTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/GsonJsonItemReaderFunctionalTests.java new file mode 100644 index 0000000000..6753f37d2a --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/GsonJsonItemReaderFunctionalTests.java @@ -0,0 +1,38 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import com.google.gson.JsonSyntaxException; + +import org.springframework.batch.item.json.domain.Trade; + +/** + * @author Mahmoud Ben Hassine + */ +public class GsonJsonItemReaderFunctionalTests extends JsonItemReaderFunctionalTests { + + @Override + protected JsonObjectReader getJsonObjectReader() { + return new GsonJsonObjectReader<>(Trade.class); + } + + @Override + protected Class getJsonParsingException() { + return JsonSyntaxException.class; + } + +} diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/JacksonJsonFileItemWriterFunctionalTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/JacksonJsonFileItemWriterFunctionalTests.java new file mode 100644 index 0000000000..d800bc04af --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/JacksonJsonFileItemWriterFunctionalTests.java @@ -0,0 +1,48 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.SerializationFeature; + +import org.springframework.batch.item.json.domain.Trade; + +/** + * @author Mahmoud Ben Hassine + */ +public class JacksonJsonFileItemWriterFunctionalTests extends JsonFileItemWriterFunctionalTests { + + @Override + protected JsonObjectMarshaller getJsonObjectMarshaller() { + return new JacksonJsonObjectMarshaller<>(); + } + + @Override + protected JsonObjectMarshaller getJsonObjectMarshallerWithPrettyPrint() { + ObjectMapper objectMapper = new ObjectMapper(); + objectMapper.enable(SerializationFeature.INDENT_OUTPUT); + JacksonJsonObjectMarshaller jsonObjectMarshaller = new JacksonJsonObjectMarshaller<>(); + jsonObjectMarshaller.setObjectMapper(objectMapper); + return jsonObjectMarshaller; + } + + @Override + protected String getExpectedPrettyPrintedFile() { + return "expected-trades-jackson-pretty-print.json"; + } + +} diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/JacksonJsonItemReaderFunctionalTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/JacksonJsonItemReaderFunctionalTests.java new file mode 100644 index 0000000000..d146d49481 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/JacksonJsonItemReaderFunctionalTests.java @@ -0,0 +1,38 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import com.fasterxml.jackson.core.JsonParseException; + +import org.springframework.batch.item.json.domain.Trade; + +/** + * @author Mahmoud Ben Hassine + */ +public class JacksonJsonItemReaderFunctionalTests extends JsonItemReaderFunctionalTests { + + @Override + protected JsonObjectReader getJsonObjectReader() { + return new JacksonJsonObjectReader<>(Trade.class); + } + + @Override + protected Class getJsonParsingException() { + return JsonParseException.class; + } + +} diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/JsonFileItemWriterFunctionalTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/JsonFileItemWriterFunctionalTests.java new file mode 100644 index 0000000000..ba7da86856 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/JsonFileItemWriterFunctionalTests.java @@ -0,0 +1,311 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import java.io.File; +import java.io.FileInputStream; +import java.math.BigDecimal; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.Arrays; +import java.util.Collections; +import java.util.List; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.batch.item.json.builder.JsonFileItemWriterBuilder; +import org.springframework.batch.item.json.domain.Trade; +import org.springframework.batch.support.transaction.ResourcelessTransactionManager; +import org.springframework.core.io.FileSystemResource; +import org.springframework.core.io.Resource; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.transaction.support.TransactionCallback; +import org.springframework.transaction.support.TransactionTemplate; +import org.springframework.util.DigestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * @author Mahmoud Ben Hassine + */ +public abstract class JsonFileItemWriterFunctionalTests { + + private static final String EXPECTED_FILE_DIRECTORY = "src/test/resources/org/springframework/batch/item/json/"; + + private Resource resource; + private List items; + private ExecutionContext executionContext; + private Trade trade1 = new Trade("123", 5, new BigDecimal("10.5"), "foo"); + private Trade trade2 = new Trade("456", 10, new BigDecimal("20.5"), "bar"); + private Trade trade3 = new Trade("789", 15, new BigDecimal("30.5"), "foobar"); + private Trade trade4 = new Trade("987", 20, new BigDecimal("40.5"), "barfoo"); + + protected abstract JsonObjectMarshaller getJsonObjectMarshaller(); + protected abstract JsonObjectMarshaller getJsonObjectMarshallerWithPrettyPrint(); + protected abstract String getExpectedPrettyPrintedFile(); + + private JsonFileItemWriter writer; + + @Before + public void setUp() throws Exception { + Path outputFilePath = Paths.get("build", "trades.json"); + Files.deleteIfExists(outputFilePath); + this.resource = new FileSystemResource(outputFilePath.toFile()); + this.executionContext = new ExecutionContext(); + this.items = Arrays.asList(this.trade1, this.trade2); + this.writer = new JsonFileItemWriterBuilder() + .name("tradesItemWriter") + .resource(this.resource) + .jsonObjectMarshaller(getJsonObjectMarshaller()) + .build(); + } + + @Test + public void testJsonWriting() throws Exception { + // when + this.writer.open(this.executionContext); + this.writer.write(this.items); + this.writer.close(); + + // then + assertFileEquals( + new File(EXPECTED_FILE_DIRECTORY + "expected-trades.json"), + this.resource.getFile()); + } + + @Test + public void testJsonWritingWithMultipleWrite() throws Exception { + // when + this.writer.open(this.executionContext); + this.writer.write(this.items); + this.writer.write(Arrays.asList(trade3, trade4)); + this.writer.close(); + + // then + assertFileEquals( + new File(EXPECTED_FILE_DIRECTORY + "expected-trades-with-multiple-writes.json"), + this.resource.getFile()); + } + + @Test + public void testJsonWritingWithPrettyPrinting() throws Exception { + // given + this.writer = new JsonFileItemWriterBuilder() + .name("tradesItemWriter") + .resource(this.resource) + .jsonObjectMarshaller(getJsonObjectMarshallerWithPrettyPrint()) + .build(); + + // when + this.writer.open(this.executionContext); + this.writer.write(this.items); + this.writer.close(); + + // when + assertFileEquals( + new File(EXPECTED_FILE_DIRECTORY + getExpectedPrettyPrintedFile()), + this.resource.getFile()); + } + + @Test + public void testJsonWritingWithEnclosingObject() throws Exception { + // given + this.writer.setHeaderCallback(writer -> writer.write("{\"trades\":[")); + this.writer.setFooterCallback(writer -> writer.write(JsonFileItemWriter.DEFAULT_LINE_SEPARATOR + "]}")); + + // when + this.writer.open(this.executionContext); + this.writer.write(this.items); + this.writer.close(); + + // then + assertFileEquals( + new File(EXPECTED_FILE_DIRECTORY + "expected-trades-with-wrapper-object.json"), + this.resource.getFile()); + } + + @Test + public void testForcedWrite() throws Exception { + // given + this.writer.setForceSync(true); + + // when + this.writer.open(this.executionContext); + this.writer.write(Collections.singletonList(this.trade1)); + this.writer.close(); + + // then + assertFileEquals( + new File(EXPECTED_FILE_DIRECTORY + "expected-trades1.json"), + this.resource.getFile()); + } + + @Test + public void testWriteWithDelete() throws Exception { + // given + this.writer.setShouldDeleteIfExists(true); + + // when + this.writer.open(this.executionContext); + this.writer.write(Collections.singletonList(this.trade1)); + this.writer.close(); + this.writer.open(this.executionContext); + this.writer.write(Collections.singletonList(this.trade2)); + this.writer.close(); + + // then + assertFileEquals( + new File(EXPECTED_FILE_DIRECTORY + "expected-trades2.json"), + this.resource.getFile()); + } + + @Test + public void testRestart() throws Exception { + this.writer.open(this.executionContext); + // write some lines + this.writer.write(Collections.singletonList(this.trade1)); + // get restart data + this.writer.update(this.executionContext); + // close template + this.writer.close(); + + // init with correct data + this.writer.open(this.executionContext); + // write more lines + this.writer.write(Collections.singletonList(this.trade2)); + // get statistics + this.writer.update(this.executionContext); + // close template + this.writer.close(); + + // verify what was written to the file + assertFileEquals( + new File(EXPECTED_FILE_DIRECTORY+ "expected-trades.json"), + this.resource.getFile()); + + // 2 lines were written to the file in total + assertEquals(2, this.executionContext.getLong("tradesItemWriter.written")); + } + + @Test + public void testTransactionalRestart() throws Exception { + this.writer.open(this.executionContext); + PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); + + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + // write some lines + this.writer.write(Collections.singletonList(this.trade1)); + } + catch (Exception e) { + throw new UnexpectedInputException("Could not write data", e); + } + // get restart data + this.writer.update(this.executionContext); + return null; + }); + // close template + this.writer.close(); + + // init with correct data + this.writer.open(this.executionContext); + + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + // write more lines + this.writer.write(Collections.singletonList(this.trade2)); + } + catch (Exception e) { + throw new UnexpectedInputException("Could not write data", e); + } + // get restart data + this.writer.update(this.executionContext); + return null; + }); + // close template + this.writer.close(); + + // verify what was written to the file + assertFileEquals( + new File(EXPECTED_FILE_DIRECTORY+ "expected-trades.json"), + this.resource.getFile()); + + // 2 lines were written to the file in total + assertEquals(2, this.executionContext.getLong("tradesItemWriter.written")); + } + + @Test + public void testItemMarshallingFailure() throws Exception { + this.writer.setJsonObjectMarshaller(item -> { + throw new IllegalArgumentException("Bad item"); + }); + this.writer.open(this.executionContext); + try { + this.writer.write(Collections.singletonList(this.trade1)); + fail(); + } + catch (IllegalArgumentException iae) { + assertEquals("Bad item", iae.getMessage()); + } + finally { + this.writer.close(); + } + + assertFileEquals( + new File(EXPECTED_FILE_DIRECTORY + "empty-trades.json"), + this.resource.getFile()); + } + + @Test + /* + * If append=true a new output file should still be created on the first run (not restart). + */ + public void testAppendToNotYetExistingFile() throws Exception { + Resource toBeCreated = new FileSystemResource("build/FlatFileItemWriterTests.out"); + + File outputFile = toBeCreated.getFile(); //enable easy content reading and auto-delete the file + + assertFalse("output file does not exist yet", toBeCreated.exists()); + this.writer.setResource(toBeCreated); + this.writer.setAppendAllowed(true); + this.writer.afterPropertiesSet(); + + this.writer.open(this.executionContext); + assertTrue("output file was created", toBeCreated.exists()); + + this.writer.write(Collections.singletonList(this.trade1)); + this.writer.close(); + assertFileEquals( + new File(EXPECTED_FILE_DIRECTORY + "expected-trades1.json"), + outputFile); + outputFile.delete(); + } + + private void assertFileEquals(File expected, File actual) throws Exception { + String expectedHash = DigestUtils.md5DigestAsHex(new FileInputStream(expected)); + String actualHash = DigestUtils.md5DigestAsHex(new FileInputStream(actual)); + Assert.assertEquals(expectedHash, actualHash); + } +} diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/JsonItemReaderFunctionalTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/JsonItemReaderFunctionalTests.java new file mode 100644 index 0000000000..82d7ad9175 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/JsonItemReaderFunctionalTests.java @@ -0,0 +1,132 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import java.math.BigDecimal; + +import org.hamcrest.Matchers; +import org.junit.Assert; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.ParseException; +import org.springframework.batch.item.json.builder.JsonItemReaderBuilder; +import org.springframework.batch.item.json.domain.Trade; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.ClassPathResource; + +import static org.hamcrest.Matchers.instanceOf; + +/** + * @author Mahmoud Ben Hassine + */ +public abstract class JsonItemReaderFunctionalTests { + + @Rule + public ExpectedException expectedException = ExpectedException.none(); + + protected abstract JsonObjectReader getJsonObjectReader(); + + protected abstract Class getJsonParsingException(); + + @Test + public void testJsonReading() throws Exception { + JsonItemReader itemReader = new JsonItemReaderBuilder() + .jsonObjectReader(getJsonObjectReader()) + .resource(new ClassPathResource("org/springframework/batch/item/json/trades.json")) + .name("tradeJsonItemReader") + .build(); + + itemReader.open(new ExecutionContext()); + + Trade trade = itemReader.read(); + Assert.assertNotNull(trade); + Assert.assertEquals("123", trade.getIsin()); + Assert.assertEquals("foo", trade.getCustomer()); + Assert.assertEquals(new BigDecimal("1.2"), trade.getPrice()); + Assert.assertEquals(1, trade.getQuantity()); + + trade = itemReader.read(); + Assert.assertNotNull(trade); + Assert.assertEquals("456", trade.getIsin()); + Assert.assertEquals("bar", trade.getCustomer()); + Assert.assertEquals(new BigDecimal("1.4"), trade.getPrice()); + Assert.assertEquals(2, trade.getQuantity()); + + trade = itemReader.read(); + Assert.assertNotNull(trade); + Assert.assertEquals("789", trade.getIsin()); + Assert.assertEquals("foobar", trade.getCustomer()); + Assert.assertEquals(new BigDecimal("1.6"), trade.getPrice()); + Assert.assertEquals(3, trade.getQuantity()); + + trade = itemReader.read(); + Assert.assertNotNull(trade); + Assert.assertEquals("100", trade.getIsin()); + Assert.assertEquals("barfoo", trade.getCustomer()); + Assert.assertEquals(new BigDecimal("1.8"), trade.getPrice()); + Assert.assertEquals(4, trade.getQuantity()); + + trade = itemReader.read(); + Assert.assertNull(trade); + } + + @Test + public void testEmptyResource() throws Exception { + JsonItemReader itemReader = new JsonItemReaderBuilder() + .jsonObjectReader(getJsonObjectReader()) + .resource(new ByteArrayResource("[]".getBytes())) + .name("tradeJsonItemReader") + .build(); + + itemReader.open(new ExecutionContext()); + + Trade trade = itemReader.read(); + Assert.assertNull(trade); + } + + @Test + public void testInvalidResourceFormat() { + this.expectedException.expect(ItemStreamException.class); + this.expectedException.expectMessage("Failed to initialize the reader"); + this.expectedException.expectCause(instanceOf(IllegalStateException.class)); + JsonItemReader itemReader = new JsonItemReaderBuilder() + .jsonObjectReader(getJsonObjectReader()) + .resource(new ByteArrayResource("{}, {}".getBytes())) + .name("tradeJsonItemReader") + .build(); + + itemReader.open(new ExecutionContext()); + } + + @Test + public void testInvalidResourceContent() throws Exception { + this.expectedException.expect(ParseException.class); + this.expectedException.expectCause(Matchers.instanceOf(getJsonParsingException())); + JsonItemReader itemReader = new JsonItemReaderBuilder() + .jsonObjectReader(getJsonObjectReader()) + .resource(new ByteArrayResource("[{]".getBytes())) + .name("tradeJsonItemReader") + .build(); + itemReader.open(new ExecutionContext()); + + itemReader.read(); + } +} diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/domain/Trade.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/domain/Trade.java new file mode 100644 index 0000000000..151292a832 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/json/domain/Trade.java @@ -0,0 +1,124 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.json.domain; + +import java.math.BigDecimal; + +/** + * @author Mahmoud Ben Hassine + */ +public class Trade { + + private String isin = ""; + + private long quantity = 0; + + private BigDecimal price = new BigDecimal(0); + + private String customer = ""; + + public Trade() { + } + + public Trade(String isin, long quantity, BigDecimal price, String customer) { + this.isin = isin; + this.quantity = quantity; + this.price = price; + this.customer = customer; + } + + public void setCustomer(String customer) { + this.customer = customer; + } + + public void setIsin(String isin) { + this.isin = isin; + } + + public void setPrice(BigDecimal price) { + this.price = price; + } + + public void setQuantity(long quantity) { + this.quantity = quantity; + } + + public String getIsin() { + return isin; + } + + public BigDecimal getPrice() { + return price; + } + + public long getQuantity() { + return quantity; + } + + public String getCustomer() { + return customer; + } + + @Override + public String toString() { + return "Trade: [isin=" + this.isin + ",quantity=" + this.quantity + ",price=" + this.price + ",customer=" + + this.customer + "]"; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + ((customer == null) ? 0 : customer.hashCode()); + result = prime * result + ((isin == null) ? 0 : isin.hashCode()); + result = prime * result + ((price == null) ? 0 : price.hashCode()); + result = prime * result + (int) (quantity ^ (quantity >>> 32)); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) + return true; + if (obj == null) + return false; + if (getClass() != obj.getClass()) + return false; + Trade other = (Trade) obj; + if (customer == null) { + if (other.customer != null) + return false; + } + else if (!customer.equals(other.customer)) + return false; + if (isin == null) { + if (other.isin != null) + return false; + } + else if (!isin.equals(other.isin)) + return false; + if (price == null) { + if (other.price != null) + return false; + } + else if (!price.equals(other.price)) + return false; + if (quantity != other.quantity) + return false; + return true; + } + +} diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/sample/Foo.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/sample/Foo.java index 9688f920ae..6c8e95db9c 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/sample/Foo.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/sample/Foo.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.sample; import javax.persistence.Entity; @@ -47,6 +62,7 @@ public void setId(int id) { this.id = id; } + @Override public String toString() { return "Foo[id=" +id +",name=" + name + ",value=" + value + "]"; } diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/sample/Person.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/sample/Person.java new file mode 100644 index 0000000000..70d855611c --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/sample/Person.java @@ -0,0 +1,68 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.sample; + +import java.util.Objects; +import javax.persistence.Entity; +import javax.persistence.Id; +import javax.persistence.Table; + +@Entity +@Table(name = "person") +public class Person { + + @Id + private int id; + private String name; + + private Person() { + } + + public Person(int id, String name) { + this.id = id; + this.name = name; + } + + public int getId() { + return id; + } + + public void setId(int id) { + this.id = id; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + @Override + public boolean equals(Object o) { + if (this == o) return true; + if (o == null || getClass() != o.getClass()) return false; + Person person = (Person) o; + return id == person.id && + Objects.equals(name, person.name); + } + + @Override + public int hashCode() { + return Objects.hash(id, name); + } +} diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/AbstractStaxEventReaderItemReaderTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/AbstractStaxEventReaderItemReaderTests.java index 60694c7dcb..86f5cb0dc9 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/AbstractStaxEventReaderItemReaderTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/AbstractStaxEventReaderItemReaderTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2011 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; import static org.junit.Assert.assertEquals; @@ -17,7 +32,7 @@ public abstract class AbstractStaxEventReaderItemReaderTests { - protected StaxEventItemReader reader = new StaxEventItemReader(); + protected StaxEventItemReader reader = new StaxEventItemReader<>(); @Before public void setUp() throws Exception { @@ -31,7 +46,7 @@ public void testRead() throws Exception { reader.setResource(new ClassPathResource(ClassUtils.addResourcePathToPackagePath(getClass(), "input.xml"))); reader.open(new ExecutionContext()); Trade result; - List results = new ArrayList(); + List results = new ArrayList<>(); while ((result = reader.read()) != null) { results.add(result); } @@ -44,7 +59,7 @@ public void testReadNested() throws Exception { .addResourcePathToPackagePath(getClass(), "input-nested.xml"))); reader.open(new ExecutionContext()); Trade result; - List results = new ArrayList(); + List results = new ArrayList<>(); while ((result = reader.read()) != null) { results.add(result); } diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/AbstractStaxEventWriterItemWriterTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/AbstractStaxEventWriterItemWriterTests.java index 53a098120a..2e4a2d94f1 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/AbstractStaxEventWriterItemWriterTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/AbstractStaxEventWriterItemWriterTests.java @@ -1,20 +1,36 @@ +/* + * Copyright 2010-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; import java.io.File; -import java.io.FileReader; import java.math.BigDecimal; import java.util.ArrayList; import java.util.List; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; -import org.custommonkey.xmlunit.XMLAssert; -import org.custommonkey.xmlunit.XMLUnit; import org.junit.After; import org.junit.Before; import org.junit.Test; +import org.xmlunit.builder.Input; +import org.xmlunit.diff.DefaultNodeMatcher; +import org.xmlunit.diff.ElementSelectors; +import org.xmlunit.matchers.CompareMatcher; + import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.item.xml.StaxEventItemWriter; import org.springframework.batch.item.xml.domain.Trade; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; import org.springframework.core.io.ClassPathResource; @@ -27,13 +43,15 @@ import org.springframework.util.ClassUtils; import org.springframework.util.StopWatch; +import static org.junit.Assert.assertThat; + public abstract class AbstractStaxEventWriterItemWriterTests { private Log logger = LogFactory.getLog(getClass()); private static final int MAX_WRITE = 100; - protected StaxEventItemWriter writer = new StaxEventItemWriter(); + protected StaxEventItemWriter writer = new StaxEventItemWriter<>(); private Resource resource; @@ -41,6 +59,7 @@ public abstract class AbstractStaxEventWriterItemWriterTests { protected Resource expected = new ClassPathResource("expected-output.xml", getClass()); + @SuppressWarnings("serial") protected List objects = new ArrayList() { { add(new Trade("isin1", 1, new BigDecimal(1.0), "customer1")); @@ -52,13 +71,15 @@ public abstract class AbstractStaxEventWriterItemWriterTests { /** * Write list of domain objects and check the output file. */ + @SuppressWarnings("resource") @Test public void testWrite() throws Exception { StopWatch stopWatch = new StopWatch(getClass().getSimpleName()); stopWatch.start(); for (int i = 0; i < MAX_WRITE; i++) { - new TransactionTemplate(new ResourcelessTransactionManager()).execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { + new TransactionTemplate(new ResourcelessTransactionManager()).execute(new TransactionCallback() { + @Override + public Void doInTransaction(TransactionStatus status) { try { writer.write(objects); } @@ -75,11 +96,11 @@ public Object doInTransaction(TransactionStatus status) { writer.close(); stopWatch.stop(); logger.info("Timing for XML writer: " + stopWatch); - XMLUnit.setIgnoreWhitespace(true); - // String content = FileUtils.readFileToString(resource.getFile()); - // System.err.println(content); - XMLAssert.assertXMLEqual(new FileReader(expected.getFile()), new FileReader(resource.getFile())); + assertThat( + Input.from(expected.getFile()), + CompareMatcher.isSimilarTo(Input.from(resource.getFile())) + .withNodeMatcher(new DefaultNodeMatcher(ElementSelectors.byNameAndText))); } @Before diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/CastorMarshallingTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/CastorMarshallingTests.java deleted file mode 100644 index e801d0282b..0000000000 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/CastorMarshallingTests.java +++ /dev/null @@ -1,20 +0,0 @@ -package org.springframework.batch.item.xml; - -import org.springframework.core.io.ClassPathResource; -import org.springframework.oxm.Marshaller; -import org.springframework.oxm.castor.CastorMarshaller; - -public class CastorMarshallingTests extends AbstractStaxEventWriterItemWriterTests { - - protected Marshaller getMarshaller() throws Exception { - - CastorMarshaller marshaller = new CastorMarshaller(); - // marshaller.setTargetClass(Trade.class); - marshaller.setMappingLocation(new ClassPathResource("mapping-castor.xml", getClass())); - // there is no way to call - // org.exolab.castor.xml.Marshaller.setSupressXMLDeclaration(); - marshaller.afterPropertiesSet(); - return marshaller; - } - -} diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/CastorUnmarshallingTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/CastorUnmarshallingTests.java deleted file mode 100644 index ba002f1e63..0000000000 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/CastorUnmarshallingTests.java +++ /dev/null @@ -1,18 +0,0 @@ -package org.springframework.batch.item.xml; - -import org.springframework.core.io.ClassPathResource; -import org.springframework.oxm.Unmarshaller; -import org.springframework.oxm.castor.CastorMarshaller; - -public class CastorUnmarshallingTests extends AbstractStaxEventReaderItemReaderTests { - - protected Unmarshaller getUnmarshaller() throws Exception { - CastorMarshaller unmarshaller = new CastorMarshaller(); - unmarshaller.setMappingLocation(new ClassPathResource("mapping-castor.xml", getClass())); - // alternatively target class can be set - //unmarshaller.setTargetClass(Trade.class); - unmarshaller.afterPropertiesSet(); - return unmarshaller; - } - -} diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2MarshallingTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2MarshallingTests.java index 7b37bb439b..c9cf2b2d10 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2MarshallingTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2MarshallingTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; import static org.junit.Assert.assertTrue; @@ -5,6 +20,7 @@ import java.io.StringWriter; import java.math.BigDecimal; +import javax.xml.XMLConstants; import javax.xml.transform.OutputKeys; import javax.xml.transform.Source; import javax.xml.transform.Transformer; @@ -17,6 +33,7 @@ public class Jaxb2MarshallingTests extends AbstractStaxEventWriterItemWriterTests { + @Override protected Marshaller getMarshaller() throws Exception { Jaxb2Marshaller marshaller = new Jaxb2Marshaller(); @@ -32,7 +49,10 @@ protected Marshaller getMarshaller() throws Exception { public static String getTextFromSource(Source source) { try { - Transformer transformer = TransformerFactory.newInstance().newTransformer(); + TransformerFactory transformerFactory = TransformerFactory.newInstance(); + transformerFactory.setAttribute(XMLConstants.ACCESS_EXTERNAL_DTD, ""); + transformerFactory.setAttribute(XMLConstants.ACCESS_EXTERNAL_STYLESHEET, ""); + Transformer transformer = transformerFactory.newTransformer(); transformer.setOutputProperty(OutputKeys.INDENT, "yes"); StreamResult stream = new StreamResult(new StringWriter()); transformer.transform(source, stream); diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2NamespaceMarshallingTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2NamespaceMarshallingTests.java index acba0ff7a9..3021774723 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2NamespaceMarshallingTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2NamespaceMarshallingTests.java @@ -1,25 +1,36 @@ +/* + * Copyright 2010-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; -import static org.junit.Assert.assertTrue; - import java.io.File; -import java.io.FileReader; import java.io.StringWriter; import java.math.BigDecimal; import java.util.ArrayList; import java.util.List; - import javax.xml.transform.stream.StreamResult; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; -import org.custommonkey.xmlunit.XMLAssert; -import org.custommonkey.xmlunit.XMLUnit; import org.junit.After; import org.junit.Before; import org.junit.Test; +import org.xmlunit.builder.Input; +import org.xmlunit.matchers.CompareMatcher; + import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.item.xml.StaxEventItemWriter; import org.springframework.batch.item.xml.domain.QualifiedTrade; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; import org.springframework.core.io.ClassPathResource; @@ -33,13 +44,16 @@ import org.springframework.util.ClassUtils; import org.springframework.util.StopWatch; +import static org.junit.Assert.assertThat; +import static org.junit.Assert.assertTrue; + public class Jaxb2NamespaceMarshallingTests { private Log logger = LogFactory.getLog(getClass()); private static final int MAX_WRITE = 100; - private StaxEventItemWriter writer = new StaxEventItemWriter(); + private StaxEventItemWriter writer = new StaxEventItemWriter<>(); private Resource resource; @@ -47,6 +61,7 @@ public class Jaxb2NamespaceMarshallingTests { private Resource expected = new ClassPathResource("expected-qualified-output.xml", getClass()); + @SuppressWarnings("serial") private List objects = new ArrayList() { { add(new QualifiedTrade("isin1", 1, new BigDecimal(1.0), "customer1")); @@ -58,13 +73,15 @@ public class Jaxb2NamespaceMarshallingTests { /** * Write list of domain objects and check the output file. */ + @SuppressWarnings("resource") @Test public void testWrite() throws Exception { StopWatch stopWatch = new StopWatch(getClass().getSimpleName()); stopWatch.start(); for (int i = 0; i < MAX_WRITE; i++) { - new TransactionTemplate(new ResourcelessTransactionManager()).execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { + new TransactionTemplate(new ResourcelessTransactionManager()).execute(new TransactionCallback() { + @Override + public Void doInTransaction(TransactionStatus status) { try { writer.write(objects); } @@ -81,11 +98,10 @@ public Object doInTransaction(TransactionStatus status) { writer.close(); stopWatch.stop(); logger.info("Timing for XML writer: " + stopWatch); - XMLUnit.setIgnoreWhitespace(true); - // String content = FileUtils.readFileToString(resource.getFile()); - // System.err.println(content); - XMLAssert.assertXMLEqual(new FileReader(expected.getFile()), new FileReader(resource.getFile())); + assertThat( + Input.from(expected.getFile()), + CompareMatcher.isSimilarTo(Input.from(resource.getFile())).normalizeWhitespace()); } @Before diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2NamespaceUnmarshallingTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2NamespaceUnmarshallingTests.java index 420009be96..fb6814a9f4 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2NamespaceUnmarshallingTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2NamespaceUnmarshallingTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2011 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; import static org.junit.Assert.assertEquals; @@ -24,7 +39,7 @@ public class Jaxb2NamespaceUnmarshallingTests { - private StaxEventItemReader reader = new StaxEventItemReader(); + private StaxEventItemReader reader = new StaxEventItemReader<>(); private Resource resource = new ClassPathResource(ClassUtils.addResourcePathToPackagePath(getClass(), "domain/trades.xml")); @@ -51,7 +66,7 @@ public void testUnmarshal() throws Exception { @Test public void testRead() throws Exception { QualifiedTrade result; - List results = new ArrayList(); + List results = new ArrayList<>(); while ((result = reader.read()) != null) { results.add(result); } diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2UnmarshallingTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2UnmarshallingTests.java index 6e4f772e30..7de16603f0 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2UnmarshallingTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/Jaxb2UnmarshallingTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; import org.springframework.batch.item.xml.domain.Trade; @@ -6,6 +21,7 @@ public class Jaxb2UnmarshallingTests extends AbstractStaxEventReaderItemReaderTests { + @Override protected Unmarshaller getUnmarshaller() throws Exception { reader.setFragmentRootElementName("trade"); diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/XStreamMarshallingTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/XStreamMarshallingTests.java index 2d58d12bcc..8d4ddd2c4f 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/XStreamMarshallingTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/XStreamMarshallingTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; import org.springframework.batch.item.xml.domain.Trade; @@ -9,6 +24,7 @@ public class XStreamMarshallingTests extends AbstractStaxEventWriterItemWriterTests { + @Override protected Marshaller getMarshaller() throws Exception { XStreamMarshaller marshaller = new XStreamMarshaller(); // marshaller.addAlias("trade", Trade.class); diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/XStreamUnmarshallingTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/XStreamUnmarshallingTests.java index 731cbe200c..90b4e2a3b9 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/XStreamUnmarshallingTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/XStreamUnmarshallingTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; import java.math.BigDecimal; @@ -10,9 +25,10 @@ public class XStreamUnmarshallingTests extends AbstractStaxEventReaderItemReaderTests { + @Override protected Unmarshaller getUnmarshaller() throws Exception { XStreamMarshaller unmarshaller = new XStreamMarshaller(); - Map> aliasesMap = new HashMap>(); + Map> aliasesMap = new HashMap<>(); aliasesMap.put("trade", Trade.class); aliasesMap.put("isin", String.class); aliasesMap.put("customer", String.class); diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/domain/QualifiedTrade.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/domain/QualifiedTrade.java index d014bf74ab..dc4a08614d 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/domain/QualifiedTrade.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/domain/QualifiedTrade.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml.domain; import java.math.BigDecimal; @@ -70,6 +85,7 @@ public String getCustomer() { return customer; } + @Override public String toString() { return "Trade: [isin=" + this.isin + ",quantity=" + this.quantity + ",price=" + this.price + ",customer=" + this.customer + "]"; diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/domain/Trade.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/domain/Trade.java index 286f1c76e6..a9ad7452d0 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/domain/Trade.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/item/xml/domain/Trade.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml.domain; import java.math.BigDecimal; @@ -61,6 +76,7 @@ public String getCustomer() { return customer; } + @Override public String toString() { return "Trade: [isin=" + this.isin + ",quantity=" + this.quantity + ",price=" + this.price + ",customer=" + this.customer + "]"; diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/jms/ExternalRetryInBatchTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/jms/ExternalRetryInBatchTests.java index 4db6913aae..dc01e623e5 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/jms/ExternalRetryInBatchTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/jms/ExternalRetryInBatchTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,14 +16,6 @@ package org.springframework.batch.jms; -import static org.junit.Assert.assertEquals; - -import java.util.ArrayList; -import java.util.Collections; -import java.util.List; - -import javax.sql.DataSource; - import org.junit.After; import org.junit.Before; import org.junit.Test; @@ -38,6 +30,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jms.core.JmsTemplate; +import org.springframework.lang.Nullable; import org.springframework.retry.RecoveryCallback; import org.springframework.retry.RetryCallback; import org.springframework.retry.RetryContext; @@ -51,6 +44,13 @@ import org.springframework.transaction.support.TransactionCallback; import org.springframework.transaction.support.TransactionTemplate; +import javax.sql.DataSource; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +import static org.junit.Assert.assertEquals; + @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "/org/springframework/batch/jms/jms-context.xml") public class ExternalRetryInBatchTests { @@ -82,6 +82,8 @@ public void onSetUp() throws Exception { jmsTemplate.convertAndSend("queue", "foo"); jmsTemplate.convertAndSend("queue", "bar"); provider = new ItemReader() { + @Nullable + @Override public String read() { String text = (String) jmsTemplate.receiveAndConvert("queue"); list.add(text); @@ -98,13 +100,13 @@ public void onTearDown() throws Exception { } private void assertInitialState() { - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(0, count); } - private List list = new ArrayList(); + private List list = new ArrayList<>(); - private List recovered = new ArrayList(); + private List recovered = new ArrayList<>(); @Test public void testExternalRetryRecoveryInBatch() throws Exception { @@ -119,12 +121,14 @@ public void testExternalRetryRecoveryInBatch() throws Exception { // *internal* retry policy. for (int i = 0; i < 4; i++) { try { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + @Override + public Void doInTransaction(TransactionStatus status) { try { repeatTemplate.iterate(new RepeatCallback() { + @Override public RepeatStatus doInIteration(RepeatContext context) throws Exception { final String item = provider.read(); @@ -133,7 +137,8 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { return RepeatStatus.FINISHED; } - RetryCallback callback = new RetryCallback() { + RetryCallback callback = new RetryCallback() { + @Override public String doWithRetry(RetryContext context) throws Exception { // No need for transaction here: the whole batch will roll // back. When it comes back for recovery this code is not @@ -146,6 +151,7 @@ public String doWithRetry(RetryContext context) throws Exception { }; RecoveryCallback recoveryCallback = new RecoveryCallback() { + @Override public String recover(RetryContext context) { // aggressive commit on a recovery RepeatSynchronizationManager.setCompleteOnly(); @@ -188,7 +194,7 @@ public String recover(RetryContext context) { assertEquals(2, recovered.size()); // The database portion committed once... - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(0, count); // ... and so did the message session. @@ -200,7 +206,7 @@ public String recover(RetryContext context) { private List getMessages() { String next = ""; - List msgs = new ArrayList(); + List msgs = new ArrayList<>(); while (next != null) { next = (String) jmsTemplate.receiveAndConvert("queue"); if (next != null) diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/repeat/jms/AsynchronousTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/repeat/jms/AsynchronousTests.java index 686ec0cb7c..6c764d1a53 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/repeat/jms/AsynchronousTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/repeat/jms/AsynchronousTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,33 +16,37 @@ package org.springframework.batch.repeat.jms; -import static org.junit.Assert.*; - import java.util.ArrayList; import java.util.List; - import javax.jms.JMSException; import javax.jms.Message; import javax.jms.Session; import javax.jms.TextMessage; import javax.sql.DataSource; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + import org.springframework.batch.container.jms.BatchMessageListenerContainer; import org.springframework.batch.jms.ExternalRetryInBatchTests; +import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jms.core.JmsTemplate; import org.springframework.jms.listener.SessionAwareMessageListener; -import org.springframework.util.ClassUtils; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.test.annotation.DirtiesContext; import org.springframework.test.context.ContextConfiguration; -import org.springframework.beans.factory.annotation.Autowired; -import org.junit.runner.RunWith; -import org.junit.Before; -import org.junit.After; -import org.junit.Test; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.util.ClassUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "/org/springframework/batch/jms/jms-context.xml") +@DirtiesContext public class AsynchronousTests { protected String[] getConfigLocations() { @@ -95,10 +99,10 @@ public void onTearDown() throws Exception { } } - private volatile List list = new ArrayList(); + private volatile List list = new ArrayList<>(); private void assertInitialState() { - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(0, count); } @@ -107,7 +111,8 @@ public void testSunnyDay() throws Exception { assertInitialState(); - container.setMessageListener(new SessionAwareMessageListener() { + container.setMessageListener(new SessionAwareMessageListener() { + @Override public void onMessage(Message message, Session session) throws JMSException { list.add(message.toString()); String text = ((TextMessage) message).getText(); @@ -116,7 +121,7 @@ public void onMessage(Message message, Session session) throws JMSException { }); container.initializeProxy(); - + container.start(); // Need to sleep for at least a second here... @@ -129,7 +134,7 @@ public void onMessage(Message message, Session session) throws JMSException { String foo = (String) jmsTemplate.receiveAndConvert("queue"); assertEquals(null, foo); - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(2, count); } @@ -138,11 +143,12 @@ public void onMessage(Message message, Session session) throws JMSException { public void testRollback() throws Exception { assertInitialState(); - + // Prevent us from being overwhelmed after rollback container.setRecoveryInterval(500); - container.setMessageListener(new SessionAwareMessageListener() { + container.setMessageListener(new SessionAwareMessageListener() { + @Override public void onMessage(Message message, Session session) throws JMSException { list.add(message.toString()); final String text = ((TextMessage) message).getText(); @@ -153,7 +159,7 @@ public void onMessage(Message message, Session session) throws JMSException { } } }); - + container.initializeProxy(); container.start(); @@ -169,13 +175,13 @@ public void onMessage(Message message, Session session) throws JMSException { assertTrue(list.size() >= 1); String text = ""; - List msgs = new ArrayList(); + List msgs = new ArrayList<>(); while (text != null) { text = (String) jmsTemplate.receiveAndConvert("queue"); msgs.add(text); } - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(0, count); assertTrue("Foo not on queue", msgs.contains("foo")); @@ -183,9 +189,9 @@ public void onMessage(Message message, Session session) throws JMSException { } /** - * @param list - * @param timeout - * @throws InterruptedException + * @param list resource to monitor + * @param timeout how long to monitor for + * @throws InterruptedException If interrupted while waiting */ private void waitFor(List list, int size, int timeout) throws InterruptedException { int count = 0; diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/repeat/jms/SynchronousTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/repeat/jms/SynchronousTests.java index 821c940068..77320e7797 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/repeat/jms/SynchronousTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/repeat/jms/SynchronousTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.repeat.RepeatCallback; import org.springframework.batch.repeat.RepeatContext; import org.springframework.batch.repeat.RepeatStatus; @@ -41,6 +42,7 @@ import org.springframework.jms.connection.SessionProxy; import org.springframework.jms.core.JmsTemplate; import org.springframework.jms.core.SessionCallback; +import org.springframework.test.annotation.DirtiesContext; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.test.context.transaction.BeforeTransaction; @@ -51,6 +53,7 @@ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "/org/springframework/batch/jms/jms-context.xml") +@DirtiesContext public class SynchronousTests implements ApplicationContextAware { @Autowired @@ -66,6 +69,9 @@ public class SynchronousTests implements ApplicationContextAware { private ApplicationContext applicationContext; + private List list = new ArrayList<>(); + + @Override public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { this.applicationContext = applicationContext; } @@ -89,12 +95,10 @@ public void onSetUpBeforeTransaction() throws Exception { } private void assertInitialState() { - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(0, count); } - List list = new ArrayList(); - @Transactional @Test public void testCommit() throws Exception { @@ -102,6 +106,7 @@ public void testCommit() throws Exception { assertInitialState(); repeatTemplate.iterate(new RepeatCallback() { + @Override public RepeatStatus doInIteration(RepeatContext context) throws Exception { String text = (String) jmsTemplate.receiveAndConvert("queue"); list.add(text); @@ -110,9 +115,12 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { } }); - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(2, count); + assertTrue(list.contains("foo")); + assertTrue(list.contains("bar")); + String text = (String) jmsTemplate.receiveAndConvert("queue"); assertEquals(null, text); @@ -121,11 +129,15 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { @Test public void testFullRollback() throws Exception { + onSetUpBeforeTransaction(); + assertInitialState(); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - public Object doInTransaction(org.springframework.transaction.TransactionStatus status) { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + @Override + public Void doInTransaction(org.springframework.transaction.TransactionStatus status) { repeatTemplate.iterate(new RepeatCallback() { + @Override public RepeatStatus doInIteration(RepeatContext context) throws Exception { String text = (String) jmsTemplate.receiveAndConvert("queue"); list.add(text); @@ -140,14 +152,14 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { }); String text = ""; - List msgs = new ArrayList(); + List msgs = new ArrayList<>(); while (text != null) { text = (String) jmsTemplate.receiveAndConvert("queue"); msgs.add(text); } // The database portion rolled back... - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(0, count); // ... and so did the message session. The rollback should have restored @@ -155,7 +167,8 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { assertTrue("Foo not on queue", msgs.contains("foo")); } - @Transactional @Test + @Transactional + @Test public void testPartialRollback() throws Exception { // The JmsTemplate is used elsewhere outside a transaction, so @@ -167,10 +180,12 @@ public void testPartialRollback() throws Exception { assertInitialState(); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - public Object doInTransaction(org.springframework.transaction.TransactionStatus status) { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + @Override + public Void doInTransaction(org.springframework.transaction.TransactionStatus status) { repeatTemplate.iterate(new RepeatCallback() { + @Override public RepeatStatus doInIteration(RepeatContext context) throws Exception { String text = (String) txJmsTemplate.receiveAndConvert("queue"); list.add(text); @@ -181,8 +196,9 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { // Simulate a message system failure before the main transaction // commits... - txJmsTemplate.execute(new SessionCallback() { - public Object doInJms(Session session) throws JMSException { + txJmsTemplate.execute(new SessionCallback() { + @Override + public Void doInJms(Session session) throws JMSException { try { assertTrue("Not a SessionProxy - wrong spring version?", session instanceof SessionProxy); ((SessionProxy) session).getTargetSession().rollback(); @@ -203,14 +219,14 @@ public Object doInJms(Session session) throws JMSException { }); String text = ""; - List msgs = new ArrayList(); + List msgs = new ArrayList<>(); while (text != null) { text = (String) txJmsTemplate.receiveAndConvert("queue"); msgs.add(text); } // The database portion committed... - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(2, count); // ...but the JMS session rolled back, so the message is still there diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplateBulkAsynchronousTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplateBulkAsynchronousTests.java index 897d0ebac7..dabbb30c2f 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplateBulkAsynchronousTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplateBulkAsynchronousTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -82,12 +82,13 @@ public void setUp() { template.setTaskExecutor(taskExecutor); template.setThrottleLimit(throttleLimit); - items = Collections.synchronizedList(new ArrayList()); + items = Collections.synchronizedList(new ArrayList<>()); callback = new RepeatCallback() { private volatile AtomicInteger count = new AtomicInteger(0); + @Override public RepeatStatus doInIteration(RepeatContext context) throws Exception { int position = count.incrementAndGet(); @@ -240,6 +241,7 @@ public void testErrorThrownByCallback() throws Exception { private volatile AtomicInteger count = new AtomicInteger(0); + @Override public RepeatStatus doInIteration(RepeatContext context) throws Exception { int position = count.incrementAndGet(); diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/retry/jms/ExternalRetryTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/retry/jms/ExternalRetryTests.java index 42eece1496..df41e25ec5 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/retry/jms/ExternalRetryTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/retry/jms/ExternalRetryTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,15 +16,6 @@ package org.springframework.batch.retry.jms; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.fail; - -import java.util.ArrayList; -import java.util.Collections; -import java.util.List; - -import javax.sql.DataSource; - import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; @@ -33,6 +24,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jms.core.JmsTemplate; +import org.springframework.lang.Nullable; import org.springframework.retry.RecoveryCallback; import org.springframework.retry.RetryCallback; import org.springframework.retry.RetryContext; @@ -45,6 +37,14 @@ import org.springframework.transaction.support.TransactionCallback; import org.springframework.transaction.support.TransactionTemplate; +import javax.sql.DataSource; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; + @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "/org/springframework/batch/jms/jms-context.xml") public class ExternalRetryTests { @@ -72,6 +72,8 @@ public void onSetUp() throws Exception { jdbcTemplate.execute("delete from T_BARS"); jmsTemplate.convertAndSend("queue", "foo"); provider = new ItemReader() { + @Nullable + @Override public String read() { String text = (String) jmsTemplate.receiveAndConvert("queue"); list.add(text); @@ -82,13 +84,13 @@ public String read() { } private void assertInitialState() { - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(0, count); } - private List list = new ArrayList(); + private List list = new ArrayList<>(); - private List recovered = new ArrayList(); + private List recovered = new ArrayList<>(); /* * Message processing is successful on the second attempt but must receive @@ -100,7 +102,8 @@ public void testExternalRetrySuccessOnSecondAttempt() throws Exception { assertInitialState(); final ItemWriter writer = new ItemWriter() { - public void write(final List texts) { + @Override + public void write(final List texts) { for (Object text : texts) { @@ -116,11 +119,13 @@ public void write(final List texts) { }; try { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + @Override public Object doInTransaction(TransactionStatus status) { try { final Object item = provider.read(); - RetryCallback callback = new RetryCallback() { + RetryCallback callback = new RetryCallback() { + @Override public Object doWithRetry(RetryContext context) throws Exception { writer.write(Collections.singletonList(item)); return null; @@ -144,11 +149,13 @@ public Object doWithRetry(RetryContext context) throws Exception { } - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + @Override public Object doInTransaction(TransactionStatus status) { try { final String item = provider.read(); - RetryCallback callback = new RetryCallback() { + RetryCallback callback = new RetryCallback() { + @Override public Object doWithRetry(RetryContext context) throws Exception { writer.write(Collections.singletonList(item)); return null; @@ -165,7 +172,7 @@ public Object doWithRetry(RetryContext context) throws Exception { List msgs = getMessages(); // The database portion committed once... - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(1, count); // ... and so did the message session. @@ -181,7 +188,8 @@ public void testExternalRetryWithRecovery() throws Exception { assertInitialState(); final String item = provider.read(); - final RetryCallback callback = new RetryCallback() { + final RetryCallback callback = new RetryCallback() { + @Override public String doWithRetry(RetryContext context) throws Exception { jdbcTemplate.update("INSERT into T_BARS (id,name,foo_date) values (?,?,null)", list.size(), item); throw new RuntimeException("Rollback!"); @@ -189,6 +197,7 @@ public String doWithRetry(RetryContext context) throws Exception { }; final RecoveryCallback recoveryCallback = new RecoveryCallback() { + @Override public String recover(RetryContext context) { recovered.add(item); return item; @@ -199,8 +208,9 @@ public String recover(RetryContext context) { for (int i = 0; i < 4; i++) { try { - result = (String) new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { + result = new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + @Override + public String doInTransaction(TransactionStatus status) { try { return retryTemplate.execute(callback, recoveryCallback, new DefaultRetryState(item)); } @@ -230,7 +240,7 @@ public Object doInTransaction(TransactionStatus status) { assertEquals(1, recovered.size()); // The database portion committed once... - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(0, count); // ... and so did the message session. @@ -240,7 +250,7 @@ public Object doInTransaction(TransactionStatus status) { private List getMessages() { String next = ""; - List msgs = new ArrayList(); + List msgs = new ArrayList<>(); while (next != null) { next = (String) jmsTemplate.receiveAndConvert("queue"); if (next != null) diff --git a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/retry/jms/SynchronousTests.java b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/retry/jms/SynchronousTests.java index 73afa89b4f..3662be81f0 100644 --- a/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/retry/jms/SynchronousTests.java +++ b/spring-batch-infrastructure-tests/src/test/java/org/springframework/batch/retry/jms/SynchronousTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,16 +16,6 @@ package org.springframework.batch.retry.jms; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - -import java.util.ArrayList; -import java.util.List; - -import javax.sql.DataSource; - import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; @@ -48,6 +38,15 @@ import org.springframework.transaction.support.TransactionTemplate; import org.springframework.util.ClassUtils; +import javax.sql.DataSource; +import java.util.ArrayList; +import java.util.List; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "/org/springframework/batch/jms/jms-context.xml") public class SynchronousTests { @@ -98,11 +97,11 @@ public void afterTransaction() { } private void assertInitialState() { - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(0, count); } - List list = new ArrayList(); + List list = new ArrayList<>(); /* * Message processing is successful on the second attempt without having to @@ -124,13 +123,15 @@ public void testInternalRetrySuccessOnSecondAttempt() throws Exception { final String text = (String) jmsTemplate.receiveAndConvert("queue"); assertNotNull(text); - retryTemplate.execute(new RetryCallback() { + retryTemplate.execute(new RetryCallback() { + @Override public String doWithRetry(RetryContext status) throws Exception { TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager); transactionTemplate.setPropagationBehavior(TransactionTemplate.PROPAGATION_NESTED); - return (String) transactionTemplate.execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { + return transactionTemplate.execute(new TransactionCallback() { + @Override + public String doInTransaction(TransactionStatus status) { list.add(text); System.err.println("Inserting: [" + list.size() + "," + text + "]"); @@ -151,7 +152,7 @@ public Object doInTransaction(TransactionStatus status) { List msgs = getMessages(); // The database portion committed once... - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(1, count); // ... and so did the message session. @@ -167,20 +168,22 @@ public void testInternalRetrySuccessOnSecondAttemptWithItemProvider() throws Exc assertInitialState(); - JmsItemReader provider = new JmsItemReader(); + JmsItemReader provider = new JmsItemReader<>(); // provider.setItemType(Message.class); jmsTemplate.setDefaultDestinationName("queue"); provider.setJmsTemplate(jmsTemplate); - final Object item = provider.read(); + final String item = (String) provider.read(); - retryTemplate.execute(new RetryCallback() { + retryTemplate.execute(new RetryCallback() { + @Override public String doWithRetry(RetryContext context) throws Exception { TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager); transactionTemplate.setPropagationBehavior(TransactionTemplate.PROPAGATION_NESTED); - return (String) transactionTemplate.execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { + return transactionTemplate.execute(new TransactionCallback() { + @Override + public String doInTransaction(TransactionStatus status) { list.add(item); System.err.println("Inserting: [" + list.size() + "," + item + "]"); @@ -202,7 +205,7 @@ public Object doInTransaction(TransactionStatus status) { List msgs = getMessages(); // The database portion committed once... - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(1, count); // ... and so did the message session. @@ -229,19 +232,22 @@ public void testInternalRetrySuccessOnFirstAttemptRollbackOuter() throws Excepti TransactionTemplate outerTxTemplate = new TransactionTemplate(transactionManager); outerTxTemplate.setPropagationBehavior(TransactionTemplate.PROPAGATION_REQUIRES_NEW); - outerTxTemplate.execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus outerStatus) { + outerTxTemplate.execute(new TransactionCallback() { + @Override + public Void doInTransaction(TransactionStatus outerStatus) { final String text = (String) jmsTemplate.receiveAndConvert("queue"); try { - retryTemplate.execute(new RetryCallback() { + retryTemplate.execute(new RetryCallback() { + @Override public String doWithRetry(RetryContext status) throws Exception { TransactionTemplate nestedTxTemplate = new TransactionTemplate(transactionManager); nestedTxTemplate.setPropagationBehavior(TransactionTemplate.PROPAGATION_NESTED); - return (String) nestedTxTemplate.execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus nestedStatus) { + return nestedTxTemplate.execute(new TransactionCallback() { + @Override + public String doInTransaction(TransactionStatus nestedStatus) { list.add(text); System.err.println("Inserting: [" + list.size() + "," + text + "]"); @@ -258,7 +264,7 @@ public Object doInTransaction(TransactionStatus nestedStatus) { } // The nested database transaction has committed... - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(1, count); // force rollback... @@ -268,12 +274,12 @@ public Object doInTransaction(TransactionStatus nestedStatus) { } }); - // Verify the state after stransactional processing is complete + // Verify the state after transactional processing is complete List msgs = getMessages(); // The database portion rolled back... - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(0, count); // ... and so did the message session. @@ -290,14 +296,16 @@ public void testExternalRetrySuccessOnSecondAttempt() throws Exception { assertInitialState(); - retryTemplate.execute(new RetryCallback() { + retryTemplate.execute(new RetryCallback() { + @Override public String doWithRetry(RetryContext status) throws Exception { // use REQUIRES_NEW so that the retry executes in its own transaction TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager); transactionTemplate.setPropagationBehavior(TransactionTemplate.PROPAGATION_REQUIRES_NEW); - return (String) transactionTemplate.execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { + return transactionTemplate.execute(new TransactionCallback() { + @Override + public String doInTransaction(TransactionStatus status) { // The receive is inside the retry and the // transaction... @@ -315,12 +323,12 @@ public Object doInTransaction(TransactionStatus status) { } }); - // Verify the state after stransactional processing is complete + // Verify the state after transactional processing is complete List msgs = getMessages(); // The database portion committed once... - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(1, count); // ... and so did the message session. @@ -338,14 +346,16 @@ public void testExternalRetryFailOnSecondAttempt() throws Exception { try { - retryTemplate.execute(new RetryCallback() { + retryTemplate.execute(new RetryCallback() { + @Override public String doWithRetry(RetryContext status) throws Exception { // use REQUIRES_NEW so that the retry executes in its own transaction TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager); transactionTemplate.setPropagationBehavior(TransactionTemplate.PROPAGATION_REQUIRES_NEW); - return (String) transactionTemplate.execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { + return transactionTemplate.execute(new TransactionCallback() { + @Override + public String doInTransaction(TransactionStatus status) { // The receive is inside the retry and the // transaction... @@ -377,7 +387,7 @@ public Object doInTransaction(TransactionStatus status) { List msgs = getMessages(); // The database portion rolled back... - int count = jdbcTemplate.queryForInt("select count(*) from T_BARS"); + int count = jdbcTemplate.queryForObject("select count(*) from T_BARS", Integer.class); assertEquals(0, count); // ... and so did the message session. @@ -386,7 +396,7 @@ public Object doInTransaction(TransactionStatus status) { private List getMessages() { String next = ""; - List msgs = new ArrayList(); + List msgs = new ArrayList<>(); while (next != null) { next = (String) jmsTemplate.receiveAndConvert("queue"); if (next != null) diff --git a/spring-batch-infrastructure-tests/src/test/java/test/jdbc/datasource/DataSourceInitializer.java b/spring-batch-infrastructure-tests/src/test/java/test/jdbc/datasource/DataSourceInitializer.java index db1f9ad857..4b67fbd8a8 100644 --- a/spring-batch-infrastructure-tests/src/test/java/test/jdbc/datasource/DataSourceInitializer.java +++ b/spring-batch-infrastructure-tests/src/test/java/test/jdbc/datasource/DataSourceInitializer.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -56,6 +56,7 @@ public void setInitialize(boolean initialize) { this.initialize = initialize; } + @Override public void destroy() throws Exception { if (!initialized) { return; @@ -76,8 +77,9 @@ public void destroy() throws Exception { } } + @Override public void afterPropertiesSet() throws Exception { - Assert.notNull(dataSource); + Assert.notNull(dataSource, "DataSource is required"); logger.info("Initializing with scripts: " + Arrays.asList(initScripts)); if (!initialized && initialize) { try { @@ -103,9 +105,8 @@ private void doExecuteScript(final Resource scriptResource) { final JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); String[] scripts; try { - @SuppressWarnings("unchecked") String[] list = StringUtils.delimitedListToStringArray(stripComments(IOUtils.readLines(scriptResource - .getInputStream())), ";"); + .getInputStream(), "UTF-8")), ";"); scripts = list; } catch (IOException e) { @@ -115,9 +116,10 @@ private void doExecuteScript(final Resource scriptResource) { final String script = scripts[i].trim(); TransactionTemplate transactionTemplate = new TransactionTemplate(new DataSourceTransactionManager( dataSource)); - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { + @Override + public Void doInTransaction(TransactionStatus status) { if (StringUtils.hasText(script)) { try { jdbcTemplate.execute(script); @@ -137,10 +139,10 @@ public Object doInTransaction(TransactionStatus status) { } private String stripComments(List list) { - StringBuffer buffer = new StringBuffer(); + StringBuilder buffer = new StringBuilder(); for (String line : list) { if (!line.startsWith("//") && !line.startsWith("--")) { - buffer.append(line + "\n"); + buffer.append(line).append("\n"); } } return buffer.toString(); diff --git a/spring-batch-infrastructure-tests/src/test/resources/META-INF/persistence.xml b/spring-batch-infrastructure-tests/src/test/resources/META-INF/persistence.xml index f2ab28139d..c5f634cbf6 100644 --- a/spring-batch-infrastructure-tests/src/test/resources/META-INF/persistence.xml +++ b/spring-batch-infrastructure-tests/src/test/resources/META-INF/persistence.xml @@ -1,6 +1,6 @@ diff --git a/spring-batch-infrastructure-tests/src/test/resources/batch-derby.properties b/spring-batch-infrastructure-tests/src/test/resources/batch-derby.properties index f35dc1a112..d7328b0255 100644 --- a/spring-batch-infrastructure-tests/src/test/resources/batch-derby.properties +++ b/spring-batch-infrastructure-tests/src/test/resources/batch-derby.properties @@ -1,7 +1,7 @@ # Placeholders batch.* # for Derby: batch.jdbc.driver=org.apache.derby.jdbc.EmbeddedDriver -batch.jdbc.url=jdbc:derby:derby-home/test;create=true +batch.jdbc.url=jdbc:derby:build/derby-home/test;create=true batch.jdbc.user=app batch.jdbc.password= batch.jdbc.testWhileIdle=false @@ -12,4 +12,4 @@ batch.business.schema.script=classpath:/org/springframework/batch/jms/init.sql batch.data.source.init=true batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.DerbyMaxValueIncrementer batch.database.incrementer.parent=columnIncrementerParent -batch.verify.cursor.position=false \ No newline at end of file +batch.verify.cursor.position=false diff --git a/spring-batch-infrastructure-tests/src/test/resources/batch-hsql.properties b/spring-batch-infrastructure-tests/src/test/resources/batch-hsql.properties index d179b0d85d..548eb7f8f8 100644 --- a/spring-batch-infrastructure-tests/src/test/resources/batch-hsql.properties +++ b/spring-batch-infrastructure-tests/src/test/resources/batch-hsql.properties @@ -1,7 +1,7 @@ # Placeholders batch.* # for HSQLDB: batch.jdbc.driver=org.hsqldb.jdbcDriver -batch.jdbc.url=jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true +batch.jdbc.url=jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true;hsqldb.tx=mvcc # use this one for a separate server process so you can inspect the results # (or add it to system properties with -D to override at run time). # batch.jdbc.url=jdbc:hsqldb:hsql://localhost:9005/samples diff --git a/spring-batch-infrastructure-tests/src/test/resources/batch-postgres.properties b/spring-batch-infrastructure-tests/src/test/resources/batch-postgres.properties index 11cd0e0885..fa15935f50 100644 --- a/spring-batch-infrastructure-tests/src/test/resources/batch-postgres.properties +++ b/spring-batch-infrastructure-tests/src/test/resources/batch-postgres.properties @@ -9,6 +9,6 @@ batch.jdbc.validationQuery= batch.schema.script=classpath:org/springframework/batch/item/database/init-foo-schema-postgres.sql batch.business.schema.script=classpath:/org/springframework/batch/jms/init.sql batch.data.source.init=true -batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.PostgreSQLSequenceMaxValueIncrementer +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.PostgresSequenceMaxValueIncrementer batch.database.incrementer.parent=sequenceIncrementerParent batch.verify.cursor.position=true diff --git a/spring-batch-infrastructure-tests/src/test/resources/data-source-context.xml b/spring-batch-infrastructure-tests/src/test/resources/data-source-context.xml index 2779dfb6e5..d7faa1953e 100644 --- a/spring-batch-infrastructure-tests/src/test/resources/data-source-context.xml +++ b/spring-batch-infrastructure-tests/src/test/resources/data-source-context.xml @@ -1,6 +1,6 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> @@ -13,7 +13,7 @@ - + diff --git a/spring-batch-infrastructure-tests/src/test/resources/log4j.properties b/spring-batch-infrastructure-tests/src/test/resources/log4j.properties index a12e7324cb..da9add047b 100644 --- a/spring-batch-infrastructure-tests/src/test/resources/log4j.properties +++ b/spring-batch-infrastructure-tests/src/test/resources/log4j.properties @@ -1,7 +1,7 @@ log4j.rootCategory=INFO, stdout -log4j.appender.stdout=org.apache.log4j.ConsoleAppender -log4j.appender.stdout.layout=org.apache.log4j.PatternLayout +log4j.appender.stdout=org.apache.logging.log4j.core.appender.ConsoleAppender +log4j.appender.stdout.layout=org.apache.logging.log4j.core.layout.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %t %c{1}:%L - %m%n log4j.category.org.apache.activemq=ERROR diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/Foo-write.hbm.xml b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/Foo-write.hbm.xml deleted file mode 100644 index 612a29924f..0000000000 --- a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/Foo-write.hbm.xml +++ /dev/null @@ -1,15 +0,0 @@ - - - - - - - - - - - - - \ No newline at end of file diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/Foo.hbm.xml b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/Foo.hbm.xml index 4c4b7bbbb0..7f05cc9ed6 100644 --- a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/Foo.hbm.xml +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/Foo.hbm.xml @@ -1,7 +1,7 @@ + "https://hibernate.org/dtd/hibernate-mapping-3.0.dtd"> diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests-context.xml b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests-context.xml index ac902d174f..964de7c89c 100644 --- a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests-context.xml +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd+http://www.springframework.org/schema/util%20https://www.springframework.org/schema/util/spring-util-3.1.xsd"> diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests-context.xml b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests-context.xml index 0e1d8476ae..262f3dbfe2 100644 --- a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests-context.xml +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests-context.xml @@ -1,8 +1,7 @@ - + diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/data-source-context.xml b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/data-source-context.xml index 74b74ec6b6..38d4073019 100644 --- a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/data-source-context.xml +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/data-source-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd+http://www.springframework.org/schema/util%20https://www.springframework.org/schema/util/spring-util-3.1.xsd"> diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/ibatis-config.xml b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/ibatis-config.xml deleted file mode 100644 index f23807357e..0000000000 --- a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/ibatis-config.xml +++ /dev/null @@ -1,12 +0,0 @@ - - - - - - - - - - \ No newline at end of file diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/ibatis-foo.xml b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/ibatis-foo.xml deleted file mode 100644 index 307e03e5f8..0000000000 --- a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/database/ibatis-foo.xml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - insert INTO T_WRITE_FOOS (ID, NAME, VALUE) VALUES (#id#, #name#, #value#) - - - - update T_WRITE_FOOS set NAME = #name#, VALUE = #value# where ID = #id# - - - - delete from T_WRITE_FOOS where ID = #id# - - - \ No newline at end of file diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/empty-trades.json b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/empty-trades.json new file mode 100644 index 0000000000..41b42e677b --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/empty-trades.json @@ -0,0 +1,3 @@ +[ + +] diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades-gson-pretty-print.json b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades-gson-pretty-print.json new file mode 100644 index 0000000000..51fb2644e1 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades-gson-pretty-print.json @@ -0,0 +1,14 @@ +[ + { + "isin": "123", + "quantity": 5, + "price": 10.5, + "customer": "foo" +}, + { + "isin": "456", + "quantity": 10, + "price": 20.5, + "customer": "bar" +} +] diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades-jackson-pretty-print.json b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades-jackson-pretty-print.json new file mode 100644 index 0000000000..36a470f62f --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades-jackson-pretty-print.json @@ -0,0 +1,14 @@ +[ + { + "isin" : "123", + "quantity" : 5, + "price" : 10.5, + "customer" : "foo" +}, + { + "isin" : "456", + "quantity" : 10, + "price" : 20.5, + "customer" : "bar" +} +] diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades-with-multiple-writes.json b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades-with-multiple-writes.json new file mode 100644 index 0000000000..f010ec6e86 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades-with-multiple-writes.json @@ -0,0 +1,6 @@ +[ + {"isin":"123","quantity":5,"price":10.5,"customer":"foo"}, + {"isin":"456","quantity":10,"price":20.5,"customer":"bar"}, + {"isin":"789","quantity":15,"price":30.5,"customer":"foobar"}, + {"isin":"987","quantity":20,"price":40.5,"customer":"barfoo"} +] diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades-with-wrapper-object.json b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades-with-wrapper-object.json new file mode 100644 index 0000000000..e8cfeca829 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades-with-wrapper-object.json @@ -0,0 +1,4 @@ +{"trades":[ + {"isin":"123","quantity":5,"price":10.5,"customer":"foo"}, + {"isin":"456","quantity":10,"price":20.5,"customer":"bar"} +]} \ No newline at end of file diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades.json b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades.json new file mode 100644 index 0000000000..49a4b7bcb2 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades.json @@ -0,0 +1,4 @@ +[ + {"isin":"123","quantity":5,"price":10.5,"customer":"foo"}, + {"isin":"456","quantity":10,"price":20.5,"customer":"bar"} +] diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades1.json b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades1.json new file mode 100644 index 0000000000..ae3c0b08b1 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades1.json @@ -0,0 +1,3 @@ +[ + {"isin":"123","quantity":5,"price":10.5,"customer":"foo"} +] diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades2.json b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades2.json new file mode 100644 index 0000000000..e7ac199033 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/expected-trades2.json @@ -0,0 +1,3 @@ +[ + {"isin":"456","quantity":10,"price":20.5,"customer":"bar"} +] diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/trades.json b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/trades.json new file mode 100644 index 0000000000..d3780a48d4 --- /dev/null +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/item/json/trades.json @@ -0,0 +1,26 @@ +[ + { + "isin": "123", + "quantity": 1, + "price": 1.2, + "customer": "foo" + }, + { + "isin": "456", + "quantity": 2, + "price": 1.4, + "customer": "bar" + }, + { + "isin": "789", + "quantity": 3, + "price": 1.6, + "customer": "foobar" + }, + { + "isin": "100", + "quantity": 4, + "price": 1.8, + "customer": "barfoo" + } +] diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/jms/init.sql b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/jms/init.sql index 655789e8ff..dc8893d01b 100644 --- a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/jms/init.sql +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/jms/init.sql @@ -1,3 +1,5 @@ +DROP TABLE T_BARS; + create table T_BARS ( id integer not null primary key, name varchar(80), diff --git a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/jms/jms-context.xml b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/jms/jms-context.xml index 89a9dc646c..d6dfcabf4d 100644 --- a/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/jms/jms-context.xml +++ b/spring-batch-infrastructure-tests/src/test/resources/org/springframework/batch/jms/jms-context.xml @@ -1,11 +1,8 @@ - + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -20,10 +17,13 @@ - - - vm://localhost - + + + + + + + - - org.springframework.batch.repeat.RepeatOperations - + - + @@ -95,7 +93,7 @@ - vm://localhost + vm://localhost?jms.prefetchPolicy.all=0 diff --git a/spring-batch-infrastructure-tests/template.mf b/spring-batch-infrastructure-tests/template.mf deleted file mode 100755 index 2d2b26bfa8..0000000000 --- a/spring-batch-infrastructure-tests/template.mf +++ /dev/null @@ -1,11 +0,0 @@ -Manifest-Version: 1.0 -Bundle-SymbolicName: org.springframework.batch.infrastructure.tests -Bundle-Name: Spring Batch Infrastructure Tests -Bundle-Vendor: Spring -Bundle-Version: ${version} -Bundle-ManifestVersion: 2 -Ignored-Existing-Headers: - Import-Package, - Export-Package -Import-Template: - *;version="0" diff --git a/spring-batch-infrastructure/.springBeans b/spring-batch-infrastructure/.springBeans index 5f547405dc..de3e57ab66 100644 --- a/spring-batch-infrastructure/.springBeans +++ b/spring-batch-infrastructure/.springBeans @@ -1,26 +1,25 @@ - - - 1 - - - - - - - src/test/resources/org/springframework/batch/item/database/data-source-context.xml - src/test/resources/org/springframework/batch/item/file/mapping/bean-wrapper.xml - src/test/resources/org/springframework/batch/item/adapter/delegating-item-processor.xml - src/test/resources/org/springframework/batch/item/adapter/delegating-item-provider.xml - src/test/resources/org/springframework/batch/item/adapter/delegating-item-writer.xml - src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests-context.xml - src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderConfigTests-context.xml - src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderParameterTests-context.xml - src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests-context.xml - src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderParameterTests-context.xml - src/test/resources/org/springframework/batch/item/adapter/pe-delegating-item-writer.xml - src/test/resources/org/springframework/batch/retry/interceptor/retry-transaction-test.xml - src/test/resources/org/springframework/batch/item/database/stored-procedure-context.xml - - - - + + + 1 + + + + + + + src/test/resources/org/springframework/batch/item/database/data-source-context.xml + src/test/resources/org/springframework/batch/item/file/mapping/bean-wrapper.xml + src/test/resources/org/springframework/batch/item/adapter/delegating-item-processor.xml + src/test/resources/org/springframework/batch/item/adapter/delegating-item-provider.xml + src/test/resources/org/springframework/batch/item/adapter/delegating-item-writer.xml + src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests-context.xml + src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderConfigTests-context.xml + src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderParameterTests-context.xml + src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests-context.xml + src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderParameterTests-context.xml + src/test/resources/org/springframework/batch/item/adapter/pe-delegating-item-writer.xml + src/test/resources/org/springframework/batch/item/database/stored-procedure-context.xml + + + + diff --git a/spring-batch-infrastructure/pom.xml b/spring-batch-infrastructure/pom.xml deleted file mode 100644 index 674a1bb4c9..0000000000 --- a/spring-batch-infrastructure/pom.xml +++ /dev/null @@ -1,246 +0,0 @@ - - - 4.0.0 - spring-batch-infrastructure - jar - Infrastructure - - - http://static.springframework.org/spring-batch/${project.artifactId} - - org.springframework.batch - spring-batch-parent - 2.2.2.BUILD-SNAPSHOT - ../spring-batch-parent - - - - - org.apache.maven.plugins - maven-antrun-plugin - - - - - - org.apache.maven.plugins - maven-surefire-plugin - - junit:junit - - - - - - - - junit - junit - - - org.aspectj - aspectjrt - test - - - org.aspectj - aspectjweaver - test - - - cglib - cglib-nodep - true - - - org.apache.geronimo.specs - geronimo-jms_1.1_spec - true - - - org.slf4j - slf4j-log4j12 - true - - - log4j - log4j - test - - - commons-io - commons-io - test - - - commons-dbcp - commons-dbcp - test - - - org.codehaus.jackson - jackson-mapper-asl - 1.0.1 - true - - - org.hsqldb - hsqldb - test - - - com.h2database - h2 - test - - - org.apache.derby - derby - test - - - org.hibernate - hibernate-core - true - - - org.hibernate - hibernate-entitymanager - true - - - org.hibernate - hibernate-annotations - true - - - org.hibernate - hibernate-validator - true - - - org.apache.geronimo.specs - geronimo-jta_1.1_spec - true - - - org.apache.ibatis - ibatis-sqlmap - true - - - javax.mail - mail - 1.4 - true - - - org.springframework - spring-oxm - true - - - org.springframework - spring-core - - - org.springframework - spring-aop - true - - - org.springframework - spring-context - true - - - org.springframework - spring-context-support - true - - - org.springframework - spring-jdbc - true - - - org.springframework - spring-jms - true - - - org.springframework - spring-orm - true - - - org.springframework.retry - spring-retry - - - org.springframework - spring-test - - - org.springframework - spring-tx - true - - - org.springframework.data - spring-data-commons - true - - - org.springframework.data - spring-data-mongodb - true - - - org.springframework.data - spring-data-neo4j - true - - - org.springframework.data - spring-data-gemfire - true - - - org.springframework.data - spring-data-redis - true - - - org.codehaus.woodstox - woodstox-core-asl - true - - - org.springframework.amqp - spring-amqp - true - - - org.springframework.amqp - spring-rabbit - true - - - org.mockito - mockito-all - test - - - - - - org.codehaus.mojo - emma-maven-plugin - 1.0-alpha-1 - - - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ExecutionContext.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ExecutionContext.java index 3b40126f87..20259a7f65 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ExecutionContext.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ExecutionContext.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,7 +22,7 @@ import java.util.Set; import java.util.concurrent.ConcurrentHashMap; -import org.springframework.util.Assert; +import org.springframework.lang.Nullable; /** * Object representing a context for an {@link ItemStream}. It is a thin wrapper @@ -34,6 +34,7 @@ * * @author Lucas Ward * @author Douglas Kaminsky + * @author Mahmoud Ben Hassine */ @SuppressWarnings("serial") public class ExecutionContext implements Serializable { @@ -47,7 +48,7 @@ public class ExecutionContext implements Serializable { * internal map. */ public ExecutionContext() { - map = new ConcurrentHashMap(); + this.map = new ConcurrentHashMap<>(); } /** @@ -56,11 +57,14 @@ public ExecutionContext() { * @param map Initial contents of context. */ public ExecutionContext(Map map) { - this.map = new ConcurrentHashMap(map); + this.map = new ConcurrentHashMap<>(map); } /** - * @param executionContext + * Initializes a new {@link ExecutionContext} with the contents of another + * {@code ExecutionContext}. + * + * @param executionContext containing the entries to be copied to this current context. */ public ExecutionContext(ExecutionContext executionContext) { this(); @@ -73,13 +77,14 @@ public ExecutionContext(ExecutionContext executionContext) { } /** - * Adds a String value to the context. + * Adds a String value to the context. Putting null + * value for a given key removes the key. * * @param key Key to add to context * @param value Value to associate with key */ - public void putString(String key, String value) { + public void putString(String key, @Nullable String value) { put(key, value); } @@ -92,7 +97,7 @@ public void putString(String key, String value) { */ public void putLong(String key, long value) { - put(key, Long.valueOf(value)); + put(key, value); } /** @@ -102,7 +107,7 @@ public void putLong(String key, long value) { * @param value Value to associate with key */ public void putInt(String key, int value) { - put(key, Integer.valueOf(value)); + put(key, value); } /** @@ -113,25 +118,24 @@ public void putInt(String key, int value) { */ public void putDouble(String key, double value) { - put(key, Double.valueOf(value)); + put(key, value); } /** - * Add an Object value to the context (must be Serializable). Putting - * null value for a given key removes the key. + * Add an Object value to the context. Putting null + * value for a given key removes the key. * * @param key Key to add to context * @param value Value to associate with key */ - public void put(String key, Object value) { + public void put(String key, @Nullable Object value) { if (value != null) { - Assert.isInstanceOf(Serializable.class, value, "Value: [ " + value + "must be serializable."); - Object result = map.put(key, value); - dirty = result==null || result!=null && !result.equals(value); + Object result = this.map.put(key, value); + this.dirty = result==null || result!=null && !result.equals(value); } else { - Object result = map.remove(key); - dirty = result!=null; + Object result = this.map.remove(key); + this.dirty = result!=null; } } @@ -143,7 +147,7 @@ public void put(String key, Object value) { * @return True if "put" operation has occurred since flag was last cleared */ public boolean isDirty() { - return dirty; + return this.dirty; } /** @@ -163,15 +167,15 @@ public String getString(String key) { * * @param key The key to get a value for * @param defaultString Default to return if key is not represented - * @return The String value if key is repreesnted, specified + * @return The String value if key is represented, specified * default otherwise */ public String getString(String key, String defaultString) { - if (!map.containsKey(key)) { + if (!containsKey(key)) { return defaultString; } - return (String) readAndValidate(key, String.class); + return getString(key); } /** @@ -182,7 +186,7 @@ public String getString(String key, String defaultString) { */ public long getLong(String key) { - return ((Long) readAndValidate(key, Long.class)).longValue(); + return (Long) readAndValidate(key, Long.class); } /** @@ -195,11 +199,11 @@ public long getLong(String key) { * default otherwise */ public long getLong(String key, long defaultLong) { - if (!map.containsKey(key)) { + if (!containsKey(key)) { return defaultLong; } - return ((Long) readAndValidate(key, Long.class)).longValue(); + return getLong(key); } /** @@ -210,7 +214,7 @@ public long getLong(String key, long defaultLong) { */ public int getInt(String key) { - return ((Integer) readAndValidate(key, Integer.class)).intValue(); + return (Integer) readAndValidate(key, Integer.class); } /** @@ -223,11 +227,11 @@ public int getInt(String key) { * default otherwise */ public int getInt(String key, int defaultInt) { - if (!map.containsKey(key)) { + if (!containsKey(key)) { return defaultInt; } - return ((Integer) readAndValidate(key, Integer.class)).intValue(); + return getInt(key); } /** @@ -237,7 +241,7 @@ public int getInt(String key, int defaultInt) { * @return The Double value */ public double getDouble(String key) { - return ((Double) readAndValidate(key, Double.class)).doubleValue(); + return (Double) readAndValidate(key, Double.class); } /** @@ -250,21 +254,23 @@ public double getDouble(String key) { * default otherwise */ public double getDouble(String key, double defaultDouble) { - if (!map.containsKey(key)) { + if (!containsKey(key)) { return defaultDouble; } - return ((Double) readAndValidate(key, Double.class)).doubleValue(); + return getDouble(key); } /** * Getter for the value represented by the provided key. * * @param key The key to get a value for - * @return The value represented by the given key + * @return The value represented by the given key or {@code null} if the key + * is not present */ + @Nullable public Object get(String key) { - return map.get(key); + return this.map.get(key); } /** @@ -277,7 +283,7 @@ public Object get(String key) { */ private Object readAndValidate(String key, Class type) { - Object value = map.get(key); + Object value = get(key); if (!type.isInstance(value)) { throw new ClassCastException("Value for key=[" + key + "] is not of type: [" + type + "], it is [" @@ -294,14 +300,14 @@ private Object readAndValidate(String key, Class type) { * @see java.util.Map#isEmpty() */ public boolean isEmpty() { - return map.isEmpty(); + return this.map.isEmpty(); } /** * Clears the dirty flag. */ public void clearDirtyFlag() { - dirty = false; + this.dirty = false; } /** @@ -311,7 +317,7 @@ public void clearDirtyFlag() { * @see java.util.Map#entrySet() */ public Set> entrySet() { - return map.entrySet(); + return this.map.entrySet(); } /** @@ -322,16 +328,20 @@ public Set> entrySet() { * @see java.util.Map#containsKey(Object) */ public boolean containsKey(String key) { - return map.containsKey(key); + return this.map.containsKey(key); } /** * Removes the mapping for a key from this context if it is present. * + * @param key {@link String} that identifies the entry to be removed from the context. + * @return the value that was removed from the context. + * * @see java.util.Map#remove(Object) */ + @Nullable public Object remove(String key) { - return map.remove(key); + return this.map.remove(key); } /** @@ -342,7 +352,7 @@ public Object remove(String key) { * @see java.util.Map#containsValue(Object) */ public boolean containsValue(Object value) { - return map.containsValue(value); + return this.map.containsValue(value); } /* @@ -369,7 +379,7 @@ public boolean equals(Object obj) { */ @Override public int hashCode() { - return map.hashCode(); + return this.map.hashCode(); } /* @@ -379,7 +389,7 @@ public int hashCode() { */ @Override public String toString() { - return map.toString(); + return this.map.toString(); } /** @@ -389,7 +399,7 @@ public String toString() { * @see java.util.Map#size() */ public int size() { - return map.size(); + return this.map.size(); } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemCountAware.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemCountAware.java index 0abfec542d..a3d0a96c11 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemCountAware.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemCountAware.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemProcessor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemProcessor.java index e253f0e1d1..9eeabebfdb 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemProcessor.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemProcessor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,6 +16,8 @@ package org.springframework.batch.item; +import org.springframework.lang.Nullable; + /** * Interface for item transformation. Given an item as input, this interface provides * an extension point which allows for the application of business logic in an item @@ -25,6 +27,7 @@ * * @author Robert Kasanicky * @author Dave Syer + * @author Mahmoud Ben Hassine */ public interface ItemProcessor { @@ -34,9 +37,11 @@ public interface ItemProcessor { * should not continue. * * @param item to be processed - * @return potentially modified or new item for continued processing, null if processing of the + * @return potentially modified or new item for continued processing, {@code null} if processing of the * provided item should not continue. - * @throws Exception + * + * @throws Exception thrown if exception occurs during processing. */ + @Nullable O process(I item) throws Exception; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemReader.java index 4d04769bce..62fa8c9d98 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,15 +16,17 @@ package org.springframework.batch.item; +import org.springframework.lang.Nullable; + /** - * Strategy interface for providing the data.
      + * Strategy interface for providing the data.
      * * Implementations are expected to be stateful and will be called multiple times * for each batch, with each call to {@link #read()} returning a different value - * and finally returning null when all input data is exhausted.
      + * and finally returning null when all input data is exhausted.
      * - * Implementations need *not* be thread safe and clients of a {@link ItemReader} - * need to be aware that this is the case.
      + * Implementations need not be thread-safe and clients of a {@link ItemReader} + * need to be aware that this is the case.
      * * A richer interface (e.g. with a look ahead or peek) is not feasible because * we need to support transactions in an asynchronous batch. @@ -32,6 +34,7 @@ * @author Rob Harrop * @author Dave Syer * @author Lucas Ward + * @author Mahmoud Ben Hassine * @since 1.0 */ public interface ItemReader { @@ -52,7 +55,10 @@ public interface ItemReader { * with the input data. Assume potentially transient, so subsequent calls to * read might succeed. * @throws Exception if an there is a non-specific error. + * @return T the item to be processed or {@code null} if the data source is + * exhausted */ + @Nullable T read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemReaderException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemReaderException.java index 70ff3d2b32..498678abf1 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemReaderException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemReaderException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,7 @@ * * @author Ben Hale */ +@SuppressWarnings("serial") public abstract class ItemReaderException extends RuntimeException { /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStream.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStream.java index 7bfa163d97..021333ff1c 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStream.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStream.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,7 +30,9 @@ public interface ItemStream { /** * Open the stream for the provided {@link ExecutionContext}. - * + * + * @param executionContext current step's {@link org.springframework.batch.item.ExecutionContext}. Will be the + * executionContext from the last run of the step on a restart. * @throws IllegalArgumentException if context is null */ void open(ExecutionContext executionContext) throws ItemStreamException; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamException.java index 508f389a11..a187629ef3 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,10 +21,11 @@ * @author Dave Syer * @author Lucas Ward */ +@SuppressWarnings("serial") public class ItemStreamException extends RuntimeException { /** - * @param message + * @param message the String that contains a detailed message. */ public ItemStreamException(String message) { super(message); @@ -34,6 +35,7 @@ public ItemStreamException(String message) { * Constructs a new instance with a message and nested exception. * * @param msg the exception message. + * @param nested the cause of the exception. * */ public ItemStreamException(String msg, Throwable nested) { @@ -42,6 +44,8 @@ public ItemStreamException(String msg, Throwable nested) { /** * Constructs a new instance with a nested exception and empty message. + * + * @param nested the cause of the exception. */ public ItemStreamException(Throwable nested) { super(nested); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamReader.java index 9f04fd6f61..6cf96ef5fc 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamReader.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamSupport.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamSupport.java index a52f9b3366..87cf485572 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamSupport.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamSupport.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -51,6 +51,17 @@ public void open(ExecutionContext executionContext) { @Override public void update(ExecutionContext executionContext) { } + + /** + * The name of the component which will be used as a stem for keys in the + * {@link ExecutionContext}. Subclasses should provide a default value, e.g. + * the short form of the class name. + * + * @param name the name for the component + */ + public void setName(String name) { + this.setExecutionContextName(name); + } protected void setExecutionContextName(String name) { executionContextUserSupport.setName(name); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamWriter.java index 797cab7f4d..47670a17d8 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemStreamWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemWriter.java index 06e61e106b..3279a5c6e4 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -41,7 +41,8 @@ public interface ItemWriter { /** * Process the supplied data element. Will not be called with any null items * in normal operation. - * + * + * @param items items to be written * @throws Exception if there are errors. The framework will catch the * exception and convert or rethrow it as appropriate. */ diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemWriterException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemWriterException.java index 3025dca1fd..67190090d6 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemWriterException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ItemWriterException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,7 @@ * * @author Ben Hale */ +@SuppressWarnings("serial") public abstract class ItemWriterException extends RuntimeException { /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/KeyValueItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/KeyValueItemWriter.java index 08a5bd3df9..df2fed925b 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/KeyValueItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/KeyValueItemWriter.java @@ -4,7 +4,7 @@ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the @@ -59,7 +59,8 @@ public void write(List items) throws Exception { /** * Set the {@link Converter} to use to derive the key from the item - * @param itemKeyMapper + * + * @param itemKeyMapper the {@link Converter} used to derive a key from an item. */ public void setItemKeyMapper(Converter itemKeyMapper) { this.itemKeyMapper = itemKeyMapper; @@ -67,7 +68,9 @@ public void setItemKeyMapper(Converter itemKeyMapper) { /** * Sets the delete flag to have the item writer perform deletes - * @param delete + * + * @param delete if true {@link ItemWriter} will perform deletes, + * if false not to perform deletes. */ public void setDelete(boolean delete) { this.delete = delete; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/NonTransientResourceException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/NonTransientResourceException.java index 31f73dc88a..4e514199a3 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/NonTransientResourceException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/NonTransientResourceException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,7 @@ * * @author Dave Syer */ +@SuppressWarnings("serial") public class NonTransientResourceException extends ItemReaderException { /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ParseException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ParseException.java index a537a2b560..f3ab5fc68e 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ParseException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ParseException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,11 +16,12 @@ package org.springframework.batch.item; /** - * Exception indicating that an error has been encountered parsing io, typically from a file. + * Exception indicating that an error has been encountered parsing IO, typically from a file. * * @author Lucas Ward * @author Ben Hale */ +@SuppressWarnings("serial") public class ParseException extends ItemReaderException { /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/PeekableItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/PeekableItemReader.java index 995fb64d1e..6058f02e96 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/PeekableItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/PeekableItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2010 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,6 +15,8 @@ */ package org.springframework.batch.item; +import org.springframework.lang.Nullable; + /** *

      * A specialisation of {@link ItemReader} that allows the user to look ahead @@ -33,6 +35,7 @@ *

      * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public interface PeekableItemReader extends ItemReader { @@ -41,9 +44,10 @@ public interface PeekableItemReader extends ItemReader { * Get the next item that would be returned by {@link #read()}, without * affecting the result of {@link #read()}. * - * @return the next item + * @return the next item or {@code null} if the data source is exhausted * @throws Exception if there is a problem */ + @Nullable T peek() throws Exception, UnexpectedInputException, ParseException; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ReaderNotOpenException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ReaderNotOpenException.java index 8f1db5a32d..6f17bec101 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ReaderNotOpenException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ReaderNotOpenException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ * * @author Ben Hale */ +@SuppressWarnings("serial") public class ReaderNotOpenException extends ItemReaderException { /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ResourceAware.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ResourceAware.java index b1762d6700..926f3a3732 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ResourceAware.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ResourceAware.java @@ -1,3 +1,18 @@ +/* + * Copyright 2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item; import org.springframework.core.io.Resource; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/SpELItemKeyMapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/SpELItemKeyMapper.java index f4c1b8d97e..59a588c32b 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/SpELItemKeyMapper.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/SpELItemKeyMapper.java @@ -4,7 +4,7 @@ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/UnexpectedInputException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/UnexpectedInputException.java index fe3588c2cd..86a4efd166 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/UnexpectedInputException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/UnexpectedInputException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,6 +23,7 @@ * @author Dave Syer * @author Ben Hale */ +@SuppressWarnings("serial") public class UnexpectedInputException extends ItemReaderException { /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/WriteFailedException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/WriteFailedException.java index 4d46b6aeeb..d82771ff8f 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/WriteFailedException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/WriteFailedException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ * @author Lucas Ward * @author Ben Hale */ +@SuppressWarnings("serial") public class WriteFailedException extends ItemWriterException { /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/WriterNotOpenException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/WriterNotOpenException.java index 5bbe60d0be..30c26b378a 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/WriterNotOpenException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/WriterNotOpenException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,10 +17,11 @@ /** * Exception indicating that an {@link ItemWriter} needed to be opened before being - * written to.. + * written to. * * @author Lucas Ward */ +@SuppressWarnings("serial") public class WriterNotOpenException extends ItemWriterException { /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/AbstractMethodInvokingDelegator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/AbstractMethodInvokingDelegator.java index 516d70e65b..9585559093 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/AbstractMethodInvokingDelegator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/AbstractMethodInvokingDelegator.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,6 +24,7 @@ import org.springframework.beans.factory.InitializingBean; import org.springframework.util.Assert; +import org.springframework.util.ClassUtils; import org.springframework.util.MethodInvoker; /** @@ -37,6 +38,7 @@ * by {@link InvocationTargetThrowableWrapper}. * * @author Robert Kasanicky + * @author Mahmoud Ben Hassine */ public abstract class AbstractMethodInvokingDelegator implements InitializingBean { @@ -49,9 +51,10 @@ public abstract class AbstractMethodInvokingDelegator implements Initializing /** * Invoker the target method with arguments set by * {@link #setArguments(Object[])}. + * * @return object returned by invoked method - * @throws DynamicMethodInvocationException if the {@link MethodInvoker} - * used throws exception + * + * @throws Exception exception thrown when executing the delegate method. */ protected T invokeDelegateMethod() throws Exception { MethodInvoker invoker = createMethodInvoker(targetObject, targetMethod); @@ -61,10 +64,11 @@ protected T invokeDelegateMethod() throws Exception { /** * Invokes the target method with given argument. + * * @param object argument for the target method * @return object returned by target method - * @throws DynamicMethodInvocationException if the {@link MethodInvoker} - * used throws exception + * + * @throws Exception exception thrown when executing the delegate method. */ protected T invokeDelegateMethodWithArgument(Object object) throws Exception { MethodInvoker invoker = createMethodInvoker(targetObject, targetMethod); @@ -74,10 +78,11 @@ protected T invokeDelegateMethodWithArgument(Object object) throws Exception { /** * Invokes the target method with given arguments. + * * @param args arguments for the invoked method * @return object returned by invoked method - * @throws DynamicMethodInvocationException if the {@link MethodInvoker} - * used throws exception + * + * @throws Exception exception thrown when executing the delegate method. */ protected T invokeDelegateMethodWithArguments(Object[] args) throws Exception { MethodInvoker invoker = createMethodInvoker(targetObject, targetMethod); @@ -106,10 +111,7 @@ private T doInvoke(MethodInvoker invoker) throws Exception { try { invoker.prepare(); } - catch (ClassNotFoundException e) { - throw new DynamicMethodInvocationException(e); - } - catch (NoSuchMethodException e) { + catch (ClassNotFoundException | NoSuchMethodException e) { throw new DynamicMethodInvocationException(e); } @@ -131,8 +133,8 @@ private T doInvoke(MethodInvoker invoker) throws Exception { @Override public void afterPropertiesSet() throws Exception { - Assert.notNull(targetObject); - Assert.hasLength(targetMethod); + Assert.notNull(targetObject, "targetObject must not be null"); + Assert.hasLength(targetMethod, "targetMethod must not be empty"); Assert.state(targetClassDeclaresTargetMethod(), "target class must declare a method with matching name and parameter types"); } @@ -147,7 +149,7 @@ private boolean targetClassDeclaresTargetMethod() { Method[] memberMethods = invoker.getTargetClass().getMethods(); Method[] declaredMethods = invoker.getTargetClass().getDeclaredMethods(); - List allMethods = new ArrayList(); + List allMethods = new ArrayList<>(); allMethods.addAll(Arrays.asList(memberMethods)); allMethods.addAll(Arrays.asList(declaredMethods)); @@ -167,7 +169,7 @@ private boolean targetClassDeclaresTargetMethod() { if (arguments[j] == null) { continue; } - if (!(params[j].isAssignableFrom(arguments[j].getClass()))) { + if (!(ClassUtils.isAssignableValue(params[j], arguments[j]))) { argumentsMatchParameters = false; } } @@ -212,12 +214,21 @@ public void setArguments(Object[] arguments) { this.arguments = arguments == null ? null : Arrays.asList(arguments).toArray(); } + /** + * Return arguments. + * @return arguments + */ + protected Object[] getArguments() { + return arguments; + } + /** * Used to wrap a {@link Throwable} (not an {@link Exception}) thrown by a * reflectively-invoked delegate. * * @author Robert Kasanicky */ + @SuppressWarnings("serial") public static class InvocationTargetThrowableWrapper extends RuntimeException { public InvocationTargetThrowableWrapper(Throwable cause) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/DynamicMethodInvocationException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/DynamicMethodInvocationException.java index 895a33a8dd..96358a9ba6 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/DynamicMethodInvocationException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/DynamicMethodInvocationException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/HippyMethodInvoker.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/HippyMethodInvoker.java index 9f5bb1b039..a93a0677f8 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/HippyMethodInvoker.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/HippyMethodInvoker.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/ItemProcessorAdapter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/ItemProcessorAdapter.java index c95dfaf1a6..d5185bd162 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/ItemProcessorAdapter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/ItemProcessorAdapter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,6 +17,7 @@ package org.springframework.batch.item.adapter; import org.springframework.batch.item.ItemProcessor; +import org.springframework.lang.Nullable; /** * Invokes a custom method on a delegate plain old Java object which itself @@ -31,6 +32,7 @@ public class ItemProcessorAdapter extends AbstractMethodInvokingDelegator extends AbstractMethodInvokingDelegator imp /** * @return return value of the target method. */ + @Nullable @Override public T read() throws Exception { return invokeDelegateMethod(); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/ItemWriterAdapter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/ItemWriterAdapter.java index 0088e03ba1..a276b12ce5 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/ItemWriterAdapter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/ItemWriterAdapter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/PropertyExtractingDelegatingItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/PropertyExtractingDelegatingItemWriter.java index f7d9979835..db0d5a379e 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/PropertyExtractingDelegatingItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/PropertyExtractingDelegatingItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -62,7 +62,7 @@ public void write(List items) throws Exception { @Override public void afterPropertiesSet() throws Exception { super.afterPropertiesSet(); - Assert.notEmpty(fieldsUsedAsTargetMethodArguments); + Assert.notEmpty(fieldsUsedAsTargetMethodArguments, "fieldsUsedAsTargetMethodArguments must not be empty"); } /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/package-info.java new file mode 100644 index 0000000000..db2847283f --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Adapters for Plain Old Java Objects. + *

      + */ +@NonNullApi +package org.springframework.batch.item.adapter; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/package.html deleted file mode 100644 index a8e90f1ca0..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/adapter/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Adapters for Plain Old Java Objects. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/AmqpItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/AmqpItemReader.java index ee4d10cd35..a198c8ee0b 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/AmqpItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/AmqpItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,6 +19,7 @@ import org.springframework.amqp.core.AmqpTemplate; import org.springframework.amqp.core.Message; import org.springframework.batch.item.ItemReader; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -28,17 +29,24 @@ *

      * * @author Chris Schaefer + * @author Mahmoud Ben Hassine */ public class AmqpItemReader implements ItemReader { private final AmqpTemplate amqpTemplate; private Class itemType; + /** + * Initialize the AmqpItemReader. + * + * @param amqpTemplate the template to be used. Must not be null. + */ public AmqpItemReader(final AmqpTemplate amqpTemplate) { - Assert.notNull(amqpTemplate, "AmpqTemplate must not be null"); + Assert.notNull(amqpTemplate, "AmqpTemplate must not be null"); this.amqpTemplate = amqpTemplate; } + @Nullable @Override @SuppressWarnings("unchecked") public T read() { @@ -56,6 +64,11 @@ public T read() { return (T) result; } + /** + * Establish the itemType for the reader. + * + * @param itemType class type that will be returned by the reader. + */ public void setItemType(Class itemType) { Assert.notNull(itemType, "Item type cannot be null"); this.itemType = itemType; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/AmqpItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/AmqpItemWriter.java index ecc42c8132..691dd49ce9 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/AmqpItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/AmqpItemWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2012 the original author or authors. + * Copyright 2012-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,13 +32,14 @@ *

      * * @author Chris Schaefer + * @author Mahmoud Ben Hassine */ public class AmqpItemWriter implements ItemWriter { private final AmqpTemplate amqpTemplate; private final Log log = LogFactory.getLog(getClass()); public AmqpItemWriter(final AmqpTemplate amqpTemplate) { - Assert.notNull(amqpTemplate, "AmpqTemplate must not be null"); + Assert.notNull(amqpTemplate, "AmqpTemplate must not be null"); this.amqpTemplate = amqpTemplate; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/builder/AmqpItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/builder/AmqpItemReaderBuilder.java new file mode 100644 index 0000000000..aeb4f246d8 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/builder/AmqpItemReaderBuilder.java @@ -0,0 +1,75 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.amqp.builder; + +import org.springframework.amqp.core.AmqpTemplate; +import org.springframework.batch.item.amqp.AmqpItemReader; +import org.springframework.util.Assert; + +/** + * A builder implementation for the {@link AmqpItemReader} + * + * @author Glenn Renfro + * @since 4.0 + * @see AmqpItemReader + */ +public class AmqpItemReaderBuilder { + + private AmqpTemplate amqpTemplate; + + private Class itemType; + + /** + * Establish the amqpTemplate to be used by the AmqpItemReader. + * @param amqpTemplate the template to be used. + * @return this instance for method chaining + * @see AmqpItemReader#AmqpItemReader(AmqpTemplate) + */ + public AmqpItemReaderBuilder amqpTemplate(AmqpTemplate amqpTemplate) { + this.amqpTemplate = amqpTemplate; + + return this; + } + + /** + * Establish the itemType for the reader. + * @param itemType class type that will be returned by the reader. + * @return this instance for method chaining. + * @see AmqpItemReader#setItemType(Class) + */ + public AmqpItemReaderBuilder itemType(Class itemType) { + this.itemType = itemType; + + return this; + } + + /** + * Validates and builds a {@link AmqpItemReader}. + * + * @return a {@link AmqpItemReader} + */ + public AmqpItemReader build() { + Assert.notNull(this.amqpTemplate, "amqpTemplate is required."); + + AmqpItemReader reader = new AmqpItemReader<>(this.amqpTemplate); + if(this.itemType != null) { + reader.setItemType(this.itemType); + } + + return reader; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/builder/AmqpItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/builder/AmqpItemWriterBuilder.java new file mode 100644 index 0000000000..37433a434e --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/builder/AmqpItemWriterBuilder.java @@ -0,0 +1,58 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.amqp.builder; + +import org.springframework.amqp.core.AmqpTemplate; +import org.springframework.batch.item.amqp.AmqpItemWriter; +import org.springframework.util.Assert; + +/** + * A builder implementation for the {@link AmqpItemWriter} + * @author Glenn Renfro + * @since 4.0 + * @see AmqpItemWriter + */ +public class AmqpItemWriterBuilder { + + private AmqpTemplate amqpTemplate; + + /** + * Establish the amqpTemplate to be used by the AmqpItemWriter. + * @param amqpTemplate the template to be used. + * @return this instance for method chaining + * @see AmqpItemWriter#AmqpItemWriter(AmqpTemplate) + */ + public AmqpItemWriterBuilder amqpTemplate(AmqpTemplate amqpTemplate) { + this.amqpTemplate = amqpTemplate; + + return this; + } + + /** + * Validates and builds a {@link AmqpItemWriter}. + * + * @return a {@link AmqpItemWriter} + */ + public AmqpItemWriter build() { + Assert.notNull(this.amqpTemplate, "amqpTemplate is required."); + + AmqpItemWriter writer = new AmqpItemWriter<>(this.amqpTemplate); + + return writer; + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/builder/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/builder/package-info.java new file mode 100644 index 0000000000..250e32146f --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/builder/package-info.java @@ -0,0 +1,26 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Builders for AMQP item reader and writer. + * + * @author Mahmoud Ben Hassine + */ + +@NonNullApi +package org.springframework.batch.item.amqp.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/package-info.java new file mode 100644 index 0000000000..aa419383c0 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/amqp/package-info.java @@ -0,0 +1,10 @@ +/** + * AMQP related batch components. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.amqp; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/avro/AvroItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/avro/AvroItemReader.java new file mode 100755 index 0000000000..3725ad5e61 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/avro/AvroItemReader.java @@ -0,0 +1,180 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.avro; + +import java.io.IOException; +import java.io.InputStream; + +import org.apache.avro.Schema; +import org.apache.avro.file.DataFileStream; +import org.apache.avro.generic.GenericDatumReader; +import org.apache.avro.generic.GenericRecord; +import org.apache.avro.io.BinaryDecoder; +import org.apache.avro.io.DatumReader; +import org.apache.avro.io.DecoderFactory; +import org.apache.avro.reflect.ReflectDatumReader; +import org.apache.avro.specific.SpecificDatumReader; +import org.apache.avro.specific.SpecificRecordBase; + +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader; +import org.springframework.core.io.Resource; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * An {@link ItemReader} that deserializes data from a {@link Resource} containing serialized Avro objects. + * + * @author David Turanski + * @author Mahmoud Ben Hassine + * @since 4.2 + */ +public class AvroItemReader extends AbstractItemCountingItemStreamItemReader { + + private boolean embeddedSchema = true; + + private InputStreamReader inputStreamReader; + + private DataFileStream dataFileReader; + + private InputStream inputStream; + + private DatumReader datumReader; + + /** + * + * @param resource the {@link Resource} containing objects serialized with Avro. + * @param clazz the data type to be deserialized. + */ + public AvroItemReader(Resource resource, Class clazz) { + Assert.notNull(resource, "'resource' is required."); + Assert.notNull(clazz, "'class' is required."); + + try { + this.inputStream = resource.getInputStream(); + this.datumReader = datumReaderForClass(clazz); + } + catch (IOException e) { + throw new IllegalArgumentException(e.getMessage(), e); + } + } + + /** + * + * @param data the {@link Resource} containing the data to be read. + * @param schema the {@link Resource} containing the Avro schema. + */ + public AvroItemReader(Resource data, Resource schema) { + Assert.notNull(data, "'data' is required."); + Assert.state(data.exists(), "'data' " + data.getFilename() +" does not exist."); + Assert.notNull(schema, "'schema' is required"); + Assert.state(schema.exists(), "'schema' " + schema.getFilename() +" does not exist."); + try { + this.inputStream = data.getInputStream(); + Schema avroSchema = new Schema.Parser().parse(schema.getInputStream()); + this.datumReader = new GenericDatumReader<>(avroSchema); + } + catch (IOException e) { + throw new IllegalArgumentException(e.getMessage(), e); + } + } + + /** + * Disable or enable reading an embedded Avro schema. True by default. + * @param embeddedSchema set to false to if the input does not embed an Avro schema. + */ + public void setEmbeddedSchema(boolean embeddedSchema) { + this.embeddedSchema = embeddedSchema; + } + + + @Nullable + @Override + protected T doRead() throws Exception { + if (this.inputStreamReader != null) { + return this.inputStreamReader.read(); + } + return this.dataFileReader.hasNext()? this.dataFileReader.next(): null; + } + + @Override + protected void doOpen() throws Exception { + initializeReader(); + } + + @Override + protected void doClose() throws Exception { + if (this.inputStreamReader != null) { + this.inputStreamReader.close(); + return; + } + this.dataFileReader.close(); + } + + private void initializeReader() throws IOException { + if (this.embeddedSchema) { + this.dataFileReader = new DataFileStream<>(this.inputStream, this.datumReader); + } else { + this.inputStreamReader = createInputStreamReader(this.inputStream, this.datumReader); + } + + } + + private InputStreamReader createInputStreamReader(InputStream inputStream, DatumReader datumReader) { + return new InputStreamReader<>(inputStream, datumReader); + } + + private static DatumReader datumReaderForClass(Class clazz) { + if (SpecificRecordBase.class.isAssignableFrom(clazz)){ + return new SpecificDatumReader<>(clazz); + } + if (GenericRecord.class.isAssignableFrom(clazz)) { + return new GenericDatumReader<>(); + } + return new ReflectDatumReader<>(clazz); + } + + + private static class InputStreamReader { + private final DatumReader datumReader; + + private final BinaryDecoder binaryDecoder; + + private final InputStream inputStream; + + private InputStreamReader(InputStream inputStream, DatumReader datumReader) { + this.inputStream = inputStream; + this.datumReader = datumReader; + this.binaryDecoder = DecoderFactory.get().binaryDecoder(inputStream, null); + } + + private T read() throws Exception { + if (!this.binaryDecoder.isEnd()) { + return this.datumReader.read(null, this.binaryDecoder); + } + return null; + } + + private void close() { + try { + this.inputStream.close(); + } catch (IOException e) { + throw new ItemStreamException(e.getMessage(), e); + } + } + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/avro/AvroItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/avro/AvroItemWriter.java new file mode 100644 index 0000000000..48b54aa1d2 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/avro/AvroItemWriter.java @@ -0,0 +1,201 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.avro; + +import java.io.IOException; +import java.io.OutputStream; +import java.util.List; + +import org.apache.avro.Schema; +import org.apache.avro.file.DataFileWriter; +import org.apache.avro.generic.GenericDatumWriter; +import org.apache.avro.generic.GenericRecord; +import org.apache.avro.io.BinaryEncoder; +import org.apache.avro.io.DatumWriter; +import org.apache.avro.io.EncoderFactory; +import org.apache.avro.reflect.ReflectDatumWriter; +import org.apache.avro.specific.SpecificDatumWriter; +import org.apache.avro.specific.SpecificRecordBase; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.AbstractItemStreamItemWriter; +import org.springframework.core.io.Resource; +import org.springframework.core.io.WritableResource; +import org.springframework.util.Assert; + +/** + * An {@link ItemWriter} that serializes data to an {@link WritableResource} using Avro. + * + * This does not support restart on failure. + * + * @since 4.2 + * @author David Turanski + * @author Mahmoud Ben Hassine + */ +public class AvroItemWriter extends AbstractItemStreamItemWriter { + + private DataFileWriter dataFileWriter; + + private OutputStreamWriter outputStreamWriter; + + private WritableResource resource; + + private Resource schemaResource; + + private Class clazz; + + private boolean embedSchema = true; + + + /** + * + * @param resource a {@link WritableResource} to which the objects will be serialized. + * @param schema a {@link Resource} containing the Avro schema. + * @param clazz the data type to be serialized. + */ + public AvroItemWriter(WritableResource resource, Resource schema, Class clazz) { + this.schemaResource = schema; + this.resource = resource; + this.clazz = clazz; + } + + /** + * This constructor will create an ItemWriter that does not embedded Avro schema. + * + * @param resource a {@link WritableResource} to which the objects will be serialized. + * @param clazz the data type to be serialized. + */ + public AvroItemWriter(WritableResource resource, Class clazz) { + this(resource, null, clazz); + embedSchema = false; + } + + @Override + public void write(List items) throws Exception { + items.forEach(item -> { + try { + if (this.dataFileWriter != null) { + this.dataFileWriter.append(item); + } + else { + this.outputStreamWriter.write(item); + } + } + catch (Exception e) { + throw new ItemStreamException(e.getMessage(), e); + } + }); + } + + /** + * @see org.springframework.batch.item.ItemStream#open(ExecutionContext) + */ + @Override + public void open(ExecutionContext executionContext) { + super.open(executionContext); + try { + initializeWriter(); + } catch (IOException e) { + throw new ItemStreamException(e.getMessage(), e); + } + } + + @Override + public void close() { + try { + if (this.dataFileWriter != null) { + this.dataFileWriter.close(); + } + else { + this.outputStreamWriter.close(); + } + } + catch (IOException e) { + throw new ItemStreamException(e.getMessage(), e); + } + } + + private void initializeWriter() throws IOException { + Assert.notNull(this.resource, "'resource' is required."); + Assert.notNull(this.clazz, "'class' is required."); + + if (this.embedSchema) { + Assert.notNull(this.schemaResource, "'schema' is required."); + Assert.state(this.schemaResource.exists(), + "'schema' " + this.schemaResource.getFilename() + " does not exist."); + Schema schema; + try { + schema = new Schema.Parser().parse(this.schemaResource.getInputStream()); + } catch (IOException e) { + throw new IllegalArgumentException(e.getMessage(), e); + } + this.dataFileWriter = new DataFileWriter<>(datumWriterForClass(this.clazz)); + this.dataFileWriter.create(schema, this.resource.getOutputStream()); + } else { + this.outputStreamWriter = createOutputStreamWriter(this.resource.getOutputStream(), + datumWriterForClass(this.clazz)); + } + + } + + private static DatumWriter datumWriterForClass(Class clazz) { + if (GenericRecord.class.isAssignableFrom(clazz)) { + return new GenericDatumWriter<>(); + + } + if (SpecificRecordBase.class.isAssignableFrom(clazz)){ + return new SpecificDatumWriter<>(clazz); + } + + return new ReflectDatumWriter<>(clazz); + } + + private AvroItemWriter.OutputStreamWriter createOutputStreamWriter(OutputStream outputStream, + DatumWriter datumWriter) { + return new AvroItemWriter.OutputStreamWriter<>(outputStream, datumWriter); + } + + private static class OutputStreamWriter { + private final DatumWriter datumWriter; + + private final BinaryEncoder binaryEncoder; + + private final OutputStream outputStream; + + private OutputStreamWriter(OutputStream outputStream, DatumWriter datumWriter) { + this.outputStream = outputStream; + this.datumWriter = datumWriter; + this.binaryEncoder = EncoderFactory.get().binaryEncoder(outputStream, null); + } + + private void write(T datum) throws Exception { + this.datumWriter.write(datum, this.binaryEncoder); + this.binaryEncoder.flush(); + } + + private void close() { + try { + this.outputStream.close(); + } + catch (IOException e) { + throw new ItemStreamException(e.getMessage(), e); + } + } + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/avro/builder/AvroItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/avro/builder/AvroItemReaderBuilder.java new file mode 100644 index 0000000000..9ae3c8ef19 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/avro/builder/AvroItemReaderBuilder.java @@ -0,0 +1,204 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.avro.builder; + +import org.apache.avro.Schema; + +import org.springframework.batch.item.avro.AvroItemReader; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * A builder implementation for the {@link AvroItemReader}. + * + * @author David Turanski + * @author Mahmoud Ben Hassine + * @since 4.2 + */ +public class AvroItemReaderBuilder { + + private boolean saveState = true; + + private String name = AvroItemReader.class.getSimpleName(); + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + private Resource schema; + + private Resource resource; + + private Class type; + + private boolean embeddedSchema =true; + + + /** + * Configure a {@link Resource} containing Avro serialized objects. + * @param resource an existing Resource. + * @return The current instance of the builder. + */ + public AvroItemReaderBuilder resource(Resource resource) { + Assert.notNull(resource, "A 'resource' is required."); + Assert.state(resource.exists(), "Resource " + resource.getFilename() + " does not exist."); + this.resource = resource; + return this; + } + + + /** + * Configure an Avro {@link Schema} from a {@link Resource}. + * @param schema an existing schema Resource. + * @return The current instance of the builder. + */ + public AvroItemReaderBuilder schema(Resource schema) { + Assert.notNull(schema, "A 'schema' Resource is required."); + Assert.state(schema.exists(), "Resource " + schema.getFilename() + " does not exist."); + this.schema = schema; + return this; + } + + /** + * Configure an Avro {@link Schema} from a String. + * @param schemaString the schema String. + * @return The current instance of the builder. + */ + public AvroItemReaderBuilder schema(String schemaString) { + Assert.hasText(schemaString, "A 'schema' is required."); + this.schema = new ByteArrayResource(schemaString.getBytes()); + return this; + } + + /** + * Configure a type to be deserialized. + * @param type the class to be deserialized. + * @return The current instance of the builder. + */ + public AvroItemReaderBuilder type(Class type) { + Assert.notNull(type, "A 'type' is required."); + this.type = type; + return this; + } + + /** + * Disable or enable reading an embedded Avro schema. True by default. + * @param embeddedSchema set to false to if the input does not contain an Avro schema. + */ + public AvroItemReaderBuilder embeddedSchema(boolean embeddedSchema) { + this.embeddedSchema = embeddedSchema; + return this; + } + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public AvroItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public AvroItemReaderBuilder name(String name) { + this.name = name; + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public AvroItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public AvroItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + return this; + } + + + /** + * Build an instance of {@link AvroItemReader}. + * @return the instance; + */ + public AvroItemReader build() { + AvroItemReader avroItemReader; + + Assert.notNull(this.resource, "A 'resource' is required."); + + if (this.type != null) { + avroItemReader = buildForType(); + } + else { + avroItemReader = buildForSchema(); + } + + avroItemReader.setSaveState(this.saveState); + + if(this.saveState) { + Assert.state(StringUtils.hasText(this.name), + "A name is required when saveState is set to true."); + } + + avroItemReader.setName(this.name); + avroItemReader.setCurrentItemCount(this.currentItemCount); + avroItemReader.setMaxItemCount(this.maxItemCount); + avroItemReader.setEmbeddedSchema(this.embeddedSchema); + + return avroItemReader; + } + + + private AvroItemReader buildForType() { + Assert.isNull(this.schema, "You cannot specify a schema and 'type'."); + return new AvroItemReader<>(this.resource, this.type); + } + + private AvroItemReader buildForSchema() { + Assert.notNull(this.schema, "'schema' is required."); + return new AvroItemReader<>(this.resource, this.schema); + } + + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/avro/builder/AvroItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/avro/builder/AvroItemWriterBuilder.java new file mode 100644 index 0000000000..a929991b28 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/avro/builder/AvroItemWriterBuilder.java @@ -0,0 +1,120 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.avro.builder; + +import org.springframework.batch.item.avro.AvroItemWriter; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.Resource; +import org.springframework.core.io.WritableResource; +import org.springframework.util.Assert; + +/** + * A builder implementation for the {@link AvroItemWriter}. + * + * @author David Turanski + * @author Mahmoud Ben Hassine + * @since 4.2 + */ +public class AvroItemWriterBuilder { + private Class type; + + private WritableResource resource; + + private Resource schema; + + private String name = AvroItemWriter.class.getSimpleName(); + + /** + * + * @param resource the {@link WritableResource} used to write the serialized data. + * @return The current instance of the builder. + */ + public AvroItemWriterBuilder resource(WritableResource resource) { + Assert.notNull(resource, "A 'resource' is required."); + this.resource = resource; + return this; + } + + /** + * + * @param schema the Resource containing the schema JSON used to serialize the output. + * @return The current instance of the builder. + */ + public AvroItemWriterBuilder schema(Resource schema) { + Assert.notNull(schema, "A 'schema' is required."); + Assert.state(schema.exists(), "Resource " + schema.getFilename() + "does not exist."); + this.schema = schema; + return this; + } + + + /** + * + * @param schemaString the String containing the schema JSON used to serialize the output. + * @return The current instance of the builder. + */ + public AvroItemWriterBuilder schema(String schemaString) { + Assert.hasText(schemaString, "A 'schemaString' is required."); + this.schema = new ByteArrayResource(schemaString.getBytes()); + return this; + } + + + /** + * + * @param type the Class of objects to be serialized. + * @return The current instance of the builder. + */ + public AvroItemWriterBuilder type(Class type) { + Assert.notNull(type, "A 'type' is required."); + this.type = type; + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public AvroItemWriterBuilder name(String name) { + Assert.hasText(name, "A 'name' is required."); + this.name = name; + return this; + } + + /** + * Build an instance of {@link AvroItemWriter}. + * + * @return the instance; + */ + public AvroItemWriter build() { + + Assert.notNull(this.resource, "A 'resource' is required."); + + Assert.notNull(this.type, "A 'type' is required."); + + AvroItemWriter avroItemWriter = this.schema != null ? + new AvroItemWriter<>(this.resource, this.schema, this.type): + new AvroItemWriter<>(this.resource, this.type); + avroItemWriter.setName(this.name); + return avroItemWriter; + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/AbstractNeo4jItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/AbstractNeo4jItemReader.java new file mode 100644 index 0000000000..e3b5229c6e --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/AbstractNeo4jItemReader.java @@ -0,0 +1,204 @@ +/* + * Copyright 2012-2016 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data; + +import java.util.Map; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.neo4j.ogm.session.SessionFactory; + +import org.springframework.batch.item.ItemReader; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + *

      + * Restartable {@link ItemReader} that reads objects from the graph database Neo4j + * via a paging technique. + *

      + * + *

      + * It executes cypher queries built from the statement fragments provided to + * retrieve the requested data. The query is executed using paged requests of + * a size specified in {@link #setPageSize(int)}. Additional pages are requested + * as needed when the {@link #read()} method is called. On restart, the reader + * will begin again at the same number item it left off at. + *

      + * + *

      + * Performance is dependent on your Neo4J configuration (embedded or remote) as + * well as page size. Setting a fairly large page size and using a commit + * interval that matches the page size should provide better performance. + *

      + * + *

      + * This implementation is thread-safe between calls to + * {@link #open(org.springframework.batch.item.ExecutionContext)}, however you + * should set saveState=false if used in a multi-threaded + * environment (no restart available). + *

      + * + * @author Michael Minella + * @since 3.07 + */ +public abstract class AbstractNeo4jItemReader extends + AbstractPaginatedDataItemReader implements InitializingBean { + + protected Log logger = LogFactory.getLog(getClass()); + + private SessionFactory sessionFactory; + + private String startStatement; + private String returnStatement; + private String matchStatement; + private String whereStatement; + private String orderByStatement; + + private Class targetType; + + private Map parameterValues; + + /** + * Optional parameters to be used in the cypher query. + * + * @param parameterValues the parameter values to be used in the cypher query + */ + public void setParameterValues(Map parameterValues) { + this.parameterValues = parameterValues; + } + + protected final Map getParameterValues() { + return this.parameterValues; + } + + /** + * The start segment of the cypher query. START is prepended + * to the statement provided and should not be + * included. + * + * @param startStatement the start fragment of the cypher query. + */ + public void setStartStatement(String startStatement) { + this.startStatement = startStatement; + } + + /** + * The return statement of the cypher query. RETURN is prepended + * to the statement provided and should not be + * included + * + * @param returnStatement the return fragment of the cypher query. + */ + public void setReturnStatement(String returnStatement) { + this.returnStatement = returnStatement; + } + + /** + * An optional match fragment of the cypher query. MATCH is + * prepended to the statement provided and should not + * be included. + * + * @param matchStatement the match fragment of the cypher query + */ + public void setMatchStatement(String matchStatement) { + this.matchStatement = matchStatement; + } + + /** + * An optional where fragment of the cypher query. WHERE is + * prepended to the statement provided and should not + * be included. + * + * @param whereStatement where fragment of the cypher query + */ + public void setWhereStatement(String whereStatement) { + this.whereStatement = whereStatement; + } + + /** + * A list of properties to order the results by. This is + * required so that subsequent page requests pull back the + * segment of results correctly. ORDER BY is prepended to + * the statement provided and should not be included. + * + * @param orderByStatement order by fragment of the cypher query. + */ + public void setOrderByStatement(String orderByStatement) { + this.orderByStatement = orderByStatement; + } + + protected SessionFactory getSessionFactory() { + return sessionFactory; + } + + /** + * Establish the session factory for the reader. + * @param sessionFactory the factory to use for the reader. + */ + public void setSessionFactory(SessionFactory sessionFactory) { + this.sessionFactory = sessionFactory; + } + + /** + * The object type to be returned from each call to {@link #read()} + * + * @param targetType the type of object to return. + */ + public void setTargetType(Class targetType) { + this.targetType = targetType; + } + + protected final Class getTargetType() { + return this.targetType; + } + + protected String generateLimitCypherQuery() { + StringBuilder query = new StringBuilder(); + + query.append("START ").append(startStatement); + query.append(matchStatement != null ? " MATCH " + matchStatement : ""); + query.append(whereStatement != null ? " WHERE " + whereStatement : ""); + query.append(" RETURN ").append(returnStatement); + query.append(" ORDER BY ").append(orderByStatement); + query.append(" SKIP " + (pageSize * page)); + query.append(" LIMIT " + pageSize); + + String resultingQuery = query.toString(); + + if (logger.isDebugEnabled()) { + logger.debug(resultingQuery); + } + + return resultingQuery; + } + + /** + * Checks mandatory properties + * + * @see InitializingBean#afterPropertiesSet() + */ + @Override + public void afterPropertiesSet() throws Exception { + Assert.state(sessionFactory != null,"A SessionFactory is required"); + Assert.state(targetType != null, "The type to be returned is required"); + Assert.state(StringUtils.hasText(startStatement), "A START statement is required"); + Assert.state(StringUtils.hasText(returnStatement), "A RETURN statement is required"); + Assert.state(StringUtils.hasText(orderByStatement), "A ORDER BY statement is required"); + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/AbstractPaginatedDataItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/AbstractPaginatedDataItemReader.java index 6d75321d29..2c1604ce11 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/AbstractPaginatedDataItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/AbstractPaginatedDataItemReader.java @@ -1,10 +1,27 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.data; -import java.util.Iterator; - import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ItemStreamReader; import org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +import java.util.Iterator; /** * A base class that handles basic reading logic based on the paginated @@ -12,6 +29,7 @@ * semantics required for restartability based on those facilities. * * @author Michael Minella + * @author Glenn Renfro * @since 2.2 * @param Type of item to be read */ @@ -29,12 +47,14 @@ public abstract class AbstractPaginatedDataItemReader extends /** * The number of items to be read with each page. * - * @param pageSize the number of items + * @param pageSize the number of items. pageSize must be greater than zero. */ public void setPageSize(int pageSize) { + Assert.isTrue(pageSize > 0, "pageSize must be greater than zero"); this.pageSize = pageSize; } + @Nullable @Override protected T doRead() throws Exception { @@ -65,7 +85,7 @@ protected T doRead() throws Exception { * for the actual work of reading a page. Each time * this method is called, the resulting {@link Iterator} * should contain the items read within the next page. - *

      + *

      * If the {@link Iterator} is empty or null when it is * returned, this {@link ItemReader} will assume that the * input has been exhausted. diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/GemfireItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/GemfireItemWriter.java index 8d2868b725..e37be4bffb 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/GemfireItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/GemfireItemWriter.java @@ -4,7 +4,7 @@ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/MongoItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/MongoItemReader.java index 00231a53fb..47a3e78054 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/MongoItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/MongoItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2012 the original author or authors. + * Copyright 2012-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,6 +23,10 @@ import java.util.regex.Matcher; import java.util.regex.Pattern; +import com.mongodb.util.JSON; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemReader; import org.springframework.beans.factory.InitializingBean; @@ -36,8 +40,6 @@ import org.springframework.util.ClassUtils; import org.springframework.util.StringUtils; -import com.mongodb.util.JSON; - /** *

      * Restartable {@link ItemReader} that reads documents from MongoDB @@ -45,14 +47,23 @@ *

      * *

      - * It executes the JSON {@link #setQuery(String)} to retrieve the requested - * documents. The query is executed using paged requests specified in the + * If you set JSON String query {@link #setQuery(String)} then + * it executes the JSON to retrieve the requested documents. + *

      + * + *

      + * If you set Query object {@link #setQuery(Query)} then + * it executes the Query to retrieve the requested documents. + *

      + * + *

      + * The query is executed using paged requests specified in the * {@link #setPageSize(int)}. Additional pages are requested as needed to * provide data when the {@link #read()} method is called. *

      * *

      - * The JSON query provided supports parameter substitution via ?<index> + * The JSON String query provided supports parameter substitution via ?<index> * placeholders where the <index> indicates the index of the * parameterValue to substitute. *

      @@ -60,27 +71,41 @@ *

      * The implementation is thread-safe between calls to * {@link #open(ExecutionContext)}, but remember to use saveState=false - * if used in a multi-threaded client (no restart available. + * if used in a multi-threaded client (no restart available). *

      * * * @author Michael Minella + * @author Takaaki Iida */ public class MongoItemReader extends AbstractPaginatedDataItemReader implements InitializingBean { - + + private static final Logger log = LoggerFactory.getLogger(MongoItemReader.class); + private static final Pattern PLACEHOLDER = Pattern.compile("\\?(\\d+)"); private MongoOperations template; - private String query; + private Query query; + private String queryString; private Class type; private Sort sort; private String hint; private String fields; + private String collection; private List parameterValues; public MongoItemReader() { super(); setName(ClassUtils.getShortName(MongoItemReader.class)); } + + /** + * A Mongo Query to be used. + * + * @param query Mongo Query to be used. + */ + public void setQuery(Query query) { + this.query = query; + } /** * Used to perform operations against the MongoDB instance. Also @@ -98,10 +123,10 @@ public void setTemplate(MongoOperations template) { * via ?<index> placeholders where the <index> indicates the index of the * parameterValue to substitute. * - * @param query JSON formatted Mongo query + * @param queryString JSON formatted Mongo query */ - public void setQuery(String query) { - this.query = query; + public void setQuery(String queryString) { + this.queryString = queryString; } /** @@ -117,7 +142,7 @@ public void setTargetType(Class type) { * {@link List} of values to be substituted in for each of the * parameters in the query. * - * @param parameterValues + * @param parameterValues values */ public void setParameterValues(List parameterValues) { this.parameterValues = parameterValues; @@ -127,7 +152,7 @@ public void setParameterValues(List parameterValues) { * JSON defining the fields to be returned from the matching documents * by MongoDB. * - * @param fields JSON string that identifies the fields to sorty by. + * @param fields JSON string that identifies the fields to sort by. */ public void setFields(String fields) { this.fields = fields; @@ -143,6 +168,13 @@ public void setSort(Map sorts) { this.sort = convertToSort(sorts); } + /** + * @param collection Mongo collection to be queried. + */ + public void setCollection(String collection) { + this.collection = collection; + } + /** * JSON String telling MongoDB what index to use. * @@ -155,27 +187,42 @@ public void setHint(String hint) { @Override @SuppressWarnings("unchecked") protected Iterator doPageRead() { - - Pageable pageRequest = new PageRequest(page, pageSize, sort); - - String populatedQuery = replacePlaceholders(query, parameterValues); - - Query mongoQuery = null; - - if(StringUtils.hasText(fields)) { - mongoQuery = new BasicQuery(populatedQuery, fields); - } - else { - mongoQuery = new BasicQuery(populatedQuery); + if (queryString != null) { + Pageable pageRequest = PageRequest.of(page, pageSize, sort); + + String populatedQuery = replacePlaceholders(queryString, parameterValues); + + Query mongoQuery; + + if(StringUtils.hasText(fields)) { + mongoQuery = new BasicQuery(populatedQuery, fields); + } + else { + mongoQuery = new BasicQuery(populatedQuery); + } + + mongoQuery.with(pageRequest); + + if(StringUtils.hasText(hint)) { + mongoQuery.withHint(hint); + } + + if(StringUtils.hasText(collection)) { + return (Iterator) template.find(mongoQuery, type, collection).iterator(); + } else { + return (Iterator) template.find(mongoQuery, type).iterator(); + } + + } else { + Pageable pageRequest = PageRequest.of(page, pageSize); + query.with(pageRequest); + + if(StringUtils.hasText(collection)) { + return (Iterator) template.find(query, type, collection).iterator(); + } else { + return (Iterator) template.find(query, type).iterator(); + } } - - mongoQuery.with(pageRequest); - - if(StringUtils.hasText(hint)) { - mongoQuery.withHint(hint); - } - - return (Iterator) template.find(mongoQuery, type).iterator(); } /** @@ -183,11 +230,19 @@ protected Iterator doPageRead() { * * @see InitializingBean#afterPropertiesSet() */ + @Override public void afterPropertiesSet() throws Exception { Assert.state(template != null, "An implementation of MongoOperations is required."); Assert.state(type != null, "A type to convert the input into is required."); - Assert.state(query != null, "A query is required."); - Assert.state(sort != null, "A sort is required."); + Assert.state(queryString != null || query != null, "A query is required."); + + if (queryString != null) { + Assert.state(sort != null, "A sort is required."); + } + + if (query != null && query.getLimit() != 0) { + log.warn("PageSize in Query object was ignored. Please set it by MongoItemReader.setPageSize()."); + } } // Copied from StringBasedMongoQuery...is there a place where this type of logic is already exposed? @@ -210,12 +265,12 @@ private String getParameterWithIndex(List values, int index) { } private Sort convertToSort(Map sorts) { - List sortValues = new ArrayList(); + List sortValues = new ArrayList<>(); for (Map.Entry curSort : sorts.entrySet()) { sortValues.add(new Sort.Order(curSort.getValue(), curSort.getKey())); } - return new Sort(sortValues); + return Sort.by(sortValues); } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/MongoItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/MongoItemWriter.java index 491eb8440b..d36414ab31 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/MongoItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/MongoItemWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2012 the original author or authors. + * Copyright 2012-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -37,7 +37,7 @@ *

      * *

      - * This writer is thread safe once all properties are set (normal singleton behavior) so it can be used in multiple + * This writer is thread-safe once all properties are set (normal singleton behavior) so it can be used in multiple * concurrent transactions. *

      * @@ -46,15 +46,14 @@ */ public class MongoItemWriter implements ItemWriter, InitializingBean { - private static final String BUFFER_KEY_PREFIX = MongoItemWriter.class.getName() + ".BUFFER_KEY"; private MongoOperations template; - private final String bufferKey; + private final Object bufferKey; private String collection; private boolean delete = false; public MongoItemWriter() { super(); - this.bufferKey = BUFFER_KEY_PREFIX + "." + hashCode(); + this.bufferKey = new Object(); } /** @@ -77,6 +76,16 @@ public void setTemplate(MongoOperations template) { this.template = template; } + /** + * Get the {@link MongoOperations} to be used to save items to be written. + * This can be called by a subclass if necessary. + * + * @return template the template implementation to be used. + */ + protected MongoOperations getTemplate() { + return template; + } + /** * Set the name of the Mongo collection to be written to. * @@ -92,13 +101,14 @@ public void setCollection(String collection) { * * @see org.springframework.batch.item.ItemWriter#write(List) */ + @Override public void write(List items) throws Exception { if(!transactionActive()) { doWrite(items); return; } - List bufferedItems = getCurrentBuffer(); + List bufferedItems = getCurrentBuffer(); bufferedItems.addAll(items); } @@ -141,14 +151,15 @@ private boolean transactionActive() { return TransactionSynchronizationManager.isActualTransactionActive(); } - private List getCurrentBuffer() { + @SuppressWarnings("unchecked") + private List getCurrentBuffer() { if(!TransactionSynchronizationManager.hasResource(bufferKey)) { - TransactionSynchronizationManager.bindResource(bufferKey, new ArrayList()); + TransactionSynchronizationManager.bindResource(bufferKey, new ArrayList()); TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() { @Override public void beforeCommit(boolean readOnly) { - List items = (List) TransactionSynchronizationManager.getResource(bufferKey); + List items = (List) TransactionSynchronizationManager.getResource(bufferKey); if(!CollectionUtils.isEmpty(items)) { if(!readOnly) { @@ -166,9 +177,10 @@ public void afterCompletion(int status) { }); } - return (List) TransactionSynchronizationManager.getResource(bufferKey); + return (List) TransactionSynchronizationManager.getResource(bufferKey); } + @Override public void afterPropertiesSet() throws Exception { Assert.state(template != null, "A MongoOperations implementation is required."); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/Neo4jItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/Neo4jItemReader.java index eec6be17de..06e2224a1f 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/Neo4jItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/Neo4jItemReader.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,208 +18,32 @@ import java.util.ArrayList; import java.util.Iterator; -import java.util.Map; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.springframework.batch.item.ItemReader; -import org.springframework.beans.factory.InitializingBean; -import org.springframework.data.neo4j.conversion.DefaultConverter; -import org.springframework.data.neo4j.conversion.Result; -import org.springframework.data.neo4j.conversion.ResultConverter; -import org.springframework.data.neo4j.template.Neo4jOperations; -import org.springframework.util.Assert; -import org.springframework.util.ClassUtils; -import org.springframework.util.StringUtils; +import org.neo4j.ogm.session.Session; /** *

      - * Restartable {@link ItemReader} that reads objects from the graph database Neo4j - * via a paging technique. - *

      - * - *

      - * It executes cypher queries built from the statement fragments provided to - * retrieve the requested data. The query is executed using paged requests of - * a size specified in {@link #setPageSize(int)}. Additional pages are requested - * as needed when the {@link #read()} method is called. On restart, the reader - * will begin again at the same number item it left off at. - *

      - * - *

      - * Performance is dependent on your Neo4J configuration (embedded or remote) as - * well as page size. Setting a fairly large page size and using a commit - * interval that matches the page size should provide better performance. - *

      - * - *

      - * This implementation is thread-safe between calls to - * {@link #open(org.springframework.batch.item.ExecutionContext)}, however you - * should set saveState=false if used in a multi-threaded - * environment (no restart available). + * Extensions of the {@link AbstractNeo4jItemReader}. *

      * * @author Michael Minella - * */ -public class Neo4jItemReader extends AbstractPaginatedDataItemReader implements -InitializingBean { - - protected Log logger = LogFactory.getLog(getClass()); - - private Neo4jOperations template; - - private String startStatement; - private String returnStatement; - private String matchStatement; - private String whereStatement; - private String orderByStatement; - - private Class targetType; - - private Map parameterValues; - - private ResultConverter resultConverter; - - public Neo4jItemReader() { - setName(ClassUtils.getShortName(Neo4jItemReader.class)); - } - - /** - * The start segment of the cypher query. START is prepended - * to the statement provided and should not be - * included. - * - * @param startStatement the start fragment of the cypher query. - */ - public void setStartStatement(String startStatement) { - this.startStatement = startStatement; - } - - /** - * The return statement of the cypher query. RETURN is prepended - * to the statement provided and should not be - * included - * - * @param returnStatement the return fragment of the cypher query. - */ - public void setReturnStatement(String returnStatement) { - this.returnStatement = returnStatement; - } - - /** - * An optional match fragment of the cypher query. MATCH is - * prepended to the statement provided and should not - * be included. - * - * @param matchStatement the match fragment of the cypher query - */ - public void setMatchStatement(String matchStatement) { - this.matchStatement = matchStatement; - } - - /** - * An optional where fragement of the cypher query. WHERE is - * prepended to the statement provided and should not - * be included. - * - * @param whereStatement where fragment of the cypher query - */ - public void setWhereStatement(String whereStatement) { - this.whereStatement = whereStatement; - } - - /** - * A list of properties to order the results by. This is - * required so that subsequent page requests pull back the - * segment of results correctly. ORDER BY is prepended to - * the statement provided and should not be included. - * - * @param orderByStatement order by fragment of the cypher query. - */ - public void setOrderByStatement(String orderByStatement) { - this.orderByStatement = orderByStatement; - } - - /** - * Used to perform operations against the Neo4J database. - * - * @param template the Neo4jOperations instance to use - * @see Neo4jOperations - */ - public void setTemplate(Neo4jOperations template) { - this.template = template; - } - - /** - * The object type to be returned from each call to {@link #read()} - * - * @param targetType the type of object to return. - */ - public void setTargetType(Class targetType) { - this.targetType = targetType; - } - - /** - * Set the converter used to convert node to the targetType. By - * default, {@link DefaultConverter} is used. - * - * @param resultConverter the converter to use. - */ - public void setResultConverter(ResultConverter resultConverter) { - this.resultConverter = resultConverter; - } +public class Neo4jItemReader extends AbstractNeo4jItemReader { + @SuppressWarnings("unchecked") @Override - @SuppressWarnings({"unchecked", "rawtypes"}) protected Iterator doPageRead() { - Result> queryResults = template.query( - generateLimitCypherQuery(), parameterValues); + Session session = getSessionFactory().openSession(); + + Iterable queryResults = session.query(getTargetType(), + generateLimitCypherQuery(), + getParameterValues()); if(queryResults != null) { - if (resultConverter != null) { - return queryResults.to(targetType, resultConverter).iterator(); - } - else { - return queryResults.to(targetType).iterator(); - } + return queryResults.iterator(); } else { - return new ArrayList().iterator(); + return new ArrayList().iterator(); } } - - private String generateLimitCypherQuery() { - StringBuilder query = new StringBuilder(); - - query.append("START ").append(startStatement); - query.append(matchStatement != null ? " MATCH " + matchStatement : ""); - query.append(whereStatement != null ? " WHERE " + whereStatement : ""); - query.append(" RETURN ").append(returnStatement); - query.append(" ORDER BY ").append(orderByStatement); - query.append(" SKIP " + (pageSize * page)); - query.append(" LIMIT " + pageSize); - - String resultingQuery = query.toString(); - - if (logger.isDebugEnabled()) { - logger.debug(resultingQuery); - } - - return resultingQuery; - } - - /** - * Checks mandatory properties - * - * @see InitializingBean#afterPropertiesSet() - */ - @Override - public void afterPropertiesSet() throws Exception { - Assert.state(template != null, "A Neo4JOperations implementation is required"); - Assert.state(targetType != null, "The type to be returned is required"); - Assert.state(StringUtils.hasText(startStatement), "A START statement is required"); - Assert.state(StringUtils.hasText(returnStatement), "A RETURN statement is required"); - Assert.state(StringUtils.hasText(orderByStatement), "A ORDER BY statement is required"); - } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/Neo4jItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/Neo4jItemWriter.java index 47e5b6b042..9ce5c29160 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/Neo4jItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/Neo4jItemWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2012 the original author or authors. + * Copyright 2012-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,24 +20,27 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; +import org.neo4j.ogm.session.Session; +import org.neo4j.ogm.session.SessionFactory; + import org.springframework.batch.item.ItemWriter; import org.springframework.beans.factory.InitializingBean; -import org.springframework.data.neo4j.template.Neo4jOperations; import org.springframework.util.Assert; import org.springframework.util.CollectionUtils; /** *

      - * A {@link ItemWriter} implementation that writes to a Neo4j database using an - * implementation of Spring Data's {@link Neo4jOperations}. + * A {@link ItemWriter} implementation that writes to a Neo4j database. *

      * *

      - * This writer is thread safe once all properties are set (normal singleton + * This writer is thread-safe once all properties are set (normal singleton * behavior) so it can be used in multiple concurrent transactions. *

      * * @author Michael Minella + * @author Glenn Renfro + * @author Mahmoud Ben Hassine * */ public class Neo4jItemWriter implements ItemWriter, InitializingBean { @@ -47,19 +50,25 @@ public class Neo4jItemWriter implements ItemWriter, InitializingBean { private boolean delete = false; - private Neo4jOperations template; + private SessionFactory sessionFactory; + /** + * Boolean flag indicating whether the writer should save or delete the item at write + * time. + * @param delete true if write should delete item, false if item should be saved. + * Default is false. + */ public void setDelete(boolean delete) { this.delete = delete; } /** - * Set the {@link Neo4jOperations} to be used to save items - * - * @param template the template implementation to be used + * Establish the session factory that will be used to create {@link Session} instances + * for interacting with Neo4j. + * @param sessionFactory sessionFactory to be used. */ - public void setTemplate(Neo4jOperations template) { - this.template = template; + public void setSessionFactory(SessionFactory sessionFactory) { + this.sessionFactory = sessionFactory; } /** @@ -67,8 +76,10 @@ public void setTemplate(Neo4jOperations template) { * * @see InitializingBean#afterPropertiesSet() */ + @Override public void afterPropertiesSet() throws Exception { - Assert.state(template != null, "A Neo4JOperations implementation is required"); + Assert.state(this.sessionFactory != null, + "A SessionFactory is required"); } /** @@ -76,6 +87,7 @@ public void afterPropertiesSet() throws Exception { * * @see org.springframework.batch.item.ItemWriter#write(java.util.List) */ + @Override public void write(List items) throws Exception { if(!CollectionUtils.isEmpty(items)) { doWrite(items); @@ -83,21 +95,33 @@ public void write(List items) throws Exception { } /** - * Performs the actual write using the template. This can be overriden by + * Performs the actual write using the template. This can be overridden by * a subclass if necessary. * * @param items the list of items to be persisted. */ protected void doWrite(List items) { if(delete) { - for (T t : items) { - template.delete(t); - } + delete(items); } else { - for (T t : items) { - template.save(t); - } + save(items); + } + } + + private void delete(List items) { + Session session = this.sessionFactory.openSession(); + + for(T item : items) { + session.delete(item); + } + } + + private void save(List items) { + Session session = this.sessionFactory.openSession(); + + for (T item : items) { + session.save(item); } } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/RepositoryItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/RepositoryItemReader.java index cce7ad9afd..ae3b6b2d4e 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/RepositoryItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/RepositoryItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2012 the original author or authors. + * Copyright 2012-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.adapter.AbstractMethodInvokingDelegator.InvocationTargetThrowableWrapper; import org.springframework.batch.item.adapter.DynamicMethodInvocationException; @@ -32,6 +33,7 @@ import org.springframework.data.domain.Pageable; import org.springframework.data.domain.Sort; import org.springframework.data.repository.PagingAndSortingRepository; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.ClassUtils; import org.springframework.util.MethodInvoker; @@ -54,19 +56,28 @@ *

      * *

      - * This implementation is thread safe between calls to {@link #open(ExecutionContext)}, but remember to use + * This implementation is thread-safe between calls to {@link #open(ExecutionContext)}, but remember to use * saveState=false if used in a multi-threaded client (no restart available). *

      * + *

      It is important to note that this is a paging item reader and exceptions that are + * thrown while reading the page itself (mapping results to objects, etc in the + * {@link RepositoryItemReader#doPageRead()}) will not be skippable since this reader has + * no way of knowing if an exception should be skipped and therefore will continue to read + * the same page until the skip limit is exceeded.

      + * + *

      + * NOTE: The {@code RepositoryItemReader} only reads Java Objects i.e. non primitives. + *

      + * * @author Michael Minella * @since 2.2 */ -@SuppressWarnings("rawtypes") public class RepositoryItemReader extends AbstractItemCountingItemStreamItemReader implements InitializingBean { protected Log logger = LogFactory.getLog(getClass()); - private PagingAndSortingRepository repository; + private PagingAndSortingRepository repository; private Sort sort; @@ -76,7 +87,7 @@ public class RepositoryItemReader extends AbstractItemCountingItemStreamItemR private volatile int current = 0; - private List arguments; + private List arguments; private volatile List results; @@ -93,7 +104,7 @@ public RepositoryItemReader() { * * @param arguments list of method arguments to be passed to the repository */ - public void setArguments(List arguments) { + public void setArguments(List arguments) { this.arguments = arguments; } @@ -119,7 +130,7 @@ public void setPageSize(int pageSize) { * * @param repository underlying repository for input to be read from. */ - public void setRepository(PagingAndSortingRepository repository) { + public void setRepository(PagingAndSortingRepository repository) { this.repository = repository; } @@ -127,7 +138,7 @@ public void setRepository(PagingAndSortingRepository repository) { * Specifies what method on the repository to call. This method must take * {@link org.springframework.data.domain.Pageable} as the last argument. * - * @param methodName + * @param methodName name of the method to invoke */ public void setMethodName(String methodName) { this.methodName = methodName; @@ -140,6 +151,7 @@ public void afterPropertiesSet() throws Exception { Assert.state(sort != null, "A sort is required"); } + @Nullable @Override protected T doRead() throws Exception { @@ -174,10 +186,11 @@ protected T doRead() throws Exception { @Override protected void jumpToItem(int itemLastIndex) throws Exception { synchronized (lock) { - page = itemLastIndex / pageSize; - current = itemLastIndex % pageSize; + page = (itemLastIndex - 1) / pageSize; + current = (itemLastIndex - 1) % pageSize; results = doPageRead(); + page++; } } @@ -186,15 +199,16 @@ protected void jumpToItem(int itemLastIndex) throws Exception { * Available for overriding as needed. * * @return the list of items that make up the page - * @throws Exception + * @throws Exception Based on what the underlying method throws or related to the + * calling of the method */ @SuppressWarnings("unchecked") protected List doPageRead() throws Exception { - Pageable pageRequest = new PageRequest(page, pageSize, sort); + Pageable pageRequest = PageRequest.of(page, pageSize, sort); MethodInvoker invoker = createMethodInvoker(repository, methodName); - List parameters = new ArrayList(); + List parameters = new ArrayList<>(); if(arguments != null && arguments.size() > 0) { parameters.addAll(arguments); @@ -204,7 +218,7 @@ protected List doPageRead() throws Exception { invoker.setArguments(parameters.toArray()); - Page curPage = (Page) doInvoke(invoker); + Page curPage = (Page) doInvoke(invoker); return curPage.getContent(); } @@ -215,26 +229,28 @@ protected void doOpen() throws Exception { @Override protected void doClose() throws Exception { + synchronized (lock) { + current = 0; + page = 0; + results = null; + } } private Sort convertToSort(Map sorts) { - List sortValues = new ArrayList(); + List sortValues = new ArrayList<>(); for (Map.Entry curSort : sorts.entrySet()) { sortValues.add(new Sort.Order(curSort.getValue(), curSort.getKey())); } - return new Sort(sortValues); + return Sort.by(sortValues); } private Object doInvoke(MethodInvoker invoker) throws Exception{ try { invoker.prepare(); } - catch (ClassNotFoundException e) { - throw new DynamicMethodInvocationException(e); - } - catch (NoSuchMethodException e) { + catch (ClassNotFoundException | NoSuchMethodException e) { throw new DynamicMethodInvocationException(e); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/RepositoryItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/RepositoryItemWriter.java index 6da207ca39..5f5d18e152 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/RepositoryItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/RepositoryItemWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2012 the original author or authors. + * Copyright 2012-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,12 +31,12 @@ /** *

      - * A {@link org.springframework.batch.item.ItemReader} wrapper for a + * A {@link org.springframework.batch.item.ItemWriter} wrapper for a * {@link org.springframework.data.repository.CrudRepository} from Spring Data. *

      * *

      - * It depends on {@link org.springframework.data.repository.CrudRepository#save(Iterable)} + * It depends on {@link org.springframework.data.repository.CrudRepository#saveAll(Iterable)} * method to store the items for the chunk. Performance will be determined by that * implementation more than this writer. *

      @@ -47,23 +47,27 @@ * transactions. *

      * + *

      + * NOTE: The {@code RepositoryItemWriter} only stores Java Objects i.e. non primitives. + *

      + * * @author Michael Minella + * @author Mahmoud Ben Hassine * @since 2.2 */ -@SuppressWarnings("rawtypes") -public class RepositoryItemWriter implements ItemWriter, InitializingBean { +public class RepositoryItemWriter implements ItemWriter, InitializingBean { protected static final Log logger = LogFactory.getLog(RepositoryItemWriter.class); - private CrudRepository repository; + private CrudRepository repository; private String methodName; /** - * Specifies what method on the repository to call. This method must the type of + * Specifies what method on the repository to call. This method must have the type of * object passed to this writer as the sole argument. * - * @param methodName + * @param methodName {@link String} containing the method name. */ public void setMethodName(String methodName) { this.methodName = methodName; @@ -75,7 +79,7 @@ public void setMethodName(String methodName) { * * @param repository the Spring Data repository to be set */ - public void setRepository(CrudRepository repository) { + public void setRepository(CrudRepository repository) { this.repository = repository; } @@ -85,26 +89,28 @@ public void setRepository(CrudRepository repository) { * @see org.springframework.batch.item.ItemWriter#write(java.util.List) */ @Override - public void write(List items) throws Exception { + public void write(List items) throws Exception { if(!CollectionUtils.isEmpty(items)) { doWrite(items); } } /** - * Performs the actual write to the repository. This can be overriden by + * Performs the actual write to the repository. This can be overridden by * a subclass if necessary. * * @param items the list of items to be persisted. + * + * @throws Exception thrown if error occurs during writing. */ - protected void doWrite(List items) throws Exception { + protected void doWrite(List items) throws Exception { if (logger.isDebugEnabled()) { logger.debug("Writing to the repository with " + items.size() + " items."); } MethodInvoker invoker = createMethodInvoker(repository, methodName); - for (Object object : items) { + for (T object : items) { invoker.setArguments(new Object [] {object}); doInvoke(invoker); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/SpELMappingGemfireItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/SpELMappingGemfireItemWriter.java index bd37b4be03..a9ee5d06b3 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/SpELMappingGemfireItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/SpELMappingGemfireItemWriter.java @@ -4,7 +4,7 @@ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the @@ -29,6 +29,6 @@ public class SpELMappingGemfireItemWriter extends GemfireItemWriter SpELMappingGemfireItemWriter(String keyExpression) { super(); Assert.hasText(keyExpression, "a valid keyExpression is required."); - setItemKeyMapper(new SpELItemKeyMapper(keyExpression)); + setItemKeyMapper(new SpELItemKeyMapper<>(keyExpression)); } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/GemfireItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/GemfireItemWriterBuilder.java new file mode 100644 index 0000000000..9cb26c2b6f --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/GemfireItemWriterBuilder.java @@ -0,0 +1,95 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import org.springframework.batch.item.data.GemfireItemWriter; +import org.springframework.core.convert.converter.Converter; +import org.springframework.data.gemfire.GemfireTemplate; +import org.springframework.util.Assert; + +/** + * A builder implementation for the {@link GemfireItemWriter} + * + * @author Glenn Renfro + * @since 4.0 + * @see GemfireItemWriterBuilder + */ +public class GemfireItemWriterBuilder { + + private GemfireTemplate template; + + private Converter itemKeyMapper; + + private boolean delete; + + /** + * Establishes the GemfireTemplate the writer should use. + * @param template the {@link GemfireTemplate} to set. + * @return The current instance of the builder. + * @see GemfireItemWriter#setTemplate(GemfireTemplate) + */ + public GemfireItemWriterBuilder template(GemfireTemplate template) { + this.template = template; + + return this; + } + + /** + * Set the {@link Converter} to use to derive the key from the item. + * + * @param itemKeyMapper the Converter to use. + * @return The current instance of the builder. + * @see GemfireItemWriter#setItemKeyMapper(Converter) + */ + public GemfireItemWriterBuilder itemKeyMapper(Converter itemKeyMapper) { + this.itemKeyMapper = itemKeyMapper; + + return this; + } + + /** + * Indicates if the items being passed to the writer are to be saved or removed from + * the data store. If set to false (default), the items will be saved. If set to true, + * the items will be removed. + * + * @param delete removal indicator. + * @return The current instance of the builder. + * @see GemfireItemWriter#setDelete(boolean) + */ + public GemfireItemWriterBuilder delete(boolean delete) { + this.delete = delete; + + return this; + } + + + /** + * Validates and builds a {@link GemfireItemWriter}. + * + * @return a {@link GemfireItemWriter} + */ + public GemfireItemWriter build() { + Assert.notNull(this.template, "template is required."); + Assert.notNull(this.itemKeyMapper, "itemKeyMapper is required."); + + GemfireItemWriter writer = new GemfireItemWriter<>(); + writer.setTemplate(this.template); + writer.setItemKeyMapper(this.itemKeyMapper); + writer.setDelete(this.delete); + return writer; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/MongoItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/MongoItemReaderBuilder.java new file mode 100644 index 0000000000..3dd4d65031 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/MongoItemReaderBuilder.java @@ -0,0 +1,311 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import java.util.Arrays; +import java.util.List; +import java.util.Map; + +import org.springframework.batch.item.data.MongoItemReader; +import org.springframework.data.domain.Sort; +import org.springframework.data.mongodb.core.MongoOperations; +import org.springframework.data.mongodb.core.query.Query; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * A builder implementation for the {@link MongoItemReader} + * + * @author Glenn Renfro + * @author Mahmoud Ben Hassine + * @author Drummond Dawson + * @since 4.0 + * @see MongoItemReader + */ +public class MongoItemReaderBuilder { + private MongoOperations template; + + private String jsonQuery; + + private Class targetType; + + private Map sorts; + + private String hint; + + private String fields; + + private String collection; + + private List parameterValues; + + protected int pageSize = 10; + + private boolean saveState = true; + + private String name; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + private Query query; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public MongoItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public MongoItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public MongoItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public MongoItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * Used to perform operations against the MongoDB instance. Also handles the mapping + * of documents to objects. + * + * @param template the MongoOperations instance to use + * @see MongoOperations + * @return The current instance of the builder + * @see MongoItemReader#setTemplate(MongoOperations) + */ + public MongoItemReaderBuilder template(MongoOperations template) { + this.template = template; + + return this; + } + + /** + * A JSON formatted MongoDB jsonQuery. Parameterization of the provided jsonQuery is allowed + * via ?<index> placeholders where the <index> indicates the index of the + * parameterValue to substitute. + * + * @param query JSON formatted Mongo jsonQuery + * @return The current instance of the builder + * @see MongoItemReader#setQuery(String) + */ + public MongoItemReaderBuilder jsonQuery(String query) { + this.jsonQuery = query; + + return this; + } + + /** + * The type of object to be returned for each {@link MongoItemReader#read()} call. + * + * @param targetType the type of object to return + * @return The current instance of the builder + * @see MongoItemReader#setTargetType(Class) + */ + public MongoItemReaderBuilder targetType(Class targetType) { + this.targetType = targetType; + + return this; + } + + /** + * {@link List} of values to be substituted in for each of the parameters in the + * query. + * + * @param parameterValues values + * @return The current instance of the builder + * @see MongoItemReader#setParameterValues(List) + */ + public MongoItemReaderBuilder parameterValues(List parameterValues) { + this.parameterValues = parameterValues; + + return this; + } + + /** + * Values to be substituted in for each of the parameters in the query. + * + * @param parameterValues values + * @return The current instance of the builder + * @see MongoItemReader#setParameterValues(List) + */ + public MongoItemReaderBuilder parameterValues(Object... parameterValues) { + return parameterValues(Arrays.asList(parameterValues)); + } + + /** + * JSON defining the fields to be returned from the matching documents by MongoDB. + * + * @param fields JSON string that identifies the fields to sort by. + * @return The current instance of the builder + * @see MongoItemReader#setFields(String) + */ + public MongoItemReaderBuilder fields(String fields) { + this.fields = fields; + + return this; + } + + /** + * {@link Map} of property + * names/{@link org.springframework.data.domain.Sort.Direction} values to sort the + * input by. + * + * @param sorts map of properties and direction to sort each. + * @return The current instance of the builder + * @see MongoItemReader#setSort(Map) + */ + public MongoItemReaderBuilder sorts(Map sorts) { + this.sorts = sorts; + + return this; + } + + /** + * Establish an optional collection that can be queried. + * + * @param collection Mongo collection to be queried. + * @return The current instance of the builder + * @see MongoItemReader#setCollection(String) + */ + public MongoItemReaderBuilder collection(String collection) { + this.collection = collection; + + return this; + } + + /** + * JSON String telling MongoDB what index to use. + * + * @param hint string indicating what index to use. + * @return The current instance of the builder + * @see MongoItemReader#setHint(String) + */ + public MongoItemReaderBuilder hint(String hint) { + this.hint = hint; + + return this; + } + + /** + * The number of items to be read with each page. + * + * @param pageSize the number of items + * @return this instance for method chaining + * @see MongoItemReader#setPageSize(int) + */ + public MongoItemReaderBuilder pageSize(int pageSize) { + this.pageSize = pageSize; + + return this; + } + + /** + * Provide a Spring Data Mongo {@link Query}. This will take precedence over a JSON + * configured query. + * + * @param query Query to execute + * @return this instance for method chaining + * @see MongoItemReader#setQuery(Query) + */ + public MongoItemReaderBuilder query(Query query) { + this.query = query; + + return this; + } + + /** + * Validates and builds a {@link MongoItemReader}. + * + * @return a {@link MongoItemReader} + */ + public MongoItemReader build() { + Assert.notNull(this.template, "template is required."); + if (this.saveState) { + Assert.hasText(this.name, "A name is required when saveState is set to true"); + } + Assert.notNull(this.targetType, "targetType is required."); + Assert.state(StringUtils.hasText(this.jsonQuery) || this.query != null, "A query is required"); + + if(StringUtils.hasText(this.jsonQuery)) { + Assert.notNull(this.sorts, "sorts map is required."); + } + else { + Assert.state(this.query.getLimit() != 0, "PageSize in Query object was ignored."); + } + + MongoItemReader reader = new MongoItemReader<>(); + reader.setTemplate(this.template); + reader.setTargetType(this.targetType); + reader.setQuery(this.jsonQuery); + reader.setSort(this.sorts); + reader.setHint(this.hint); + reader.setFields(this.fields); + reader.setCollection(this.collection); + reader.setParameterValues(this.parameterValues); + reader.setQuery(this.query); + + reader.setPageSize(this.pageSize); + reader.setName(this.name); + reader.setSaveState(this.saveState); + reader.setCurrentItemCount(this.currentItemCount); + reader.setMaxItemCount(this.maxItemCount); + + return reader; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/MongoItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/MongoItemWriterBuilder.java new file mode 100644 index 0000000000..4ac0cb7703 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/MongoItemWriterBuilder.java @@ -0,0 +1,96 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import org.springframework.batch.item.data.MongoItemWriter; +import org.springframework.data.mongodb.core.MongoOperations; +import org.springframework.util.Assert; + +/** + * A builder implementation for the {@link MongoItemWriter} + * + * @author Glenn Renfro + * @since 4.0 + * @see MongoItemWriter + */ +public class MongoItemWriterBuilder { + + private MongoOperations template; + + private String collection; + + private boolean delete = false; + + /** + * Indicates if the items being passed to the writer are to be saved or removed from + * the data store. If set to false (default), the items will be saved. If set to true, + * the items will be removed. + * + * @param delete removal indicator + * @return The current instance of the builder + * @see MongoItemWriter#setDelete(boolean) + */ + public MongoItemWriterBuilder delete(boolean delete) { + this.delete = delete; + + return this; + } + + /** + * Set the {@link MongoOperations} to be used to save items to be written. + * + * @param template the template implementation to be used. + * @return The current instance of the builder + * @see MongoItemWriter#setTemplate(MongoOperations) + */ + public MongoItemWriterBuilder template(MongoOperations template) { + this.template = template; + + return this; + } + + /** + * Set the name of the Mongo collection to be written to. + * + * @param collection the name of the collection. + * @return The current instance of the builder + * @see MongoItemWriter#setCollection(String) + * + */ + public MongoItemWriterBuilder collection(String collection) { + this.collection = collection; + + return this; + } + + /** + * Validates and builds a {@link MongoItemWriter}. + * + * @return a {@link MongoItemWriter} + */ + public MongoItemWriter build() { + Assert.notNull(this.template, "template is required."); + + MongoItemWriter writer = new MongoItemWriter<>(); + writer.setTemplate(this.template); + writer.setDelete(this.delete); + writer.setCollection(this.collection); + + return writer; + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/Neo4jItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/Neo4jItemReaderBuilder.java new file mode 100644 index 0000000000..5b345406d3 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/Neo4jItemReaderBuilder.java @@ -0,0 +1,274 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import java.util.Map; + +import org.neo4j.ogm.session.SessionFactory; + +import org.springframework.batch.item.data.Neo4jItemReader; +import org.springframework.util.Assert; + +/** + * A builder for the {@link Neo4jItemReader}. + * + * @author Glenn Renfro + * @since 4.0 + * @see Neo4jItemReader + */ +public class Neo4jItemReaderBuilder { + + private SessionFactory sessionFactory; + + private String startStatement; + + private String returnStatement; + + private String matchStatement; + + private String whereStatement; + + private String orderByStatement; + + private Class targetType; + + private Map parameterValues; + + private int pageSize = 10; + + private boolean saveState = true; + + private String name; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public Neo4jItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public Neo4jItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public Neo4jItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public Neo4jItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * Establish the session factory for the reader. + * @param sessionFactory the factory to use for the reader. + * @return this instance for method chaining + * @see Neo4jItemReader#setSessionFactory(SessionFactory) + */ + public Neo4jItemReaderBuilder sessionFactory(SessionFactory sessionFactory) { + this.sessionFactory = sessionFactory; + + return this; + } + + /** + * The number of items to be read with each page. + * + * @param pageSize the number of items + * @return this instance for method chaining + * @see Neo4jItemReader#setPageSize(int) + */ + public Neo4jItemReaderBuilder pageSize(int pageSize) { + this.pageSize = pageSize; + + return this; + } + + /** + * Optional parameters to be used in the cypher query. + * + * @param parameterValues the parameter values to be used in the cypher query + * @return this instance for method chaining + * @see Neo4jItemReader#setParameterValues(Map) + */ + public Neo4jItemReaderBuilder parameterValues(Map parameterValues) { + this.parameterValues = parameterValues; + + return this; + } + + /** + * The start segment of the cypher query. START is prepended to the statement provided + * and should not be included. + * + * @param startStatement the start fragment of the cypher query. + * @return this instance for method chaining + * @see Neo4jItemReader#setStartStatement(String) + */ + public Neo4jItemReaderBuilder startStatement(String startStatement) { + this.startStatement = startStatement; + + return this; + } + + /** + * The return statement of the cypher query. RETURN is prepended to the statement + * provided and should not be included + * + * @param returnStatement the return fragment of the cypher query. + * @return this instance for method chaining + * @see Neo4jItemReader#setReturnStatement(String) + */ + public Neo4jItemReaderBuilder returnStatement(String returnStatement) { + this.returnStatement = returnStatement; + + return this; + } + + /** + * An optional match fragment of the cypher query. MATCH is prepended to the statement + * provided and should not be included. + * + * @param matchStatement the match fragment of the cypher query + * @return this instance for method chaining + * @see Neo4jItemReader#setMatchStatement(String) + */ + public Neo4jItemReaderBuilder matchStatement(String matchStatement) { + this.matchStatement = matchStatement; + + return this; + } + + /** + * An optional where fragment of the cypher query. WHERE is prepended to the statement + * provided and should not be included. + * + * @param whereStatement where fragment of the cypher query + * @return this instance for method chaining + * @see Neo4jItemReader#setWhereStatement(String) + */ + public Neo4jItemReaderBuilder whereStatement(String whereStatement) { + this.whereStatement = whereStatement; + + return this; + } + + /** + * A list of properties to order the results by. This is required so that subsequent + * page requests pull back the segment of results correctly. ORDER BY is prepended to + * the statement provided and should not be included. + * + * @param orderByStatement order by fragment of the cypher query. + * @return this instance for method chaining + * @see Neo4jItemReader#setOrderByStatement(String) + */ + public Neo4jItemReaderBuilder orderByStatement(String orderByStatement) { + this.orderByStatement = orderByStatement; + + return this; + } + + /** + * The object type to be returned from each call to {@link Neo4jItemReader#read()} + * + * @param targetType the type of object to return. + * @return this instance for method chaining + * @see Neo4jItemReader#setTargetType(Class) + */ + public Neo4jItemReaderBuilder targetType(Class targetType) { + this.targetType = targetType; + + return this; + } + + /** + * Returns a fully constructed {@link Neo4jItemReader}. + * + * @return a new {@link Neo4jItemReader} + */ + public Neo4jItemReader build() { + if (this.saveState) { + Assert.hasText(this.name, "A name is required when saveState is set to true"); + } + Assert.notNull(this.sessionFactory, "sessionFactory is required."); + Assert.notNull(this.targetType, "targetType is required."); + Assert.hasText(this.startStatement, "startStatement is required."); + Assert.hasText(this.returnStatement, "returnStatement is required."); + Assert.hasText(this.orderByStatement, "orderByStatement is required."); + Assert.isTrue(this.pageSize > 0, "pageSize must be greater than zero"); + Assert.isTrue(this.maxItemCount > 0, "maxItemCount must be greater than zero"); + Assert.isTrue(this.maxItemCount > this.currentItemCount , "maxItemCount must be greater than currentItemCount"); + + Neo4jItemReader reader = new Neo4jItemReader<>(); + reader.setMatchStatement(this.matchStatement); + reader.setOrderByStatement(this.orderByStatement); + reader.setPageSize(this.pageSize); + reader.setParameterValues(this.parameterValues); + reader.setSessionFactory(this.sessionFactory); + reader.setTargetType(this.targetType); + reader.setStartStatement(this.startStatement); + reader.setReturnStatement(this.returnStatement); + reader.setWhereStatement(this.whereStatement); + reader.setName(this.name); + reader.setSaveState(this.saveState); + reader.setCurrentItemCount(this.currentItemCount); + reader.setMaxItemCount(this.maxItemCount); + + return reader; + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/Neo4jItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/Neo4jItemWriterBuilder.java new file mode 100644 index 0000000000..b6823b137d --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/Neo4jItemWriterBuilder.java @@ -0,0 +1,77 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import org.neo4j.ogm.session.Session; +import org.neo4j.ogm.session.SessionFactory; + +import org.springframework.batch.item.data.Neo4jItemWriter; +import org.springframework.util.Assert; + +/** + * A builder implementation for the {@link Neo4jItemWriter} + * + * @author Glenn Renfro + * @since 4.0 + * @see Neo4jItemWriter + */ +public class Neo4jItemWriterBuilder { + + private boolean delete = false; + + private SessionFactory sessionFactory; + + /** + * Boolean flag indicating whether the writer should save or delete the item at write + * time. + * @param delete true if write should delete item, false if item should be saved. + * Default is false. + * @return The current instance of the builder + * @see Neo4jItemWriter#setDelete(boolean) + */ + public Neo4jItemWriterBuilder delete(boolean delete) { + this.delete = delete; + + return this; + } + + /** + * Establish the session factory that will be used to create {@link Session} instances + * for interacting with Neo4j. + * @param sessionFactory sessionFactory to be used. + * @return The current instance of the builder + * @see Neo4jItemWriter#setSessionFactory(SessionFactory) + */ + public Neo4jItemWriterBuilder sessionFactory(SessionFactory sessionFactory) { + this.sessionFactory = sessionFactory; + + return this; + } + + /** + * Validates and builds a {@link org.springframework.batch.item.data.Neo4jItemWriter}. + * + * @return a {@link Neo4jItemWriter} + */ + public Neo4jItemWriter build() { + Assert.notNull(sessionFactory, "sessionFactory is required."); + Neo4jItemWriter writer = new Neo4jItemWriter<>(); + writer.setDelete(this.delete); + writer.setSessionFactory(this.sessionFactory); + return writer; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/RepositoryItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/RepositoryItemReaderBuilder.java new file mode 100644 index 0000000000..aa9e245fe7 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/RepositoryItemReaderBuilder.java @@ -0,0 +1,324 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import java.lang.reflect.Method; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.Map; + +import org.springframework.batch.item.data.RepositoryItemReader; +import org.springframework.cglib.proxy.Enhancer; +import org.springframework.cglib.proxy.MethodInterceptor; +import org.springframework.cglib.proxy.MethodProxy; +import org.springframework.data.domain.Sort; +import org.springframework.data.repository.PagingAndSortingRepository; +import org.springframework.util.Assert; +import org.springframework.util.CollectionUtils; +import org.springframework.util.StringUtils; + +/** + * A builder implementation for the {@link RepositoryItemReader}. + * + * @author Glenn Renfro + * @author Mahmoud Ben Hassine + * @author Drummond Dawson + * @since 4.0 + * @see RepositoryItemReader + */ +public class RepositoryItemReaderBuilder { + + private PagingAndSortingRepository repository; + + private Map sorts; + + private List arguments; + + private int pageSize = 10; + + private String methodName; + + private RepositoryMethodReference repositoryMethodReference; + + private boolean saveState = true; + + private String name; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public RepositoryItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public RepositoryItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public RepositoryItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public RepositoryItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * Arguments to be passed to the data providing method. + * + * @param arguments list of method arguments to be passed to the repository. + * @return The current instance of the builder. + * @see RepositoryItemReader#setArguments(List) + */ + public RepositoryItemReaderBuilder arguments(List arguments) { + this.arguments = arguments; + + return this; + } + + /** + * Arguments to be passed to the data providing method. + * + * @param arguments the method arguments to be passed to the repository. + * @return The current instance of the builder. + * @see RepositoryItemReader#setArguments(List) + */ + public RepositoryItemReaderBuilder arguments(Object... arguments) { + return arguments(Arrays.asList(arguments)); + } + + /** + * Provides ordering of the results so that order is maintained between paged queries. + * + * @param sorts the fields to sort by and the directions. + * @return The current instance of the builder. + * @see RepositoryItemReader#setSort(Map) + */ + public RepositoryItemReaderBuilder sorts(Map sorts) { + this.sorts = sorts; + + return this; + } + + /** + * Establish the pageSize for the generated RepositoryItemReader. + * + * @param pageSize The number of items to retrieve per page. + * @return The current instance of the builder. + * @see RepositoryItemReader#setPageSize(int) + */ + public RepositoryItemReaderBuilder pageSize(int pageSize) { + this.pageSize = pageSize; + + return this; + } + + /** + * The {@link org.springframework.data.repository.PagingAndSortingRepository} + * implementation used to read input from. + * + * @param repository underlying repository for input to be read from. + * @return The current instance of the builder. + * @see RepositoryItemReader#setRepository(PagingAndSortingRepository) + */ + public RepositoryItemReaderBuilder repository(PagingAndSortingRepository repository) { + this.repository = repository; + + return this; + } + + /** + * Specifies what method on the repository to call. This method must take + * {@link org.springframework.data.domain.Pageable} as the last argument. + * + * @param methodName name of the method to invoke. + * @return The current instance of the builder. + * @see RepositoryItemReader#setMethodName(String) + */ + public RepositoryItemReaderBuilder methodName(String methodName) { + this.methodName = methodName; + + return this; + } + + /** + * Specifies a repository and the type-safe method to call for the reader. The method + * configured via this mechanism must take + * {@link org.springframework.data.domain.Pageable} as the last + * argument. This method can be used in place of {@link #repository(PagingAndSortingRepository)}, + * {@link #methodName(String)}, and {@link #arguments(List)}. + * + * Note: The repository that is used by the repositoryMethodReference must be + * non-final. + * + * @param repositoryMethodReference of the used to get a repository and type-safe + * method for use by the reader. + * @return The current instance of the builder. + * @see RepositoryItemReader#setMethodName(String) + * @see RepositoryItemReader#setRepository(PagingAndSortingRepository) + * + */ + public RepositoryItemReaderBuilder repository(RepositoryMethodReference repositoryMethodReference) { + this.repositoryMethodReference = repositoryMethodReference; + + return this; + } + + /** + * Builds the {@link RepositoryItemReader}. + * + * @return a {@link RepositoryItemReader} + */ + public RepositoryItemReader build() { + if (this.repositoryMethodReference != null) { + this.methodName = this.repositoryMethodReference.getMethodName(); + this.repository = this.repositoryMethodReference.getRepository(); + + if(CollectionUtils.isEmpty(this.arguments)) { + this.arguments = this.repositoryMethodReference.getArguments(); + } + } + + Assert.notNull(this.sorts, "sorts map is required."); + Assert.notNull(this.repository, "repository is required."); + Assert.hasText(this.methodName, "methodName is required."); + if (this.saveState) { + Assert.state(StringUtils.hasText(this.name), "A name is required when saveState is set to true."); + } + + RepositoryItemReader reader = new RepositoryItemReader<>(); + reader.setArguments(this.arguments); + reader.setRepository(this.repository); + reader.setMethodName(this.methodName); + reader.setPageSize(this.pageSize); + reader.setCurrentItemCount(this.currentItemCount); + reader.setMaxItemCount(this.maxItemCount); + reader.setSaveState(this.saveState); + reader.setSort(this.sorts); + reader.setName(this.name); + return reader; + } + + /** + * Establishes a proxy that will capture a the Repository and the associated + * methodName that will be used by the reader. + * @param The type of repository that will be used by the reader. The class must + * not be final. + */ + public static class RepositoryMethodReference { + private RepositoryMethodInterceptor repositoryInvocationHandler; + + private PagingAndSortingRepository repository; + + public RepositoryMethodReference(PagingAndSortingRepository repository) { + this.repository = repository; + this.repositoryInvocationHandler = new RepositoryMethodInterceptor(); + } + + /** + * The proxy returned prevents actual method execution and is only used to gather, + * information about the method. + * @return T is a proxy of the object passed in in the constructor + */ + @SuppressWarnings("unchecked") + public T methodIs() { + Enhancer enhancer = new Enhancer(); + enhancer.setSuperclass(this.repository.getClass()); + enhancer.setCallback(this.repositoryInvocationHandler); + return (T) enhancer.create(); + } + + PagingAndSortingRepository getRepository() { + return this.repository; + } + + String getMethodName() { + return this.repositoryInvocationHandler.getMethodName(); + } + + List getArguments() { + return this.repositoryInvocationHandler.getArguments(); + } + } + + private static class RepositoryMethodInterceptor implements MethodInterceptor { + private String methodName; + + private List arguments; + + @Override + public Object intercept(Object o, Method method, Object[] objects, MethodProxy methodProxy) throws Throwable { + this.methodName = method.getName(); + if (objects != null && objects.length > 1) { + arguments = new ArrayList<>(Arrays.asList(objects)); + // remove last entry because that will be provided by the + // RepositoryItemReader + arguments.remove(objects.length - 1); + } + return null; + } + + String getMethodName() { + return this.methodName; + } + + List getArguments() { + return arguments; + } + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/RepositoryItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/RepositoryItemWriterBuilder.java new file mode 100644 index 0000000000..da01090941 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/RepositoryItemWriterBuilder.java @@ -0,0 +1,169 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + + +import java.lang.reflect.Method; + +import org.springframework.batch.item.data.RepositoryItemWriter; +import org.springframework.cglib.proxy.Enhancer; +import org.springframework.cglib.proxy.MethodInterceptor; +import org.springframework.cglib.proxy.MethodProxy; +import org.springframework.data.repository.CrudRepository; +import org.springframework.util.Assert; + +/** + * A builder implementation for the {@link RepositoryItemWriter}. + * + * @author Glenn Renfro + * @author Mahmoud Ben Hassine + * @since 4.0 + * @see RepositoryItemWriter + */ +public class RepositoryItemWriterBuilder { + + private CrudRepository repository; + + private String methodName; + + private RepositoryMethodReference repositoryMethodReference; + + /** + * Specifies what method on the repository to call. This method must have the type of + * object passed to this writer as the sole argument. + * + * @param methodName the name of the method to be used for saving the item. + * @return The current instance of the builder. + * @see RepositoryItemWriter#setMethodName(String) + */ + public RepositoryItemWriterBuilder methodName(String methodName) { + this.methodName = methodName; + + return this; + } + + /** + * Set the {@link org.springframework.data.repository.CrudRepository} implementation + * for persistence + * + * @param repository the Spring Data repository to be set + * @return The current instance of the builder. + * @see RepositoryItemWriter#setRepository(CrudRepository) + */ + public RepositoryItemWriterBuilder repository(CrudRepository repository) { + this.repository = repository; + + return this; + } + + /** + * Specifies a repository and the type-safe method to call for the writer. The method + * configured via this mechanism must take + * {@link org.springframework.data.domain.Pageable} as the last + * argument. This method can be used in place of {@link #repository(CrudRepository)}, + * {@link #methodName(String)}}. + * + * Note: The repository that is used by the repositoryMethodReference must be + * non-final. + * + * @param repositoryMethodReference of the used to get a repository and type-safe + * method for use by the writer. + * @return The current instance of the builder. + * @see RepositoryItemWriter#setMethodName(String) + * @see RepositoryItemWriter#setRepository(CrudRepository) + * + */ + public RepositoryItemWriterBuilder repository(RepositoryItemWriterBuilder.RepositoryMethodReference repositoryMethodReference) { + this.repositoryMethodReference = repositoryMethodReference; + + return this; + } + + /** + * Builds the {@link RepositoryItemWriter}. + * + * @return a {@link RepositoryItemWriter} + */ + public RepositoryItemWriter build() { + if (this.repositoryMethodReference != null) { + this.methodName = this.repositoryMethodReference.getMethodName(); + this.repository = this.repositoryMethodReference.getRepository(); + } + + Assert.hasText(this.methodName, "methodName is required."); + Assert.notNull(this.repository, "repository is required."); + + RepositoryItemWriter writer = new RepositoryItemWriter<>(); + writer.setMethodName(this.methodName); + writer.setRepository(this.repository); + return writer; + } + + /** + * Establishes a proxy that will capture a the Repository and the associated + * methodName that will be used by the writer. + * @param The type of repository that will be used by the writer. The class must + * not be final. + */ + public static class RepositoryMethodReference { + private RepositoryMethodInterceptor repositoryInvocationHandler; + + private CrudRepository repository; + + public RepositoryMethodReference(CrudRepository repository) { + this.repository = repository; + this.repositoryInvocationHandler = new RepositoryMethodInterceptor(); + } + + /** + * The proxy returned prevents actual method execution and is only used to gather, + * information about the method. + * @return T is a proxy of the object passed in in the constructor + */ + @SuppressWarnings("unchecked") + public T methodIs() { + Enhancer enhancer = new Enhancer(); + enhancer.setSuperclass(this.repository.getClass()); + enhancer.setCallback(this.repositoryInvocationHandler); + return (T) enhancer.create(); + } + + CrudRepository getRepository() { + return this.repository; + } + + String getMethodName() { + return this.repositoryInvocationHandler.getMethodName(); + } + } + + private static class RepositoryMethodInterceptor implements MethodInterceptor { + private String methodName; + + @Override + public Object intercept(Object o, Method method, Object[] objects, + MethodProxy methodProxy) throws Throwable { + this.methodName = method.getName(); + return null; + } + + String getMethodName() { + return this.methodName; + } + + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/package-info.java new file mode 100644 index 0000000000..d1aa6bacce --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/builder/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Builders for Spring Data item readers and writers. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.data.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/package-info.java new file mode 100644 index 0000000000..b4029d2f33 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/data/package-info.java @@ -0,0 +1,10 @@ +/** + * Spring Data related readers and writers. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.data; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/AbstractCursorItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/AbstractCursorItemReader.java index a7f51fc6ae..d37040cc9a 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/AbstractCursorItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/AbstractCursorItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,6 +40,7 @@ import org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator; import org.springframework.jdbc.support.SQLExceptionTranslator; import org.springframework.jdbc.support.SQLStateSQLExceptionTranslator; +import org.springframework.lang.Nullable; import org.springframework.transaction.support.TransactionSynchronizationManager; import org.springframework.util.Assert; @@ -63,7 +64,7 @@ * then you must wrap the DataSource in a {@link ExtendedConnectionDataSourceProxy} to prevent the * connection from being closed and released after each commit performed as part of the step processing. * You must also use a JDBC driver supporting JDBC 3.0 or later since the cursor will be opened with the - * additional option of 'HOLD_CUSORS_OVER_COMMIT' enabled. + * additional option of 'HOLD_CURSORS_OVER_COMMIT' enabled. *

      * *

      @@ -102,6 +103,8 @@ * @author Peter Zozom * @author Robert Kasanicky * @author Thomas Risberg + * @author Michael Minella + * @author Mahmoud Ben Hassine */ public abstract class AbstractCursorItemReader extends AbstractItemCountingItemStreamItemReader implements InitializingBean { @@ -134,6 +137,9 @@ public abstract class AbstractCursorItemReader extends AbstractItemCountingIt private boolean useSharedExtendedConnection = false; + private Boolean connectionAutoCommit; + + private boolean initialConnectionAutoCommit; public AbstractCursorItemReader() { super(); @@ -142,7 +148,7 @@ public AbstractCursorItemReader() { /** * Assert that mandatory properties are set. * - * @throws IllegalArgumentException if either data source or sql properties + * @throws IllegalArgumentException if either data source or SQL properties * not set. */ @Override @@ -153,7 +159,7 @@ public void afterPropertiesSet() throws Exception { /** * Public setter for the data source for injection purposes. * - * @param dataSource + * @param dataSource {@link javax.sql.DataSource} to be used */ public void setDataSource(DataSource dataSource) { this.dataSource = dataSource; @@ -172,7 +178,10 @@ public DataSource getDataSource() { * Prepare the given JDBC Statement (or PreparedStatement or * CallableStatement), applying statement settings such as fetch size, max * rows, and query timeout. @param stmt the JDBC Statement to prepare - * @throws SQLException + * + * @param stmt {@link java.sql.PreparedStatement} to be configured + * + * @throws SQLException if interactions with provided stmt fail * * @see #setFetchSize * @see #setMaxRows @@ -192,10 +201,10 @@ protected void applyStatementSettings(PreparedStatement stmt) throws SQLExceptio } /** - * Return the exception translator for this instance. - * * Creates a default SQLErrorCodeSQLExceptionTranslator for the specified * DataSource if none is set. + * + * @return the exception translator for this instance. */ protected SQLExceptionTranslator getExceptionTranslator() { synchronized(this) { @@ -216,7 +225,7 @@ protected SQLExceptionTranslator getExceptionTranslator() { * warnings (at debug level). * * @param statement the current statement to obtain the warnings from, if there are any. - * @throws SQLException + * @throws SQLException if interaction with provided statement fails. * * @see org.springframework.jdbc.SQLWarningException */ @@ -243,7 +252,7 @@ protected void handleWarnings(Statement statement) throws SQLWarningException, /** * Moves the cursor in the ResultSet to the position specified by the row * parameter by traversing the ResultSet. - * @param row + * @param row The index of the row to move to */ private void moveCursorToRow(int row) { try { @@ -338,7 +347,7 @@ public void setDriverSupportsAbsolute(boolean driverSupportsAbsolute) { * connection from being closed and released after each commit. * * When you set this option to true then the statement used to open the cursor - * will be created with both 'READ_ONLY' and 'HOLD_CUSORS_OVER_COMMIT' options. This allows + * will be created with both 'READ_ONLY' and 'HOLD_CURSORS_OVER_COMMIT' options. This allows * holding the cursor open over transaction start and commits performed in the step processing. * To use this feature you need a database that supports this and a JDBC driver supporting * JDBC 3.0 or later. @@ -353,10 +362,21 @@ public boolean isUseSharedExtendedConnection() { return useSharedExtendedConnection; } + /** + * Set whether "autoCommit" should be overridden for the connection used by the cursor. If not set, defaults to + * Connection / Datasource default configuration. + * + * @param autoCommit value used for {@link Connection#setAutoCommit(boolean)}. + * @since 4.0 + */ + public void setConnectionAutoCommit(boolean autoCommit) { + this.connectionAutoCommit = autoCommit; + } + public abstract String getSql(); /** - * Check the result set is in synch with the currentRow attribute. This is + * Check the result set is in sync with the currentRow attribute. This is * important to ensure that the user hasn't modified the current row. */ private void verifyCursorPosition(long expectedCurrentRow) throws SQLException { @@ -377,6 +397,11 @@ protected void doClose() throws Exception { JdbcUtils.closeResultSet(this.rs); rs = null; cleanupOnClose(); + + if(this.con != null) { + this.con.setAutoCommit(this.initialConnectionAutoCommit); + } + if (useSharedExtendedConnection && dataSource instanceof ExtendedConnectionDataSourceProxy) { ((ExtendedConnectionDataSourceProxy)dataSource).stopCloseSuppression(this.con); if (!TransactionSynchronizationManager.isActualTransactionActive()) { @@ -421,6 +446,12 @@ protected void initializeConnection() { else { this.con = dataSource.getConnection(); } + + this.initialConnectionAutoCommit = this.con.getAutoCommit(); + + if (this.connectionAutoCommit != null && this.con.getAutoCommit() != this.connectionAutoCommit) { + this.con.setAutoCommit(this.connectionAutoCommit); + } } catch (SQLException se) { close(); @@ -434,6 +465,7 @@ protected void initializeConnection() { * Read next row and map it to item, verify cursor position if * {@link #setVerifyCursorPosition(boolean)} is true. */ + @Nullable @Override protected T doRead() throws Exception { if (rs == null) { @@ -456,13 +488,14 @@ protected T doRead() throws Exception { /** * Read the cursor and map to the type of object this reader should return. This method must be - * overriden by subclasses. + * overridden by subclasses. * * @param rs The current result set * @param currentRow Current position of the result set * @return the mapped object at the cursor position - * @throws SQLException + * @throws SQLException if interactions with the current result set fail */ + @Nullable protected abstract T readCursor(ResultSet rs, int currentRow) throws SQLException; /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/AbstractPagingItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/AbstractPagingItemReader.java index 0206f17e2d..57a975e391 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/AbstractPagingItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/AbstractPagingItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,7 @@ import org.apache.commons.logging.LogFactory; import org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader; import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.ClassUtils; @@ -94,6 +95,7 @@ public void afterPropertiesSet() throws Exception { Assert.isTrue(pageSize > 0, "pageSize must be greater than zero"); } + @Nullable @Override protected T doRead() throws Exception { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/BeanPropertyItemSqlParameterSourceProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/BeanPropertyItemSqlParameterSourceProvider.java index 68c23f3e58..aa07d50b22 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/BeanPropertyItemSqlParameterSourceProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/BeanPropertyItemSqlParameterSourceProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/ExtendedConnectionDataSourceProxy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/ExtendedConnectionDataSourceProxy.java index 90f9a3c26f..8e97b8ef66 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/ExtendedConnectionDataSourceProxy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/ExtendedConnectionDataSourceProxy.java @@ -1,11 +1,11 @@ /* - * Copyright 2002-2013 the original author or authors. + * Copyright 2002-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -73,14 +73,14 @@ *

      * The connection returned will be a close-suppressing proxy instead of the * physical {@link Connection}. Be aware that you will not be able to cast this - * to a native OracleConnection or the like anymore; you need to - * use a {@link org.springframework.jdbc.support.nativejdbc.NativeJdbcExtractor}. + * to a native OracleConnection or the like anymore; you'd be required to use + * {@link java.sql.Connection#unwrap(Class)}. * * @author Thomas Risberg * @see #getConnection() * @see java.sql.Connection#close() * @see DataSourceUtils#releaseConnection - * @see org.springframework.jdbc.support.nativejdbc.NativeJdbcExtractor + * @see java.sql.Connection#unwrap(Class) * @since 2.0 */ public class ExtendedConnectionDataSourceProxy implements SmartDataSource, InitializingBean { @@ -104,15 +104,17 @@ public ExtendedConnectionDataSourceProxy() { } /** - * Constructor that takes as a parameter with the {&link DataSource} to be + * Constructor that takes as a parameter with the {@link DataSource} to be * wrapped. + * + * @param dataSource DataSource to be used */ public ExtendedConnectionDataSourceProxy(DataSource dataSource) { this.dataSource = dataSource; } /** - * Setter for the {&link DataSource} that is to be wrapped. + * Setter for the {@link DataSource} that is to be wrapped. * * @param dataSource the DataSource */ @@ -141,10 +143,7 @@ public boolean shouldClose(Connection connection) { * @return true or false */ public boolean isCloseSuppressionActive(Connection connection) { - if (connection == null) { - return false; - } - return connection.equals(closeSuppressedConnection) ? true : false; + return connection != null && connection.equals(closeSuppressedConnection); } /** @@ -208,8 +207,8 @@ private Connection initConnection(String username, String password) throws SQLEx else { target = dataSource.getConnection(); } - Connection connection = getCloseSuppressingConnectionProxy(target); - return connection; + + return getCloseSuppressingConnectionProxy(target); } @Override @@ -238,6 +237,7 @@ public void setLoginTimeout(int seconds) throws SQLException { * @param target the original Connection to wrap * @return the wrapped Connection */ + @SuppressWarnings("rawtypes") protected Connection getCloseSuppressingConnectionProxy(Connection target) { return (Connection) Proxy.newProxyInstance(ConnectionProxy.class.getClassLoader(), new Class[] { ConnectionProxy.class }, new CloseSuppressingInvocationHandler(target, this)); @@ -263,29 +263,27 @@ public CloseSuppressingInvocationHandler(Connection target, ExtendedConnectionDa public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { // Invocation on ConnectionProxy interface coming in... - if (method.getName().equals("equals")) { - // Only consider equal when proxies are identical. - return (proxy == args[0] ? Boolean.TRUE : Boolean.FALSE); - } - else if (method.getName().equals("hashCode")) { - // Use hashCode of Connection proxy. - return new Integer(System.identityHashCode(proxy)); - } - else if (method.getName().equals("close")) { - // Handle close method: don't pass the call on if we are - // suppressing close calls. - if (dataSource.completeCloseCall((Connection) proxy)) { - return null; - } - else { - target.close(); - return null; - } - } - else if (method.getName().equals("getTargetConnection")) { - // Handle getTargetConnection method: return underlying - // Connection. - return this.target; + switch (method.getName()) { + case "equals": + // Only consider equal when proxies are identical. + return (proxy == args[0] ? Boolean.TRUE : Boolean.FALSE); + case "hashCode": + // Use hashCode of Connection proxy. + return System.identityHashCode(proxy); + case "close": + // Handle close method: don't pass the call on if we are + // suppressing close calls. + if (dataSource.completeCloseCall((Connection) proxy)) { + return null; + } + else { + target.close(); + return null; + } + case "getTargetConnection": + // Handle getTargetConnection method: return underlying + // Connection. + return this.target; } // Invoke method on target Connection. @@ -304,10 +302,7 @@ else if (method.getName().equals("getTargetConnection")) { */ @Override public boolean isWrapperFor(Class iface) throws SQLException { - if (iface.isAssignableFrom(SmartDataSource.class) || iface.isAssignableFrom(dataSource.getClass())) { - return true; - } - return false; + return iface.isAssignableFrom(SmartDataSource.class) || iface.isAssignableFrom(dataSource.getClass()); } /** @@ -332,7 +327,7 @@ else if (iface.isAssignableFrom(dataSource.getClass())) { @Override public void afterPropertiesSet() throws Exception { - Assert.notNull(dataSource); + Assert.notNull(dataSource, "DataSource is required"); } /** @@ -346,14 +341,8 @@ public Logger getParentLogger() throws SQLFeatureNotSupportedException{ try { invoker.prepare(); return (Logger) invoker.invoke(); - } catch (ClassNotFoundException cnfe) { - throw new SQLFeatureNotSupportedException(cnfe); - } catch (NoSuchMethodException nsme) { + } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | InvocationTargetException nsme) { throw new SQLFeatureNotSupportedException(nsme); - } catch (IllegalAccessException iae) { - throw new SQLFeatureNotSupportedException(iae); - } catch (InvocationTargetException ite) { - throw new SQLFeatureNotSupportedException(ite); } } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernateCursorItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernateCursorItemReader.java index 3b7b8692b2..5769e267fa 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernateCursorItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernateCursorItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,6 +27,7 @@ import org.springframework.batch.item.database.orm.HibernateQueryProvider; import org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader; import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.ClassUtils; @@ -57,7 +58,7 @@ public class HibernateCursorItemReader extends AbstractItemCountingItemStreamItemReader implements InitializingBean { - private HibernateItemReaderHelper helper = new HibernateItemReaderHelper(); + private HibernateItemReaderHelper helper = new HibernateItemReaderHelper<>(); public HibernateCursorItemReader() { setName(ClassUtils.getShortName(HibernateCursorItemReader.class)); @@ -115,7 +116,7 @@ public void setFetchSize(int fetchSize) { * * @param queryProvider Hibernate query provider */ - public void setQueryProvider(HibernateQueryProvider queryProvider) { + public void setQueryProvider(HibernateQueryProvider queryProvider) { helper.setQueryProvider(queryProvider); } @@ -150,6 +151,7 @@ public void setUseStatelessSession(boolean useStatelessSession) { helper.setUseStatelessSession(useStatelessSession); } + @Nullable @Override protected T doRead() throws Exception { if (cursor.next()) { @@ -220,13 +222,14 @@ protected void jumpToItem(int itemIndex) throws Exception { @Override protected void doClose() throws Exception { - initialized = false; + if(initialized) { + if (cursor != null) { + cursor.close(); + } - if (cursor != null) { - cursor.close(); + helper.close(); } - helper.close(); - + initialized = false; } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernateItemReaderHelper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernateItemReaderHelper.java index a769f1ea67..70fe34ba8e 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernateItemReaderHelper.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernateItemReaderHelper.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,6 +15,7 @@ */ package org.springframework.batch.item.database; +import java.lang.reflect.Method; import java.util.Collection; import java.util.List; import java.util.Map; @@ -25,9 +26,11 @@ import org.hibernate.Session; import org.hibernate.SessionFactory; import org.hibernate.StatelessSession; + import org.springframework.batch.item.database.orm.HibernateQueryProvider; import org.springframework.beans.factory.InitializingBean; import org.springframework.util.Assert; +import org.springframework.util.ReflectionUtils; import org.springframework.util.StringUtils; /** @@ -35,8 +38,10 @@ * queries. * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ +@SuppressWarnings("rawtypes") public class HibernateItemReaderHelper implements InitializingBean { private SessionFactory sessionFactory; @@ -45,7 +50,7 @@ public class HibernateItemReaderHelper implements InitializingBean { private String queryName = ""; - private HibernateQueryProvider queryProvider; + private HibernateQueryProvider queryProvider; private boolean useStatelessSession = true; @@ -70,7 +75,7 @@ public void setQueryString(String queryString) { /** * @param queryProvider Hibernate query provider */ - public void setQueryProvider(HibernateQueryProvider queryProvider) { + public void setQueryProvider(HibernateQueryProvider queryProvider) { this.queryProvider = queryProvider; } @@ -104,11 +109,6 @@ public void afterPropertiesSet() throws Exception { Assert.state(StringUtils.hasText(queryString) ^ StringUtils.hasText(queryName), "queryString or queryName must be set"); } - // making sure that the appropriate (Hibernate) query provider is set - else { - Assert.state(queryProvider != null, "Hibernate query provider must be set"); - } - } /** @@ -129,6 +129,8 @@ public ScrollableResults getForwardOnlyCursor(int fetchSize, Map /** * Open appropriate type of hibernate session and create the query. + * + * @return a Hibernate Query */ public Query createQuery() { @@ -174,6 +176,8 @@ public Query createQuery() { * Scroll through the results up to the item specified. * * @param cursor the results to scroll over + * @param itemIndex index to scroll to + * @param flushInterval the number of items to scroll past before flushing */ public void jumpToItem(ScrollableResults cursor, int itemIndex, int flushInterval) { for (int i = 0; i < itemIndex; i++) { @@ -193,7 +197,9 @@ public void close() { statelessSession = null; } if (statefulSession != null) { - statefulSession.close(); + + Method close = ReflectionUtils.findMethod(Session.class, "close"); + ReflectionUtils.invokeMethod(close, statefulSession); statefulSession = null; } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernateItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernateItemWriter.java index 01b80c2e93..5857f5e973 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernateItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernateItemWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,9 +22,10 @@ import org.hibernate.Session; import org.hibernate.SessionFactory; import org.hibernate.context.spi.CurrentSessionContext; + import org.springframework.batch.item.ItemWriter; import org.springframework.beans.factory.InitializingBean; -import org.springframework.orm.hibernate3.HibernateOperations; +import org.springframework.orm.hibernate5.HibernateOperations; import org.springframework.util.Assert; /** @@ -32,10 +33,10 @@ * that are not part of the current Hibernate session. It will also flush the * session after writing (i.e. at chunk boundaries if used in a Spring Batch * TaskletStep). It will also clear the session on write - * default (see {@link #setClearSession(boolean) clearSession} property).
      - *
      + * default (see {@link #setClearSession(boolean) clearSession} property).
      + *
      * - * The writer is thread safe once properties are set (normal singleton behavior) + * The writer is thread-safe once properties are set (normal singleton behavior) * if a {@link CurrentSessionContext} that uses only one session per thread is * used. * @@ -49,7 +50,6 @@ public class HibernateItemWriter implements ItemWriter, InitializingBean { protected static final Log logger = LogFactory .getLog(HibernateItemWriter.class); - private HibernateOperations hibernateTemplate; private SessionFactory sessionFactory; private boolean clearSession = true; @@ -65,17 +65,6 @@ public void setClearSession(boolean clearSession) { this.clearSession = clearSession; } - /** - * Public setter for the {@link HibernateOperations} property. - * - * @param hibernateTemplate - * the hibernateTemplate to set - * @deprecated As of 2.2 in favor of using Hibernate's session management APIs directly - */ - public void setHibernateTemplate(HibernateOperations hibernateTemplate) { - this.hibernateTemplate = hibernateTemplate; - } - /** * Set the Hibernate SessionFactory to be used internally. * @@ -86,12 +75,12 @@ public void setSessionFactory(SessionFactory sessionFactory) { } /** - * Check mandatory properties - there must be a hibernateTemplate. + * Check mandatory properties - there must be a sessionFactory. */ @Override public void afterPropertiesSet() { - Assert.state(!(hibernateTemplate == null && sessionFactory == null), - "Either HibernateOperations or SessionFactory must be provided"); + Assert.state(sessionFactory != null, + "SessionFactory must be provided"); } /** @@ -102,19 +91,10 @@ public void afterPropertiesSet() { */ @Override public void write(List items) { - if(sessionFactory == null) { - doWrite(hibernateTemplate, items); - hibernateTemplate.flush(); - if (clearSession) { - hibernateTemplate.clear(); - } - } - else { - doWrite(sessionFactory, items); - sessionFactory.getCurrentSession().flush(); - if(clearSession) { - sessionFactory.getCurrentSession().clear(); - } + doWrite(sessionFactory, items); + sessionFactory.getCurrentSession().flush(); + if(clearSession) { + sessionFactory.getCurrentSession().clear(); } } @@ -122,9 +102,8 @@ public void write(List items) { * Do perform the actual write operation using Hibernate's API. * This can be overridden in a subclass if necessary. * - * @param items - * the list of items to use for the write - * @deprecated As of 2.2 in favor of using Hibernate's session management APIs directly + * @param sessionFactory Hibernate SessionFactory to be used + * @param items the list of items to use for the write */ protected void doWrite(SessionFactory sessionFactory, List items) { if (logger.isDebugEnabled()) { @@ -160,6 +139,7 @@ protected void doWrite(SessionFactory sessionFactory, List items) { * the list of items to use for the write * @deprecated As of 2.2 in favor of using Hibernate's session management APIs directly */ + @Deprecated protected void doWrite(HibernateOperations hibernateTemplate, List items) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernatePagingItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernatePagingItemReader.java index 68a4f6057e..92d71d5831 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernatePagingItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/HibernatePagingItemReader.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,7 +23,6 @@ import org.hibernate.StatelessSession; import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemReader; -import org.springframework.batch.item.ItemStream; import org.springframework.batch.item.database.orm.HibernateQueryProvider; import org.springframework.beans.factory.InitializingBean; import org.springframework.util.Assert; @@ -61,7 +60,7 @@ public class HibernatePagingItemReader extends AbstractPagingItemReader implements InitializingBean { - private HibernateItemReaderHelper helper = new HibernateItemReaderHelper(); + private HibernateItemReaderHelper helper = new HibernateItemReaderHelper<>(); private Map parameterValues; @@ -109,7 +108,7 @@ public void setFetchSize(int fetchSize) { * * @param queryProvider Hibernate query provider */ - public void setQueryProvider(HibernateQueryProvider queryProvider) { + public void setQueryProvider(HibernateQueryProvider queryProvider) { helper.setQueryProvider(queryProvider); } @@ -160,7 +159,7 @@ protected void doOpen() throws Exception { protected void doReadPage() { if (results == null) { - results = new CopyOnWriteArrayList(); + results = new CopyOnWriteArrayList<>(); } else { results.clear(); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/IbatisBatchItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/IbatisBatchItemWriter.java deleted file mode 100644 index 6b57aa0989..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/IbatisBatchItemWriter.java +++ /dev/null @@ -1,162 +0,0 @@ -/* - * Copyright 2006-2013 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.database; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.springframework.batch.item.ItemWriter; -import org.springframework.beans.factory.InitializingBean; -import org.springframework.dao.EmptyResultDataAccessException; -import org.springframework.dao.InvalidDataAccessResourceUsageException; -import org.springframework.orm.ibatis.SqlMapClientCallback; -import org.springframework.orm.ibatis.SqlMapClientTemplate; -import org.springframework.util.Assert; - -import com.ibatis.sqlmap.client.SqlMapClient; -import com.ibatis.sqlmap.client.SqlMapExecutor; -import com.ibatis.sqlmap.engine.execution.BatchException; -import com.ibatis.sqlmap.engine.execution.BatchResult; - -/** - * {@link ItemWriter} that uses the batching features from - * SqlMapClientTemplate to execute a batch of statements for all items - * provided.
      - * - * The user must provide an iBATIS statement id that points to the SQL statement defined - * in the iBATIS SqlMap configuration.
      - * - * It is expected that {@link #write(List)} is called inside a transaction.
      - * - * The writer is thread safe after its properties are set (normal singleton - * behavior), so it can be used to write in multiple concurrent transactions. - * - * @author Thomas Risberg - * @since 2.0 - */ -public class IbatisBatchItemWriter implements ItemWriter, InitializingBean { - - protected static final Log logger = LogFactory.getLog(IbatisBatchItemWriter.class); - - private SqlMapClientTemplate sqlMapClientTemplate; - - private String statementId; - - private boolean assertUpdates = true; - - /** - * Public setter for the flag that determines whether an assertion is made - * that all items cause at least one row to be updated. - * - * @param assertUpdates the flag to set. Defaults to true; - */ - public void setAssertUpdates(boolean assertUpdates) { - this.assertUpdates = assertUpdates; - } - - /** - * Public setter for {@link SqlMapClient} for injection purposes. - * - * @param sqlMapClient the SqlMapClient - */ - public void setSqlMapClient(SqlMapClient sqlMapClient) { - if (sqlMapClientTemplate == null) { - this.sqlMapClientTemplate = new SqlMapClientTemplate(sqlMapClient); - } - } - - /** - * Public setter for the SqlMapClientTemplate. - * - * @param sqlMapClientTemplate the SqlMapClientTemplate - */ - public void setSqlMapClientTemplate(SqlMapClientTemplate sqlMapClientTemplate) { - this.sqlMapClientTemplate = sqlMapClientTemplate; - } - - /** - * Public setter for the statement id identifying the statement in the SqlMap - * configuration file. - * - * @param statementId the id for the statement - */ - public void setStatementId(String statementId) { - this.statementId = statementId; - } - - /** - * Check mandatory properties - there must be an SqlMapClient and a statementId. - */ - @Override - public void afterPropertiesSet() { - Assert.notNull(sqlMapClientTemplate, "A SqlMapClient or a SqlMapClientTemplate is required."); - Assert.notNull(statementId, "A statementId is required."); - } - - /* (non-Javadoc) - * @see org.springframework.batch.item.ItemWriter#write(java.util.List) - */ - @Override - public void write(final List items) { - - if (!items.isEmpty()) { - - if (logger.isDebugEnabled()) { - logger.debug("Executing batch with " + items.size() + " items."); - } - - @SuppressWarnings("unchecked") - List results = (List) sqlMapClientTemplate.execute( - new SqlMapClientCallback() { - @Override - public Object doInSqlMapClient(SqlMapExecutor executor) - throws SQLException { - executor.startBatch(); - for (T item : items) { - executor.update(statementId, item); - } - try { - return executor.executeBatchDetailed(); - } catch (BatchException e) { - throw e.getBatchUpdateException(); - } - } - }); - - if (assertUpdates) { - if (results.size() != 1) { - throw new InvalidDataAccessResourceUsageException("Batch execution returned invalid results. " + - "Expected 1 but number of BatchResult objects returned was " + results.size()); - } - - int[] updateCounts = results.get(0).getUpdateCounts(); - - for (int i = 0; i < updateCounts.length; i++) { - int value = updateCounts[i]; - if (value == 0) { - throw new EmptyResultDataAccessException("Item " + i + " of " + updateCounts.length - + " did not update any rows: [" + items.get(i) + "]", 1); - } - } - } - - } - - } - -} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/IbatisPagingItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/IbatisPagingItemReader.java deleted file mode 100644 index afbf25dfe8..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/IbatisPagingItemReader.java +++ /dev/null @@ -1,141 +0,0 @@ -/* - * Copyright 2006-2013 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.item.database; - -import java.util.HashMap; -import java.util.Map; -import java.util.concurrent.CopyOnWriteArrayList; - -import org.springframework.batch.item.ExecutionContext; -import org.springframework.orm.ibatis.SqlMapClientTemplate; -import org.springframework.util.Assert; -import org.springframework.util.ClassUtils; - -import com.ibatis.sqlmap.client.SqlMapClient; - -/** - *

      - * {@link org.springframework.batch.item.ItemReader} for reading database - * records using iBATIS in a paging fashion. - *

      - * - *

      - * It executes the query specified as the {@link #setQueryId(String)} to - * retrieve requested data. The query is executed using paged requests of a size - * specified in {@link #setPageSize(int)}. Additional pages are requested when - * needed as {@link #read()} method is called, returning an object corresponding - * to current position. Some standard query parameters are provided by the - * reader and the SQL in the named query must use some or all of these parameters - * (depending on the SQL variant) to construct a result set of the required - * size. The parameters are: - *

        - *
      • _page: the page number to be read (starting at 0)
      • - *
      • _pagesize: the size of the pages, i.e. the number of rows to - * return
      • - *
      • _skiprows: the product of _page and - * _pagesize
      • - *
      - * Failure to write the correct platform-specific SQL often results in an - * infinite loop in the reader because it keeps asking for the next page and - * gets the same result set over and over. - *

      - * - *

      - * The performance of the paging depends on the iBATIS implementation. - * Setting a fairly large page size and using a commit interval that matches the - * page size should provide better performance. - *

      - * - *

      - * The implementation is thread-safe in between calls to - * {@link #open(ExecutionContext)}, but remember to use - * saveState=false if used in a multi-threaded client (no restart - * available). - *

      - * - * @author Thomas Risberg - * @author Dave Syer - * @since 2.0 - */ -public class IbatisPagingItemReader extends AbstractPagingItemReader { - - private SqlMapClient sqlMapClient; - - private String queryId; - - private SqlMapClientTemplate sqlMapClientTemplate; - - private Map parameterValues; - - public IbatisPagingItemReader() { - setName(ClassUtils.getShortName(IbatisPagingItemReader.class)); - } - - public void setSqlMapClient(SqlMapClient sqlMapClient) { - this.sqlMapClient = sqlMapClient; - } - - public void setQueryId(String queryId) { - this.queryId = queryId; - } - - /** - * The parameter values to be used for the query execution. - * - * @param parameterValues the values keyed by the parameter named used in - * the query string. - */ - public void setParameterValues(Map parameterValues) { - this.parameterValues = parameterValues; - } - - /** - * Check mandatory properties. - * @see org.springframework.beans.factory.InitializingBean#afterPropertiesSet() - */ - @Override - public void afterPropertiesSet() throws Exception { - super.afterPropertiesSet(); - Assert.notNull(sqlMapClient); - sqlMapClientTemplate = new SqlMapClientTemplate(sqlMapClient); - Assert.notNull(queryId); - } - - @Override - @SuppressWarnings("unchecked") - protected void doReadPage() { - Map parameters = new HashMap(); - if (parameterValues != null) { - parameters.putAll(parameterValues); - } - parameters.put("_page", getPage()); - parameters.put("_pagesize", getPageSize()); - parameters.put("_skiprows", getPage() * getPageSize()); - if (results == null) { - results = new CopyOnWriteArrayList(); - } - else { - results.clear(); - } - results.addAll(sqlMapClientTemplate.queryForList(queryId, parameters)); - } - - @Override - protected void doJumpToPage(int itemIndex) { - } - -} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/ItemPreparedStatementSetter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/ItemPreparedStatementSetter.java index e3ea33f8cf..61264d2bfc 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/ItemPreparedStatementSetter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/ItemPreparedStatementSetter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,6 +31,7 @@ public interface ItemPreparedStatementSetter { /** * Set parameter values on the given PreparedStatement as determined from * the provided item. + * @param item the item to obtain the values from * @param ps the PreparedStatement to invoke setter methods on * @throws SQLException if a SQLException is encountered (i.e. there is no * need to catch SQLException) diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/ItemSqlParameterSourceProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/ItemSqlParameterSourceProvider.java index b0d30045ec..45634769a1 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/ItemSqlParameterSourceProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/ItemSqlParameterSourceProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ public interface ItemSqlParameterSourceProvider { * Provide parameter values in an {@link SqlParameterSource} based on values from * the provided item. * @param item the item to use for parameter values + * @return parameters extracted from the item */ SqlParameterSource createSqlParameterSource(T item); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcBatchItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcBatchItemWriter.java index 13d899e634..0d70214c19 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcBatchItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcBatchItemWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,11 +20,11 @@ import java.util.ArrayList; import java.util.List; import java.util.Map; - import javax.sql.DataSource; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.batch.item.ItemWriter; import org.springframework.beans.factory.InitializingBean; import org.springframework.dao.DataAccessException; @@ -41,17 +41,17 @@ * {@link NamedParameterJdbcTemplate} to execute a batch of statements for all items * provided.

      * - * The user must provide an SQL query and a special callback in the for of either - * {@link ItemPreparedStatementSetter}, or a {@link ItemSqlParameterSourceProvider}. + * The user must provide an SQL query and a special callback for either of + * {@link ItemPreparedStatementSetter} or {@link ItemSqlParameterSourceProvider}. * You can use either named parameters or the traditional '?' placeholders. If you use the * named parameter support then you should provide a {@link ItemSqlParameterSourceProvider}, * otherwise you should provide a {@link ItemPreparedStatementSetter}. * This callback would be responsible for mapping the item to the parameters needed to - * execute the SQL statement.
      + * execute the SQL statement.
      * - * It is expected that {@link #write(List)} is called inside a transaction.
      + * It is expected that {@link #write(List)} is called inside a transaction.
      * - * The writer is thread safe after its properties are set (normal singleton + * The writer is thread-safe after its properties are set (normal singleton * behavior), so it can be used to write in multiple concurrent transactions. * * @author Dave Syer @@ -63,19 +63,19 @@ public class JdbcBatchItemWriter implements ItemWriter, InitializingBean { protected static final Log logger = LogFactory.getLog(JdbcBatchItemWriter.class); - private NamedParameterJdbcOperations namedParameterJdbcTemplate; + protected NamedParameterJdbcOperations namedParameterJdbcTemplate; - private ItemPreparedStatementSetter itemPreparedStatementSetter; + protected ItemPreparedStatementSetter itemPreparedStatementSetter; - private ItemSqlParameterSourceProvider itemSqlParameterSourceProvider; + protected ItemSqlParameterSourceProvider itemSqlParameterSourceProvider; - private String sql; + protected String sql; - private boolean assertUpdates = true; + protected boolean assertUpdates = true; - private int parameterCount; + protected int parameterCount; - private boolean usingNamedParameters; + protected boolean usingNamedParameters; /** * Public setter for the flag that determines whether an assertion is made @@ -118,7 +118,7 @@ public void setItemSqlParameterSourceProvider(ItemSqlParameterSourceProvider /** * Public setter for the data source for injection purposes. * - * @param dataSource + * @param dataSource {@link javax.sql.DataSource} to use for querying against */ public void setDataSource(DataSource dataSource) { if (namedParameterJdbcTemplate == null) { @@ -142,7 +142,7 @@ public void setJdbcTemplate(NamedParameterJdbcOperations namedParameterJdbcTempl public void afterPropertiesSet() { Assert.notNull(namedParameterJdbcTemplate, "A DataSource or a NamedParameterJdbcTemplate is required."); Assert.notNull(sql, "An SQL statement is required."); - List namedParameters = new ArrayList(); + List namedParameters = new ArrayList<>(); parameterCount = JdbcParameterUtils.countParameterPlaceholders(sql, namedParameters); if (namedParameters.size() > 0) { if (parameterCount != namedParameters.size()) { @@ -158,8 +158,8 @@ public void afterPropertiesSet() { /* (non-Javadoc) * @see org.springframework.batch.item.ItemWriter#write(java.util.List) */ - @Override @SuppressWarnings({"unchecked", "rawtypes"}) + @Override public void write(final List items) throws Exception { if (!items.isEmpty()) { @@ -168,11 +168,11 @@ public void write(final List items) throws Exception { logger.debug("Executing batch with " + items.size() + " items."); } - int[] updateCounts = null; + int[] updateCounts; if (usingNamedParameters) { - if(items.get(0) instanceof Map) { - updateCounts = namedParameterJdbcTemplate.batchUpdate(sql, items.toArray(new Map[0])); + if(items.get(0) instanceof Map && this.itemSqlParameterSourceProvider == null) { + updateCounts = namedParameterJdbcTemplate.batchUpdate(sql, items.toArray(new Map[items.size()])); } else { SqlParameterSource[] batchArgs = new SqlParameterSource[items.size()]; int i = 0; @@ -183,9 +183,9 @@ public void write(final List items) throws Exception { } } else { - updateCounts = (int[]) namedParameterJdbcTemplate.getJdbcOperations().execute(sql, new PreparedStatementCallback() { + updateCounts = namedParameterJdbcTemplate.getJdbcOperations().execute(sql, new PreparedStatementCallback() { @Override - public Object doInPreparedStatement(PreparedStatement ps) throws SQLException, DataAccessException { + public int[] doInPreparedStatement(PreparedStatement ps) throws SQLException, DataAccessException { for (T item : items) { itemPreparedStatementSetter.setValues(item, ps); ps.addBatch(); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcCursorItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcCursorItemReader.java index 1404bdd78c..94a7d4c1ed 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcCursorItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcCursorItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,6 +24,7 @@ import org.springframework.jdbc.core.PreparedStatementSetter; import org.springframework.jdbc.core.RowMapper; import org.springframework.jdbc.support.JdbcUtils; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.ClassUtils; @@ -50,16 +51,15 @@ * @author Robert Kasanicky * @author Thomas Risberg */ -@SuppressWarnings("rawtypes") public class JdbcCursorItemReader extends AbstractCursorItemReader { - PreparedStatement preparedStatement; + private PreparedStatement preparedStatement; - PreparedStatementSetter preparedStatementSetter; + private PreparedStatementSetter preparedStatementSetter; - String sql; + private String sql; - RowMapper rowMapper; + private RowMapper rowMapper; public JdbcCursorItemReader() { super(); @@ -69,9 +69,9 @@ public JdbcCursorItemReader() { /** * Set the RowMapper to be used for all calls to read(). * - * @param rowMapper + * @param rowMapper the mapper used to map each item */ - public void setRowMapper(RowMapper rowMapper) { + public void setRowMapper(RowMapper rowMapper) { this.rowMapper = rowMapper; } @@ -80,7 +80,7 @@ public void setRowMapper(RowMapper rowMapper) { * should be a complete and valid SQL statement, as it will be run directly * without any modification. * - * @param sql + * @param sql SQL statement */ public void setSql(String sql) { this.sql = sql; @@ -90,7 +90,7 @@ public void setSql(String sql) { * Set the PreparedStatementSetter to use if any parameter values that need * to be set in the supplied query. * - * @param preparedStatementSetter + * @param preparedStatementSetter PreparedStatementSetter responsible for filling out the statement */ public void setPreparedStatementSetter(PreparedStatementSetter preparedStatementSetter) { this.preparedStatementSetter = preparedStatementSetter; @@ -99,7 +99,7 @@ public void setPreparedStatementSetter(PreparedStatementSetter preparedStatement /** * Assert that mandatory properties are set. * - * @throws IllegalArgumentException if either data source or sql properties + * @throws IllegalArgumentException if either data source or SQL properties * not set. */ @Override @@ -135,10 +135,10 @@ protected void openCursor(Connection con) { } + @Nullable @Override - @SuppressWarnings("unchecked") protected T readCursor(ResultSet rs, int currentRow) throws SQLException { - return (T) rowMapper.mapRow(rs, currentRow); + return rowMapper.mapRow(rs, currentRow); } /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcPagingItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcPagingItemReader.java index 3ba6c8089b..889cdcd7b4 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcPagingItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcPagingItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -51,7 +51,8 @@ * needed as {@link #read()} method is called, returning an object corresponding * to current position. On restart it uses the last sort key value to locate the * first page to read (so it doesn't matter if the successfully processed items - * have been removed or modified). + * have been removed or modified). It is important to have a unique key constraint + * on the sort key to guarantee that no data is lost between executions. *

      * *

      @@ -71,6 +72,7 @@ * @author Thomas Risberg * @author Dave Syer * @author Michael Minella + * @author Mahmoud Ben Hassine * @since 2.0 */ public class JdbcPagingItemReader extends AbstractPagingItemReader implements InitializingBean { @@ -86,8 +88,7 @@ public class JdbcPagingItemReader extends AbstractPagingItemReader impleme private NamedParameterJdbcTemplate namedParameterJdbcTemplate; - @SuppressWarnings("rawtypes") - private RowMapper rowMapper; + private RowMapper rowMapper; private String firstPageSql; @@ -136,11 +137,10 @@ public void setQueryProvider(PagingQueryProvider queryProvider) { * by the reader. * * @param rowMapper a - * {@link org.springframework.jdbc.core.simple.ParameterizedRowMapper} + * {@link RowMapper} * implementation */ - @SuppressWarnings("rawtypes") - public void setRowMapper(RowMapper rowMapper) { + public void setRowMapper(RowMapper rowMapper) { this.rowMapper = rowMapper; } @@ -165,14 +165,14 @@ public void setParameterValues(Map parameterValues) { @Override public void afterPropertiesSet() throws Exception { super.afterPropertiesSet(); - Assert.notNull(dataSource); + Assert.notNull(dataSource, "DataSource may not be null"); JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); if (fetchSize != VALUE_NOT_SET) { jdbcTemplate.setFetchSize(fetchSize); } jdbcTemplate.setMaxRows(getPageSize()); namedParameterJdbcTemplate = new NamedParameterJdbcTemplate(jdbcTemplate); - Assert.notNull(queryProvider); + Assert.notNull(queryProvider, "QueryProvider may not be null"); queryProvider.init(dataSource); this.firstPageSql = queryProvider.generateFirstPageQuery(getPageSize()); this.remainingPagesSql = queryProvider.generateRemainingPagesQuery(getPageSize()); @@ -182,7 +182,7 @@ public void afterPropertiesSet() throws Exception { @SuppressWarnings("unchecked") protected void doReadPage() { if (results == null) { - results = new CopyOnWriteArrayList(); + results = new CopyOnWriteArrayList<>(); } else { results.clear(); @@ -255,7 +255,7 @@ public void open(ExecutionContext executionContext) { startAfterValues = (Map) executionContext.get(getExecutionContextKey(START_AFTER_VALUE)); if(startAfterValues == null) { - startAfterValues = new LinkedHashMap(); + startAfterValues = new LinkedHashMap<>(); } } @@ -263,40 +263,31 @@ public void open(ExecutionContext executionContext) { } @Override - @SuppressWarnings({"unchecked", "rawtypes"}) protected void doJumpToPage(int itemIndex) { /* * Normally this would be false (the startAfterValue is enough * information to restart from. */ + // TODO: this is dead code, startAfterValues is never null - see #open(ExecutionContext) if (startAfterValues == null && getPage() > 0) { - String jumpToItemSql; - jumpToItemSql = queryProvider.generateJumpToItemQuery(itemIndex, getPageSize()); + String jumpToItemSql = queryProvider.generateJumpToItemQuery(itemIndex, getPageSize()); if (logger.isDebugEnabled()) { logger.debug("SQL used for jumping: [" + jumpToItemSql + "]"); } - - RowMapper startMapper = new RowMapper() { - @Override - public Object mapRow(ResultSet rs, int i) throws SQLException { - return rs.getObject(1); - } - }; + if (this.queryProvider.isUsingNamedParameters()) { - startAfterValues = (Map) namedParameterJdbcTemplate.queryForObject(jumpToItemSql, - getParameterMap(parameterValues, startAfterValues), startMapper); + startAfterValues = namedParameterJdbcTemplate.queryForMap(jumpToItemSql, getParameterMap(parameterValues, null)); } else { - startAfterValues = (Map) getJdbcTemplate().queryForObject(jumpToItemSql, - getParameterList(parameterValues, startAfterValues).toArray(), startMapper); + startAfterValues = getJdbcTemplate().queryForMap(jumpToItemSql, getParameterList(parameterValues, null).toArray()); } } } private Map getParameterMap(Map values, Map sortKeyValues) { - Map parameterMap = new LinkedHashMap(); + Map parameterMap = new LinkedHashMap<>(); if (values != null) { parameterMap.putAll(values); } @@ -312,14 +303,14 @@ private Map getParameterMap(Map values, Map getParameterList(Map values, Map sortKeyValue) { - SortedMap sm = new TreeMap(); + SortedMap sm = new TreeMap<>(); if (values != null) { sm.putAll(values); } - List parameterList = new ArrayList(); + List parameterList = new ArrayList<>(); parameterList.addAll(sm.values()); if (sortKeyValue != null && sortKeyValue.size() > 0) { - List> keys = new ArrayList>(sortKeyValue.entrySet()); + List> keys = new ArrayList<>(sortKeyValue.entrySet()); for(int i = 0; i < keys.size(); i++) { for(int j = 0; j < i; j++) { @@ -336,11 +327,10 @@ private List getParameterList(Map values, Map { @Override - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { - startAfterValues = new LinkedHashMap(); + public T mapRow(ResultSet rs, int rowNum) throws SQLException { + startAfterValues = new LinkedHashMap<>(); for (Map.Entry sortKey : queryProvider.getSortKeys().entrySet()) { startAfterValues.put(sortKey.getKey(), rs.getObject(sortKey.getKey())); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcParameterUtils.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcParameterUtils.java index 51553b8595..c0ceaf7d58 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcParameterUtils.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JdbcParameterUtils.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,7 +36,7 @@ public class JdbcParameterUtils { * sql. The character placeholder is not counted if it appears * within a literal, that is, surrounded by single or double quotes. This method will * count traditional placeholders in the form of a question mark ('?') as well as - * named parameters indicated with a leading ':' or '&'. + * named parameters indicated with a leading ':' or '&'. * * The code for this method is taken from an early version of the * {@link org.springframework.jdbc.core.namedparam.NamedParameterUtils} @@ -45,6 +45,8 @@ public class JdbcParameterUtils { * suite the batch processing requirements. * * @param sql String to search in. Returns 0 if the given String is null. + * @param namedParameterHolder holder for the named parameters + * @return the number of named parameter placeholders */ public static int countParameterPlaceholders(String sql, List namedParameterHolder ) { if (sql == null) { @@ -53,7 +55,7 @@ public static int countParameterPlaceholders(String sql, List namedParam char[] statement = sql.toCharArray(); boolean withinQuotes = false; - Map namedParameters = new HashMap(); + Map namedParameters = new HashMap<>(); char currentQuote = '-'; int parameterCount = 0; int i = 0; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JpaItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JpaItemWriter.java index cafe2dd7b5..aabadceb43 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JpaItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JpaItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,11 +16,6 @@ package org.springframework.batch.item.database; -import java.util.List; - -import javax.persistence.EntityManager; -import javax.persistence.EntityManagerFactory; - import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.item.ItemWriter; @@ -29,18 +24,22 @@ import org.springframework.orm.jpa.EntityManagerFactoryUtils; import org.springframework.util.Assert; +import javax.persistence.EntityManager; +import javax.persistence.EntityManagerFactory; +import java.util.List; + /** * {@link org.springframework.batch.item.ItemWriter} that is using a JPA * EntityManagerFactory to merge any Entities that aren't part of the * persistence context. * - * It is required that {@link #write(List)} is called inside a transaction.
      + * It is required that {@link #write(List)} is called inside a transaction.
      * * The reader must be configured with an * {@link javax.persistence.EntityManagerFactory} that is capable of * participating in Spring managed transactions. * - * The writer is thread safe after its properties are set (normal singleton + * The writer is thread-safe after its properties are set (normal singleton * behaviour), so it can be used to write in multiple concurrent transactions. * * @author Thomas Risberg @@ -51,6 +50,7 @@ public class JpaItemWriter implements ItemWriter, InitializingBean { protected static final Log logger = LogFactory.getLog(JpaItemWriter.class); private EntityManagerFactory entityManagerFactory; + private boolean usePersist = false; /** * Set the EntityManager to be used internally. @@ -60,6 +60,15 @@ public class JpaItemWriter implements ItemWriter, InitializingBean { public void setEntityManagerFactory(EntityManagerFactory entityManagerFactory) { this.entityManagerFactory = entityManagerFactory; } + + /** + * Set whether the EntityManager should perform a persist instead of a merge. + * + * @param usePersist whether to use persist instead of merge. + */ + public void setUsePersist(boolean usePersist) { + this.usePersist = usePersist; + } /** * Check mandatory properties - there must be an entityManagerFactory. @@ -99,16 +108,21 @@ protected void doWrite(EntityManager entityManager, List items) { } if (!items.isEmpty()) { - long mergeCount = 0; + long addedToContextCount = 0; for (T item : items) { if (!entityManager.contains(item)) { - entityManager.merge(item); - mergeCount++; + if(usePersist) { + entityManager.persist(item); + } + else { + entityManager.merge(item); + } + addedToContextCount++; } } if (logger.isDebugEnabled()) { - logger.debug(mergeCount + " entities merged."); - logger.debug((items.size() - mergeCount) + " entities found in persistence context."); + logger.debug(addedToContextCount + " entities " + (usePersist ? " persisted." : "merged.")); + logger.debug((items.size() - addedToContextCount) + " entities found in persistence context."); } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JpaPagingItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JpaPagingItemReader.java index 9500384fa2..e346261acc 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JpaPagingItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/JpaPagingItemReader.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -89,7 +89,7 @@ public class JpaPagingItemReader extends AbstractPagingItemReader { private EntityManager entityManager; - private final Map jpaPropertyMap = new HashMap(); + private final Map jpaPropertyMap = new HashMap<>(); private String queryString; @@ -136,7 +136,7 @@ public void setParameterValues(Map parameterValues) { * particular transaction. (e.g. Hibernate with a JTA transaction). NOTE: may cause * problems in guaranteeing the object consistency in the EntityManagerFactory. * - * @param transacted + * @param transacted indicator */ public void setTransacted(boolean transacted) { this.transacted = transacted; @@ -147,12 +147,8 @@ public void afterPropertiesSet() throws Exception { super.afterPropertiesSet(); if (queryProvider == null) { - Assert.notNull(entityManagerFactory); - Assert.hasLength(queryString); - } - // making sure that the appropriate (JPA) query provider is set - else { - Assert.isTrue(queryProvider != null, "JPA query provider must be set"); + Assert.notNull(entityManagerFactory, "EntityManager is required when queryProvider is null"); + Assert.hasLength(queryString, "Query string is required when queryProvider is null"); } } @@ -209,7 +205,7 @@ protected void doReadPage() { } if (results == null) { - results = new CopyOnWriteArrayList(); + results = new CopyOnWriteArrayList<>(); } else { results.clear(); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/Order.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/Order.java index ad7d60df8e..a4edf5bc91 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/Order.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/Order.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/PagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/PagingQueryProvider.java index 00bffc2d70..dc26c90754 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/PagingQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/PagingQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,7 +17,6 @@ package org.springframework.batch.item.database; import java.util.Map; - import javax.sql.DataSource; @@ -35,6 +34,7 @@ public interface PagingQueryProvider { * Initialize the query provider using the provided {@link DataSource} if necessary. * * @param dataSource DataSource to use for any initialization + * @throws Exception for errors when initializing */ void init(DataSource dataSource) throws Exception; @@ -95,4 +95,11 @@ public interface PagingQueryProvider { * @return The string to be used for a parameterized query. */ String getSortKeyPlaceHolder(String keyName); + + /** + * The sort key (unique single column name) without alias. + * + * @return the sort key used to order the query (without alias) + */ + Map getSortKeysWithoutAliases(); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/StoredProcedureItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/StoredProcedureItemReader.java index e63387c649..27210a0553 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/StoredProcedureItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/StoredProcedureItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ import org.springframework.jdbc.core.SqlParameter; import org.springframework.jdbc.core.metadata.CallMetaDataContext; import org.springframework.jdbc.support.JdbcUtils; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.ClassUtils; @@ -55,7 +56,6 @@ * * @author Thomas Risberg */ -@SuppressWarnings("rawtypes") public class StoredProcedureItemReader extends AbstractCursorItemReader { private CallableStatement callableStatement; @@ -66,7 +66,7 @@ public class StoredProcedureItemReader extends AbstractCursorItemReader { private String callString; - private RowMapper rowMapper; + private RowMapper rowMapper; private SqlParameter[] parameters = new SqlParameter[0]; @@ -82,9 +82,9 @@ public StoredProcedureItemReader() { /** * Set the RowMapper to be used for all calls to read(). * - * @param rowMapper + * @param rowMapper the RowMapper to use to map the results */ - public void setRowMapper(RowMapper rowMapper) { + public void setRowMapper(RowMapper rowMapper) { this.rowMapper = rowMapper; } @@ -93,7 +93,7 @@ public void setRowMapper(RowMapper rowMapper) { * should be a complete and valid SQL statement, as it will be run directly * without any modification. * - * @param sprocedureName + * @param sprocedureName the SQL used to call the statement */ public void setProcedureName(String sprocedureName) { this.procedureName = sprocedureName; @@ -103,7 +103,7 @@ public void setProcedureName(String sprocedureName) { * Set the PreparedStatementSetter to use if any parameter values that need * to be set in the supplied query. * - * @param preparedStatementSetter + * @param preparedStatementSetter used to populate the SQL */ public void setPreparedStatementSetter(PreparedStatementSetter preparedStatementSetter) { this.preparedStatementSetter = preparedStatementSetter; @@ -121,6 +121,8 @@ public void setParameters(SqlParameter[] parameters) { /** * Set whether this stored procedure is a function. + * + * @param function indicator */ public void setFunction(boolean function) { this.function = function; @@ -140,7 +142,7 @@ public void setRefCursorPosition(int refCursorPosition) { /** * Assert that mandatory properties are set. * - * @throws IllegalArgumentException if either data source or sql properties + * @throws IllegalArgumentException if either data source or SQL properties * not set. */ @Override @@ -170,7 +172,10 @@ protected void openCursor(Connection con) { SqlParameter cursorParameter = callContext.createReturnResultSetParameter("cursor", rowMapper); this.callString = callContext.createCallString(); - log.debug("Call string is: " + callString); + + if (log.isDebugEnabled()) { + log.debug("Call string is: " + callString); + } int cursorSqlType = Types.OTHER; if (function) { @@ -226,10 +231,10 @@ protected void openCursor(Connection con) { } + @Nullable @Override - @SuppressWarnings("unchecked") protected T readCursor(ResultSet rs, int currentRow) throws SQLException { - return (T) rowMapper.mapRow(rs, currentRow); + return rowMapper.mapRow(rs, currentRow); } /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/HibernateCursorItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/HibernateCursorItemReaderBuilder.java new file mode 100644 index 0000000000..408f9404e0 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/HibernateCursorItemReaderBuilder.java @@ -0,0 +1,299 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.util.Map; + +import org.hibernate.SessionFactory; + +import org.springframework.batch.item.database.HibernateCursorItemReader; +import org.springframework.batch.item.database.orm.HibernateNativeQueryProvider; +import org.springframework.batch.item.database.orm.HibernateQueryProvider; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * This is a builder for the {@link HibernateCursorItemReader}. When configuring, one of + * the following should be provided (listed in order of precedence): + *
        + *
      • {@link #queryProvider(HibernateQueryProvider)}
      • + *
      • {@link #queryName(String)}
      • + *
      • {@link #queryString(String)}
      • + *
      • {@link #nativeQuery(String)} and {@link #entityClass(Class)}
      • + *
      + * + * @author Michael Minella + * @author Glenn Renfro + * @author Mahmoud Ben Hassine + * @since 4.0 + * @see HibernateCursorItemReader + */ +public class HibernateCursorItemReaderBuilder { + + private Map parameterValues; + + private String queryName; + + private int fetchSize; + + private HibernateQueryProvider queryProvider; + + private String queryString; + + private SessionFactory sessionFactory; + + private boolean useStatelessSession; + + private String nativeQuery; + + private Class nativeClass; + + private boolean saveState = true; + + private String name; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public HibernateCursorItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public HibernateCursorItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public HibernateCursorItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public HibernateCursorItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * A map of parameter values to be set on the query. The key of the map is the name + * of the parameter to be set with the value being the value to be set. + * + * @param parameterValues map of values + * @return this instance for method chaining + * @see HibernateCursorItemReader#setParameterValues(Map) + */ + public HibernateCursorItemReaderBuilder parameterValues(Map parameterValues) { + this.parameterValues = parameterValues; + + return this; + } + + /** + * The name of the Hibernate named query to be executed for this reader. + * + * @param queryName name of the query to execute + * @return this instance for method chaining + * @see HibernateCursorItemReader#setQueryName(String) + */ + public HibernateCursorItemReaderBuilder queryName(String queryName) { + this.queryName = queryName; + + return this; + } + + /** + * The number of items to be returned with each round trip to the database. Used + * internally by Hibernate. + * + * @param fetchSize number of records to return per fetch + * @return this instance for method chaining + * @see HibernateCursorItemReader#setFetchSize(int) + */ + public HibernateCursorItemReaderBuilder fetchSize(int fetchSize) { + this.fetchSize = fetchSize; + + return this; + } + + /** + * A query provider. This should be set only if {@link #queryString(String)} and + * {@link #queryName(String)} have not been set. + * + * @param queryProvider the query provider + * @return this instance for method chaining + * @see HibernateCursorItemReader#setQueryProvider(HibernateQueryProvider) + */ + public HibernateCursorItemReaderBuilder queryProvider(HibernateQueryProvider queryProvider) { + this.queryProvider = queryProvider; + + return this; + } + + /** + * The HQL query string to execute. This should only be set if + * {@link #queryProvider(HibernateQueryProvider)} and {@link #queryName(String)} have + * not been set. + * + * @param queryString the HQL query + * @return this instance for method chaining + * @see HibernateCursorItemReader#setQueryString(String) + */ + public HibernateCursorItemReaderBuilder queryString(String queryString) { + this.queryString = queryString; + + return this; + } + + /** + * The Hibernate {@link SessionFactory} to execute the query against. + * + * @param sessionFactory the session factory + * @return this instance for method chaining + * @see HibernateCursorItemReader#setSessionFactory(SessionFactory) + */ + public HibernateCursorItemReaderBuilder sessionFactory(SessionFactory sessionFactory) { + this.sessionFactory = sessionFactory; + + return this; + } + + /** + * Indicator for whether to use a {@link org.hibernate.StatelessSession} + * (true) or a {@link org.hibernate.Session} (false). + * + * @param useStatelessSession Defaults to false + * @return this instance for method chaining + * @see HibernateCursorItemReader#setUseStatelessSession(boolean) + */ + public HibernateCursorItemReaderBuilder useStatelessSession(boolean useStatelessSession) { + this.useStatelessSession = useStatelessSession; + + return this; + } + + /** + * Used to configure a {@link HibernateNativeQueryProvider}. This is ignored if + * + * @param nativeQuery {@link String} containing the native query. + * @return this instance for method chaining + */ + public HibernateCursorItemReaderBuilder nativeQuery(String nativeQuery) { + this.nativeQuery = nativeQuery; + + return this; + } + + public HibernateCursorItemReaderBuilder entityClass(Class nativeClass) { + this.nativeClass = nativeClass; + + return this; + } + + /** + * Returns a fully constructed {@link HibernateCursorItemReader}. + * + * @return a new {@link HibernateCursorItemReader} + */ + public HibernateCursorItemReader build() { + Assert.state(this.fetchSize >= 0, "fetchSize must not be negative"); + Assert.state(this.sessionFactory != null, "A SessionFactory must be provided"); + + if(this.saveState) { + Assert.state(StringUtils.hasText(this.name), + "A name is required when saveState is set to true."); + } + + HibernateCursorItemReader reader = new HibernateCursorItemReader<>(); + + reader.setFetchSize(this.fetchSize); + reader.setParameterValues(this.parameterValues); + + if(this.queryProvider != null) { + reader.setQueryProvider(this.queryProvider); + } + else if(StringUtils.hasText(this.queryName)) { + reader.setQueryName(this.queryName); + } + else if(StringUtils.hasText(this.queryString)) { + reader.setQueryString(this.queryString); + } + else if(StringUtils.hasText(this.nativeQuery) && this.nativeClass != null) { + HibernateNativeQueryProvider provider = new HibernateNativeQueryProvider<>(); + provider.setSqlQuery(this.nativeQuery); + provider.setEntityClass(this.nativeClass); + + try { + provider.afterPropertiesSet(); + } + catch (Exception e) { + throw new IllegalStateException("Unable to initialize the HibernateNativeQueryProvider", e); + } + + reader.setQueryProvider(provider); + } + else { + throw new IllegalStateException("A HibernateQueryProvider, queryName, queryString, " + + "or both the nativeQuery and entityClass must be configured"); + } + + reader.setSessionFactory(this.sessionFactory); + reader.setUseStatelessSession(this.useStatelessSession); + reader.setCurrentItemCount(this.currentItemCount); + reader.setMaxItemCount(this.maxItemCount); + reader.setName(this.name); + reader.setSaveState(this.saveState); + + return reader; + } + + } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/HibernateItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/HibernateItemWriterBuilder.java new file mode 100644 index 0000000000..e08343a7a8 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/HibernateItemWriterBuilder.java @@ -0,0 +1,78 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import org.hibernate.SessionFactory; + +import org.springframework.batch.item.database.HibernateItemWriter; +import org.springframework.util.Assert; + +/** + * A builder for the {@link HibernateItemWriter} + * + * @author Michael Minella + * @since 4.0 + * @see HibernateItemWriter + */ +public class HibernateItemWriterBuilder { + + private boolean clearSession = true; + + private SessionFactory sessionFactory; + + /** + * If set to false, the {@link org.hibernate.Session} will not be cleared at the end + * of the chunk. + * + * @param clearSession defaults to true + * @return this instance for method chaining + * @see HibernateItemWriter#setClearSession(boolean) + */ + public HibernateItemWriterBuilder clearSession(boolean clearSession) { + this.clearSession = clearSession; + + return this; + } + + /** + * The Hibernate {@link SessionFactory} to obtain a session from. Required. + * + * @param sessionFactory the {@link SessionFactory} + * @return this instance for method chaining + * @see HibernateItemWriter#setSessionFactory(SessionFactory) + */ + public HibernateItemWriterBuilder sessionFactory(SessionFactory sessionFactory) { + this.sessionFactory = sessionFactory; + + return this; + } + + /** + * Returns a fully built {@link HibernateItemWriter} + * + * @return the writer + */ + public HibernateItemWriter build() { + Assert.state(this.sessionFactory != null, + "SessionFactory must be provided"); + + HibernateItemWriter writer = new HibernateItemWriter<>(); + writer.setSessionFactory(this.sessionFactory); + writer.setClearSession(this.clearSession); + + return writer; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/HibernatePagingItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/HibernatePagingItemReaderBuilder.java new file mode 100644 index 0000000000..b440c70070 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/HibernatePagingItemReaderBuilder.java @@ -0,0 +1,286 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.util.Map; + +import org.hibernate.SessionFactory; + +import org.springframework.batch.item.database.HibernatePagingItemReader; +import org.springframework.batch.item.database.orm.HibernateQueryProvider; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * A builder for the {@link HibernatePagingItemReader}. When configuring, only one of the + * following should be provided: + *
        + *
      • {@link #queryString(String)}
      • + *
      • {@link #queryName(String)}
      • + *
      • {@link #queryProvider(HibernateQueryProvider)}
      • + *
      + * + * @author Michael Minella + * @author Glenn Renfro + * @author Mahmoud Ben Hassine + * @since 4.0 + * @see HibernatePagingItemReader + */ +public class HibernatePagingItemReaderBuilder { + + private int pageSize = 10; + + private Map parameterValues; + + private String queryName; + + private int fetchSize; + + private HibernateQueryProvider queryProvider; + + private String queryString; + + private SessionFactory sessionFactory; + + private boolean statelessSession = true; + + private boolean saveState = true; + + private String name; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public HibernatePagingItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public HibernatePagingItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public HibernatePagingItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public HibernatePagingItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * The number of records to request per page/query. Defaults to 10. Must be greater + * than zero. + * + * @param pageSize number of items + * @return this instance for method chaining + * @see HibernatePagingItemReader#setPageSize(int) + */ + public HibernatePagingItemReaderBuilder pageSize(int pageSize) { + this.pageSize = pageSize; + + return this; + } + + /** + * A map of parameter values to be set on the query. The key of the map is the name + * of the parameter to be set with the value being the value to be set. + * + * @param parameterValues map of values + * @return this instance for method chaining + * @see HibernatePagingItemReader#setParameterValues(Map) + */ + public HibernatePagingItemReaderBuilder parameterValues(Map parameterValues) { + this.parameterValues = parameterValues; + + return this; + } + + /** + * The name of the Hibernate named query to be executed for this reader. + * + * @param queryName name of the query to execute + * @return this instance for method chaining + * @see HibernatePagingItemReader#setQueryName(String) + */ + public HibernatePagingItemReaderBuilder queryName(String queryName) { + this.queryName = queryName; + + return this; + } + + /** + * Fetch size used internally by Hibernate to limit amount of data fetched + * from database per round trip. + * + * @param fetchSize number of records + * @return this instance for method chaining + * @see HibernatePagingItemReader#setFetchSize(int) + */ + public HibernatePagingItemReaderBuilder fetchSize(int fetchSize) { + this.fetchSize = fetchSize; + + return this; + } + + /** + * A query provider. This should be set only if {@link #queryString(String)} and + * {@link #queryName(String)} have not been set. + * + * @param queryProvider the query provider + * @return this instance for method chaining + * @see HibernatePagingItemReader#setQueryProvider(HibernateQueryProvider) + */ + public HibernatePagingItemReaderBuilder queryProvider(HibernateQueryProvider queryProvider) { + this.queryProvider = queryProvider; + + return this; + } + + /** + * The HQL query string to execute. This should only be set if + * {@link #queryProvider(HibernateQueryProvider)} and {@link #queryName(String)} have + * not been set. + * + * @param queryString the HQL query + * @return this instance for method chaining + * @see HibernatePagingItemReader#setQueryString(String) + */ + public HibernatePagingItemReaderBuilder queryString(String queryString) { + this.queryString = queryString; + + return this; + } + + /** + * The Hibernate {@link SessionFactory} to execute the query against. + * + * @param sessionFactory the session factory + * @return this instance for method chaining + * @see HibernatePagingItemReader#setSessionFactory(SessionFactory) + */ + public HibernatePagingItemReaderBuilder sessionFactory(SessionFactory sessionFactory) { + this.sessionFactory = sessionFactory; + + return this; + } + + /** + * Indicator for whether to use a {@link org.hibernate.StatelessSession} + * (true) or a {@link org.hibernate.Session} (false). + * + * @param useStatelessSession Defaults to false + * @return this instance for method chaining + * @see HibernatePagingItemReader#setUseStatelessSession(boolean) + * @deprecated This method is deprecated in favor of + * {@link HibernatePagingItemReaderBuilder#useStatelessSession} and will be + * removed in a future version. + */ + @Deprecated + public HibernatePagingItemReaderBuilder useSatelessSession(boolean useStatelessSession) { + return useStatelessSession(useStatelessSession); + } + + /** + * Indicator for whether to use a {@link org.hibernate.StatelessSession} + * (true) or a {@link org.hibernate.Session} (false). + * + * @param useStatelessSession Defaults to false + * @return this instance for method chaining + * @see HibernatePagingItemReader#setUseStatelessSession(boolean) + */ + public HibernatePagingItemReaderBuilder useStatelessSession(boolean useStatelessSession) { + this.statelessSession = useStatelessSession; + + return this; + } + + /** + * Returns a fully constructed {@link HibernatePagingItemReader}. + * + * @return a new {@link HibernatePagingItemReader} + */ + public HibernatePagingItemReader build() { + Assert.notNull(this.sessionFactory, "A SessionFactory must be provided"); + Assert.state(this.fetchSize >= 0, "fetchSize must not be negative"); + + if(this.saveState) { + Assert.hasText(this.name, + "A name is required when saveState is set to true"); + } + + if(this.queryProvider == null) { + Assert.state(StringUtils.hasText(queryString) ^ StringUtils.hasText(queryName), + "queryString or queryName must be set"); + } + + HibernatePagingItemReader reader = new HibernatePagingItemReader<>(); + + reader.setSessionFactory(this.sessionFactory); + reader.setSaveState(this.saveState); + reader.setMaxItemCount(this.maxItemCount); + reader.setCurrentItemCount(this.currentItemCount); + reader.setName(this.name); + reader.setFetchSize(this.fetchSize); + reader.setParameterValues(this.parameterValues); + reader.setQueryName(this.queryName); + reader.setQueryProvider(this.queryProvider); + reader.setQueryString(this.queryString); + reader.setPageSize(this.pageSize); + reader.setUseStatelessSession(this.statelessSession); + + return reader; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JdbcBatchItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JdbcBatchItemWriterBuilder.java new file mode 100644 index 0000000000..b93d207b2b --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JdbcBatchItemWriterBuilder.java @@ -0,0 +1,200 @@ +/* + * Copyright 2016 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.math.BigInteger; +import java.util.Map; +import javax.sql.DataSource; + +import org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider; +import org.springframework.batch.item.database.ItemPreparedStatementSetter; +import org.springframework.batch.item.database.ItemSqlParameterSourceProvider; +import org.springframework.batch.item.database.JdbcBatchItemWriter; +import org.springframework.batch.item.database.support.ColumnMapItemPreparedStatementSetter; +import org.springframework.jdbc.core.namedparam.NamedParameterJdbcOperations; +import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate; +import org.springframework.util.Assert; + +/** + * A builder implementation for the {@link JdbcBatchItemWriter}. + * + * @author Michael Minella + * @since 4.0 + * @see JdbcBatchItemWriter + */ +public class JdbcBatchItemWriterBuilder { + + private boolean assertUpdates = true; + + private String sql; + + private ItemPreparedStatementSetter itemPreparedStatementSetter; + + private ItemSqlParameterSourceProvider itemSqlParameterSourceProvider; + + private DataSource dataSource; + + private NamedParameterJdbcOperations namedParameterJdbcTemplate; + + private BigInteger mapped = new BigInteger("0"); + + /** + * Configure the {@link DataSource} to be used. + * + * @param dataSource the DataSource + * @return The current instance of the builder for chaining. + * @see JdbcBatchItemWriter#setDataSource(DataSource) + */ + public JdbcBatchItemWriterBuilder dataSource(DataSource dataSource) { + this.dataSource = dataSource; + + return this; + } + + /** + * If set to true, confirms that every insert results in the update of at least one + * row in the database. Defaults to true. + * + * @param assertUpdates boolean indicator + * @return The current instance of the builder for chaining + * @see JdbcBatchItemWriter#setAssertUpdates(boolean) + */ + public JdbcBatchItemWriterBuilder assertUpdates(boolean assertUpdates) { + this.assertUpdates = assertUpdates; + + return this; + } + + /** + * Set the SQL statement to be used for each item's updates. This is a required + * field. + * + * @param sql SQL string + * @return The current instance of the builder for chaining + * @see JdbcBatchItemWriter#setSql(String) + */ + public JdbcBatchItemWriterBuilder sql(String sql) { + this.sql = sql; + + return this; + } + + /** + * Configures a {@link ItemPreparedStatementSetter} for use by the writer. This + * should only be used if {@link #columnMapped()} isn't called. + * + * @param itemPreparedStatementSetter The {@link ItemPreparedStatementSetter} + * @return The current instance of the builder for chaining + * @see JdbcBatchItemWriter#setItemPreparedStatementSetter(ItemPreparedStatementSetter) + */ + public JdbcBatchItemWriterBuilder itemPreparedStatementSetter(ItemPreparedStatementSetter itemPreparedStatementSetter) { + this.itemPreparedStatementSetter = itemPreparedStatementSetter; + + return this; + } + + /** + * Configures a {@link ItemSqlParameterSourceProvider} for use by the writer. This + * should only be used if {@link #beanMapped()} isn't called. + * + * @param itemSqlParameterSourceProvider The {@link ItemSqlParameterSourceProvider} + * @return The current instance of the builder for chaining + * @see JdbcBatchItemWriter#setItemSqlParameterSourceProvider(ItemSqlParameterSourceProvider) + */ + public JdbcBatchItemWriterBuilder itemSqlParameterSourceProvider(ItemSqlParameterSourceProvider itemSqlParameterSourceProvider) { + this.itemSqlParameterSourceProvider = itemSqlParameterSourceProvider; + + return this; + } + + /** + * The {@link NamedParameterJdbcOperations} instance to use. If one isn't provided, + * a {@link DataSource} is required. + * + * @param namedParameterJdbcOperations The template + * @return The current instance of the builder for chaining + */ + public JdbcBatchItemWriterBuilder namedParametersJdbcTemplate(NamedParameterJdbcOperations namedParameterJdbcOperations) { + this.namedParameterJdbcTemplate = namedParameterJdbcOperations; + + return this; + } + + /** + * Creates a {@link ColumnMapItemPreparedStatementSetter} to be used as your + * {@link ItemPreparedStatementSetter}. + * + * NOTE: The item type for this {@link org.springframework.batch.item.ItemWriter} must + * be castable to Map<String,Object>>. + * + * @return The current instance of the builder for chaining + * @see ColumnMapItemPreparedStatementSetter + */ + public JdbcBatchItemWriterBuilder columnMapped() { + this.mapped = this.mapped.setBit(0); + + return this; + } + + /** + * Creates a {@link BeanPropertyItemSqlParameterSourceProvider} to be used as your + * {@link ItemSqlParameterSourceProvider}. + * + * @return The current instance of the builder for chaining + * @see BeanPropertyItemSqlParameterSourceProvider + */ + public JdbcBatchItemWriterBuilder beanMapped() { + this.mapped = this.mapped.setBit(1); + + return this; + } + + /** + * Validates configuration and builds the {@link JdbcBatchItemWriter}. + * + * @return a {@link JdbcBatchItemWriter} + */ + @SuppressWarnings("unchecked") + public JdbcBatchItemWriter build() { + Assert.state(this.dataSource != null || this.namedParameterJdbcTemplate != null, + "Either a DataSource or a NamedParameterJdbcTemplate is required"); + + Assert.notNull(this.sql, "A SQL statement is required"); + int mappedValue = this.mapped.intValue(); + Assert.state(mappedValue != 3, + "Either an item can be mapped via db column or via bean spec, can't be both"); + + JdbcBatchItemWriter writer = new JdbcBatchItemWriter<>(); + writer.setSql(this.sql); + writer.setAssertUpdates(this.assertUpdates); + writer.setItemSqlParameterSourceProvider(this.itemSqlParameterSourceProvider); + writer.setItemPreparedStatementSetter(this.itemPreparedStatementSetter); + + if(mappedValue == 1) { + ((JdbcBatchItemWriter>)writer).setItemPreparedStatementSetter(new ColumnMapItemPreparedStatementSetter()); + } else if(mappedValue == 2) { + writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>()); + } + + if(this.dataSource != null) { + this.namedParameterJdbcTemplate = new NamedParameterJdbcTemplate(this.dataSource); + } + + writer.setJdbcTemplate(this.namedParameterJdbcTemplate); + + return writer; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JdbcCursorItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JdbcCursorItemReaderBuilder.java new file mode 100644 index 0000000000..9d56869b1e --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JdbcCursorItemReaderBuilder.java @@ -0,0 +1,362 @@ +/* + * Copyright 2016-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.util.List; +import javax.sql.DataSource; + +import org.springframework.batch.item.database.AbstractCursorItemReader; +import org.springframework.batch.item.database.JdbcCursorItemReader; +import org.springframework.jdbc.core.ArgumentPreparedStatementSetter; +import org.springframework.jdbc.core.ArgumentTypePreparedStatementSetter; +import org.springframework.jdbc.core.BeanPropertyRowMapper; +import org.springframework.jdbc.core.PreparedStatementSetter; +import org.springframework.jdbc.core.RowMapper; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * Builder for the {@link JdbcCursorItemReader} + * + * @author Michael Minella + * @author Glenn Renfro + * @author Drummond Dawson + * @author Mahmoud Ben Hassine + * @since 4.0 + */ +public class JdbcCursorItemReaderBuilder { + + private DataSource dataSource; + + private int fetchSize = AbstractCursorItemReader.VALUE_NOT_SET; + + private int maxRows = AbstractCursorItemReader.VALUE_NOT_SET; + + private int queryTimeout = AbstractCursorItemReader.VALUE_NOT_SET; + + private boolean ignoreWarnings; + + private boolean verifyCursorPosition; + + private boolean driverSupportsAbsolute; + + private boolean useSharedExtendedConnection; + + private PreparedStatementSetter preparedStatementSetter; + + private String sql; + + private RowMapper rowMapper; + + private boolean saveState = true; + + private String name; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public JdbcCursorItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public JdbcCursorItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public JdbcCursorItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public JdbcCursorItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * The {@link DataSource} to read from + * + * @param dataSource a relational data base + * @return this instance for method chaining + * @see JdbcCursorItemReader#setDataSource(DataSource) + */ + public JdbcCursorItemReaderBuilder dataSource(DataSource dataSource) { + this.dataSource = dataSource; + + return this; + } + + /** + * A hint to the driver as to how many rows to return with each fetch. + * + * @param fetchSize the hint + * @return this instance for method chaining + * @see JdbcCursorItemReader#setFetchSize(int) + */ + public JdbcCursorItemReaderBuilder fetchSize(int fetchSize) { + this.fetchSize = fetchSize; + + return this; + } + + /** + * The max number of rows the {@link java.sql.ResultSet} can contain + * + * @param maxRows the max + * @return this instance for method chaining + * @see JdbcCursorItemReader#setMaxRows(int) + */ + public JdbcCursorItemReaderBuilder maxRows(int maxRows) { + this.maxRows = maxRows; + + return this; + } + + /** + * The time in milliseconds for the query to timeout + * + * @param queryTimeout timeout + * @return this instance for method chaining + * @see JdbcCursorItemReader#setQueryTimeout(int) + */ + public JdbcCursorItemReaderBuilder queryTimeout(int queryTimeout) { + this.queryTimeout = queryTimeout; + + return this; + } + + public JdbcCursorItemReaderBuilder ignoreWarnings(boolean ignoreWarnings) { + this.ignoreWarnings = ignoreWarnings; + + return this; + } + + /** + * Indicates if the reader should verify the current position of the + * {@link java.sql.ResultSet} after being passed to the {@link RowMapper}. Defaults + * to true. + * + * @param verifyCursorPosition indicator + * @return this instance for method chaining + * @see JdbcCursorItemReader#setVerifyCursorPosition(boolean) + */ + public JdbcCursorItemReaderBuilder verifyCursorPosition(boolean verifyCursorPosition) { + this.verifyCursorPosition = verifyCursorPosition; + + return this; + } + + /** + * Indicates if the JDBC driver supports setting the absolute row on the + * {@link java.sql.ResultSet}. + * + * @param driverSupportsAbsolute indicator + * @return this instance for method chaining + * @see JdbcCursorItemReader#setDriverSupportsAbsolute(boolean) + */ + public JdbcCursorItemReaderBuilder driverSupportsAbsolute(boolean driverSupportsAbsolute) { + this.driverSupportsAbsolute = driverSupportsAbsolute; + + return this; + } + + /** + * Indicates that the connection used for the cursor is being used by all other + * processing, therefor part of the same transaction. + * + * @param useSharedExtendedConnection indicator + * @return this instance for method chaining + * @see JdbcCursorItemReader#setUseSharedExtendedConnection(boolean) + */ + public JdbcCursorItemReaderBuilder useSharedExtendedConnection(boolean useSharedExtendedConnection) { + this.useSharedExtendedConnection = useSharedExtendedConnection; + + return this; + } + + /** + * Configures the provided {@link PreparedStatementSetter} to be used to populate any + * arguments in the SQL query to be executed for the reader. + * + * @param preparedStatementSetter setter + * @return this instance for method chaining + * @see JdbcCursorItemReader#setPreparedStatementSetter(PreparedStatementSetter) + */ + public JdbcCursorItemReaderBuilder preparedStatementSetter(PreparedStatementSetter preparedStatementSetter) { + this.preparedStatementSetter = preparedStatementSetter; + + return this; + } + + /** + * Configures a {@link PreparedStatementSetter} that will use the array as the values + * to be set on the query to be executed for this reader. + * + * @param args values to set on the reader query + * @return this instance for method chaining + */ + public JdbcCursorItemReaderBuilder queryArguments(Object... args) { + this.preparedStatementSetter = new ArgumentPreparedStatementSetter(args); + + return this; + } + + /** + * Configures a {@link PreparedStatementSetter} that will use the Object [] as the + * values to be set on the query to be executed for this reader. The int[] will + * provide the types ({@link java.sql.Types}) for each of the values provided. + * + * @param args values to set on the query + * @param types the type for each value in the args array + * @return this instance for method chaining + */ + public JdbcCursorItemReaderBuilder queryArguments(Object[] args, int[] types) { + this.preparedStatementSetter = new ArgumentTypePreparedStatementSetter(args, types); + + return this; + } + + /** + * Configures a {@link PreparedStatementSetter} that will use the List as the values + * to be set on the query to be executed for this reader. + * + * @param args values to set on the query + * @return this instance for method chaining + */ + public JdbcCursorItemReaderBuilder queryArguments(List args) { + Assert.notNull(args, "The list of arguments must not be null"); + this.preparedStatementSetter = new ArgumentPreparedStatementSetter(args.toArray()); + + return this; + } + + /** + * The query to be executed for this reader + * + * @param sql query + * @return this instance for method chaining + * @see JdbcCursorItemReader#setSql(String) + */ + public JdbcCursorItemReaderBuilder sql(String sql) { + this.sql = sql; + + return this; + } + + /** + * The {@link RowMapper} used to map the results of the cursor to each item. + * + * @param rowMapper {@link RowMapper} + * @return this instance for method chaining + * @see JdbcCursorItemReader#setRowMapper(RowMapper) + */ + public JdbcCursorItemReaderBuilder rowMapper(RowMapper rowMapper) { + this.rowMapper = rowMapper; + + return this; + } + + /** + * Creates a {@link BeanPropertyRowMapper} to be used as your + * {@link RowMapper}. + * + * @param mappedClass the class for the row mapper + * @return this instance for method chaining + * @see BeanPropertyRowMapper + */ + public JdbcCursorItemReaderBuilder beanRowMapper(Class mappedClass) { + this.rowMapper = new BeanPropertyRowMapper<>(mappedClass); + + return this; + } + + /** + * Validates configuration and builds a new reader instance. + * + * @return a fully constructed {@link JdbcCursorItemReader} + */ + public JdbcCursorItemReader build() { + if(this.saveState) { + Assert.hasText(this.name, + "A name is required when saveState is set to true"); + } + + Assert.hasText(this.sql, "A query is required"); + Assert.notNull(this.dataSource, "A datasource is required"); + Assert.notNull(this.rowMapper, "A rowmapper is required"); + + JdbcCursorItemReader reader = new JdbcCursorItemReader<>(); + + if(StringUtils.hasText(this.name)) { + reader.setName(this.name); + } + + reader.setSaveState(this.saveState); + reader.setPreparedStatementSetter(this.preparedStatementSetter); + reader.setRowMapper(this.rowMapper); + reader.setSql(this.sql); + reader.setCurrentItemCount(this.currentItemCount); + reader.setDataSource(this.dataSource); + reader.setDriverSupportsAbsolute(this.driverSupportsAbsolute); + reader.setFetchSize(this.fetchSize); + reader.setIgnoreWarnings(this.ignoreWarnings); + reader.setMaxItemCount(this.maxItemCount); + reader.setMaxRows(this.maxRows); + reader.setQueryTimeout(this.queryTimeout); + reader.setUseSharedExtendedConnection(this.useSharedExtendedConnection); + reader.setVerifyCursorPosition(this.verifyCursorPosition); + + return reader; + } + } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JdbcPagingItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JdbcPagingItemReaderBuilder.java new file mode 100644 index 0000000000..27a2caba0b --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JdbcPagingItemReaderBuilder.java @@ -0,0 +1,370 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.util.Map; +import javax.sql.DataSource; + +import org.springframework.batch.item.database.JdbcPagingItemReader; +import org.springframework.batch.item.database.Order; +import org.springframework.batch.item.database.PagingQueryProvider; +import org.springframework.batch.item.database.support.AbstractSqlPagingQueryProvider; +import org.springframework.batch.item.database.support.Db2PagingQueryProvider; +import org.springframework.batch.item.database.support.DerbyPagingQueryProvider; +import org.springframework.batch.item.database.support.H2PagingQueryProvider; +import org.springframework.batch.item.database.support.HsqlPagingQueryProvider; +import org.springframework.batch.item.database.support.MySqlPagingQueryProvider; +import org.springframework.batch.item.database.support.OraclePagingQueryProvider; +import org.springframework.batch.item.database.support.PostgresPagingQueryProvider; +import org.springframework.batch.item.database.support.SqlServerPagingQueryProvider; +import org.springframework.batch.item.database.support.SqlitePagingQueryProvider; +import org.springframework.batch.item.database.support.SybasePagingQueryProvider; +import org.springframework.batch.support.DatabaseType; +import org.springframework.jdbc.core.RowMapper; +import org.springframework.jdbc.support.MetaDataAccessException; +import org.springframework.util.Assert; + +/** + * This is a builder for the {@link JdbcPagingItemReader}. When configuring, either a + * {@link PagingQueryProvider} or the SQL fragments should be provided. If the SQL + * fragments are provided, the metadata from the provided {@link DataSource} will be used + * to create a PagingQueryProvider for you. If both are provided, the PagingQueryProvider + * will be used. + * + * @author Michael Minella + * @author Glenn Renfro + * @since 4.0 + * @see JdbcPagingItemReader + */ +public class JdbcPagingItemReaderBuilder { + + private DataSource dataSource; + + private int fetchSize = JdbcPagingItemReader.VALUE_NOT_SET; + + private PagingQueryProvider queryProvider; + + private RowMapper rowMapper; + + private Map parameterValues; + + private int pageSize = 10; + + private String groupClause; + + private String selectClause; + + private String fromClause; + + private String whereClause; + + private Map sortKeys; + + private boolean saveState = true; + + private String name; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public JdbcPagingItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public JdbcPagingItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public JdbcPagingItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public JdbcPagingItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * The {@link DataSource} to query against. Required. + * + * @param dataSource the {@link DataSource} + * @return this instance for method chaining + * @see JdbcPagingItemReader#setDataSource(DataSource) + */ + public JdbcPagingItemReaderBuilder dataSource(DataSource dataSource) { + this.dataSource = dataSource; + + return this; + } + + /** + * A hint to the underlying RDBMS as to how many records to return with each fetch. + * + * @param fetchSize number of records + * @return this instance for method chaining + * @see JdbcPagingItemReader#setFetchSize(int) + */ + public JdbcPagingItemReaderBuilder fetchSize(int fetchSize) { + this.fetchSize = fetchSize; + + return this; + } + + /** + * The {@link RowMapper} used to map the query results to objects. Required. + * + * @param rowMapper a {@link RowMapper} implementation + * @return this instance for method chaining + * @see JdbcPagingItemReader#setRowMapper(RowMapper) + */ + public JdbcPagingItemReaderBuilder rowMapper(RowMapper rowMapper) { + this.rowMapper = rowMapper; + + return this; + } + + /** + * A {@link Map} of values to set on the SQL's prepared statement. + * + * @param parameterValues Map of values + * @return this instance for method chaining + * @see JdbcPagingItemReader#setParameterValues(Map) + */ + public JdbcPagingItemReaderBuilder parameterValues(Map parameterValues) { + this.parameterValues = parameterValues; + + return this; + } + + /** + * The number of records to request per page/query. Defaults to 10. Must be greater + * than zero. + * + * @param pageSize number of items + * @return this instance for method chaining + * @see JdbcPagingItemReader#setPageSize(int) + */ + public JdbcPagingItemReaderBuilder pageSize(int pageSize) { + this.pageSize = pageSize; + + return this; + } + + /** + * The SQL GROUP BY clause for a db specific @{@link PagingQueryProvider}. + * This is only used if a PagingQueryProvider is not provided. + * + * @param groupClause the SQL clause + * @return this instance for method chaining + * @see AbstractSqlPagingQueryProvider#setGroupClause(String) + */ + public JdbcPagingItemReaderBuilder groupClause(String groupClause) { + this.groupClause = groupClause; + + return this; + } + + /** + * The SQL SELECT clause for a db specific {@link PagingQueryProvider}. + * This is only used if a PagingQueryProvider is not provided. + * + * @param selectClause the SQL clause + * @return this instance for method chaining + * @see AbstractSqlPagingQueryProvider#setSelectClause(String) + */ + public JdbcPagingItemReaderBuilder selectClause(String selectClause) { + this.selectClause = selectClause; + + return this; + } + + /** + * The SQL FROM clause for a db specific {@link PagingQueryProvider}. + * This is only used if a PagingQueryProvider is not provided. + * + * @param fromClause the SQL clause + * @return this instance for method chaining + * @see AbstractSqlPagingQueryProvider#setFromClause(String) + */ + public JdbcPagingItemReaderBuilder fromClause(String fromClause) { + this.fromClause = fromClause; + + return this; + } + + /** + * The SQL WHERE clause for a db specific {@link PagingQueryProvider}. + * This is only used if a PagingQueryProvider is not provided. + * + * @param whereClause the SQL clause + * @return this instance for method chaining + * @see AbstractSqlPagingQueryProvider#setWhereClause(String) + */ + public JdbcPagingItemReaderBuilder whereClause(String whereClause) { + this.whereClause = whereClause; + + return this; + } + + /** + * The keys to sort by. These keys must create a unique key. + * + * @param sortKeys keys to sort by and the direction for each. + * @return this instance for method chaining + * @see AbstractSqlPagingQueryProvider#setSortKeys(Map) + */ + public JdbcPagingItemReaderBuilder sortKeys(Map sortKeys) { + this.sortKeys = sortKeys; + + return this; + } + + /** + * A {@link PagingQueryProvider} to provide the queries required. If provided, the + * SQL fragments configured via {@link #selectClause(String)}, + * {@link #fromClause(String)}, {@link #whereClause(String)}, {@link #groupClause}, + * and {@link #sortKeys(Map)} are ignored. + * + * @param provider the db specific query provider + * @return this instance for method chaining + * @see JdbcPagingItemReader#setQueryProvider(PagingQueryProvider) + */ + public JdbcPagingItemReaderBuilder queryProvider(PagingQueryProvider provider) { + this.queryProvider = provider; + + return this; + } + + /** + * Provides a completely built instance of the {@link JdbcPagingItemReader} + * + * @return a {@link JdbcPagingItemReader} + */ + public JdbcPagingItemReader build() { + Assert.isTrue(this.pageSize > 0, "pageSize must be greater than zero"); + Assert.notNull(this.dataSource, "dataSource is required"); + + if(this.saveState) { + Assert.hasText(this.name, + "A name is required when saveState is set to true"); + } + + JdbcPagingItemReader reader = new JdbcPagingItemReader<>(); + + reader.setMaxItemCount(this.maxItemCount); + reader.setCurrentItemCount(this.currentItemCount); + reader.setName(this.name); + reader.setSaveState(this.saveState); + reader.setDataSource(this.dataSource); + reader.setFetchSize(this.fetchSize); + reader.setParameterValues(this.parameterValues); + + if(this.queryProvider == null) { + Assert.hasLength(this.selectClause, "selectClause is required when not providing a PagingQueryProvider"); + Assert.hasLength(this.fromClause, "fromClause is required when not providing a PagingQueryProvider"); + Assert.notEmpty(this.sortKeys, "sortKeys are required when not providing a PagingQueryProvider"); + + reader.setQueryProvider(determineQueryProvider(this.dataSource)); + } + else { + reader.setQueryProvider(this.queryProvider); + } + + reader.setRowMapper(this.rowMapper); + reader.setPageSize(this.pageSize); + + return reader; + } + + private PagingQueryProvider determineQueryProvider(DataSource dataSource) { + + try { + DatabaseType databaseType = DatabaseType.fromMetaData(dataSource); + + AbstractSqlPagingQueryProvider provider; + + switch (databaseType) { + + case DERBY: provider = new DerbyPagingQueryProvider(); break; + case DB2: + case DB2VSE: + case DB2ZOS: + case DB2AS400: provider = new Db2PagingQueryProvider(); break; + case H2: provider = new H2PagingQueryProvider(); break; + case HSQL: provider = new HsqlPagingQueryProvider(); break; + case SQLSERVER: provider = new SqlServerPagingQueryProvider(); break; + case MYSQL: provider = new MySqlPagingQueryProvider(); break; + case ORACLE: provider = new OraclePagingQueryProvider(); break; + case POSTGRES: provider = new PostgresPagingQueryProvider(); break; + case SYBASE: provider = new SybasePagingQueryProvider(); break; + case SQLITE: provider = new SqlitePagingQueryProvider(); break; + default: + throw new IllegalArgumentException("Unable to determine PagingQueryProvider type " + + "from database type: " + databaseType); + } + + provider.setSelectClause(this.selectClause); + provider.setFromClause(this.fromClause); + provider.setWhereClause(this.whereClause); + provider.setGroupClause(this.groupClause); + provider.setSortKeys(this.sortKeys); + + return provider; + } + catch (MetaDataAccessException e) { + throw new IllegalArgumentException("Unable to determine PagingQueryProvider type", e); + } + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JpaItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JpaItemWriterBuilder.java new file mode 100644 index 0000000000..ee9b2afbf4 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JpaItemWriterBuilder.java @@ -0,0 +1,61 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import javax.persistence.EntityManagerFactory; + +import org.springframework.batch.item.database.JpaItemWriter; +import org.springframework.util.Assert; + +/** + * A builder for the {@link JpaItemWriter}. + * + * @author Mahmoud Ben Hassine + * @since 4.1 + * @see JpaItemWriter + */ +public class JpaItemWriterBuilder { + + private EntityManagerFactory entityManagerFactory; + + /** + * The JPA {@link EntityManagerFactory} to obtain an entity manager from. Required. + * + * @param entityManagerFactory the {@link EntityManagerFactory} + * @return this instance for method chaining + * @see JpaItemWriter#setEntityManagerFactory(EntityManagerFactory) + */ + public JpaItemWriterBuilder entityManagerFactory(EntityManagerFactory entityManagerFactory) { + this.entityManagerFactory = entityManagerFactory; + + return this; + } + + /** + * Returns a fully built {@link JpaItemWriter}. + * + * @return the writer + */ + public JpaItemWriter build() { + Assert.state(this.entityManagerFactory != null, + "EntityManagerFactory must be provided"); + + JpaItemWriter writer = new JpaItemWriter<>(); + writer.setEntityManagerFactory(this.entityManagerFactory); + + return writer; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JpaPagingItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JpaPagingItemReaderBuilder.java new file mode 100644 index 0000000000..22393fcfc0 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/JpaPagingItemReaderBuilder.java @@ -0,0 +1,229 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.util.Map; +import javax.persistence.EntityManagerFactory; + +import org.springframework.batch.item.database.JpaPagingItemReader; +import org.springframework.batch.item.database.orm.JpaQueryProvider; +import org.springframework.util.Assert; + +/** + * Creates a fully qualified JpaPagingItemReader. + * + * @author Michael Minella + * @author Glenn Renfro + * + * @since 4.0 + */ + +public class JpaPagingItemReaderBuilder { + + private int pageSize = 10; + + private EntityManagerFactory entityManagerFactory; + + private Map parameterValues; + + private boolean transacted = true; + + private String queryString; + + private JpaQueryProvider queryProvider; + + private boolean saveState = true; + + private String name; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public JpaPagingItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public JpaPagingItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public JpaPagingItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public JpaPagingItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * The number of records to request per page/query. Defaults to 10. Must be greater + * than zero. + * + * @param pageSize number of items + * @return this instance for method chaining + * @see JpaPagingItemReader#setPageSize(int) + */ + public JpaPagingItemReaderBuilder pageSize(int pageSize) { + this.pageSize = pageSize; + + return this; + } + + /** + * A map of parameter values to be set on the query. The key of the map is the name + * of the parameter to be set with the value being the value to be set. + * + * @param parameterValues map of values + * @return this instance for method chaining + * @see JpaPagingItemReader#setParameterValues(Map) + */ + public JpaPagingItemReaderBuilder parameterValues(Map parameterValues) { + this.parameterValues = parameterValues; + + return this; + } + + /** + * A query provider. This should be set only if {@link #queryString(String)} have not + * been set. + * + * @param queryProvider the query provider + * @return this instance for method chaining + * @see JpaPagingItemReader#setQueryProvider(JpaQueryProvider) + */ + public JpaPagingItemReaderBuilder queryProvider(JpaQueryProvider queryProvider) { + this.queryProvider = queryProvider; + + return this; + } + + /** + * The HQL query string to execute. This should only be set if + * {@link #queryProvider(JpaQueryProvider)} has not been set. + * + * @param queryString the HQL query + * @return this instance for method chaining + * @see JpaPagingItemReader#setQueryString(String) + */ + public JpaPagingItemReaderBuilder queryString(String queryString) { + this.queryString = queryString; + + return this; + } + + /** + * Indicates if a transaction should be created around the read (true by default). + * Can be set to false in cases where JPA implementation doesn't support a particular + * transaction, however this may cause object inconsistency in the EntityManagerFactory. + * + * @param transacted defaults to true + * @return this instance for method chaining + * @see JpaPagingItemReader#setTransacted(boolean) + */ + public JpaPagingItemReaderBuilder transacted(boolean transacted) { + this.transacted = transacted; + + return this; + } + + /** + * The {@link EntityManagerFactory} to be used for executing the configured + * {@link #queryString}. + * + * @param entityManagerFactory {@link EntityManagerFactory} used to create + * {@link javax.persistence.EntityManager} + * @return this instance for method chaining + */ + public JpaPagingItemReaderBuilder entityManagerFactory(EntityManagerFactory entityManagerFactory) { + this.entityManagerFactory = entityManagerFactory; + + return this; + } + + /** + * Returns a fully constructed {@link JpaPagingItemReader}. + * + * @return a new {@link JpaPagingItemReader} + */ + public JpaPagingItemReader build() { + Assert.isTrue(this.pageSize > 0, "pageSize must be greater than zero"); + Assert.notNull(this.entityManagerFactory, "An EntityManagerFactory is required"); + + if(this.saveState) { + Assert.hasText(this.name, + "A name is required when saveState is set to true"); + } + + if(this.queryProvider == null) { + Assert.hasLength(this.queryString, "Query string is required when queryProvider is null"); + } + + JpaPagingItemReader reader = new JpaPagingItemReader<>(); + + reader.setQueryString(this.queryString); + reader.setPageSize(this.pageSize); + reader.setParameterValues(this.parameterValues); + reader.setEntityManagerFactory(this.entityManagerFactory); + reader.setQueryProvider(this.queryProvider); + reader.setTransacted(this.transacted); + reader.setCurrentItemCount(this.currentItemCount); + reader.setMaxItemCount(this.maxItemCount); + reader.setSaveState(this.saveState); + reader.setName(this.name); + + return reader; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/StoredProcedureItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/StoredProcedureItemReaderBuilder.java new file mode 100644 index 0000000000..b87145e6e9 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/StoredProcedureItemReaderBuilder.java @@ -0,0 +1,362 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import javax.sql.DataSource; + +import org.springframework.batch.item.database.AbstractCursorItemReader; + +import org.springframework.batch.item.database.StoredProcedureItemReader; +import org.springframework.jdbc.core.PreparedStatementSetter; +import org.springframework.jdbc.core.RowMapper; +import org.springframework.jdbc.core.SqlParameter; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * A fluent builder API for the configuration of a {@link StoredProcedureItemReader}. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @author Drummond Dawson + * @since 4.0.0 + * @see StoredProcedureItemReader + */ +public class StoredProcedureItemReaderBuilder { + + public static final int VALUE_NOT_SET = -1; + + private int currentItemCount = 0; + + private int maxItemCount = Integer.MAX_VALUE; + + private boolean saveState = true; + + private DataSource dataSource; + + private int fetchSize = VALUE_NOT_SET; + + private int maxRows = VALUE_NOT_SET; + + private int queryTimeout = VALUE_NOT_SET; + + private boolean ignoreWarnings = true; + + private boolean verifyCursorPosition = true; + + private boolean driverSupportsAbsolute = false; + + private boolean useSharedExtendedConnection = false; + + private PreparedStatementSetter preparedStatementSetter; + + private RowMapper rowMapper; + + private String procedureName; + + private SqlParameter[] parameters = new SqlParameter[0]; + + private boolean function = false; + + private int refCursorPosition = 0; + + private String name; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public StoredProcedureItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public StoredProcedureItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public StoredProcedureItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public StoredProcedureItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * The {@link DataSource} to read from + * + * @param dataSource a relational data base + * @return this instance for method chaining + * @see StoredProcedureItemReader#setDataSource(DataSource) + */ + public StoredProcedureItemReaderBuilder dataSource(DataSource dataSource) { + this.dataSource = dataSource; + + return this; + } + + /** + * A hint to the driver as to how many rows to return with each fetch. + * + * @param fetchSize the hint + * @return this instance for method chaining + * @see StoredProcedureItemReader#setFetchSize(int) + */ + public StoredProcedureItemReaderBuilder fetchSize(int fetchSize) { + this.fetchSize = fetchSize; + + return this; + } + + /** + * The max number of rows the {@link java.sql.ResultSet} can contain + * + * @param maxRows the max + * @return this instance for method chaining + * @see StoredProcedureItemReader#setMaxRows(int) + */ + public StoredProcedureItemReaderBuilder maxRows(int maxRows) { + this.maxRows = maxRows; + + return this; + } + + /** + * The time in milliseconds for the query to timeout + * + * @param queryTimeout timeout + * @return this instance for method chaining + * @see StoredProcedureItemReader#setQueryTimeout(int) + */ + public StoredProcedureItemReaderBuilder queryTimeout(int queryTimeout) { + this.queryTimeout = queryTimeout; + + return this; + } + + /** + * Indicates if SQL warnings should be ignored or if an exception should be thrown. + * + * @param ignoreWarnings indicator. Defaults to true + * @return this instance for method chaining + * @see AbstractCursorItemReader#setIgnoreWarnings(boolean) + */ + public StoredProcedureItemReaderBuilder ignoreWarnings(boolean ignoreWarnings) { + this.ignoreWarnings = ignoreWarnings; + + return this; + } + + /** + * Indicates if the reader should verify the current position of the + * {@link java.sql.ResultSet} after being passed to the {@link RowMapper}. Defaults + * to true. + * + * @param verifyCursorPosition indicator + * @return this instance for method chaining + * @see StoredProcedureItemReader#setVerifyCursorPosition(boolean) + */ + public StoredProcedureItemReaderBuilder verifyCursorPosition(boolean verifyCursorPosition) { + this.verifyCursorPosition = verifyCursorPosition; + + return this; + } + + /** + * Indicates if the JDBC driver supports setting the absolute row on the + * {@link java.sql.ResultSet}. + * + * @param driverSupportsAbsolute indicator + * @return this instance for method chaining + * @see StoredProcedureItemReader#setDriverSupportsAbsolute(boolean) + */ + public StoredProcedureItemReaderBuilder driverSupportsAbsolute(boolean driverSupportsAbsolute) { + this.driverSupportsAbsolute = driverSupportsAbsolute; + + return this; + } + + /** + * Indicates that the connection used for the cursor is being used by all other + * processing, therefor part of the same transaction. + * + * @param useSharedExtendedConnection indicator + * @return this instance for method chaining + * @see StoredProcedureItemReader#setUseSharedExtendedConnection(boolean) + */ + public StoredProcedureItemReaderBuilder useSharedExtendedConnection(boolean useSharedExtendedConnection) { + this.useSharedExtendedConnection = useSharedExtendedConnection; + + return this; + } + + /** + * Configures the provided {@link PreparedStatementSetter} to be used to populate any + * arguments in the SQL query to be executed for the reader. + * + * @param preparedStatementSetter setter + * @return this instance for method chaining + * @see StoredProcedureItemReader#setPreparedStatementSetter(PreparedStatementSetter) + */ + public StoredProcedureItemReaderBuilder preparedStatementSetter(PreparedStatementSetter preparedStatementSetter) { + this.preparedStatementSetter = preparedStatementSetter; + + return this; + } + + /** + * The {@link RowMapper} used to map the results of the cursor to each item. + * + * @param rowMapper {@link RowMapper} + * @return this instance for method chaining + * @see StoredProcedureItemReader#setRowMapper(RowMapper) + */ + public StoredProcedureItemReaderBuilder rowMapper(RowMapper rowMapper) { + this.rowMapper = rowMapper; + + return this; + } + + /** + * The name of the stored procedure to execute + * + * @param procedureName name of the procedure + * @return this instance for method chaining + * @see StoredProcedureItemReader#setProcedureName(String) + */ + public StoredProcedureItemReaderBuilder procedureName(String procedureName) { + this.procedureName = procedureName; + + return this; + } + + /** + * SQL parameters to be set when executing the stored procedure + * + * @param parameters parameters to be set + * @return this instance for method chaining + * @see StoredProcedureItemReader#setParameters(SqlParameter[]) + */ + public StoredProcedureItemReaderBuilder parameters(SqlParameter... parameters) { + this.parameters = parameters; + + return this; + } + + /** + * Indicates the stored procedure is a function + * + * @return this instance for method chaining + * @see StoredProcedureItemReader#setFunction(boolean) + */ + public StoredProcedureItemReaderBuilder function() { + this.function = true; + + return this; + } + + /** + * The parameter position of the REF CURSOR. Only used for Oracle and PostgreSQL that + * use REF CURSORs. For any other database, this should remain as the default (0). + * + * @param refCursorPosition the parameter position + * @return this instance for method chaining + * @see StoredProcedureItemReader#setRefCursorPosition(int) + */ + public StoredProcedureItemReaderBuilder refCursorPosition(int refCursorPosition) { + this.refCursorPosition = refCursorPosition; + + return this; + } + + /** + * Validates configuration and builds a new reader instance + * + * @return a fully constructed {@link StoredProcedureItemReader} + */ + public StoredProcedureItemReader build() { + if(this.saveState) { + Assert.hasText(this.name, + "A name is required when saveSate is set to true"); + } + + Assert.notNull(this.procedureName, "The name of the stored procedure must be provided"); + Assert.notNull(this.dataSource, "A datasource is required"); + Assert.notNull(this.rowMapper, "A rowmapper is required"); + + StoredProcedureItemReader itemReader = new StoredProcedureItemReader<>(); + + if(StringUtils.hasText(this.name)) { + itemReader.setName(this.name); + } + + itemReader.setProcedureName(this.procedureName); + itemReader.setRowMapper(this.rowMapper); + itemReader.setParameters(this.parameters); + itemReader.setPreparedStatementSetter(this.preparedStatementSetter); + itemReader.setFunction(this.function); + itemReader.setRefCursorPosition(this.refCursorPosition); + itemReader.setCurrentItemCount(this.currentItemCount); + itemReader.setDataSource(this.dataSource); + itemReader.setDriverSupportsAbsolute(this.driverSupportsAbsolute); + itemReader.setFetchSize(this.fetchSize); + itemReader.setIgnoreWarnings(this.ignoreWarnings); + itemReader.setMaxItemCount(this.maxItemCount); + itemReader.setMaxRows(this.maxRows); + itemReader.setQueryTimeout(this.queryTimeout); + itemReader.setSaveState(this.saveState); + itemReader.setUseSharedExtendedConnection(this.useSharedExtendedConnection); + itemReader.setVerifyCursorPosition(this.verifyCursorPosition); + + return itemReader; + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/package-info.java new file mode 100644 index 0000000000..13b752fc84 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/builder/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Builders for database item readers and writers. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.database.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/AbstractHibernateQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/AbstractHibernateQueryProvider.java index f2e7440370..0de7ce35af 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/AbstractHibernateQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/AbstractHibernateQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,7 +16,7 @@ package org.springframework.batch.item.database.orm; -import org.hibernate.Query; +import org.hibernate.query.Query; import org.hibernate.Session; import org.hibernate.StatelessSession; @@ -35,7 +35,7 @@ * @since 2.1 * */ -public abstract class AbstractHibernateQueryProvider implements HibernateQueryProvider { +public abstract class AbstractHibernateQueryProvider implements HibernateQueryProvider { private StatelessSession statelessSession; private Session statefulSession; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/AbstractJpaQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/AbstractJpaQueryProvider.java index 118dc34bb9..1664ccc6e8 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/AbstractJpaQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/AbstractJpaQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -43,7 +43,7 @@ public abstract class AbstractJpaQueryProvider implements JpaQueryProvider, Init * {@link HibernateQueryProvider} to participate in a user's managed transaction. *

      * - * @param entityManager + * @param entityManager EntityManager to use */ @Override public void setEntityManager(EntityManager entityManager) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/HibernateNativeQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/HibernateNativeQueryProvider.java index ffacfdd321..1963894287 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/HibernateNativeQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/HibernateNativeQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,8 +16,9 @@ package org.springframework.batch.item.database.orm; -import org.hibernate.Query; -import org.hibernate.SQLQuery; +import org.hibernate.query.NativeQuery; +import org.hibernate.query.Query; + import org.springframework.util.Assert; import org.springframework.util.StringUtils; @@ -32,7 +33,7 @@ * * @param entity returned by executing the query */ -public class HibernateNativeQueryProvider extends AbstractHibernateQueryProvider { +public class HibernateNativeQueryProvider extends AbstractHibernateQueryProvider { private String sqlQuery; @@ -40,18 +41,19 @@ public class HibernateNativeQueryProvider extends AbstractHibernateQueryProvi /** *

      - * Create an {@link SQLQuery} from the session provided (preferring + * Create an {@link NativeQuery} from the session provided (preferring * stateless if both are available). *

      */ @Override - public SQLQuery createQuery() { + @SuppressWarnings("unchecked") + public NativeQuery createQuery() { if (isStatelessSession()) { - return getStatelessSession().createSQLQuery(sqlQuery).addEntity(entityClass); + return getStatelessSession().createNativeQuery(sqlQuery).addEntity(entityClass); } else { - return getStatefulSession().createSQLQuery(sqlQuery).addEntity(entityClass); + return getStatefulSession().createNativeQuery(sqlQuery).addEntity(entityClass); } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/HibernateQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/HibernateQueryProvider.java index a879d72c06..ef823a5bd9 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/HibernateQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/HibernateQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,7 +16,7 @@ package org.springframework.batch.item.database.orm; -import org.hibernate.Query; +import org.hibernate.query.Query; import org.hibernate.Session; import org.hibernate.StatelessSession; import org.springframework.batch.item.ItemReader; @@ -33,7 +33,7 @@ * @since 2.1 * */ -public interface HibernateQueryProvider { +public interface HibernateQueryProvider { /** *

      @@ -43,7 +43,7 @@ public interface HibernateQueryProvider { * * @return created query */ - Query createQuery(); + Query createQuery(); /** *

      diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/JpaNativeQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/JpaNativeQueryProvider.java index 6a9be866b2..c9501d4088 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/JpaNativeQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/JpaNativeQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/JpaQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/JpaQueryProvider.java index 4b2b1276c0..b23e8d848f 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/JpaQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/JpaQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -42,7 +42,7 @@ public interface JpaQueryProvider { /** * Provide an {@link EntityManager} for the query to be built. * - * @param entityManager + * @param entityManager to be used by the {@link JpaQueryProvider}. */ void setEntityManager(EntityManager entityManager); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/package-info.java new file mode 100644 index 0000000000..97d032d38b --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/orm/package-info.java @@ -0,0 +1,10 @@ +/** + * Support classes for components using various ORM related technologies. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.database.orm; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/package-info.java new file mode 100644 index 0000000000..801309e020 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of database based item readers and writers. + *

      + */ +@NonNullApi +package org.springframework.batch.item.database; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/package.html deleted file mode 100644 index c73e51f880..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of database based item readers and writers. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/AbstractSqlPagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/AbstractSqlPagingQueryProvider.java index c1b89b1d2a..265bd692fc 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/AbstractSqlPagingQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/AbstractSqlPagingQueryProvider.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,7 +20,6 @@ import java.util.LinkedHashMap; import java.util.List; import java.util.Map; - import javax.sql.DataSource; import org.springframework.batch.item.database.JdbcParameterUtils; @@ -43,11 +42,14 @@ * Provides properties and preparation for the mandatory "selectClause" and * "fromClause" as well as for the optional "whereClause". Also provides * property for the mandatory "sortKeys". Note: The columns that make up - * the sort key must be a true key and not just a column to order by. + * the sort key must be a true key and not just a column to order by. It is important + * to have a unique key constraint on the sort key to guarantee that no data is lost + * between executions. * * @author Thomas Risberg * @author Dave Syer * @author Michael Minella + * @author Mahmoud Ben Hassine * @since 2.0 */ public abstract class AbstractSqlPagingQueryProvider implements PagingQueryProvider { @@ -58,7 +60,7 @@ public abstract class AbstractSqlPagingQueryProvider implements PagingQueryProvi private String whereClause; - private Map sortKeys = new LinkedHashMap(); + private Map sortKeys = new LinkedHashMap<>(); private String groupClause; @@ -147,7 +149,7 @@ public void setSortKeys(Map sortKeys) { } /** - * A Map of sort columns as the key and boolean for ascending/descending (assending = true). + * A Map<String, Boolean> of sort columns as the key and boolean for ascending/descending (ascending = true). * * @return sortKey key to use to sort and limit page content */ @@ -183,7 +185,7 @@ public String getSortKeyPlaceHolder(String keyName) { */ @Override public void init(DataSource dataSource) throws Exception { - Assert.notNull(dataSource); + Assert.notNull(dataSource, "A DataSource is required"); Assert.hasLength(selectClause, "selectClause must be specified"); Assert.hasLength(fromClause, "fromClause must be specified"); Assert.notEmpty(sortKeys, "sortKey must be specified"); @@ -196,7 +198,7 @@ public void init(DataSource dataSource) throws Exception { if(groupClause != null) { sql.append(" GROUP BY ").append(groupClause); } - List namedParameters = new ArrayList(); + List namedParameters = new ArrayList<>(); parameterCount = JdbcParameterUtils.countParameterPlaceholders(sql.toString(), namedParameters); if (namedParameters.size() > 0) { if (parameterCount != namedParameters.size()) { @@ -249,4 +251,27 @@ private String removeKeyWord(String keyWord, String clause) { } } + /** + * + * @return sortKey key to use to sort and limit page content (without alias) + */ + @Override + public Map getSortKeysWithoutAliases() { + Map sortKeysWithoutAliases = new LinkedHashMap<>(); + + for (Map.Entry sortKeyEntry : sortKeys.entrySet()) { + String key = sortKeyEntry.getKey(); + int separator = key.indexOf('.'); + if (separator > 0) { + int columnIndex = separator + 1; + if (columnIndex < key.length()) { + sortKeysWithoutAliases.put(key.substring(columnIndex), sortKeyEntry.getValue()); + } + } else { + sortKeysWithoutAliases.put(sortKeyEntry.getKey(), sortKeyEntry.getValue()); + } + } + + return sortKeysWithoutAliases; + } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/ColumnMapItemPreparedStatementSetter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/ColumnMapItemPreparedStatementSetter.java index 6cb7d98153..dde96eda99 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/ColumnMapItemPreparedStatementSetter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/ColumnMapItemPreparedStatementSetter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,18 +16,18 @@ package org.springframework.batch.item.database.support; -import java.sql.PreparedStatement; -import java.sql.SQLException; -import java.util.Map; - import org.springframework.batch.item.database.ItemPreparedStatementSetter; import org.springframework.jdbc.core.ColumnMapRowMapper; import org.springframework.jdbc.core.SqlTypeValue; import org.springframework.jdbc.core.StatementCreatorUtils; import org.springframework.util.Assert; +import java.sql.PreparedStatement; +import java.sql.SQLException; +import java.util.Map; + /** - *

      Implementation of the {@link ItemPreparedStatementSetter} interface that assumes all + *

      Implementation of the {@link ItemPreparedStatementSetter} interface that assumes all * keys are contained within a {@link Map} with the column name as the key. It assumes nothing * about ordering, and assumes that the order the entry set can be iterated over is the same as * the PreparedStatement should be set.

      diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/DataFieldMaxValueIncrementerFactory.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/DataFieldMaxValueIncrementerFactory.java index 7157c897dc..764c4b70b7 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/DataFieldMaxValueIncrementerFactory.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/DataFieldMaxValueIncrementerFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -41,11 +41,16 @@ public interface DataFieldMaxValueIncrementerFactory { /** * Returns boolean indicated whether or not the provided string is supported by this * factory. + * + * @param databaseType {@link String} containing the database type. + * @return true if the incrementerType is supported by this database type. Else false is returned. */ public boolean isSupportedIncrementerType(String databaseType); /** * Returns the list of supported database incrementer types + * + * @return an array of {@link String}s containing the supported incrementer types. */ public String[] getSupportedIncrementerTypes(); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/Db2PagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/Db2PagingQueryProvider.java index a2c46f73ae..7add56f068 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/Db2PagingQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/Db2PagingQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/DefaultDataFieldMaxValueIncrementerFactory.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/DefaultDataFieldMaxValueIncrementerFactory.java index 83ff2f5069..d95c6574b6 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/DefaultDataFieldMaxValueIncrementerFactory.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/DefaultDataFieldMaxValueIncrementerFactory.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,40 +15,46 @@ */ package org.springframework.batch.item.database.support; -import static org.springframework.batch.support.DatabaseType.DB2; -import static org.springframework.batch.support.DatabaseType.DB2ZOS; -import static org.springframework.batch.support.DatabaseType.DERBY; -import static org.springframework.batch.support.DatabaseType.H2; -import static org.springframework.batch.support.DatabaseType.HSQL; -import static org.springframework.batch.support.DatabaseType.MYSQL; -import static org.springframework.batch.support.DatabaseType.ORACLE; -import static org.springframework.batch.support.DatabaseType.POSTGRES; -import static org.springframework.batch.support.DatabaseType.SQLSERVER; -import static org.springframework.batch.support.DatabaseType.SYBASE; - import java.util.ArrayList; import java.util.List; - import javax.sql.DataSource; import org.springframework.batch.support.DatabaseType; -import org.springframework.jdbc.support.incrementer.DB2MainframeSequenceMaxValueIncrementer; -import org.springframework.jdbc.support.incrementer.DB2SequenceMaxValueIncrementer; +import org.springframework.jdbc.support.incrementer.Db2LuwMaxValueIncrementer; +import org.springframework.jdbc.support.incrementer.Db2MainframeMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.DataFieldMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.DerbyMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.H2SequenceMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.HsqlMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.MySQLMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.OracleSequenceMaxValueIncrementer; -import org.springframework.jdbc.support.incrementer.PostgreSQLSequenceMaxValueIncrementer; +import org.springframework.jdbc.support.incrementer.PostgresSequenceMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.SqlServerMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.SybaseMaxValueIncrementer; +import static org.springframework.batch.support.DatabaseType.DB2; +import static org.springframework.batch.support.DatabaseType.DB2AS400; +import static org.springframework.batch.support.DatabaseType.DB2ZOS; +import static org.springframework.batch.support.DatabaseType.DERBY; +import static org.springframework.batch.support.DatabaseType.H2; +import static org.springframework.batch.support.DatabaseType.HSQL; +import static org.springframework.batch.support.DatabaseType.MYSQL; +import static org.springframework.batch.support.DatabaseType.ORACLE; +import static org.springframework.batch.support.DatabaseType.POSTGRES; +import static org.springframework.batch.support.DatabaseType.SQLITE; +import static org.springframework.batch.support.DatabaseType.SQLSERVER; +import static org.springframework.batch.support.DatabaseType.SYBASE; + /** * Default implementation of the {@link DataFieldMaxValueIncrementerFactory} * interface. Valid database types are given by the {@link DatabaseType} enum. + * + * Note: For MySql databases, the + * {@link MySQLMaxValueIncrementer#setUseNewConnection(boolean)} will be set to true. * * @author Lucas Ward + * @author Michael Minella + * @author Drummond Dawson * @see DatabaseType */ public class DefaultDataFieldMaxValueIncrementerFactory implements DataFieldMaxValueIncrementerFactory { @@ -73,15 +79,15 @@ public DefaultDataFieldMaxValueIncrementerFactory(DataSource dataSource) { this.dataSource = dataSource; } - @Override + @Override public DataFieldMaxValueIncrementer getIncrementer(String incrementerType, String incrementerName) { DatabaseType databaseType = DatabaseType.valueOf(incrementerType.toUpperCase()); - if (databaseType == DB2) { - return new DB2SequenceMaxValueIncrementer(dataSource, incrementerName); + if (databaseType == DB2 || databaseType == DB2AS400) { + return new Db2LuwMaxValueIncrementer(dataSource, incrementerName); } else if (databaseType == DB2ZOS) { - return new DB2MainframeSequenceMaxValueIncrementer(dataSource, incrementerName); + return new Db2MainframeMaxValueIncrementer(dataSource, incrementerName); } else if (databaseType == DERBY) { return new DerbyMaxValueIncrementer(dataSource, incrementerName, incrementerColumnName); @@ -93,13 +99,18 @@ else if (databaseType == H2) { return new H2SequenceMaxValueIncrementer(dataSource, incrementerName); } else if (databaseType == MYSQL) { - return new MySQLMaxValueIncrementer(dataSource, incrementerName, incrementerColumnName); + MySQLMaxValueIncrementer mySQLMaxValueIncrementer = new MySQLMaxValueIncrementer(dataSource, incrementerName, incrementerColumnName); + mySQLMaxValueIncrementer.setUseNewConnection(true); + return mySQLMaxValueIncrementer; } else if (databaseType == ORACLE) { return new OracleSequenceMaxValueIncrementer(dataSource, incrementerName); } else if (databaseType == POSTGRES) { - return new PostgreSQLSequenceMaxValueIncrementer(dataSource, incrementerName); + return new PostgresSequenceMaxValueIncrementer(dataSource, incrementerName); + } + else if (databaseType == SQLITE) { + return new SqliteMaxValueIncrementer(dataSource, incrementerName, incrementerColumnName); } else if (databaseType == SQLSERVER) { return new SqlServerMaxValueIncrementer(dataSource, incrementerName, incrementerColumnName); @@ -108,13 +119,12 @@ else if (databaseType == SYBASE) { return new SybaseMaxValueIncrementer(dataSource, incrementerName, incrementerColumnName); } throw new IllegalArgumentException("databaseType argument was not on the approved list"); - } - + @Override public boolean isSupportedIncrementerType(String incrementerType) { for (DatabaseType type : DatabaseType.values()) { - if (type.name().equals(incrementerType.toUpperCase())) { + if (type.name().equalsIgnoreCase(incrementerType)) { return true; } } @@ -125,7 +135,7 @@ public boolean isSupportedIncrementerType(String incrementerType) { @Override public String[] getSupportedIncrementerTypes() { - List types = new ArrayList(); + List types = new ArrayList<>(); for (DatabaseType type : DatabaseType.values()) { types.add(type.name()); @@ -133,4 +143,4 @@ public String[] getSupportedIncrementerTypes() { return types.toArray(new String[types.size()]); } -} +} \ No newline at end of file diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/DerbyPagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/DerbyPagingQueryProvider.java index 0de89956d0..5f2490c613 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/DerbyPagingQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/DerbyPagingQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -47,7 +47,7 @@ public void init(DataSource dataSource) throws Exception { } } - // derby version numbering is M.m.f.p [ {alpha|beta} ] see http://db.apache.org/derby/papers/versionupgrade.html#Basic+Numbering+Scheme + // derby version numbering is M.m.f.p [ {alpha|beta} ] see https://db.apache.org/derby/papers/versionupgrade.html#Basic+Numbering+Scheme private boolean isDerbyVersionSupported(String version) { String[] minimalVersionParts = MINIMAL_DERBY_VERSION.split("\\."); String[] versionParts = version.split("[\\. ]"); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/H2PagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/H2PagingQueryProvider.java index 712028cb60..0fabb0d125 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/H2PagingQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/H2PagingQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/HsqlPagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/HsqlPagingQueryProvider.java index 198651b21c..ee117e7866 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/HsqlPagingQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/HsqlPagingQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/resource/ListPreparedStatementSetter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/ListPreparedStatementSetter.java similarity index 79% rename from spring-batch-core/src/main/java/org/springframework/batch/core/resource/ListPreparedStatementSetter.java rename to spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/ListPreparedStatementSetter.java index 8b367bb540..b5c60c9f29 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/resource/ListPreparedStatementSetter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/ListPreparedStatementSetter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -13,7 +13,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.springframework.batch.core.resource; +package org.springframework.batch.item.database.support; import java.sql.PreparedStatement; import java.sql.SQLException; @@ -35,14 +35,25 @@ * item in the list will be the first bind variable set. (i.e. it will * correspond to the first '?' in the SQL statement) * + * @deprecated use {@link org.springframework.jdbc.core.ArgumentPreparedStatementSetter} + * instead. + * * @author Lucas Ward + * @author Mahmoud Ben Hassine * */ +@Deprecated public class ListPreparedStatementSetter implements PreparedStatementSetter, InitializingBean { private List parameters; + public ListPreparedStatementSetter() {} + + public ListPreparedStatementSetter(List parameters) { + this.parameters = parameters; + } + @Override public void setValues(PreparedStatement ps) throws SQLException { for (int i = 0; i < parameters.size(); i++) { @@ -54,7 +65,11 @@ public void setValues(PreparedStatement ps) throws SQLException { * The parameter values that will be set on the PreparedStatement. * It is assumed that their order in the List is the order of the parameters * in the PreparedStatement. + * + * @param parameters list containing the parameter values to be used. + * @deprecated In favor of the constructor */ + @Deprecated public void setParameters(List parameters) { this.parameters = parameters; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/MySqlPagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/MySqlPagingQueryProvider.java index 5714e3df40..7fee3c1e7f 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/MySqlPagingQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/MySqlPagingQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/OraclePagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/OraclePagingQueryProvider.java index 4217e7f771..9e2e902637 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/OraclePagingQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/OraclePagingQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/PostgresPagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/PostgresPagingQueryProvider.java index 326b897e02..5449b1f1aa 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/PostgresPagingQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/PostgresPagingQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlPagingQueryProviderFactoryBean.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlPagingQueryProviderFactoryBean.java index 592e961144..dad72b3261 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlPagingQueryProviderFactoryBean.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlPagingQueryProviderFactoryBean.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,13 +16,16 @@ package org.springframework.batch.item.database.support; import static org.springframework.batch.support.DatabaseType.DB2; +import static org.springframework.batch.support.DatabaseType.DB2VSE; import static org.springframework.batch.support.DatabaseType.DB2ZOS; +import static org.springframework.batch.support.DatabaseType.DB2AS400; import static org.springframework.batch.support.DatabaseType.DERBY; import static org.springframework.batch.support.DatabaseType.H2; import static org.springframework.batch.support.DatabaseType.HSQL; import static org.springframework.batch.support.DatabaseType.MYSQL; import static org.springframework.batch.support.DatabaseType.ORACLE; import static org.springframework.batch.support.DatabaseType.POSTGRES; +import static org.springframework.batch.support.DatabaseType.SQLITE; import static org.springframework.batch.support.DatabaseType.SQLSERVER; import static org.springframework.batch.support.DatabaseType.SYBASE; @@ -48,8 +51,7 @@ * @author Dave Syer * @author Michael Minella */ -@SuppressWarnings("rawtypes") -public class SqlPagingQueryProviderFactoryBean implements FactoryBean { +public class SqlPagingQueryProviderFactoryBean implements FactoryBean { private DataSource dataSource; @@ -65,18 +67,21 @@ public class SqlPagingQueryProviderFactoryBean implements FactoryBean { private Map sortKeys; - private Map providers = new HashMap(); + private Map providers = new HashMap<>(); { providers.put(DB2, new Db2PagingQueryProvider()); + providers.put(DB2VSE, new Db2PagingQueryProvider()); providers.put(DB2ZOS, new Db2PagingQueryProvider()); + providers.put(DB2AS400, new Db2PagingQueryProvider()); providers.put(DERBY,new DerbyPagingQueryProvider()); providers.put(HSQL,new HsqlPagingQueryProvider()); providers.put(H2,new H2PagingQueryProvider()); providers.put(MYSQL,new MySqlPagingQueryProvider()); providers.put(ORACLE,new OraclePagingQueryProvider()); providers.put(POSTGRES,new PostgresPagingQueryProvider()); + providers.put(SQLITE, new SqlitePagingQueryProvider()); providers.put(SQLSERVER,new SqlServerPagingQueryProvider()); providers.put(SYBASE,new SybasePagingQueryProvider()); } @@ -133,7 +138,7 @@ public void setSortKeys(Map sortKeys) { public void setSortKey(String key) { Assert.doesNotContain(key, ",", "String setter is valid for a single ASC key only"); - Map keys = new LinkedHashMap(); + Map keys = new LinkedHashMap<>(); keys.put(key, Order.ASCENDING); this.sortKeys = keys; @@ -146,7 +151,7 @@ public void setSortKey(String key) { * @see FactoryBean#getObject() */ @Override - public Object getObject() throws Exception { + public PagingQueryProvider getObject() throws Exception { DatabaseType type; try { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlPagingQueryUtils.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlPagingQueryUtils.java index 06d16c3465..f3b9990cd1 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlPagingQueryUtils.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlPagingQueryUtils.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2015 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,7 +40,7 @@ public class SqlPagingQueryUtils { * * @param provider {@link AbstractSqlPagingQueryProvider} providing the * implementation specifics - * @param remainingPageQuery is this query for the ramining pages (true) as + * @param remainingPageQuery is this query for the remaining pages (true) as * opposed to the first page (false) * @param limitClause the implementation specific limit clause to be used * @return the generated query @@ -63,7 +63,7 @@ public static String generateLimitSqlQuery(AbstractSqlPagingQueryProvider provid * * @param provider {@link AbstractSqlPagingQueryProvider} providing the * implementation specifics - * @param remainingPageQuery is this query for the ramining pages (true) as + * @param remainingPageQuery is this query for the remaining pages (true) as * opposed to the first page (false) * @param limitClause the implementation specific limit clause to be used * @return the generated query @@ -91,7 +91,7 @@ public static String generateLimitGroupedSqlQuery(AbstractSqlPagingQueryProvider * * @param provider {@link AbstractSqlPagingQueryProvider} providing the * implementation specifics - * @param remainingPageQuery is this query for the ramining pages (true) as + * @param remainingPageQuery is this query for the remaining pages (true) as * opposed to the first page (false) * @param topClause the implementation specific top clause to be used * @return the generated query @@ -113,7 +113,7 @@ public static String generateTopSqlQuery(AbstractSqlPagingQueryProvider provider * * @param provider {@link AbstractSqlPagingQueryProvider} providing the * implementation specifics - * @param remainingPageQuery is this query for the ramining pages (true) as + * @param remainingPageQuery is this query for the remaining pages (true) as * opposed to the first page (false) * @param topClause the implementation specific top clause to be used * @return the generated query @@ -156,6 +156,7 @@ public static String generateRowNumSqlQuery(AbstractSqlPagingQueryProvider provi * * @param provider {@link AbstractSqlPagingQueryProvider} providing the * implementation specifics + * @param selectClause {@link String} containing the select portion of the query. * @param remainingPageQuery is this query for the remaining pages (true) as * opposed to the first page (false) * @param rowNumClause the implementation specific row num clause to be used @@ -242,14 +243,26 @@ public static String generateTopJumpToQuery(AbstractSqlPagingQueryProvider provi /** * Generates ORDER BY attributes based on the sort keys. * - * @param provider + * @param provider the {@link AbstractSqlPagingQueryProvider} to be used for + * used for pagination. * @return a String that can be appended to an ORDER BY clause. */ public static String buildSortClause(AbstractSqlPagingQueryProvider provider) { + return buildSortClause(provider.getSortKeys()); + } + + /** + * Generates ORDER BY attributes based on the sort keys. + * + * @param sortKeys {@link Map} where the key is the name of the column to be + * sorted and the value contains the {@link Order}. + * @return a String that can be appended to an ORDER BY clause. + */ + public static String buildSortClause(Map sortKeys) { StringBuilder builder = new StringBuilder(); String prefix = ""; - for (Map.Entry sortKey : provider.getSortKeys().entrySet()) { + for (Map.Entry sortKey : sortKeys.entrySet()) { builder.append(prefix); prefix = ", "; @@ -270,13 +283,15 @@ public static String buildSortClause(AbstractSqlPagingQueryProvider provider) { /** * Appends the where conditions required to query for the subsequent pages. * - * @param provider - * @param sql + * @param provider the {@link AbstractSqlPagingQueryProvider} to be used for + * pagination. + * @param sql {@link StringBuilder} containing the sql to be used for the + * query. */ public static void buildSortConditions( AbstractSqlPagingQueryProvider provider, StringBuilder sql) { - List> keys = new ArrayList>(provider.getSortKeys().entrySet()); - List clauses = new ArrayList(); + List> keys = new ArrayList<>(provider.getSortKeys().entrySet()); + List clauses = new ArrayList<>(); for(int i = 0; i < keys.size(); i++) { StringBuilder clause = new StringBuilder(); @@ -342,8 +357,9 @@ private static void buildWhereClause(AbstractSqlPagingQueryProvider provider, bo if (remainingPageQuery) { sql.append(" WHERE "); if (provider.getWhereClause() != null) { + sql.append("("); sql.append(provider.getWhereClause()); - sql.append(" AND "); + sql.append(") AND "); } buildSortConditions(provider, sql); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlServerPagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlServerPagingQueryProvider.java index f3cbec5862..5913f2652c 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlServerPagingQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlServerPagingQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,7 +19,7 @@ import org.springframework.util.StringUtils; /** - * Sql Server implementation of a + * SQL Server implementation of a * {@link org.springframework.batch.item.database.PagingQueryProvider} using * database specific features. * diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlWindowingPagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlWindowingPagingQueryProvider.java index 476f3b41ad..56e5dc549e 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlWindowingPagingQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlWindowingPagingQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,6 +16,7 @@ package org.springframework.batch.item.database.support; +import java.util.LinkedHashMap; import java.util.Map; import org.springframework.batch.item.database.Order; @@ -102,7 +103,7 @@ public String generateJumpToItemQuery(int itemIndex, int pageSize) { StringBuilder sql = new StringBuilder(); sql.append("SELECT "); - buildSortKeySelect(sql); + buildSortKeySelect(sql, getSortKeysReplaced(extractTableAlias())); sql.append(" FROM ( "); sql.append("SELECT "); buildSortKeySelect(sql); @@ -115,14 +116,30 @@ public String generateJumpToItemQuery(int itemIndex, int pageSize) { sql.append(getOverSubstituteClauseEnd()); sql.append(") ").append(getSubQueryAlias()).append("WHERE ").append(extractTableAlias()).append( "ROW_NUMBER = ").append(lastRowNum); - sql.append(" ORDER BY ").append(SqlPagingQueryUtils.buildSortClause(this)); + sql.append(" ORDER BY ").append(SqlPagingQueryUtils.buildSortClause(getSortKeysReplaced(extractTableAlias()))); return sql.toString(); } + private Map getSortKeysReplaced(Object qualifierReplacement) { + final String newQualifier = "" + qualifierReplacement; + final Map sortKeys = new LinkedHashMap<>(); + for (Map.Entry sortKey : getSortKeys().entrySet()) { + sortKeys.put(sortKey.getKey().replaceFirst("^.*\\.", newQualifier), sortKey.getValue()); + } + return sortKeys; + } + private void buildSortKeySelect(StringBuilder sql) { + buildSortKeySelect(sql, null); + } + + private void buildSortKeySelect(StringBuilder sql, Map sortKeys) { String prefix = ""; - for (Map.Entry sortKey : getSortKeys().entrySet()) { + if (sortKeys == null) { + sortKeys = getSortKeys(); + } + for (Map.Entry sortKey : sortKeys.entrySet()) { sql.append(prefix); prefix = ", "; sql.append(sortKey.getKey()); @@ -132,7 +149,7 @@ private void buildSortKeySelect(StringBuilder sql) { protected String getOverClause() { StringBuilder sql = new StringBuilder(); - sql.append(" ORDER BY ").append(SqlPagingQueryUtils.buildSortClause(this)); + sql.append(" ORDER BY ").append(buildSortClause(this)); return sql.toString(); } @@ -144,4 +161,16 @@ protected String getOverSubstituteClauseStart() { protected String getOverSubstituteClauseEnd() { return ""; } + + + /** + * Generates ORDER BY attributes based on the sort keys. + * + * @param provider + * @return a String that can be appended to an ORDER BY clause. + */ + private String buildSortClause(AbstractSqlPagingQueryProvider provider) { + return SqlPagingQueryUtils.buildSortClause(provider.getSortKeysWithoutAliases()); + } + } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqliteMaxValueIncrementer.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqliteMaxValueIncrementer.java new file mode 100644 index 0000000000..5e00046f62 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqliteMaxValueIncrementer.java @@ -0,0 +1,70 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.support; + +import java.sql.Connection; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.sql.Statement; + +import javax.sql.DataSource; + +import org.springframework.dao.DataAccessResourceFailureException; +import org.springframework.jdbc.datasource.DataSourceUtils; +import org.springframework.jdbc.support.JdbcUtils; +import org.springframework.jdbc.support.incrementer.AbstractColumnMaxValueIncrementer; + +/** + * Implemented as a package-private class since it is required for SQLite support, but + * should ideally be in Spring JDBC. + * + * @author Luke Taylor + * @since 3.0 + */ +class SqliteMaxValueIncrementer extends AbstractColumnMaxValueIncrementer { + + public SqliteMaxValueIncrementer(DataSource dataSource, String incrementerName, String columnName) { + super(dataSource, incrementerName, columnName); + } + + /* (non-Javadoc) + * @see org.springframework.jdbc.support.incrementer.AbstractDataFieldMaxValueIncrementer#getNextKey() + */ + @Override + protected long getNextKey() { + Connection con = DataSourceUtils.getConnection(getDataSource()); + Statement stmt = null; + try { + stmt = con.createStatement(); + DataSourceUtils.applyTransactionTimeout(stmt, getDataSource()); + stmt.executeUpdate("insert into " + getIncrementerName() + " values(null)"); + ResultSet rs = stmt.executeQuery("select max(rowid) from " + getIncrementerName()); + if (!rs.next()) { + throw new DataAccessResourceFailureException("rowid query failed after executing an update"); + } + long nextKey = rs.getLong(1); + stmt.executeUpdate("delete from " + getIncrementerName() + " where " + getColumnName() + " < " + nextKey); + return nextKey; + } + catch (SQLException ex) { + throw new DataAccessResourceFailureException("Could not obtain rowid", ex); + } + finally { + JdbcUtils.closeStatement(stmt); + DataSourceUtils.releaseConnection(con, getDataSource()); + } + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlitePagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlitePagingQueryProvider.java new file mode 100644 index 0000000000..0ffc58d64f --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SqlitePagingQueryProvider.java @@ -0,0 +1,67 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.database.support; + +import org.springframework.util.StringUtils; + +/** + * SQLite implementation of a {@link org.springframework.batch.item.database.PagingQueryProvider} using database specific + * features. + * + * @author Luke Taylor + * @since 3.0.0 + */ +public class SqlitePagingQueryProvider extends AbstractSqlPagingQueryProvider { + /* (non-Javadoc) + * @see org.springframework.batch.item.database.support.AbstractSqlPagingQueryProvider#generateFirstPageQuery(int) + */ + @Override + public String generateFirstPageQuery(int pageSize) { + return SqlPagingQueryUtils.generateLimitSqlQuery(this, false, buildLimitClause(pageSize)); + } + + /* (non-Javadoc) + * @see org.springframework.batch.item.database.support.AbstractSqlPagingQueryProvider#generateRemainingPagesQuery(int) + */ + @Override + public String generateRemainingPagesQuery(int pageSize) { + if(StringUtils.hasText(getGroupClause())) { + return SqlPagingQueryUtils.generateLimitGroupedSqlQuery(this, true, buildLimitClause(pageSize)); + } + else { + return SqlPagingQueryUtils.generateLimitSqlQuery(this, true, buildLimitClause(pageSize)); + } + } + + /* (non-Javadoc) + * @see org.springframework.batch.item.database.support.AbstractSqlPagingQueryProvider#generateJumpToItemQuery(int, int) + */ + @Override + public String generateJumpToItemQuery(int itemIndex, int pageSize) { + int page = itemIndex / pageSize; + int offset = (page * pageSize) - 1; + offset = offset<0 ? 0 : offset; + + String limitClause = new StringBuilder().append("LIMIT ").append(offset).append(", 1").toString(); + return SqlPagingQueryUtils.generateLimitJumpToQuery(this, limitClause); + } + + private String buildLimitClause(int pageSize) { + return new StringBuilder().append("LIMIT ").append(pageSize).toString(); + } +} + diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SybasePagingQueryProvider.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SybasePagingQueryProvider.java index 0615e5c86b..88e00e9a3e 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SybasePagingQueryProvider.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/SybasePagingQueryProvider.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/package-info.java new file mode 100644 index 0000000000..67d58af49e --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/database/support/package-info.java @@ -0,0 +1,10 @@ +/** + * Support classes for database specific semantics. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.database.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/BufferedReaderFactory.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/BufferedReaderFactory.java index 194a6aec34..f8d3aef955 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/BufferedReaderFactory.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/BufferedReaderFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/DefaultBufferedReaderFactory.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/DefaultBufferedReaderFactory.java index f32940121d..ad63a099e5 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/DefaultBufferedReaderFactory.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/DefaultBufferedReaderFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileFooterCallback.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileFooterCallback.java index 5c55838660..e0694c44e9 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileFooterCallback.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileFooterCallback.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,10 @@ public interface FlatFileFooterCallback { /** * Write contents to a file using the supplied {@link Writer}. It is not * required to flush the writer inside this method. + * + * @param writer the {@link Writer} to be used to write the footer. + * + * @throws IOException if error occurs during writing. */ void writeFooter(Writer writer) throws IOException; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileHeaderCallback.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileHeaderCallback.java index b4374cdbf4..941b6a4d8f 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileHeaderCallback.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileHeaderCallback.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,15 +20,20 @@ import java.io.IOException; /** - * Callback interface for writing to a header to a file. + * Callback interface for writing a header to a file. * * @author Robert Kasanicky + * @author Mahmoud Ben Hassine */ public interface FlatFileHeaderCallback { /** * Write contents to a file using the supplied {@link Writer}. It is not * required to flush the writer inside this method. + * + * @param writer the {@link Writer} to be used to write the header. + * + * @throws IOException if error occurs during writing. */ void writeHeader(Writer writer) throws IOException; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileItemReader.java index cb3308cf55..1d1cf84636 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ import org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader; import org.springframework.beans.factory.InitializingBean; import org.springframework.core.io.Resource; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.ClassUtils; import org.springframework.util.StringUtils; @@ -122,7 +123,7 @@ public void setEncoding(String encoding) { /** * Factory for the {@link BufferedReader} that will be used to extract lines from the file. The default is fine for - * plain text files, but this is a useful strategy for binary files where the standard BufferedReaader from java.io + * plain text files, but this is a useful strategy for binary files where the standard BufferedReader from java.io * is limiting. * * @param bufferedReaderFactory the bufferedReaderFactory to set @@ -164,6 +165,7 @@ public void setRecordSeparatorPolicy(RecordSeparatorPolicy recordSeparatorPolicy * @return string corresponding to logical record according to * {@link #setRecordSeparatorPolicy(RecordSeparatorPolicy)} (might span multiple lines in file). */ + @Nullable @Override protected T doRead() throws Exception { if (noInput) { @@ -189,6 +191,7 @@ protected T doRead() throws Exception { /** * @return next line (skip comments).getCurrentResource */ + @Nullable private String readLine() { if (reader == null) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileItemWriter.java index 2664c102cb..2443908427 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileItemWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,28 +16,10 @@ package org.springframework.batch.item.file; -import java.io.BufferedWriter; -import java.io.File; -import java.io.FileOutputStream; -import java.io.IOException; -import java.io.Writer; -import java.nio.channels.Channels; -import java.nio.channels.FileChannel; -import java.nio.charset.UnsupportedCharsetException; import java.util.List; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.item.ItemStream; -import org.springframework.batch.item.ItemStreamException; -import org.springframework.batch.item.WriteFailedException; -import org.springframework.batch.item.WriterNotOpenException; import org.springframework.batch.item.file.transform.LineAggregator; -import org.springframework.batch.item.support.AbstractItemStreamItemWriter; -import org.springframework.batch.item.util.FileUtils; -import org.springframework.batch.support.transaction.TransactionAwareBufferedWriter; -import org.springframework.beans.factory.InitializingBean; +import org.springframework.batch.item.support.AbstractFileItemWriter; import org.springframework.core.io.Resource; import org.springframework.util.Assert; import org.springframework.util.ClassUtils; @@ -45,56 +27,22 @@ /** * This class is an item writer that writes data to a file or stream. The writer * also provides restart. The location of the output file is defined by a - * {@link Resource} and must represent a writable file.
      + * {@link Resource} and must represent a writable file.
      * - * Uses buffered writer to improve performance.
      + * Uses buffered writer to improve performance.
      * - * The implementation is *not* thread-safe. + * The implementation is not thread-safe. * * @author Waseem Malik * @author Tomas Slanina * @author Robert Kasanicky * @author Dave Syer * @author Michael Minella + * @author Mahmoud Ben Hassine */ -public class FlatFileItemWriter extends AbstractItemStreamItemWriter implements ResourceAwareItemWriterItemStream, -InitializingBean { +public class FlatFileItemWriter extends AbstractFileItemWriter { - private static final boolean DEFAULT_TRANSACTIONAL = true; - - protected static final Log logger = LogFactory.getLog(FlatFileItemWriter.class); - - private static final String DEFAULT_LINE_SEPARATOR = System.getProperty("line.separator"); - - private static final String WRITTEN_STATISTICS_NAME = "written"; - - private static final String RESTART_DATA_NAME = "current.count"; - - private Resource resource; - - private OutputState state = null; - - private LineAggregator lineAggregator; - - private boolean saveState = true; - - private boolean forceSync = false; - - private boolean shouldDeleteIfExists = true; - - private boolean shouldDeleteIfEmpty = false; - - private String encoding = OutputState.DEFAULT_CHARSET; - - private FlatFileHeaderCallback headerCallback; - - private FlatFileFooterCallback footerCallback; - - private String lineSeparator = DEFAULT_LINE_SEPARATOR; - - private boolean transactional = DEFAULT_TRANSACTIONAL; - - private boolean append = false; + protected LineAggregator lineAggregator; public FlatFileItemWriter() { this.setExecutionContextName(ClassUtils.getShortName(FlatFileItemWriter.class)); @@ -113,28 +61,6 @@ public void afterPropertiesSet() throws Exception { } } - /** - * Flag to indicate that changes should be force-synced to disk on flush. - * Defaults to false, which means that even with a local disk changes could - * be lost if the OS crashes in between a write and a cache flush. Setting - * to true may result in slower performance for usage patterns involving many - * frequent writes. - * - * @param forceSync the flag value to set - */ - public void setForceSync(boolean forceSync) { - this.forceSync = forceSync; - } - - /** - * Public setter for the line separator. Defaults to the System property - * line.separator. - * @param lineSeparator the line separator to set - */ - public void setLineSeparator(String lineSeparator) { - this.lineSeparator = lineSeparator; - } - /** * Public setter for the {@link LineAggregator}. This will be used to * translate the item into a line for output. @@ -145,515 +71,13 @@ public void setLineAggregator(LineAggregator lineAggregator) { this.lineAggregator = lineAggregator; } - /** - * Setter for resource. Represents a file that can be written. - * - * @param resource - */ @Override - public void setResource(Resource resource) { - this.resource = resource; - } - - /** - * Sets encoding for output template. - */ - public void setEncoding(String newEncoding) { - this.encoding = newEncoding; - } - - /** - * Flag to indicate that the target file should be deleted if it already - * exists, otherwise it will be created. Defaults to true, so no appending - * except on restart. If set to false and {@link #setAppendAllowed(boolean) - * appendAllowed} is also false then there will be an exception when the - * stream is opened to prevent existing data being potentially corrupted. - * - * @param shouldDeleteIfExists the flag value to set - */ - public void setShouldDeleteIfExists(boolean shouldDeleteIfExists) { - this.shouldDeleteIfExists = shouldDeleteIfExists; - } - - /** - * Flag to indicate that the target file should be appended if it already - * exists. If this flag is set then the flag - * {@link #setShouldDeleteIfExists(boolean) shouldDeleteIfExists} is - * automatically set to false, so that flag should not be set explicitly. - * Defaults value is false. - * - * @param append the flag value to set - */ - public void setAppendAllowed(boolean append) { - this.append = append; - this.shouldDeleteIfExists = false; - } - - /** - * Flag to indicate that the target file should be deleted if no lines have - * been written (other than header and footer) on close. Defaults to false. - * - * @param shouldDeleteIfEmpty the flag value to set - */ - public void setShouldDeleteIfEmpty(boolean shouldDeleteIfEmpty) { - this.shouldDeleteIfEmpty = shouldDeleteIfEmpty; - } - - /** - * Set the flag indicating whether or not state should be saved in the - * provided {@link ExecutionContext} during the {@link ItemStream} call to - * update. Setting this to false means that it will always start at the - * beginning on a restart. - * - * @param saveState - */ - public void setSaveState(boolean saveState) { - this.saveState = saveState; - } - - /** - * headerCallback will be called before writing the first item to file. - * Newline will be automatically appended after the header is written. - */ - public void setHeaderCallback(FlatFileHeaderCallback headerCallback) { - this.headerCallback = headerCallback; - } - - /** - * footerCallback will be called after writing the last item to file, but - * before the file is closed. - */ - public void setFooterCallback(FlatFileFooterCallback footerCallback) { - this.footerCallback = footerCallback; - } - - /** - * Flag to indicate that writing to the buffer should be delayed if a - * transaction is active. Defaults to true. - */ - public void setTransactional(boolean transactional) { - this.transactional = transactional; - } - - /** - * Writes out a string followed by a "new line", where the format of the new - * line separator is determined by the underlying operating system. If the - * input is not a String and a converter is available the converter will be - * applied and then this method recursively called with the result. If the - * input is an array or collection each value will be written to a separate - * line (recursively calling this method for each value). If no converter is - * supplied the input object's toString method will be used.
      - * - * @param items list of items to be written to output stream - * @throws Exception if the transformer or file output fail, - * WriterNotOpenException if the writer has not been initialized. - */ - @Override - public void write(List items) throws Exception { - - if (!getOutputState().isInitialized()) { - throw new WriterNotOpenException("Writer must be open before it can be written to"); - } - - if (logger.isDebugEnabled()) { - logger.debug("Writing to flat file with " + items.size() + " items."); - } - - OutputState state = getOutputState(); - + public String doWrite(List items) { StringBuilder lines = new StringBuilder(); - int lineCount = 0; for (T item : items) { - lines.append(lineAggregator.aggregate(item) + lineSeparator); - lineCount++; - } - try { - state.write(lines.toString()); + lines.append(this.lineAggregator.aggregate(item)).append(this.lineSeparator); } - catch (IOException e) { - throw new WriteFailedException("Could not write data. The file may be corrupt.", e); - } - state.linesWritten += lineCount; - } - - /** - * @see ItemStream#close() - */ - @Override - public void close() { - super.close(); - if (state != null) { - try { - if (footerCallback != null && state.outputBufferedWriter != null) { - footerCallback.writeFooter(state.outputBufferedWriter); - state.outputBufferedWriter.flush(); - } - } - catch (IOException e) { - throw new ItemStreamException("Failed to write footer before closing", e); - } - finally { - state.close(); - if (state.linesWritten == 0 && shouldDeleteIfEmpty) { - try { - resource.getFile().delete(); - } - catch (IOException e) { - throw new ItemStreamException("Failed to delete empty file on close", e); - } - } - state = null; - } - } - } - - /** - * Initialize the reader. This method may be called multiple times before - * close is called. - * - * @see ItemStream#open(ExecutionContext) - */ - @Override - public void open(ExecutionContext executionContext) throws ItemStreamException { - super.open(executionContext); - - Assert.notNull(resource, "The resource must be set"); - - if (!getOutputState().isInitialized()) { - doOpen(executionContext); - } - } - - private void doOpen(ExecutionContext executionContext) throws ItemStreamException { - OutputState outputState = getOutputState(); - if (executionContext.containsKey(getExecutionContextKey(RESTART_DATA_NAME))) { - outputState.restoreFrom(executionContext); - } - try { - outputState.initializeBufferedWriter(); - } - catch (IOException ioe) { - throw new ItemStreamException("Failed to initialize writer", ioe); - } - if (outputState.lastMarkedByteOffsetPosition == 0 && !outputState.appending) { - if (headerCallback != null) { - try { - headerCallback.writeHeader(outputState.outputBufferedWriter); - outputState.write(lineSeparator); - } - catch (IOException e) { - throw new ItemStreamException("Could not write headers. The file may be corrupt.", e); - } - } - } - } - - /** - * @see ItemStream#update(ExecutionContext) - */ - @Override - public void update(ExecutionContext executionContext) { - super.update(executionContext); - if (state == null) { - throw new ItemStreamException("ItemStream not open or already closed."); - } - - Assert.notNull(executionContext, "ExecutionContext must not be null"); - - if (saveState) { - - try { - executionContext.putLong(getExecutionContextKey(RESTART_DATA_NAME), state.position()); - } - catch (IOException e) { - throw new ItemStreamException("ItemStream does not return current position properly", e); - } - - executionContext.putLong(getExecutionContextKey(WRITTEN_STATISTICS_NAME), state.linesWritten); - } - } - - // Returns object representing state. - private OutputState getOutputState() { - if (state == null) { - File file; - try { - file = resource.getFile(); - } - catch (IOException e) { - throw new ItemStreamException("Could not convert resource to file: [" + resource + "]", e); - } - Assert.state(!file.exists() || file.canWrite(), "Resource is not writable: [" + resource + "]"); - state = new OutputState(); - state.setDeleteIfExists(shouldDeleteIfExists); - state.setAppendAllowed(append); - state.setEncoding(encoding); - } - return state; - } - - /** - * The name of the component which will be used as a stem for keys in the - * {@link ExecutionContext}. Subclasses should provide a default value, e.g. - * the short form of the class name. - * - * @param name the name for the component - */ - public void setName(String name) { - this.setExecutionContextName(name); - } - - /** - * Encapsulates the runtime state of the writer. All state changing - * operations on the writer go through this class. - */ - private class OutputState { - // default encoding for writing to output files - set to UTF-8. - private static final String DEFAULT_CHARSET = "UTF-8"; - - private FileOutputStream os; - - // The bufferedWriter over the file channel that is actually written - Writer outputBufferedWriter; - - FileChannel fileChannel; - - // this represents the charset encoding (if any is needed) for the - // output file - String encoding = DEFAULT_CHARSET; - - boolean restarted = false; - - long lastMarkedByteOffsetPosition = 0; - - long linesWritten = 0; - - boolean shouldDeleteIfExists = true; - - boolean initialized = false; - - private boolean append = false; - - private boolean appending = false; - - /** - * Return the byte offset position of the cursor in the output file as a - * long integer. - */ - public long position() throws IOException { - long pos = 0; - - if (fileChannel == null) { - return 0; - } - - outputBufferedWriter.flush(); - pos = fileChannel.position(); - if (transactional) { - pos += ((TransactionAwareBufferedWriter) outputBufferedWriter).getBufferSize(); - } - - return pos; - - } - - /** - * @param append - */ - public void setAppendAllowed(boolean append) { - this.append = append; - } - - /** - * @param executionContext - */ - public void restoreFrom(ExecutionContext executionContext) { - lastMarkedByteOffsetPosition = executionContext.getLong(getExecutionContextKey(RESTART_DATA_NAME)); - linesWritten = executionContext.getLong(getExecutionContextKey(WRITTEN_STATISTICS_NAME)); - if (shouldDeleteIfEmpty && linesWritten == 0) { - // previous execution deleted the output file because no items were written - restarted = false; - lastMarkedByteOffsetPosition = 0; - } else { - restarted = true; - } - } - - /** - * @param shouldDeleteIfExists - */ - public void setDeleteIfExists(boolean shouldDeleteIfExists) { - this.shouldDeleteIfExists = shouldDeleteIfExists; - } - - /** - * @param encoding - */ - public void setEncoding(String encoding) { - this.encoding = encoding; - } - - /** - * Close the open resource and reset counters. - */ - public void close() { - - initialized = false; - restarted = false; - try { - if (outputBufferedWriter != null) { - outputBufferedWriter.close(); - } - } - catch (IOException ioe) { - throw new ItemStreamException("Unable to close the the ItemWriter", ioe); - } - finally { - if (!transactional) { - closeStream(); - } - } - } - - private void closeStream() { - try { - if (fileChannel != null) { - fileChannel.close(); - } - } - catch (IOException ioe) { - throw new ItemStreamException("Unable to close the the ItemWriter", ioe); - } - finally { - try { - if (os != null) { - os.close(); - } - } - catch (IOException ioe) { - throw new ItemStreamException("Unable to close the the ItemWriter", ioe); - } - } - } - - /** - * @param line - * @throws IOException - */ - public void write(String line) throws IOException { - if (!initialized) { - initializeBufferedWriter(); - } - - outputBufferedWriter.write(line); - outputBufferedWriter.flush(); - } - - /** - * Truncate the output at the last known good point. - * - * @throws IOException - */ - public void truncate() throws IOException { - fileChannel.truncate(lastMarkedByteOffsetPosition); - fileChannel.position(lastMarkedByteOffsetPosition); - } - - /** - * Creates the buffered writer for the output file channel based on - * configuration information. - * @throws IOException - */ - private void initializeBufferedWriter() throws IOException { - - File file = resource.getFile(); - FileUtils.setUpOutputFile(file, restarted, append, shouldDeleteIfExists); - - os = new FileOutputStream(file.getAbsolutePath(), true); - fileChannel = os.getChannel(); - - outputBufferedWriter = getBufferedWriter(fileChannel, encoding); - outputBufferedWriter.flush(); - - if (append) { - // Bug in IO library? This doesn't work... - // lastMarkedByteOffsetPosition = fileChannel.position(); - if (file.length() > 0) { - appending = true; - // Don't write the headers again - } - } - - Assert.state(outputBufferedWriter != null); - // in case of restarting reset position to last committed point - if (restarted) { - checkFileSize(); - truncate(); - } - - initialized = true; - } - - public boolean isInitialized() { - return initialized; - } - - /** - * Returns the buffered writer opened to the beginning of the file - * specified by the absolute path name contained in absoluteFileName. - */ - private Writer getBufferedWriter(FileChannel fileChannel, String encoding) { - try { - final FileChannel channel = fileChannel; - if (transactional) { - TransactionAwareBufferedWriter writer = new TransactionAwareBufferedWriter(channel, new Runnable() { - @Override - public void run() { - closeStream(); - } - }); - - writer.setEncoding(encoding); - writer.setForceSync(forceSync); - return writer; - } - else { - Writer writer = new BufferedWriter(Channels.newWriter(fileChannel, encoding)) { - @Override - public void flush() throws IOException { - super.flush(); - if (forceSync) { - channel.force(false); - } - } - }; - - return writer; - } - } - catch (UnsupportedCharsetException ucse) { - throw new ItemStreamException("Bad encoding configuration for output file " + fileChannel, ucse); - } - } - - /** - * Checks (on setState) to make sure that the current output file's size - * is not smaller than the last saved commit point. If it is, then the - * file has been damaged in some way and whole task must be started over - * again from the beginning. - * @throws IOException if there is an IO problem - */ - private void checkFileSize() throws IOException { - long size = -1; - - outputBufferedWriter.flush(); - size = fileChannel.size(); - - if (size < lastMarkedByteOffsetPosition) { - throw new ItemStreamException("Current file size is smaller than size at last commit"); - } - } - + return lines.toString(); } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileParseException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileParseException.java index 1533e95153..f3b2f65d38 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileParseException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/FlatFileParseException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,6 +27,7 @@ * @author Lucas Ward * @author Ben Hale */ +@SuppressWarnings("serial") public class FlatFileParseException extends ParseException { private String input; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/LineCallbackHandler.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/LineCallbackHandler.java index 1c3e34923a..f6f6a5170f 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/LineCallbackHandler.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/LineCallbackHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/LineMapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/LineMapper.java index 3c7a39d1e4..ddf0efe3f0 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/LineMapper.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/LineMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -42,7 +42,7 @@ public interface LineMapper { * @param line to be mapped * @param lineNumber of the current line * @return mapped object of type T - * @throws Exception if error occured while parsing. + * @throws Exception if error occurred while parsing. */ T mapLine(String line, int lineNumber) throws Exception; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/MultiResourceItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/MultiResourceItemReader.java index a45bf08880..df3d0ead4e 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/MultiResourceItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/MultiResourceItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ import org.springframework.batch.item.UnexpectedInputException; import org.springframework.batch.item.support.AbstractItemStreamItemReader; import org.springframework.core.io.Resource; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.ClassUtils; @@ -42,6 +43,7 @@ * * @author Robert Kasanicky * @author Lucas Ward + * @author Mahmoud Ben Hassine */ public class MultiResourceItemReader extends AbstractItemStreamItemReader { @@ -90,6 +92,7 @@ public MultiResourceItemReader() { /** * Reads the next item, jumping to next resource if necessary. */ + @Nullable @Override public T read() throws Exception, UnexpectedInputException, ParseException { @@ -150,7 +153,11 @@ private T readFromDelegate() throws Exception { @Override public void close() throws ItemStreamException { super.close(); - delegate.close(); + + if(!this.noInput) { + delegate.close(); + } + noInput = false; } @@ -217,7 +224,8 @@ public void setDelegate(ResourceAwareItemReaderItemStream delegate) * Set the boolean indicating whether or not state should be saved in the provided {@link ExecutionContext} during * the {@link ItemStream} call to update. * - * @param saveState + * @param saveState true to update ExecutionContext. False do not update + * ExecutionContext. */ public void setSaveState(boolean saveState) { this.saveState = saveState; @@ -239,6 +247,12 @@ public void setResources(Resource[] resources) { this.resources = Arrays.asList(resources).toArray(new Resource[resources.length]); } + /** + * Getter for the current resource. + * @return the current resource or {@code null} if all resources have been + * processed or the first resource has not been assigned yet. + */ + @Nullable public Resource getCurrentResource() { if (currentResource >= resources.length || currentResource < 0) { return null; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/MultiResourceItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/MultiResourceItemWriter.java index abf2a91377..0c32631a58 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/MultiResourceItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/MultiResourceItemWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -91,6 +91,8 @@ public void write(List items) throws Exception { /** * Allows customization of the suffix of the created resources based on the * index. + * + * @param suffixCreator {@link ResourceSuffixCreator} to be used by the writer. */ public void setResourceSuffixCreator(ResourceSuffixCreator suffixCreator) { this.suffixCreator = suffixCreator; @@ -99,6 +101,9 @@ public void setResourceSuffixCreator(ResourceSuffixCreator suffixCreator) { /** * After this limit is exceeded the next chunk will be written into newly * created resource. + * + * @param itemCountLimitPerResource int item threshold used to determine when a new + * resource should be created. */ public void setItemCountLimitPerResource(int itemCountLimitPerResource) { this.itemCountLimitPerResource = itemCountLimitPerResource; @@ -106,6 +111,9 @@ public void setItemCountLimitPerResource(int itemCountLimitPerResource) { /** * Delegate used for actual writing of the output. + * + * @param delegate {@link ResourceAwareItemWriterItemStream} that will be used + * to write the output. */ public void setDelegate(ResourceAwareItemWriterItemStream delegate) { this.delegate = delegate; @@ -116,18 +124,26 @@ public void setDelegate(ResourceAwareItemWriterItemStream delegate) { * the same directory and use the same name as this prototype with appended * suffix (according to * {@link #setResourceSuffixCreator(ResourceSuffixCreator)}. + * + * @param resource The prototype resource. */ public void setResource(Resource resource) { this.resource = resource; } + + /** + * Indicates that the state of the reader will be saved after each commit. + * + * @param saveState true the state is saved. + */ public void setSaveState(boolean saveState) { this.saveState = saveState; } @Override public void close() throws ItemStreamException { - super.close(); + super.close(); resourceIndex = 1; currentResourceItemCount = 0; if (opened) { @@ -137,7 +153,7 @@ public void close() throws ItemStreamException { @Override public void open(ExecutionContext executionContext) throws ItemStreamException { - super.open(executionContext); + super.open(executionContext); resourceIndex = executionContext.getInt(getExecutionContextKey(RESOURCE_INDEX_KEY), 1); currentResourceItemCount = executionContext.getInt(getExecutionContextKey(CURRENT_RESOURCE_ITEM_COUNT), 0); @@ -151,6 +167,7 @@ public void open(ExecutionContext executionContext) throws ItemStreamException { if (executionContext.containsKey(getExecutionContextKey(CURRENT_RESOURCE_ITEM_COUNT))) { // It's a restart delegate.open(executionContext); + opened = true; } else { opened = false; @@ -159,7 +176,7 @@ public void open(ExecutionContext executionContext) throws ItemStreamException { @Override public void update(ExecutionContext executionContext) throws ItemStreamException { - super.update(executionContext); + super.update(executionContext); if (saveState) { if (opened) { delegate.update(executionContext); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/NonTransientFlatFileException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/NonTransientFlatFileException.java index 7175f43d8b..9aa5b074bd 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/NonTransientFlatFileException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/NonTransientFlatFileException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ * * @author Dave Syer */ +@SuppressWarnings("serial") public class NonTransientFlatFileException extends NonTransientResourceException { private String input; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourceAwareItemReaderItemStream.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourceAwareItemReaderItemStream.java index 6f8fdbf0e7..3a4261ad60 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourceAwareItemReaderItemStream.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourceAwareItemReaderItemStream.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourceAwareItemWriterItemStream.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourceAwareItemWriterItemStream.java index 000fd873e4..da4296a458 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourceAwareItemWriterItemStream.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourceAwareItemWriterItemStream.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourceSuffixCreator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourceSuffixCreator.java index 97f01ad85c..d9581bf09c 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourceSuffixCreator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourceSuffixCreator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourcesItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourcesItemReader.java index f8dcbd7e96..30e9df74ca 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourcesItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/ResourcesItemReader.java @@ -1,14 +1,30 @@ +/* + * Copyright 2009-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file; -import java.util.Arrays; -import java.util.concurrent.atomic.AtomicInteger; - import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ItemStreamException; import org.springframework.batch.item.support.AbstractItemStreamItemReader; import org.springframework.core.io.Resource; import org.springframework.core.io.support.ResourceArrayPropertyEditor; +import org.springframework.lang.Nullable; + +import java.util.Arrays; +import java.util.concurrent.atomic.AtomicInteger; /** * {@link ItemReader} which produces {@link Resource} instances from an array. @@ -16,14 +32,15 @@ * pattern (e.g. mydir/*.txt, which can then be converted by Spring * to an array of Resources by the ApplicationContext. * - *
      - *
      + *
      + *
      * - * Thread safe between calls to {@link #open(ExecutionContext)}. The + * Thread-safe between calls to {@link #open(ExecutionContext)}. The * {@link ExecutionContext} is not accurate in a multi-threaded environment, so * do not rely on that data for restart (i.e. always open with a fresh context). * * @author Dave Syer + * @author Mahmoud Ben Hassine * * @see ResourceArrayPropertyEditor * @@ -53,9 +70,10 @@ public void setResources(Resource[] resources) { /** * Increments a counter and returns the next {@link Resource} instance from - * the input, or null if none remain. + * the input, or {@code null} if none remain. */ @Override + @Nullable public synchronized Resource read() throws Exception { int index = counter.incrementAndGet() - 1; if (index >= resources.length) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/SimpleBinaryBufferedReaderFactory.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/SimpleBinaryBufferedReaderFactory.java index 1f1b5b1dca..b6a747368d 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/SimpleBinaryBufferedReaderFactory.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/SimpleBinaryBufferedReaderFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -44,7 +44,7 @@ public class SimpleBinaryBufferedReaderFactory implements BufferedReaderFactory private String lineEnding = DEFAULT_LINE_ENDING; /** - * @param lineEnding + * @param lineEnding {@link String} indicating what defines the end of a "line". */ public void setLineEnding(String lineEnding) { this.lineEnding = lineEnding; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/SimpleResourceSuffixCreator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/SimpleResourceSuffixCreator.java index 01fe937b1a..74641814f5 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/SimpleResourceSuffixCreator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/SimpleResourceSuffixCreator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/FlatFileItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/FlatFileItemReaderBuilder.java new file mode 100644 index 0000000000..9f43f685aa --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/FlatFileItemReaderBuilder.java @@ -0,0 +1,803 @@ +/* + * Copyright 2016-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.file.builder; + +import java.beans.PropertyEditor; +import java.math.BigInteger; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashMap; +import java.util.HashSet; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import org.springframework.batch.item.file.BufferedReaderFactory; +import org.springframework.batch.item.file.DefaultBufferedReaderFactory; +import org.springframework.batch.item.file.FlatFileItemReader; +import org.springframework.batch.item.file.LineCallbackHandler; +import org.springframework.batch.item.file.LineMapper; +import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper; +import org.springframework.batch.item.file.mapping.DefaultLineMapper; +import org.springframework.batch.item.file.mapping.FieldSetMapper; +import org.springframework.batch.item.file.separator.RecordSeparatorPolicy; +import org.springframework.batch.item.file.separator.SimpleRecordSeparatorPolicy; +import org.springframework.batch.item.file.transform.DefaultFieldSetFactory; +import org.springframework.batch.item.file.transform.DelimitedLineTokenizer; +import org.springframework.batch.item.file.transform.FieldSetFactory; +import org.springframework.batch.item.file.transform.FixedLengthTokenizer; +import org.springframework.batch.item.file.transform.LineTokenizer; +import org.springframework.batch.item.file.transform.Range; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * A builder implementation for the {@link FlatFileItemReader}. + * + * @author Michael Minella + * @author Glenn Renfro + * @author Mahmoud Ben Hassine + * @author Drummond Dawson + * @since 4.0 + * @see FlatFileItemReader + */ +public class FlatFileItemReaderBuilder { + + protected Log logger = LogFactory.getLog(getClass()); + + private boolean strict = true; + + private String encoding = FlatFileItemReader.DEFAULT_CHARSET; + + private RecordSeparatorPolicy recordSeparatorPolicy = + new SimpleRecordSeparatorPolicy(); + + private BufferedReaderFactory bufferedReaderFactory = + new DefaultBufferedReaderFactory(); + + private Resource resource; + + private List comments = new ArrayList<>(); + + private int linesToSkip = 0; + + private LineCallbackHandler skippedLinesCallback; + + private LineMapper lineMapper; + + private FieldSetMapper fieldSetMapper; + + private LineTokenizer lineTokenizer; + + private DelimitedBuilder delimitedBuilder; + + private FixedLengthBuilder fixedLengthBuilder; + + private Class targetType; + + private String prototypeBeanName; + + private BeanFactory beanFactory; + + private Map, PropertyEditor> customEditors = new HashMap<>(); + + private int distanceLimit = 5; + + private boolean beanMapperStrict = true; + + private BigInteger tokenizerValidator = new BigInteger("0"); + + private boolean saveState = true; + + private String name; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public FlatFileItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public FlatFileItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public FlatFileItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public FlatFileItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * Add a string to the list of Strings that indicate commented lines. + * + * @param comment the string to define a commented line. + * @return The current instance of the builder. + * @see FlatFileItemReader#setComments(String[]) + */ + public FlatFileItemReaderBuilder addComment(String comment) { + this.comments.add(comment); + return this; + } + + /** + * An array of Strings that indicate lines that are comments (and therefore skipped by + * the reader. + * + * @param comments an array of strings to identify comments. + * @return The current instance of the builder. + * @see FlatFileItemReader#setComments(String[]) + */ + public FlatFileItemReaderBuilder comments(String... comments) { + this.comments.addAll(Arrays.asList(comments)); + return this; + } + + /** + * Configure a custom {@link RecordSeparatorPolicy} for the reader. + * + * @param policy custom policy + * @return The current instance of the builder. + * @see FlatFileItemReader#setRecordSeparatorPolicy(RecordSeparatorPolicy) + */ + public FlatFileItemReaderBuilder recordSeparatorPolicy(RecordSeparatorPolicy policy) { + this.recordSeparatorPolicy = policy; + return this; + } + + /** + * Configure a custom {@link BufferedReaderFactory} for the reader. + * + * @param factory custom factory + * @return The current instance of the builder. + * @see FlatFileItemReader#setBufferedReaderFactory(BufferedReaderFactory) + */ + public FlatFileItemReaderBuilder bufferedReaderFactory(BufferedReaderFactory factory) { + this.bufferedReaderFactory = factory; + return this; + } + + + /** + * The {@link Resource} to be used as input. + * + * @param resource the input to the reader. + * @return The current instance of the builder. + * @see FlatFileItemReader#setResource(Resource) + */ + public FlatFileItemReaderBuilder resource(Resource resource) { + this.resource = resource; + return this; + } + + /** + * Configure if the reader should be in strict mode (require the input {@link Resource} + * to exist). + * + * @param strict true if the input file is required to exist. + * @return The current instance of the builder. + * @see FlatFileItemReader#setStrict(boolean) + */ + public FlatFileItemReaderBuilder strict(boolean strict) { + this.strict = strict; + return this; + } + + /** + * Configure the encoding used by the reader to read the input source. + * Default value is {@link FlatFileItemReader#DEFAULT_CHARSET}. + * + * @param encoding to use to read the input source. + * @return The current instance of the builder. + * @see FlatFileItemReader#setEncoding(String) + */ + public FlatFileItemReaderBuilder encoding(String encoding) { + this.encoding = encoding; + return this; + } + + /** + * The number of lines to skip at the beginning of reading the file. + * + * @param linesToSkip number of lines to be skipped. + * @return The current instance of the builder. + * @see FlatFileItemReader#setLinesToSkip(int) + */ + public FlatFileItemReaderBuilder linesToSkip(int linesToSkip) { + this.linesToSkip = linesToSkip; + return this; + } + + /** + * A callback to be called for each line that is skipped. + * + * @param callback the callback + * @return The current instance of the builder. + * @see FlatFileItemReader#setSkippedLinesCallback(LineCallbackHandler) + */ + public FlatFileItemReaderBuilder skippedLinesCallback(LineCallbackHandler callback) { + this.skippedLinesCallback = callback; + return this; + } + + /** + * A {@link LineMapper} implementation to be used. + * + * @param lineMapper {@link LineMapper} + * @return The current instance of the builder. + * @see FlatFileItemReader#setLineMapper(LineMapper) + */ + public FlatFileItemReaderBuilder lineMapper(LineMapper lineMapper) { + this.lineMapper = lineMapper; + return this; + } + + /** + * A {@link FieldSetMapper} implementation to be used. + * + * @param mapper a {@link FieldSetMapper} + * @return The current instance of the builder. + * @see DefaultLineMapper#setFieldSetMapper(FieldSetMapper) + */ + public FlatFileItemReaderBuilder fieldSetMapper(FieldSetMapper mapper) { + this.fieldSetMapper = mapper; + return this; + } + + /** + * A {@link LineTokenizer} implementation to be used. + * + * @param tokenizer a {@link LineTokenizer} + * @return The current instance of the builder. + * @see DefaultLineMapper#setLineTokenizer(LineTokenizer) + */ + public FlatFileItemReaderBuilder lineTokenizer(LineTokenizer tokenizer) { + updateTokenizerValidation(tokenizer, 0); + + this.lineTokenizer = tokenizer; + return this; + } + + /** + * Returns an instance of a {@link DelimitedBuilder} for building a + * {@link DelimitedLineTokenizer}. The {@link DelimitedLineTokenizer} configured by + * this builder will only be used if one is not explicitly configured via + * {@link FlatFileItemReaderBuilder#lineTokenizer} + * + * @return a {@link DelimitedBuilder} + * + */ + public DelimitedBuilder delimited() { + this.delimitedBuilder = new DelimitedBuilder<>(this); + updateTokenizerValidation(this.delimitedBuilder, 1); + return this.delimitedBuilder; + } + + /** + * Returns an instance of a {@link FixedLengthBuilder} for building a + * {@link FixedLengthTokenizer}. The {@link FixedLengthTokenizer} configured by this + * builder will only be used if the {@link FlatFileItemReaderBuilder#lineTokenizer} + * has not been configured. + * + * @return a {@link FixedLengthBuilder} + */ + public FixedLengthBuilder fixedLength() { + this.fixedLengthBuilder = new FixedLengthBuilder<>(this); + updateTokenizerValidation(this.fixedLengthBuilder, 2); + return this.fixedLengthBuilder; + } + + /** + * The class that will represent the "item" to be returned from the reader. This + * class is used via the {@link BeanWrapperFieldSetMapper}. If more complex logic is + * required, providing your own {@link FieldSetMapper} via + * {@link FlatFileItemReaderBuilder#fieldSetMapper} is required. + * + * @param targetType The class to map to + * @return The current instance of the builder. + * @see BeanWrapperFieldSetMapper#setTargetType(Class) + */ + public FlatFileItemReaderBuilder targetType(Class targetType) { + this.targetType = targetType; + return this; + } + + /** + * Configures the id of a prototype scoped bean to be used as the item returned by the + * reader. + * + * @param prototypeBeanName the name of a prototype scoped bean + * @return The current instance of the builder. + * @see BeanWrapperFieldSetMapper#setPrototypeBeanName(String) + */ + public FlatFileItemReaderBuilder prototypeBeanName(String prototypeBeanName) { + this.prototypeBeanName = prototypeBeanName; + return this; + } + + /** + * Configures the {@link BeanFactory} used to create the beans that are returned as + * items. + * + * @param beanFactory a {@link BeanFactory} + * @return The current instance of the builder. + * @see BeanWrapperFieldSetMapper#setBeanFactory(BeanFactory) + */ + public FlatFileItemReaderBuilder beanFactory(BeanFactory beanFactory) { + this.beanFactory = beanFactory; + return this; + } + + /** + * Register custom type converters for beans being mapped. + * + * @param customEditors a {@link Map} of editors + * @return The current instance of the builder. + * @see BeanWrapperFieldSetMapper#setCustomEditors(Map) + */ + public FlatFileItemReaderBuilder customEditors(Map, PropertyEditor> customEditors) { + if(customEditors != null) { + this.customEditors.putAll(customEditors); + } + + return this; + } + + /** + * Configures the maximum tolerance between the actual spelling of a field's name and + * the property's name. + * + * @param distanceLimit distance limit to set + * @return The current instance of the builder. + * @see BeanWrapperFieldSetMapper#setDistanceLimit(int) + */ + public FlatFileItemReaderBuilder distanceLimit(int distanceLimit) { + this.distanceLimit = distanceLimit; + return this; + } + + /** + * If set to true, mapping will fail if the {@link org.springframework.batch.item.file.transform.FieldSet} + * contains fields that cannot be mapped to the bean. + * + * @param beanMapperStrict defaults to false + * @return The current instance of the builder. + * @see BeanWrapperFieldSetMapper#setStrict(boolean) + */ + public FlatFileItemReaderBuilder beanMapperStrict(boolean beanMapperStrict) { + this.beanMapperStrict = beanMapperStrict; + return this; + } + + /** + * Builds the {@link FlatFileItemReader}. + * + * @return a {@link FlatFileItemReader} + */ + public FlatFileItemReader build() { + if(this.saveState) { + Assert.state(StringUtils.hasText(this.name), + "A name is required when saveState is set to true."); + } + + if(this.resource == null) { + logger.debug("The resource is null. This is only a valid scenario when " + + "injecting it later as in when using the MultiResourceItemReader"); + } + + Assert.notNull(this.recordSeparatorPolicy, "A RecordSeparatorPolicy is required."); + Assert.notNull(this.bufferedReaderFactory, "A BufferedReaderFactory is required."); + int validatorValue = this.tokenizerValidator.intValue(); + + FlatFileItemReader reader = new FlatFileItemReader<>(); + + if(StringUtils.hasText(this.name)) { + reader.setName(this.name); + } + + if(StringUtils.hasText(this.encoding)) { + reader.setEncoding(this.encoding); + } + + reader.setResource(this.resource); + + if(this.lineMapper != null) { + reader.setLineMapper(this.lineMapper); + } + else { + Assert.state(validatorValue == 1 || validatorValue == 2 || validatorValue == 4, + "Only one LineTokenizer option may be configured"); + + DefaultLineMapper lineMapper = new DefaultLineMapper<>(); + + if(this.lineTokenizer != null && this.fieldSetMapper != null) { + lineMapper.setLineTokenizer(this.lineTokenizer); + } + else if(this.fixedLengthBuilder != null) { + lineMapper.setLineTokenizer(this.fixedLengthBuilder.build()); + } + else if(this.delimitedBuilder != null) { + lineMapper.setLineTokenizer(this.delimitedBuilder.build()); + } + else { + throw new IllegalStateException("No LineTokenizer implementation was provided."); + } + + if(this.targetType != null || StringUtils.hasText(this.prototypeBeanName)) { + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + mapper.setTargetType(this.targetType); + mapper.setPrototypeBeanName(this.prototypeBeanName); + mapper.setStrict(this.beanMapperStrict); + mapper.setBeanFactory(this.beanFactory); + mapper.setDistanceLimit(this.distanceLimit); + mapper.setCustomEditors(this.customEditors); + try { + mapper.afterPropertiesSet(); + } + catch (Exception e) { + throw new IllegalStateException("Unable to initialize BeanWrapperFieldSetMapper", e); + } + + lineMapper.setFieldSetMapper(mapper); + } + else if(this.fieldSetMapper != null) { + lineMapper.setFieldSetMapper(this.fieldSetMapper); + } + else { + throw new IllegalStateException("No FieldSetMapper implementation was provided."); + } + + reader.setLineMapper(lineMapper); + } + + reader.setLinesToSkip(this.linesToSkip); + + if(!this.comments.isEmpty()) { + reader.setComments(this.comments.toArray(new String[this.comments.size()])); + } + + reader.setSkippedLinesCallback(this.skippedLinesCallback); + reader.setRecordSeparatorPolicy(this.recordSeparatorPolicy); + reader.setBufferedReaderFactory(this.bufferedReaderFactory); + reader.setMaxItemCount(this.maxItemCount); + reader.setCurrentItemCount(this.currentItemCount); + reader.setSaveState(this.saveState); + reader.setStrict(this.strict); + + return reader; + } + + private void updateTokenizerValidation(Object tokenizer, int index) { + if(tokenizer != null) { + this.tokenizerValidator = this.tokenizerValidator.flipBit(index); + } + else { + this.tokenizerValidator = this.tokenizerValidator.clearBit(index); + } + } + + /** + * A builder for constructing a {@link DelimitedLineTokenizer} + * + * @param the type of the parent {@link FlatFileItemReaderBuilder} + */ + public static class DelimitedBuilder { + private FlatFileItemReaderBuilder parent; + + private List names = new ArrayList<>(); + + private String delimiter; + + private Character quoteCharacter; + + private List includedFields = new ArrayList<>(); + + private FieldSetFactory fieldSetFactory = new DefaultFieldSetFactory(); + + private boolean strict = true; + + protected DelimitedBuilder(FlatFileItemReaderBuilder parent) { + this.parent = parent; + } + + /** + * Define the delimiter for the file. + * + * @param delimiter String used as a delimiter between fields. + * @return The instance of the builder for chaining. + * @see DelimitedLineTokenizer#setDelimiter(String) + */ + public DelimitedBuilder delimiter(String delimiter) { + this.delimiter = delimiter; + return this; + } + + /** + * Define the character used to quote fields. + * + * @param quoteCharacter char used to define quoted fields + * @return The instance of the builder for chaining. + * @see DelimitedLineTokenizer#setQuoteCharacter(char) + */ + public DelimitedBuilder quoteCharacter(char quoteCharacter) { + this.quoteCharacter = quoteCharacter; + return this; + } + + /** + * A list of indices of the fields within a delimited file to be included + * + * @param fields indices of the fields + * @return The instance of the builder for chaining. + * @see DelimitedLineTokenizer#setIncludedFields(int[]) + */ + public DelimitedBuilder includedFields(Integer... fields) { + this.includedFields.addAll(Arrays.asList(fields)); + return this; + } + + /** + * Add an index to the list of fields to be included from the file + * + * @param field the index to be included + * @return The instance of the builder for chaining. + * @see DelimitedLineTokenizer#setIncludedFields(int[]) + */ + public DelimitedBuilder addIncludedField(int field) { + this.includedFields.add(field); + return this; + } + + /** + * A factory for creating the resulting + * {@link org.springframework.batch.item.file.transform.FieldSet}. Defaults to + * {@link DefaultFieldSetFactory}. + * + * @param fieldSetFactory Factory for creating {@link org.springframework.batch.item.file.transform.FieldSet} + * @return The instance of the builder for chaining. + * @see DelimitedLineTokenizer#setFieldSetFactory(FieldSetFactory) + */ + public DelimitedBuilder fieldSetFactory(FieldSetFactory fieldSetFactory) { + this.fieldSetFactory = fieldSetFactory; + return this; + } + + /** + * Names of each of the fields within the fields that are returned in the order + * they occur within the delimited file. Required. + * + * @param names names of each field + * @return The parent {@link FlatFileItemReaderBuilder} + * @see DelimitedLineTokenizer#setNames(String[]) + */ + public FlatFileItemReaderBuilder names(String... names) { + this.names.addAll(Arrays.asList(names)); + return this.parent; + } + + /** + * Returns a {@link DelimitedLineTokenizer} + * + * @return {@link DelimitedLineTokenizer} + */ + public DelimitedLineTokenizer build() { + Assert.notNull(this.fieldSetFactory, "A FieldSetFactory is required."); + Assert.notEmpty(this.names, "A list of field names is required"); + + DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer(); + + tokenizer.setNames(this.names.toArray(new String[this.names.size()])); + + if(StringUtils.hasLength(this.delimiter)) { + tokenizer.setDelimiter(this.delimiter); + } + + if(this.quoteCharacter != null) { + tokenizer.setQuoteCharacter(this.quoteCharacter); + } + + if(!this.includedFields.isEmpty()) { + Set deDupedFields = new HashSet<>(this.includedFields.size()); + deDupedFields.addAll(this.includedFields); + deDupedFields.remove(null); + + int [] fields = new int[deDupedFields.size()]; + Iterator iterator = deDupedFields.iterator(); + for(int i = 0; i < fields.length; i++) { + fields[i] = iterator.next(); + } + + tokenizer.setIncludedFields(fields); + } + + tokenizer.setFieldSetFactory(this.fieldSetFactory); + tokenizer.setStrict(this.strict); + + try { + tokenizer.afterPropertiesSet(); + } + catch (Exception e) { + throw new IllegalStateException("Unable to initialize DelimitedLineTokenizer", e); + } + + return tokenizer; + } + } + + /** + * A builder for constructing a {@link FixedLengthTokenizer} + * + * @param the type of the parent {@link FlatFileItemReaderBuilder} + */ + public static class FixedLengthBuilder { + private FlatFileItemReaderBuilder parent; + + private List ranges = new ArrayList<>(); + + private List names = new ArrayList<>(); + + private boolean strict = true; + + private FieldSetFactory fieldSetFactory = new DefaultFieldSetFactory(); + + protected FixedLengthBuilder(FlatFileItemReaderBuilder parent) { + this.parent = parent; + } + + /** + * The column ranges for each field + * + * @param ranges column ranges + * @return This instance for chaining + * @see FixedLengthTokenizer#setColumns(Range[]) + */ + public FixedLengthBuilder columns(Range... ranges) { + this.ranges.addAll(Arrays.asList(ranges)); + return this; + } + + /** + * Add a column range to the existing list + * + * @param range a new column range + * @return This instance for chaining + * @see FixedLengthTokenizer#setColumns(Range[]) + */ + public FixedLengthBuilder addColumns(Range range) { + this.ranges.add(range); + return this; + } + + /** + * Insert a column range to the existing list + * + * @param range a new column range + * @param index index to add it at + * @return This instance for chaining + * @see FixedLengthTokenizer#setColumns(Range[]) + */ + public FixedLengthBuilder addColumns(Range range, int index) { + this.ranges.add(index, range); + return this; + } + + /** + * The names of the fields to be parsed from the file. Required. + * + * @param names names of fields + * @return The parent builder + * @see FixedLengthTokenizer#setNames(String[]) + */ + public FlatFileItemReaderBuilder names(String... names) { + this.names.addAll(Arrays.asList(names)); + return this.parent; + } + + /** + * Boolean indicating if the number of tokens in a line must match the number of + * fields (ranges) configured. Defaults to true. + * + * @param strict defaults to true + * @return This instance for chaining + * @see FixedLengthTokenizer#setStrict(boolean) + */ + public FixedLengthBuilder strict(boolean strict) { + this.strict = strict; + return this; + } + + /** + * A factory for creating the resulting + * {@link org.springframework.batch.item.file.transform.FieldSet}. Defaults to + * {@link DefaultFieldSetFactory}. + * @param fieldSetFactory Factory for creating {@link org.springframework.batch.item.file.transform.FieldSet} + * @return The instance of the builder for chaining. + * @see FixedLengthTokenizer#setFieldSetFactory(FieldSetFactory) + */ + public FixedLengthBuilder fieldSetFactory(FieldSetFactory fieldSetFactory) { + this.fieldSetFactory = fieldSetFactory; + return this; + } + + /** + * Returns a {@link FixedLengthTokenizer} + * + * @return a {@link FixedLengthTokenizer} + */ + public FixedLengthTokenizer build() { + Assert.notNull(this.fieldSetFactory, "A FieldSetFactory is required."); + Assert.notEmpty(this.names, "A list of field names is required."); + Assert.notEmpty(this.ranges, "A list of column ranges is required."); + + FixedLengthTokenizer tokenizer = new FixedLengthTokenizer(); + + tokenizer.setNames(this.names.toArray(new String[this.names.size()])); + tokenizer.setColumns(this.ranges.toArray(new Range[this.ranges.size()])); + tokenizer.setFieldSetFactory(this.fieldSetFactory); + tokenizer.setStrict(this.strict); + + return tokenizer; + } + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/FlatFileItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/FlatFileItemWriterBuilder.java new file mode 100644 index 0000000000..43e3006876 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/FlatFileItemWriterBuilder.java @@ -0,0 +1,526 @@ +/* + * Copyright 2016-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.file.builder; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.Locale; + +import org.springframework.batch.item.file.FlatFileFooterCallback; +import org.springframework.batch.item.file.FlatFileHeaderCallback; +import org.springframework.batch.item.file.FlatFileItemWriter; +import org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor; +import org.springframework.batch.item.file.transform.DelimitedLineAggregator; +import org.springframework.batch.item.file.transform.FieldExtractor; +import org.springframework.batch.item.file.transform.FormatterLineAggregator; +import org.springframework.batch.item.file.transform.LineAggregator; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * A builder implementation for the {@link FlatFileItemWriter} + * + * @author Michael Minella + * @author Glenn Renfro + * @author Mahmoud Ben Hassine + * @author Drummond Dawson + * @since 4.0 + * @see FlatFileItemWriter + */ +public class FlatFileItemWriterBuilder { + + private Resource resource; + + private boolean forceSync = false; + + private String lineSeparator = FlatFileItemWriter.DEFAULT_LINE_SEPARATOR; + + private LineAggregator lineAggregator; + + private String encoding = FlatFileItemWriter.DEFAULT_CHARSET; + + private boolean shouldDeleteIfExists = true; + + private boolean append = false; + + private boolean shouldDeleteIfEmpty = false; + + private FlatFileHeaderCallback headerCallback; + + private FlatFileFooterCallback footerCallback; + + private boolean transactional = FlatFileItemWriter.DEFAULT_TRANSACTIONAL; + + private boolean saveState = true; + + private String name; + + private DelimitedBuilder delimitedBuilder; + + private FormattedBuilder formattedBuilder; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public FlatFileItemWriterBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public FlatFileItemWriterBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * The {@link Resource} to be used as output. + * + * @param resource the output of the writer. + * @return The current instance of the builder. + * @see FlatFileItemWriter#setResource(Resource) + */ + public FlatFileItemWriterBuilder resource(Resource resource) { + this.resource = resource; + + return this; + } + + /** + * A flag indicating that changes should be force-synced to disk on flush. Defaults + * to false. + * + * @param forceSync value to set the flag to + * @return The current instance of the builder. + * @see FlatFileItemWriter#setForceSync(boolean) + */ + public FlatFileItemWriterBuilder forceSync(boolean forceSync) { + this.forceSync = forceSync; + + return this; + } + + /** + * String used to separate lines in output. Defaults to the System property + * line.separator. + * + * @param lineSeparator value to use for a line separator + * @return The current instance of the builder. + * @see FlatFileItemWriter#setLineSeparator(String) + */ + public FlatFileItemWriterBuilder lineSeparator(String lineSeparator) { + this.lineSeparator = lineSeparator; + + return this; + } + + /** + * Line aggregator used to build the String version of each item. + * + * @param lineAggregator {@link LineAggregator} implementation + * @return The current instance of the builder. + * @see FlatFileItemWriter#setLineAggregator(LineAggregator) + */ + public FlatFileItemWriterBuilder lineAggregator(LineAggregator lineAggregator) { + this.lineAggregator = lineAggregator; + + return this; + } + + /** + * Encoding used for output. + * + * @param encoding encoding type. + * @return The current instance of the builder. + * @see FlatFileItemWriter#setEncoding(String) + */ + public FlatFileItemWriterBuilder encoding(String encoding) { + this.encoding = encoding; + + return this; + } + + /** + * If set to true, once the step is complete, if the resource previously provided is + * empty, it will be deleted. + * + * @param shouldDelete defaults to false + * @return The current instance of the builder + * @see FlatFileItemWriter#setShouldDeleteIfEmpty(boolean) + */ + public FlatFileItemWriterBuilder shouldDeleteIfEmpty(boolean shouldDelete) { + this.shouldDeleteIfEmpty = shouldDelete; + + return this; + } + + /** + * If set to true, upon the start of the step, if the resource already exists, it will + * be deleted and recreated. + * + * @param shouldDelete defaults to true + * @return The current instance of the builder + * @see FlatFileItemWriter#setShouldDeleteIfExists(boolean) + */ + public FlatFileItemWriterBuilder shouldDeleteIfExists(boolean shouldDelete) { + this.shouldDeleteIfExists = shouldDelete; + + return this; + } + + /** + * If set to true and the file exists, the output will be appended to the existing + * file. + * + * @param append defaults to false + * @return The current instance of the builder + * @see FlatFileItemWriter#setAppendAllowed(boolean) + */ + public FlatFileItemWriterBuilder append(boolean append) { + this.append = append; + + return this; + } + + /** + * A callback for header processing. + * + * @param callback {@link FlatFileHeaderCallback} impl + * @return The current instance of the builder + * @see FlatFileItemWriter#setHeaderCallback(FlatFileHeaderCallback) + */ + public FlatFileItemWriterBuilder headerCallback(FlatFileHeaderCallback callback) { + this.headerCallback = callback; + + return this; + } + + /** + * A callback for footer processing + * @param callback {@link FlatFileFooterCallback} impl + * @return The current instance of the builder + * @see FlatFileItemWriter#setFooterCallback(FlatFileFooterCallback) + */ + public FlatFileItemWriterBuilder footerCallback(FlatFileFooterCallback callback) { + this.footerCallback = callback; + + return this; + } + + /** + * If set to true, the flushing of the buffer is delayed while a transaction is active. + * + * @param transactional defaults to true + * @return The current instance of the builder + * @see FlatFileItemWriter#setTransactional(boolean) + */ + public FlatFileItemWriterBuilder transactional(boolean transactional) { + this.transactional = transactional; + + return this; + } + + /** + * Returns an instance of a {@link DelimitedBuilder} for building a + * {@link DelimitedLineAggregator}. The {@link DelimitedLineAggregator} configured by + * this builder will only be used if one is not explicitly configured via + * {@link FlatFileItemWriterBuilder#lineAggregator} + * + * @return a {@link DelimitedBuilder} + * + */ + public DelimitedBuilder delimited() { + this.delimitedBuilder = new DelimitedBuilder<>(this); + return this.delimitedBuilder; + } + + /** + * Returns an instance of a {@link FormattedBuilder} for building a + * {@link FormatterLineAggregator}. The {@link FormatterLineAggregator} configured by + * this builder will only be used if one is not explicitly configured via + * {@link FlatFileItemWriterBuilder#lineAggregator} + * + * @return a {@link FormattedBuilder} + * + */ + public FormattedBuilder formatted() { + this.formattedBuilder = new FormattedBuilder<>(this); + return this.formattedBuilder; + } + + /** + * A builder for constructing a {@link FormatterLineAggregator}. + * + * @param the type of the parent {@link FlatFileItemWriterBuilder} + */ + public static class FormattedBuilder { + + private FlatFileItemWriterBuilder parent; + + private String format; + + private Locale locale = Locale.getDefault(); + + private int maximumLength = 0; + + private int minimumLength = 0; + + private FieldExtractor fieldExtractor; + + private List names = new ArrayList<>(); + + protected FormattedBuilder(FlatFileItemWriterBuilder parent) { + this.parent = parent; + } + + /** + * Set the format string used to aggregate items + * @param format used to aggregate items + * @return The instance of the builder for chaining. + */ + public FormattedBuilder format(String format) { + this.format = format; + return this; + } + + /** + * Set the locale. + * @param locale to use + * @return The instance of the builder for chaining. + */ + public FormattedBuilder locale(Locale locale) { + this.locale = locale; + return this; + } + + /** + * Set the minimum length of the formatted string. If this is not set + * the default is to allow any length. + * @param minimumLength of the formatted string + * @return The instance of the builder for chaining. + */ + public FormattedBuilder minimumLength(int minimumLength) { + this.minimumLength = minimumLength; + return this; + } + + /** + * Set the maximum length of the formatted string. If this is not set + * the default is to allow any length. + * @param maximumLength of the formatted string + * @return The instance of the builder for chaining. + */ + public FormattedBuilder maximumLength(int maximumLength) { + this.maximumLength = maximumLength; + return this; + } + + /** + * Set the {@link FieldExtractor} to use to extract fields from each item. + * @param fieldExtractor to use to extract fields from each item + * @return The current instance of the builder + */ + public FlatFileItemWriterBuilder fieldExtractor(FieldExtractor fieldExtractor) { + this.fieldExtractor = fieldExtractor; + return this.parent; + } + + /** + * Names of each of the fields within the fields that are returned in the order + * they occur within the formatted file. These names will be used to create + * a {@link BeanWrapperFieldExtractor} only if no explicit field extractor + * is set via {@link FormattedBuilder#fieldExtractor(FieldExtractor)}. + * + * @param names names of each field + * @return The parent {@link FlatFileItemWriterBuilder} + * @see BeanWrapperFieldExtractor#setNames(String[]) + */ + public FlatFileItemWriterBuilder names(String... names) { + this.names.addAll(Arrays.asList(names)); + return this.parent; + } + + public FormatterLineAggregator build() { + Assert.notNull(this.format, "A format is required"); + Assert.isTrue((this.names != null && !this.names.isEmpty()) || this.fieldExtractor != null, + "A list of field names or a field extractor is required"); + + FormatterLineAggregator formatterLineAggregator = new FormatterLineAggregator<>(); + formatterLineAggregator.setFormat(this.format); + formatterLineAggregator.setLocale(this.locale); + formatterLineAggregator.setMinimumLength(this.minimumLength); + formatterLineAggregator.setMaximumLength(this.maximumLength); + + if (this.fieldExtractor == null) { + BeanWrapperFieldExtractor beanWrapperFieldExtractor = new BeanWrapperFieldExtractor<>(); + beanWrapperFieldExtractor.setNames(this.names.toArray(new String[this.names.size()])); + try { + beanWrapperFieldExtractor.afterPropertiesSet(); + } + catch (Exception e) { + throw new IllegalStateException("Unable to initialize FormatterLineAggregator", e); + } + this.fieldExtractor = beanWrapperFieldExtractor; + } + + formatterLineAggregator.setFieldExtractor(this.fieldExtractor); + return formatterLineAggregator; + } + } + + /** + * A builder for constructing a {@link DelimitedLineAggregator} + * + * @param the type of the parent {@link FlatFileItemWriterBuilder} + */ + public static class DelimitedBuilder { + + private FlatFileItemWriterBuilder parent; + + private List names = new ArrayList<>(); + + private String delimiter = ","; + + private FieldExtractor fieldExtractor; + + protected DelimitedBuilder(FlatFileItemWriterBuilder parent) { + this.parent = parent; + } + + /** + * Define the delimiter for the file. + * + * @param delimiter String used as a delimiter between fields. + * @return The instance of the builder for chaining. + * @see DelimitedLineAggregator#setDelimiter(String) + */ + public DelimitedBuilder delimiter(String delimiter) { + this.delimiter = delimiter; + return this; + } + + /** + * Names of each of the fields within the fields that are returned in the order + * they occur within the delimited file. These names will be used to create + * a {@link BeanWrapperFieldExtractor} only if no explicit field extractor + * is set via {@link DelimitedBuilder#fieldExtractor(FieldExtractor)}. + * + * @param names names of each field + * @return The parent {@link FlatFileItemWriterBuilder} + * @see BeanWrapperFieldExtractor#setNames(String[]) + */ + public FlatFileItemWriterBuilder names(String... names) { + this.names.addAll(Arrays.asList(names)); + return this.parent; + } + + /** + * Set the {@link FieldExtractor} to use to extract fields from each item. + * @param fieldExtractor to use to extract fields from each item + * @return The parent {@link FlatFileItemWriterBuilder} + */ + public FlatFileItemWriterBuilder fieldExtractor(FieldExtractor fieldExtractor) { + this.fieldExtractor = fieldExtractor; + return this.parent; + } + + public DelimitedLineAggregator build() { + Assert.isTrue((this.names != null && !this.names.isEmpty()) || this.fieldExtractor != null, + "A list of field names or a field extractor is required"); + + DelimitedLineAggregator delimitedLineAggregator = new DelimitedLineAggregator<>(); + if (StringUtils.hasLength(this.delimiter)) { + delimitedLineAggregator.setDelimiter(this.delimiter); + } + + if (this.fieldExtractor == null) { + BeanWrapperFieldExtractor beanWrapperFieldExtractor = new BeanWrapperFieldExtractor<>(); + beanWrapperFieldExtractor.setNames(this.names.toArray(new String[this.names.size()])); + try { + beanWrapperFieldExtractor.afterPropertiesSet(); + } + catch (Exception e) { + throw new IllegalStateException("Unable to initialize DelimitedLineAggregator", e); + } + this.fieldExtractor = beanWrapperFieldExtractor; + } + + delimitedLineAggregator.setFieldExtractor(this.fieldExtractor); + return delimitedLineAggregator; + } + } + + /** + * Validates and builds a {@link FlatFileItemWriter}. + * + * @return a {@link FlatFileItemWriter} + */ + public FlatFileItemWriter build() { + + Assert.isTrue(this.lineAggregator != null || this.delimitedBuilder != null || this.formattedBuilder != null, + "A LineAggregator or a DelimitedBuilder or a FormattedBuilder is required"); + Assert.notNull(this.resource, "A Resource is required"); + + if(this.saveState) { + Assert.hasText(this.name, "A name is required when saveState is true"); + } + + FlatFileItemWriter writer = new FlatFileItemWriter<>(); + + writer.setName(this.name); + writer.setAppendAllowed(this.append); + writer.setEncoding(this.encoding); + writer.setFooterCallback(this.footerCallback); + writer.setForceSync(this.forceSync); + writer.setHeaderCallback(this.headerCallback); + if (this.lineAggregator == null) { + Assert.state(this.delimitedBuilder == null || this.formattedBuilder == null, + "Either a DelimitedLineAggregator or a FormatterLineAggregator should be provided, but not both"); + if (this.delimitedBuilder != null) { + this.lineAggregator = this.delimitedBuilder.build(); + } + else { + this.lineAggregator = this.formattedBuilder.build(); + } + } + writer.setLineAggregator(this.lineAggregator); + writer.setLineSeparator(this.lineSeparator); + writer.setResource(this.resource); + writer.setSaveState(this.saveState); + writer.setShouldDeleteIfEmpty(this.shouldDeleteIfEmpty); + writer.setShouldDeleteIfExists(this.shouldDeleteIfExists); + writer.setTransactional(this.transactional); + + return writer; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/MultiResourceItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/MultiResourceItemReaderBuilder.java new file mode 100644 index 0000000000..7dffefd264 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/MultiResourceItemReaderBuilder.java @@ -0,0 +1,164 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.file.builder; + +import java.util.Comparator; + +import org.springframework.batch.item.file.MultiResourceItemReader; +import org.springframework.batch.item.file.ResourceAwareItemReaderItemStream; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * A builder implementation for the {@link MultiResourceItemReader}. + * + * @author Glenn Renfro + * @author Drummond Dawson + * @since 4.0 + * @see MultiResourceItemReader + */ +public class MultiResourceItemReaderBuilder { + + private ResourceAwareItemReaderItemStream delegate; + + private Resource[] resources; + + private boolean strict = false; + + private Comparator comparator; + + private boolean saveState = true; + + private String name; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public MultiResourceItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public MultiResourceItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * The array of resources that the {@link MultiResourceItemReader} will use to + * retrieve items. + * + * @param resources the array of resources to use. + * @return this instance for method chaining. + * + * @see MultiResourceItemReader#setResources(Resource[]) + */ + public MultiResourceItemReaderBuilder resources(Resource... resources) { + this.resources = resources; + + return this; + } + + /** + * Establishes the delegate to use for reading the resources provided. + * + * @param delegate reads items from single {@link Resource}. + * @return this instance for method chaining. + * + * @see MultiResourceItemReader#setDelegate(ResourceAwareItemReaderItemStream) + */ + public MultiResourceItemReaderBuilder delegate(ResourceAwareItemReaderItemStream delegate) { + this.delegate = delegate; + + return this; + } + + /** + * In strict mode the reader will throw an exception on + * {@link MultiResourceItemReader#open(org.springframework.batch.item.ExecutionContext)} + * if there are no resources to read. + * + * @param strict false by default. + * @return this instance for method chaining. + * @see MultiResourceItemReader#setStrict(boolean) + */ + public MultiResourceItemReaderBuilder setStrict(boolean strict) { + this.strict = strict; + + return this; + } + + /** + * Used to order the injected resources, by default compares + * {@link Resource#getFilename()} values. + * + * @param comparator the comparator to use for ordering resources. + * @return this instance for method chaining. + * @see MultiResourceItemReader#setComparator(Comparator) + */ + public MultiResourceItemReaderBuilder comparator(Comparator comparator) { + this.comparator = comparator; + + return this; + } + + /** + * Builds the {@link MultiResourceItemReader}. + * + * @return a {@link MultiResourceItemReader} + */ + public MultiResourceItemReader build() { + Assert.notNull(this.resources, "resources array is required."); + Assert.notNull(this.delegate, "delegate is required."); + if (this.saveState) { + Assert.state(StringUtils.hasText(this.name), "A name is required when saveState is set to true."); + } + + MultiResourceItemReader reader = new MultiResourceItemReader<>(); + reader.setResources(this.resources); + reader.setDelegate(this.delegate); + reader.setSaveState(this.saveState); + reader.setStrict(this.strict); + + if (comparator != null) { + reader.setComparator(this.comparator); + } + if (StringUtils.hasText(this.name)) { + reader.setName(this.name); + } + + return reader; + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/MultiResourceItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/MultiResourceItemWriterBuilder.java new file mode 100644 index 0000000000..ebaccf770c --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/MultiResourceItemWriterBuilder.java @@ -0,0 +1,156 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.file.builder; + +import org.springframework.batch.item.file.MultiResourceItemWriter; +import org.springframework.batch.item.file.ResourceAwareItemWriterItemStream; +import org.springframework.batch.item.file.ResourceSuffixCreator; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; + +/** + * A builder implementation for the {@link MultiResourceItemWriter}. + * + * @author Glenn Renfro + * @author Glenn Renfro + * @since 4.0 + * @see MultiResourceItemWriter + */ +public class MultiResourceItemWriterBuilder { + + private Resource resource; + + private ResourceAwareItemWriterItemStream delegate; + + private int itemCountLimitPerResource = Integer.MAX_VALUE; + + private ResourceSuffixCreator suffixCreator; + + private boolean saveState = true; + + private String name; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public MultiResourceItemWriterBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public MultiResourceItemWriterBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Allows customization of the suffix of the created resources based on the index. + * + * @param suffixCreator the customizable ResourceSuffixCreator to use. + * @return The current instance of the builder. + * @see MultiResourceItemWriter#setResourceSuffixCreator(ResourceSuffixCreator) + */ + public MultiResourceItemWriterBuilder resourceSuffixCreator(ResourceSuffixCreator suffixCreator) { + this.suffixCreator = suffixCreator; + + return this; + } + + /** + * After this limit is exceeded the next chunk will be written into newly created + * resource. + * + * @param itemCountLimitPerResource the max numbers of items to be written per chunk. + * @return The current instance of the builder. + * @see MultiResourceItemWriter#setItemCountLimitPerResource(int) + */ + public MultiResourceItemWriterBuilder itemCountLimitPerResource(int itemCountLimitPerResource) { + this.itemCountLimitPerResource = itemCountLimitPerResource; + + return this; + } + + /** + * Delegate used for actual writing of the output. + * @param delegate The delegate to use for writing. + * @return The current instance of the builder. + * @see MultiResourceItemWriter#setDelegate(ResourceAwareItemWriterItemStream) + */ + public MultiResourceItemWriterBuilder delegate(ResourceAwareItemWriterItemStream delegate) { + this.delegate = delegate; + + return this; + } + + /** + * Prototype for output resources. Actual output files will be created in the same + * directory and use the same name as this prototype with appended suffix (according + * to {@link MultiResourceItemWriter#setResourceSuffixCreator(ResourceSuffixCreator)}. + * + * @param resource the prototype resource to use as the basis for creating resources. + * @return The current instance of the builder. + * @see MultiResourceItemWriter#setResource(Resource) + */ + public MultiResourceItemWriterBuilder resource(Resource resource) { + this.resource = resource; + + return this; + } + + /** + * Builds the {@link MultiResourceItemWriter}. + * + * @return a {@link MultiResourceItemWriter} + */ + public MultiResourceItemWriter build() { + Assert.notNull(this.resource, "resource is required."); + Assert.notNull(this.delegate, "delegate is required."); + + if(this.saveState) { + org.springframework.util.Assert.hasText(this.name, "A name is required when saveState is true."); + } + + MultiResourceItemWriter writer = new MultiResourceItemWriter<>(); + writer.setResource(this.resource); + writer.setDelegate(this.delegate); + writer.setItemCountLimitPerResource(this.itemCountLimitPerResource); + if(this.suffixCreator != null) { + writer.setResourceSuffixCreator(this.suffixCreator); + } + writer.setSaveState(this.saveState); + writer.setName(this.name); + + return writer; + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/package-info.java new file mode 100644 index 0000000000..6ceabb3b91 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/builder/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Builders for file item readers and writers. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.file.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/ArrayFieldSetMapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/ArrayFieldSetMapper.java index b089b0c0f9..6d6dbb38dc 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/ArrayFieldSetMapper.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/ArrayFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapper.java index 0525cd3922..25026e86b3 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapper.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapper.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,6 +36,8 @@ import org.springframework.beans.factory.BeanFactory; import org.springframework.beans.factory.BeanFactoryAware; import org.springframework.beans.factory.InitializingBean; +import org.springframework.beans.factory.config.CustomEditorConfigurer; +import org.springframework.core.convert.ConversionService; import org.springframework.util.Assert; import org.springframework.util.ReflectionUtils; import org.springframework.validation.BindException; @@ -45,26 +47,30 @@ * {@link FieldSetMapper} implementation based on bean property paths. The * {@link FieldSet} to be mapped should have field name meta data corresponding * to bean property paths in an instance of the desired type. The instance is - * created and initialized either by referring to to a prototype object by bean + * created and initialized either by referring to a prototype object by bean * name in the enclosing BeanFactory, or by providing a class to instantiate - * reflectively.
      - *
      + * reflectively.
      + *
      * * Nested property paths, including indexed properties in maps and collections, * can be referenced by the {@link FieldSet} names. They will be converted to * nested bean properties inside the prototype. The {@link FieldSet} and the * prototype are thus tightly coupled by the fields that are available and those * that can be initialized. If some of the nested properties are optional (e.g. - * collection members) they need to be removed by a post processor.
      - *
      + * collection members) they need to be removed by a post processor.
      + *
      * * To customize the way that {@link FieldSet} values are converted to the * desired type for injecting into the prototype there are several choices. You * can inject {@link PropertyEditor} instances directly through the * {@link #setCustomEditors(Map) customEditors} property, or you can override * the {@link #createBinder(Object)} and {@link #initBinder(DataBinder)} - * methods, or you can provide a custom {@link FieldSet} implementation.
      - *
      + * methods, or you can provide a custom {@link FieldSet} implementation. + * You can also use a {@link ConversionService} to convert to the desired type + * through the {@link #setConversionService(ConversionService) conversionService} + * property. + *
      + *
      * * Property name matching is "fuzzy" in the sense that it tolerates close * matches, as long as the match is unique. For instance: @@ -85,6 +91,7 @@ * match is found. If more than one match is found there will be an error. * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public class BeanWrapperFieldSetMapper extends DefaultPropertyEditorRegistrar implements FieldSetMapper, @@ -96,12 +103,16 @@ public class BeanWrapperFieldSetMapper extends DefaultPropertyEditorRegistrar private BeanFactory beanFactory; - private ConcurrentMap> propertiesMatched = new ConcurrentHashMap>(); + private ConcurrentMap> propertiesMatched = new ConcurrentHashMap<>(); private int distanceLimit = 5; private boolean strict = true; + private ConversionService conversionService; + + private boolean isCustomEditorsSet; + /* * (non-Javadoc) * @@ -143,7 +154,7 @@ public void setPrototypeBeanName(String name) { /** * Public setter for the type of bean to create instead of using a prototype * bean. An object of this type will be created from its default constructor - * for every call to {@link #mapFieldSet(FieldSet)}.
      + * for every call to {@link #mapFieldSet(FieldSet)}.
      * * Either this property or the prototype bean name must be specified, but * not both. @@ -166,6 +177,7 @@ public void setTargetType(Class type) { public void afterPropertiesSet() throws Exception { Assert.state(name != null || type != null, "Either name or type must be provided."); Assert.state(name == null || type == null, "Both name and type cannot be specified together."); + Assert.state(!this.isCustomEditorsSet || this.conversionService == null, "Both customEditor and conversionService cannot be specified together."); } /** @@ -198,7 +210,7 @@ public T mapFieldSet(FieldSet fs) throws BindException { * {@link #initBinder(DataBinder)} and * {@link #registerCustomEditors(PropertyEditorRegistry)}. * - * @param target + * @param target Object to bind to * @return a {@link DataBinder} that can be used to bind properties to the * target. */ @@ -207,6 +219,9 @@ protected DataBinder createBinder(Object target) { binder.setIgnoreUnknownFields(!this.strict); initBinder(binder); registerCustomEditors(binder); + if(this.conversionService != null) { + binder.setConversionService(this.conversionService); + } return binder; } @@ -232,10 +247,7 @@ private T getBean() { try { return type.newInstance(); } - catch (InstantiationException e) { - ReflectionUtils.handleReflectionException(e); - } - catch (IllegalAccessException e) { + catch (InstantiationException | IllegalAccessException e) { ReflectionUtils.handleReflectionException(e); } // should not happen @@ -243,22 +255,25 @@ private T getBean() { } /** - * @param bean - * @param properties - * @return + * @param bean Object to get properties for + * @param properties Properties to retrieve */ - @SuppressWarnings({ "unchecked", "rawtypes" }) private Properties getBeanProperties(Object bean, Properties properties) { + if (this.distanceLimit == 0) { + return properties; + } + Class cls = bean.getClass(); // Map from field names to property names DistanceHolder distanceKey = new DistanceHolder(cls, distanceLimit); if (!propertiesMatched.containsKey(distanceKey)) { - propertiesMatched.putIfAbsent(distanceKey, new ConcurrentHashMap()); + propertiesMatched.putIfAbsent(distanceKey, new ConcurrentHashMap<>()); } - Map matches = new HashMap(propertiesMatched.get(distanceKey)); + Map matches = new HashMap<>(propertiesMatched.get(distanceKey)); + @SuppressWarnings({ "unchecked", "rawtypes" }) Set keys = new HashSet(properties.keySet()); for (String key : keys) { @@ -285,7 +300,7 @@ private Properties getBeanProperties(Object bean, Properties properties) { } } - propertiesMatched.replace(distanceKey, new ConcurrentHashMap(matches)); + propertiesMatched.replace(distanceKey, new ConcurrentHashMap<>(matches)); return properties; } @@ -357,10 +372,7 @@ private Object getPropertyValue(Object bean, String nestedName) { nestedValue = wrapper.getPropertyType(nestedName).newInstance(); wrapper.setPropertyValue(nestedName, nestedValue); } - catch (InstantiationException e) { - ReflectionUtils.handleReflectionException(e); - } - catch (IllegalAccessException e) { + catch (InstantiationException | IllegalAccessException e) { ReflectionUtils.handleReflectionException(e); } } @@ -378,12 +390,37 @@ private void switchPropertyNames(Properties properties, String oldName, String n * {@link #mapFieldSet(FieldSet)} will fail of the FieldSet contains fields * that cannot be mapped to the bean. * - * @param strict + * @param strict indicator */ public void setStrict(boolean strict) { this.strict = strict; } + + /** + * Public setter for the 'conversionService' property. + * {@link #createBinder(Object)} will use it if not null. + * + * @param conversionService {@link ConversionService} to be used for type conversions + */ + public void setConversionService(ConversionService conversionService) { + this.conversionService = conversionService; + } + + /** + * Specify the {@link PropertyEditor custom editors} to register. + * + * + * @param customEditors a map of Class to PropertyEditor (or class name to + * PropertyEditor). + * @see CustomEditorConfigurer#setCustomEditors(Map) + */ + @Override + public void setCustomEditors(Map customEditors) { + this.isCustomEditorsSet = true; + super.setCustomEditors(customEditors); + } + private static class DistanceHolder { private final Class cls; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/DefaultLineMapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/DefaultLineMapper.java index 27aac6d5bd..27357ed1a6 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/DefaultLineMapper.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/DefaultLineMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/FieldSetMapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/FieldSetMapper.java index c8a570053e..ba3327ab28 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/FieldSetMapper.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/FieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,6 +35,7 @@ public interface FieldSetMapper { * Method used to map data obtained from a {@link FieldSet} into an object. * * @param fieldSet the {@link FieldSet} to map + * @return the populated object * @throws BindException if there is a problem with the binding */ T mapFieldSet(FieldSet fieldSet) throws BindException; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/JsonLineMapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/JsonLineMapper.java index 05d427a014..861c5f46db 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/JsonLineMapper.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/JsonLineMapper.java @@ -1,13 +1,29 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file.mapping; import java.util.Map; -import org.codehaus.jackson.JsonParser; -import org.codehaus.jackson.map.MappingJsonFactory; +import com.fasterxml.jackson.core.JsonParser; +import com.fasterxml.jackson.databind.MappingJsonFactory; + import org.springframework.batch.item.file.LineMapper; /** - * Interpret a line as a Json object and parse it up to a Map. The line should be a standard Json object, starting with + * Interpret a line as a JSON object and parse it up to a Map. The line should be a standard JSON object, starting with * "{" and ending with "}" and composed of name:value pairs separated by commas. Whitespace is ignored, * e.g. * @@ -15,7 +31,7 @@ * { "foo" : "bar", "value" : 123 } * * - * The values can also be Json objects (which are converted to maps): + * The values can also be JSON objects (which are converted to maps): * *
        * { "foo": "bar", "map": { "one": 1, "two": 2}}
      @@ -36,7 +52,7 @@ public class JsonLineMapper implements LineMapper> {
           @Override
       	public Map mapLine(String line, int lineNumber) throws Exception {
       		Map result;
      -		JsonParser parser = factory.createJsonParser(line);
      +		JsonParser parser = factory.createParser(line);
       		@SuppressWarnings("unchecked")
       		Map token = parser.readValueAs(Map.class);
       		result = token;
      diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PassThroughFieldSetMapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PassThroughFieldSetMapper.java
      index e90d1b918e..2542599181 100644
      --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PassThroughFieldSetMapper.java
      +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PassThroughFieldSetMapper.java
      @@ -5,7 +5,7 @@
        * you may not use this file except in compliance with the License.
        * You may obtain a copy of the License at
        *
      - *      http://www.apache.org/licenses/LICENSE-2.0
      + *      https://www.apache.org/licenses/LICENSE-2.0
        *
        * Unless required by applicable law or agreed to in writing, software
        * distributed under the License is distributed on an "AS IS" BASIS,
      diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PassThroughLineMapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PassThroughLineMapper.java
      index be7f2e71ec..1c3f99fdbf 100644
      --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PassThroughLineMapper.java
      +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PassThroughLineMapper.java
      @@ -5,7 +5,7 @@
        * you may not use this file except in compliance with the License.
        * You may obtain a copy of the License at
        *
      - *      http://www.apache.org/licenses/LICENSE-2.0
      + *      https://www.apache.org/licenses/LICENSE-2.0
        *
        * Unless required by applicable law or agreed to in writing, software
        * distributed under the License is distributed on an "AS IS" BASIS,
      diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapper.java
      index 428ba622d4..32ed9e1f12 100644
      --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapper.java
      +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapper.java
      @@ -5,7 +5,7 @@
        * you may not use this file except in compliance with the License.
        * You may obtain a copy of the License at
        *
      - *      http://www.apache.org/licenses/LICENSE-2.0
      + *      https://www.apache.org/licenses/LICENSE-2.0
        *
        * Unless required by applicable law or agreed to in writing, software
        * distributed under the License is distributed on an "AS IS" BASIS,
      @@ -71,7 +71,7 @@ public T mapLine(String line, int lineNumber) throws Exception {
           @Override
       	public void afterPropertiesSet() throws Exception {
       		this.tokenizer.afterPropertiesSet();
      -		Assert.isTrue(this.patternMatcher != null, "The 'fieldSetMappers' property must be non-empty");
      +		Assert.isTrue(this.patternMatcher != null, "The 'patternMatcher' property must be non-null");
       	}
       
       	public void setTokenizers(Map tokenizers) {
      @@ -80,6 +80,6 @@ public void setTokenizers(Map tokenizers) {
       
       	public void setFieldSetMappers(Map> fieldSetMappers) {
       		Assert.isTrue(!fieldSetMappers.isEmpty(), "The 'fieldSetMappers' property must be non-empty");
      -		this.patternMatcher = new PatternMatcher>(fieldSetMappers);
      +		this.patternMatcher = new PatternMatcher<>(fieldSetMappers);
       	}
       }
      diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PropertyMatches.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PropertyMatches.java
      index 6004b136a4..c47f700278 100644
      --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PropertyMatches.java
      +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/PropertyMatches.java
      @@ -5,7 +5,7 @@
        * you may not use this file except in compliance with the License.
        * You may obtain a copy of the License at
        *
      - *      http://www.apache.org/licenses/LICENSE-2.0
      + *      https://www.apache.org/licenses/LICENSE-2.0
        *
        * Unless required by applicable law or agreed to in writing, software
        * distributed under the License is distributed on an "AS IS" BASIS,
      @@ -16,18 +16,18 @@
       
       package org.springframework.batch.item.file.mapping;
       
      +import org.springframework.beans.BeanUtils;
      +import org.springframework.util.ObjectUtils;
      +import org.springframework.util.StringUtils;
      +
       import java.beans.PropertyDescriptor;
       import java.util.ArrayList;
       import java.util.Collections;
       import java.util.List;
       
      -import org.springframework.beans.BeanUtils;
      -import org.springframework.util.ObjectUtils;
      -import org.springframework.util.StringUtils;
      -
       /**
        * Helper class for calculating bean property matches, according to.
      - * Used by BeanWrapperImpl to suggest alternatives for an invalid property name.
      + * Used by BeanWrapperImpl to suggest alternatives for an invalid property name.
      * * Copied and slightly modified from Spring core, * @@ -99,7 +99,7 @@ public String[] getPossibleMatches() { * indicating the possible property matches. */ public String buildErrorMessage() { - StringBuffer buf = new StringBuffer(); + StringBuilder buf = new StringBuilder(); buf.append("Bean property '"); buf.append(this.propertyName); buf.append("' is not writable or has an invalid setter method. "); @@ -134,7 +134,7 @@ else if (i == this.possibleMatches.length - 2){ * @param maxDistance the maximum distance to accept */ private String[] calculateMatches(PropertyDescriptor[] propertyDescriptors, int maxDistance) { - List candidates = new ArrayList(); + List candidates = new ArrayList<>(); for (int i = 0; i < propertyDescriptors.length; i++) { if (propertyDescriptors[i].getWriteMethod() != null) { String possibleAlternative = propertyDescriptors[i].getName(); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/package-info.java new file mode 100644 index 0000000000..3405d55d18 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of io file support mapping concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.item.file.mapping; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/package.html deleted file mode 100644 index f6a1929c8a..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/mapping/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of io file support mapping concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/package-info.java new file mode 100644 index 0000000000..4365cd486d --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of io file concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.item.file; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/package.html deleted file mode 100644 index acc13f0c61..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of io file concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/DefaultRecordSeparatorPolicy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/DefaultRecordSeparatorPolicy.java index 92d6b07457..7f8d4cd5aa 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/DefaultRecordSeparatorPolicy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/DefaultRecordSeparatorPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -45,6 +45,8 @@ public DefaultRecordSeparatorPolicy() { /** * Convenient constructor with quote character as parameter. + * + * @param quoteCharacter value used to indicate a quoted string */ public DefaultRecordSeparatorPolicy(String quoteCharacter) { this(quoteCharacter, CONTINUATION); @@ -53,6 +55,9 @@ public DefaultRecordSeparatorPolicy(String quoteCharacter) { /** * Convenient constructor with quote character and continuation marker as * parameters. + * + * @param quoteCharacter value used to indicate a quoted string + * @param continuation value used to indicate a line continuation */ public DefaultRecordSeparatorPolicy(String quoteCharacter, String continuation) { super(); @@ -113,7 +118,7 @@ public String preProcess(String line) { * contains an unterminated quote, indicating that the record is continuing * onto the next line. * - * @param result + * @param line * @return */ private boolean isQuoteUnterminated(String line) { @@ -125,7 +130,7 @@ private boolean isQuoteUnterminated(String line) { * with the continuation marker, indicating that the record is continuing * onto the next line. * - * @param result + * @param line * @return */ private boolean isContinued(String line) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/JsonRecordSeparatorPolicy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/JsonRecordSeparatorPolicy.java index def28d8ed9..0d92b3b9cc 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/JsonRecordSeparatorPolicy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/JsonRecordSeparatorPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/RecordSeparatorPolicy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/RecordSeparatorPolicy.java index 7ac05621db..01320c5acf 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/RecordSeparatorPolicy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/RecordSeparatorPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/SimpleRecordSeparatorPolicy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/SimpleRecordSeparatorPolicy.java index d00674abd5..3e50164a49 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/SimpleRecordSeparatorPolicy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/SimpleRecordSeparatorPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/SuffixRecordSeparatorPolicy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/SuffixRecordSeparatorPolicy.java index b04b400c4b..5b7be98a09 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/SuffixRecordSeparatorPolicy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/SuffixRecordSeparatorPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -38,7 +38,7 @@ public class SuffixRecordSeparatorPolicy extends DefaultRecordSeparatorPolicy { /** * Lines ending in this terminator String signal the end of a record. * - * @param suffix + * @param suffix suffix to indicate the end of a record */ public void setSuffix(String suffix) { this.suffix = suffix; @@ -48,7 +48,7 @@ public void setSuffix(String suffix) { * Flag to indicate that the decision to terminate a record should ignore * whitespace at the end of the line. * - * @param ignoreWhitespace + * @param ignoreWhitespace indicator */ public void setIgnoreWhitespace(boolean ignoreWhitespace) { this.ignoreWhitespace = ignoreWhitespace; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/package-info.java new file mode 100644 index 0000000000..87f48e90e3 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of io file support separator concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.item.file.separator; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/package.html deleted file mode 100644 index 11f311cafe..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/separator/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of io file support separator concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/AbstractLineTokenizer.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/AbstractLineTokenizer.java index 5f58642c7d..394ef73c84 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/AbstractLineTokenizer.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/AbstractLineTokenizer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,9 @@ import java.util.Arrays; import java.util.List; +import org.springframework.lang.Nullable; +import org.springframework.util.StringUtils; + /** * Abstract class handling common concerns of various {@link LineTokenizer} * implementations such as dealing with names and actual construction of @@ -28,6 +31,7 @@ * @author Dave Syer * @author Robert Kasanicky * @author Lucas Ward + * @author Michael Minella */ public abstract class AbstractLineTokenizer implements LineTokenizer { @@ -76,10 +80,25 @@ public void setFieldSetFactory(FieldSetFactory fieldSetFactory) { * Setter for column names. Optional, but if set, then all lines must have * as many or fewer tokens. * - * @param names + * @param names names of each column */ - public void setNames(String[] names) { - this.names = names==null ? null : Arrays.asList(names).toArray(new String[names.length]); + public void setNames(String... names) { + if(names == null) { + this.names = null; + } + else { + boolean valid = false; + for (String name : names) { + if(StringUtils.hasText(name)) { + valid = true; + break; + } + } + + if(valid) { + this.names = Arrays.asList(names).toArray(new String[names.length]); + } + } } /** @@ -97,31 +116,31 @@ public boolean hasNames() { * Yields the tokens resulting from the splitting of the supplied * line. * - * @param line the line to be tokenised (can be null) + * @param line the line to be tokenized (can be null) * * @return the resulting tokens */ @Override - public FieldSet tokenize(String line) { + public FieldSet tokenize(@Nullable String line) { if (line == null) { line = ""; } - List tokens = new ArrayList(doTokenize(line)); + List tokens = new ArrayList<>(doTokenize(line)); // if names are set and strict flag is false if ( ( names.length != 0 ) && ( ! strict ) ) { adjustTokenCountIfNecessary( tokens ); } - String[] values = (String[]) tokens.toArray(new String[tokens.size()]); + String[] values = tokens.toArray(new String[tokens.size()]); if (names.length == 0) { return fieldSetFactory.create(values); } else if (values.length != names.length) { - throw new IncorrectTokenCountException(names.length, values.length); + throw new IncorrectTokenCountException(names.length, values.length, line); } return fieldSetFactory.create(values, names); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/Alignment.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/Alignment.java index 2230b7bf89..cb54f40f18 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/Alignment.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/Alignment.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,10 +29,6 @@ public enum Alignment { private String code; private String label; - /** - * @param code - * @param label - */ private Alignment(String code, String label) { Assert.notNull(code, "'code' must not be null"); @@ -40,7 +36,7 @@ private Alignment(String code, String label) { this.label = label; } - public Comparable getCode() { + public Comparable getCode() { return code; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/BeanWrapperFieldExtractor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/BeanWrapperFieldExtractor.java index 9151a5b5bf..d7d0822024 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/BeanWrapperFieldExtractor.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/BeanWrapperFieldExtractor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -50,7 +50,7 @@ public void setNames(String[] names) { */ @Override public Object[] extract(T item) { - List values = new ArrayList(); + List values = new ArrayList<>(); BeanWrapper bw = new BeanWrapperImpl(item); for (String propertyName : this.names) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/ConversionException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/ConversionException.java index 904d48bb1d..5bf9a16881 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/ConversionException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/ConversionException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,10 +19,11 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class ConversionException extends RuntimeException { /** - * @param msg + * @param msg the detail message. */ public ConversionException(String msg) { super(msg); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DefaultFieldSet.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DefaultFieldSet.java index d6d1af5ea5..e140e18b35 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DefaultFieldSet.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DefaultFieldSet.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -90,7 +90,7 @@ public void setDateFormat(DateFormat dateFormat) { * @see FieldSet#readString(int) */ public DefaultFieldSet(String[] tokens) { - this.tokens = tokens == null ? null : (String[]) tokens.clone(); + this.tokens = tokens == null ? null : tokens.clone(); setNumberFormat(NumberFormat.getInstance(Locale.US)); } @@ -102,13 +102,13 @@ public DefaultFieldSet(String[] tokens) { * @see FieldSet#readString(String) */ public DefaultFieldSet(String[] tokens, String[] names) { - Assert.notNull(tokens); - Assert.notNull(names); + Assert.notNull(tokens, "Tokens must not be null"); + Assert.notNull(names, "Names must not be null"); if (tokens.length != names.length) { throw new IllegalArgumentException("Field names must be same length as values: names=" + Arrays.asList(names) + ", values=" + Arrays.asList(tokens)); } - this.tokens = (String[]) tokens.clone(); + this.tokens = tokens.clone(); this.names = Arrays.asList(names); setNumberFormat(NumberFormat.getInstance(Locale.US)); } @@ -228,7 +228,7 @@ public boolean readBoolean(int index, String trueValue) { String value = readAndTrim(index); - return trueValue.equals(value) ? true : false; + return trueValue.equals(value); } /* @@ -437,7 +437,7 @@ public float readFloat(String name) { */ @Override public double readDouble(int index) { - return (Double) parseNumber(readAndTrim(index)).doubleValue(); + return parseNumber(readAndTrim(index)).doubleValue(); } /* @@ -541,12 +541,8 @@ public Date readDate(int index) { */ @Override public Date readDate(int index, Date defaultValue) { - try { - return readDate(index); - } - catch (IllegalArgumentException e) { - return defaultValue; - } + String candidate = readAndTrim(index); + return StringUtils.hasText(candidate) ? parseDate(candidate, dateFormat) : defaultValue; } /* @@ -575,10 +571,10 @@ public Date readDate(String name) { @Override public Date readDate(String name, Date defaultValue) { try { - return readDate(name); + return readDate(indexOf(name), defaultValue); } catch (IllegalArgumentException e) { - return defaultValue; + throw new IllegalArgumentException(e.getMessage() + ", name: [" + name + "]"); } } @@ -603,12 +599,8 @@ public Date readDate(int index, String pattern) { */ @Override public Date readDate(int index, String pattern, Date defaultValue) { - try { - return readDate(index, pattern); - } - catch (IllegalArgumentException e) { - return defaultValue; - } + String candidate = readAndTrim(index); + return StringUtils.hasText(candidate) ? readDate(index, pattern) : defaultValue; } /* @@ -637,10 +629,10 @@ public Date readDate(String name, String pattern) { @Override public Date readDate(String name, String pattern, Date defaultValue) { try { - return readDate(name, pattern); + return readDate(indexOf(name), pattern, defaultValue); } catch (IllegalArgumentException e) { - return defaultValue; + throw new IllegalArgumentException(e.getMessage() + ", name: [" + name + "]"); } } @@ -657,8 +649,10 @@ public int getFieldCount() { /** * Read and trim the {@link String} value at 'index'. + * + * @param index the offset in the token array to obtain the value to be trimmed. * - * @returns null if the field value is null. + * @return null if the field value is null. */ protected String readAndTrim(int index) { String value = tokens[index]; @@ -672,9 +666,12 @@ protected String readAndTrim(int index) { } /** - * Read and trim the {@link String} value from column with given ' - * name. - * + * Retrieve the index of where a specified column is located based on the + * {@code name} parameter. + * + * @param name the value to search in the {@link List} of names. + * @return the index in the {@link List} of names where the name was found. + * * @throws IllegalArgumentException if a column with given name is not * defined. */ @@ -726,8 +723,8 @@ public int hashCode() { int result = 1; - for (int i = 0; i < tokens.length; i++) { - result = 31 * result + (tokens[i] == null ? 0 : tokens[i].hashCode()); + for (String token : tokens) { + result = 31 * result + (token == null ? 0 : token.hashCode()); } return result; @@ -748,7 +745,7 @@ public Properties getProperties() { for (int i = 0; i < tokens.length; i++) { String value = readAndTrim(i); if (value != null) { - props.setProperty((String) names.get(i), value); + props.setProperty(names.get(i), value); } } return props; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DefaultFieldSetFactory.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DefaultFieldSetFactory.java index d1a01cea5a..5e723baba4 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DefaultFieldSetFactory.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DefaultFieldSetFactory.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file.transform; import java.text.DateFormat; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DelimitedLineAggregator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DelimitedLineAggregator.java index 4d6c1a1c18..6433ba6e79 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DelimitedLineAggregator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DelimitedLineAggregator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DelimitedLineTokenizer.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DelimitedLineTokenizer.java index 084822f0e8..b912416d74 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DelimitedLineTokenizer.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/DelimitedLineTokenizer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,7 @@ import java.util.HashSet; import java.util.List; +import org.springframework.beans.factory.InitializingBean; import org.springframework.util.Assert; import org.springframework.util.StringUtils; @@ -32,8 +33,10 @@ * @author Rob Harrop * @author Dave Syer * @author Michael Minella + * @author Olivier Bourgain */ -public class DelimitedLineTokenizer extends AbstractLineTokenizer { +public class DelimitedLineTokenizer extends AbstractLineTokenizer + implements InitializingBean { /** * Convenient constant for the common case of a tab delimiter. */ @@ -57,6 +60,8 @@ public class DelimitedLineTokenizer extends AbstractLineTokenizer { private String quoteString; + private String escapedQuoteString; + private Collection includedFields = null; /** @@ -73,9 +78,10 @@ public DelimitedLineTokenizer() { /** * Create a new instance of the {@link DelimitedLineTokenizer} class. * - * @param delimiter the desired delimiter + * @param delimiter the desired delimiter. This is required */ public DelimitedLineTokenizer(String delimiter) { + Assert.notNull(delimiter, "A delimiter is required"); Assert.state(!delimiter.equals(String.valueOf(DEFAULT_QUOTE_CHARACTER)), "[" + DEFAULT_QUOTE_CHARACTER + "] is not allowed as delimiter for tokenizers."); @@ -86,7 +92,7 @@ public DelimitedLineTokenizer(String delimiter) { /** * Setter for the delimiter character. * - * @param delimiter + * @param delimiter the String used as a delimiter */ public void setDelimiter(String delimiter) { this.delimiter = delimiter; @@ -100,8 +106,8 @@ public void setDelimiter(String delimiter) { * * @param includedFields the included fields to set */ - public void setIncludedFields(int[] includedFields) { - this.includedFields = new HashSet(); + public void setIncludedFields(int... includedFields) { + this.includedFields = new HashSet<>(); for (int i : includedFields) { this.includedFields.add(i); } @@ -120,6 +126,7 @@ public void setIncludedFields(int[] includedFields) { public void setQuoteCharacter(char quoteCharacter) { this.quoteCharacter = quoteCharacter; this.quoteString = "" + quoteCharacter; + this.escapedQuoteString = "" + quoteCharacter + quoteCharacter; } /** @@ -133,22 +140,21 @@ public void setQuoteCharacter(char quoteCharacter) { @Override protected List doTokenize(String line) { - List tokens = new ArrayList(); + List tokens = new ArrayList<>(); // line is never null in current implementation // line is checked in parent: AbstractLineTokenizer.tokenize() - char[] chars = line.toCharArray(); boolean inQuoted = false; int lastCut = 0; - int length = chars.length; + int length = line.length(); int fieldCount = 0; int endIndexLastDelimiter = -1; for (int i = 0; i < length; i++) { - char currentChar = chars[i]; + char currentChar = line.charAt(i); boolean isEnd = (i == (length - 1)); - boolean isDelimiter = isDelimiter(chars, i, delimiter, endIndexLastDelimiter); + boolean isDelimiter = endsWithDelimiter(line, i, endIndexLastDelimiter); if ((isDelimiter && !inQuoted) || isEnd) { endIndexLastDelimiter = i; @@ -162,7 +168,8 @@ else if (!isEnd){ } if (includedFields == null || includedFields.contains(fieldCount)) { - String value = maybeStripQuotes(new String(chars, lastCut, endPosition)); + String value = + substringWithTrimmedWhitespaceAndQuotesIfQuotesPresent(line, lastCut, endPosition); tokens.add(value); } @@ -186,65 +193,82 @@ else if (isQuoteCharacter(currentChar)) { return tokens; } - /** - * If the string is quoted strip (possibly with whitespace outside the - * quotes (which will be stripped), replace escaped quotes inside the - * string. Quotes are escaped with double instances of the quote character. - * - * @param string - * @return the same string but stripped and unescaped if necessary - */ - private String maybeStripQuotes(String string) { - String value = string.trim(); - if (isQuoted(value)) { - value = StringUtils.replace(value, "" + quoteCharacter + quoteCharacter, "" + quoteCharacter); - int endLength = value.length() - 1; - // used to deal with empty quoted values - if (endLength == 0) { - endLength = 1; - } - value = value.substring(1, endLength); - return value; - } - return string; - } - - /** - * Is this string surrounded by quote characters? - * - * @param value - * @return true if the value starts and ends with the - * {@link #quoteCharacter} - */ - private boolean isQuoted(String value) { - if (value.startsWith(quoteString) && value.endsWith(quoteString)) { - return true; - } - return false; - } - - /** - * Is the supplied character the delimiter character? - * - * @param c the character to be checked - * @return true if the supplied character is the delimiter - * character - * @see DelimitedLineTokenizer#DelimitedLineTokenizer(char) - */ - private boolean isDelimiter(char[] chars, int i, String token, int endIndexLastDelimiter) { - boolean result = false; - - if(i-endIndexLastDelimiter >= delimiter.length()) { - if(i >= token.length() - 1) { - String end = new String(chars, (i-token.length()) + 1, token.length()); - if(token.equals(end)) { - result = true; - } - } - } - - return result; - } + /** + * Trim any leading or trailing quotes (and any leading or trailing + * whitespace before or after the quotes) from within the specified character + * array beginning at the specified offset index for the specified count. + *

      + * Quotes are escaped with double instances of the quote character. + * + * @param line the string + * @param offset index from which to begin extracting substring + * @param count length of substring + * @return a substring from the specified offset within the character array + * with any leading or trailing whitespace trimmed. + * @see String#trim() + */ + private String substringWithTrimmedWhitespaceAndQuotesIfQuotesPresent(String line, int offset, int count) { + int start = offset; + int len = count; + + while ((start < (start + len - 1)) && (line.charAt(start) <= ' ')) { + start++; + len--; + } + + while ((start < (start + len)) && ((start + len - 1 < line.length()) && (line.charAt(start + len - 1) <= ' '))) { + len--; + } + + String value; + + if ((line.length() >= 2) && (line.charAt(start) == quoteCharacter) && (line.charAt(start + len - 1) == quoteCharacter)) { + int beginIndex = start + 1; + int endIndex = len - 2; + value = line.substring(beginIndex, beginIndex + endIndex); + if (value.contains(escapedQuoteString)) { + value = StringUtils.replace(value, escapedQuoteString, quoteString); + } + } + else { + value = line.substring(offset, offset + count); + } + + return value; + } + + /** + * Do the character(s) in the specified array end, at the specified end + * index, with the delimiter character(s)? + *

      + * Checks that the specified end index is sufficiently greater than the + * specified previous delimiter end index to warrant trying to match + * another delimiter. Also checks that the specified end index is + * sufficiently large to be able to match the length of a delimiter. + * + * @param line the string + * @param end the index in up to which the delimiter should be matched + * @param previous the index of the end of the last delimiter + * @return true if the character(s) from the specified end + * match the delimiter character(s), otherwise false + * @see DelimitedLineTokenizer#DelimitedLineTokenizer(String) + */ + private boolean endsWithDelimiter(String line, int end, int previous) { + boolean result = false; + + if (end - previous >= delimiter.length()) { + if (end >= delimiter.length() - 1) { + result = true; + for (int j = 0; j < delimiter.length() && (((end - delimiter.length() + 1) + j) < line.length()); j++) { + if (delimiter.charAt(j) != line.charAt((end - delimiter.length() + 1) + j)) { + result = false; + } + } + } + } + + return result; + } /** * Is the supplied character a quote character? @@ -256,4 +280,9 @@ private boolean isDelimiter(char[] chars, int i, String token, int endIndexLastD protected boolean isQuoteCharacter(char c) { return c == quoteCharacter; } + + @Override + public void afterPropertiesSet() throws Exception { + Assert.hasLength(this.delimiter, "A delimiter is required"); + } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/ExtractorLineAggregator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/ExtractorLineAggregator.java index 48838e5bb0..dfc0c9cbf1 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/ExtractorLineAggregator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/ExtractorLineAggregator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -28,7 +28,7 @@ */ public abstract class ExtractorLineAggregator implements LineAggregator { - private FieldExtractor fieldExtractor = new PassThroughFieldExtractor(); + private FieldExtractor fieldExtractor = new PassThroughFieldExtractor<>(); /** * Public setter for the field extractor responsible for splitting an input @@ -50,7 +50,7 @@ public void setFieldExtractor(FieldExtractor fieldExtractor) { */ @Override public String aggregate(T item) { - Assert.notNull(item); + Assert.notNull(item, "Item is required"); Object[] fields = this.fieldExtractor.extract(item); // diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FieldExtractor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FieldExtractor.java index f5c902b3ca..8a23934e90 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FieldExtractor.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FieldExtractor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,12 +19,13 @@ * This class will convert an object to an array of its parts. * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public interface FieldExtractor { /** - * @param item + * @param item the object that contains the information to be extracted. * @return an array containing item's parts */ Object[] extract(T item); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FieldSet.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FieldSet.java index 30fa3c07b4..4b57676fbb 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FieldSet.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FieldSet.java @@ -1,11 +1,11 @@ -/* + /* * Copyright 2006-2007 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -57,14 +57,17 @@ public interface FieldSet { * Read the {@link String} value at index 'index'. * * @param index the field index. - * @throws IndexOutOfBoundsException if the index is out of bounds. + * @return {@link String} containing the value at the index. + * + * @throws IndexOutOfBoundsException if the {@code index} is out of bounds. */ String readString(int index); /** * Read the {@link String} value from column with given 'name'. * - * @param name the field name. + * @param name the field {@code name}. + * @return {@link String} containing the value from the specified {@code name}. */ String readString(String name); @@ -73,7 +76,9 @@ public interface FieldSet { * trailing whitespace (don't trim). * * @param index the field index. - * @throws IndexOutOfBoundsException if the index is out of bounds. + * @return {@link String} containing the value from the specified {@code index}. + * + * @throws IndexOutOfBoundsException if the {@code index} is out of bounds. */ String readRawString(int index); @@ -81,7 +86,8 @@ public interface FieldSet { * Read the {@link String} value from column with given 'name' * including trailing whitespace (don't trim). * - * @param name the field name. + * @param name the field {@code name}. + * @return {@link String} containing the value from the specified {@code name}. */ String readRawString(String name); @@ -89,15 +95,19 @@ public interface FieldSet { * Read the 'boolean' value at index 'index'. * * @param index the field index. - * @throws IndexOutOfBoundsException if the index is out of bounds. + * @return boolean containing the value from the specified {@code index}. + * + * @throws IndexOutOfBoundsException if the {@code index} is out of bounds. */ boolean readBoolean(int index); /** * Read the 'boolean' value from column with given 'name'. * - * @param name the field name. - * @throws IllegalArgumentException if a column with given name is not + * @param name the field {@code name}. + * @return boolean containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined. */ boolean readBoolean(String name); @@ -108,6 +118,8 @@ public interface FieldSet { * @param index the field index. * @param trueValue the value that signifies {@link Boolean#TRUE true}; * case-sensitive. + * @return boolean containing the value from the specified {@code index}. + * * @throws IndexOutOfBoundsException if the index is out of bounds, or if * the supplied trueValue is null. */ @@ -116,10 +128,12 @@ public interface FieldSet { /** * Read the 'boolean' value from column with given 'name'. * - * @param name the field name. + * @param name the field {@code name}. * @param trueValue the value that signifies {@link Boolean#TRUE true}; * case-sensitive. - * @throws IllegalArgumentException if a column with given name is not + * @return boolean containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined, or if the supplied trueValue is null. */ boolean readBoolean(String name, String trueValue); @@ -128,6 +142,8 @@ public interface FieldSet { * Read the 'char' value at index 'index'. * * @param index the field index. + * @return char containing the value from the specified {@code index}. + * * @throws IndexOutOfBoundsException if the index is out of bounds. */ char readChar(int index); @@ -135,8 +151,10 @@ public interface FieldSet { /** * Read the 'char' value from column with given 'name'. * - * @param name the field name. - * @throws IllegalArgumentException if a column with given name is not + * @param name the field {@code name}. + * @return char containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined. */ char readChar(String name); @@ -145,6 +163,8 @@ public interface FieldSet { * Read the 'byte' value at index 'index'. * * @param index the field index. + * @return byte containing the value from the specified {@code index}. + * * @throws IndexOutOfBoundsException if the index is out of bounds. */ byte readByte(int index); @@ -152,14 +172,17 @@ public interface FieldSet { /** * Read the 'byte' value from column with given 'name'. * - * @param name the field name. + * @param name the field {@code name}. + * @return byte containing the value from the specified {@code name}. */ byte readByte(String name); /** * Read the 'short' value at index 'index'. * - * @param index the field index. + * @param index the field {@code index}. + * @return short containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. */ short readShort(int index); @@ -167,8 +190,10 @@ public interface FieldSet { /** * Read the 'short' value from column with given 'name'. * - * @param name the field name. - * @throws IllegalArgumentException if a column with given name is not + * @param name the field {@code name}. + * @return short containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined. */ short readShort(String name); @@ -177,6 +202,8 @@ public interface FieldSet { * Read the 'int' value at index 'index'. * * @param index the field index. + * @return int containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. */ int readInt(int index); @@ -184,8 +211,10 @@ public interface FieldSet { /** * Read the 'int' value from column with given 'name'. * - * @param name the field name. - * @throws IllegalArgumentException if a column with given name is not + * @param name the field {@code name}. + * @return int containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined. */ int readInt(String name); @@ -195,7 +224,10 @@ public interface FieldSet { * using the supplied defaultValue if the field value is * blank. * - * @param index the field index.. + * @param index the field index. + * @param defaultValue the value to use if the field value is blank. + * @return int containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. */ int readInt(int index, int defaultValue); @@ -205,8 +237,11 @@ public interface FieldSet { * using the supplied defaultValue if the field value is * blank. * - * @param name the field name. - * @throws IllegalArgumentException if a column with given name is not + * @param name the field {@code name}. + * @param defaultValue the value to use if the field value is blank. + * @return int containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined. */ int readInt(String name, int defaultValue); @@ -215,6 +250,8 @@ public interface FieldSet { * Read the 'long' value at index 'index'. * * @param index the field index. + * @return long containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. */ long readLong(int index); @@ -222,8 +259,10 @@ public interface FieldSet { /** * Read the 'long' value from column with given 'name'. * - * @param name the field name. - * @throws IllegalArgumentException if a column with given name is not + * @param name the field {@code name}. + * @return long containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined. */ long readLong(String name); @@ -233,7 +272,10 @@ public interface FieldSet { * using the supplied defaultValue if the field value is * blank. * - * @param index the field index.. + * @param index the field index. + * @param defaultValue the value to use if the field value is blank. + * @return long containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. */ long readLong(int index, long defaultValue); @@ -243,8 +285,11 @@ public interface FieldSet { * using the supplied defaultValue if the field value is * blank. * - * @param name the field name. - * @throws IllegalArgumentException if a column with given name is not + * @param name the field {@code name}. + * @param defaultValue the value to use if the field value is blank. + * @return long containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined. */ long readLong(String name, long defaultValue); @@ -253,6 +298,8 @@ public interface FieldSet { * Read the 'float' value at index 'index'. * * @param index the field index. + * @return float containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. */ float readFloat(int index); @@ -260,8 +307,10 @@ public interface FieldSet { /** * Read the 'float' value from column with given 'name. * - * @param name the field name. - * @throws IllegalArgumentException if a column with given name is not + * @param name the field {@code name}. + * @return float containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined. */ float readFloat(String name); @@ -270,6 +319,8 @@ public interface FieldSet { * Read the 'double' value at index 'index'. * * @param index the field index. + * @return double containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. */ double readDouble(int index); @@ -277,8 +328,10 @@ public interface FieldSet { /** * Read the 'double' value from column with given 'name. * - * @param name the field name. - * @throws IllegalArgumentException if a column with given name is not + * @param name the field {@code name}. + * @return double containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined. */ double readDouble(String name); @@ -287,6 +340,8 @@ public interface FieldSet { * Read the {@link java.math.BigDecimal} value at index 'index'. * * @param index the field index. + * @return {@link BigDecimal} containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. */ BigDecimal readBigDecimal(int index); @@ -294,8 +349,10 @@ public interface FieldSet { /** * Read the {@link java.math.BigDecimal} value from column with given 'name. * - * @param name the field name. - * @throws IllegalArgumentException if a column with given name is not + * @param name the field {@code name}. + * @return {@link BigDecimal} containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined. */ BigDecimal readBigDecimal(String name); @@ -306,6 +363,9 @@ public interface FieldSet { * value at index 'index' is blank. * * @param index the field index. + * @param defaultValue the value to use if the field value is blank. + * @return {@link BigDecimal} containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. */ BigDecimal readBigDecimal(int index, BigDecimal defaultValue); @@ -315,9 +375,11 @@ public interface FieldSet { * returning the supplied defaultValue if the trimmed string * value at index 'index' is blank. * - * @param name the field name. + * @param name the field {@code name}. * @param defaultValue the default value to use if the field is blank - * @throws IllegalArgumentException if a column with given name is not + * @return {@link BigDecimal} containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined. */ BigDecimal readBigDecimal(String name, BigDecimal defaultValue); @@ -327,6 +389,8 @@ public interface FieldSet { * designated column index. * * @param index the field index. + * @return {@link Date} containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. * @throws IllegalArgumentException if the value is not parseable * @throws NullPointerException if the value is empty @@ -337,8 +401,10 @@ public interface FieldSet { * Read the java.sql.Date value in given format from column * with given name. * - * @param name the field name. - * @throws IllegalArgumentException if a column with given name is not + * @param name the field {@code name}. + * @return {@link Date} containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined or if the value is not parseable * @throws NullPointerException if the value is empty */ @@ -350,6 +416,8 @@ public interface FieldSet { * * @param index the field index. * @param defaultValue the default value to use if the field is blank + * @return {@link Date} containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. * @throws IllegalArgumentException if the value is not parseable * @throws NullPointerException if the value is empty @@ -360,9 +428,11 @@ public interface FieldSet { * Read the java.sql.Date value in given format from column * with given name. * - * @param name the field name. + * @param name the field {@code name}. * @param defaultValue the default value to use if the field is blank - * @throws IllegalArgumentException if a column with given name is not + * @return {@link Date} containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined. */ Date readDate(String name, Date defaultValue); @@ -373,6 +443,8 @@ public interface FieldSet { * * @param index the field index. * @param pattern the pattern describing the date and time format + * @return {@link Date} containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. * @throws IllegalArgumentException if the date cannot be parsed. * @@ -383,9 +455,11 @@ public interface FieldSet { * Read the java.sql.Date value in given format from column * with given name. * - * @param name the field name. + * @param name the field {@code name}. * @param pattern the pattern describing the date and time format - * @throws IllegalArgumentException if a column with given name is not + * @return {@link Date} containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined or if the specified field cannot be parsed * */ @@ -398,6 +472,8 @@ public interface FieldSet { * @param index the field index. * @param pattern the pattern describing the date and time format * @param defaultValue the default value to use if the field is blank + * @return {@link Date} containing the value from the specified index. + * * @throws IndexOutOfBoundsException if the index is out of bounds. * @throws IllegalArgumentException if the date cannot be parsed. * @@ -408,10 +484,12 @@ public interface FieldSet { * Read the java.sql.Date value in given format from column * with given name. * - * @param name the field name. + * @param name the field {@code name}. * @param pattern the pattern describing the date and time format * @param defaultValue the default value to use if the field is blank - * @throws IllegalArgumentException if a column with given name is not + * @return {@link Date} containing the value from the specified {@code name}. + * + * @throws IllegalArgumentException if a column with given {@code name} is not * defined or if the specified field cannot be parsed * */ @@ -419,6 +497,8 @@ public interface FieldSet { /** * Return the number of fields in this 'FieldSet'. + * + * @return int containing the number of fields in this field set. */ int getFieldCount(); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FieldSetFactory.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FieldSetFactory.java index 27f6c318ae..26d42ae670 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FieldSetFactory.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FieldSetFactory.java @@ -1,5 +1,17 @@ -/** - * +/* + * Copyright 2009-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ package org.springframework.batch.item.file.transform; @@ -14,8 +26,11 @@ public interface FieldSetFactory { /** * Create a FieldSet with named tokens. The token values can then be * retrieved either by name or by column number. + * * @param values the token values * @param names the names of the tokens + * @return an instance of {@link FieldSet}. + * * @see DefaultFieldSet#readString(String) */ FieldSet create(String[] values, String[] names); @@ -23,7 +38,10 @@ public interface FieldSetFactory { /** * Create a FieldSet with anonymous tokens. They can only be retrieved by * column number. + * * @param values the token values + * @return an instance of {@link FieldSet}. + * * @see FieldSet#readString(int) */ FieldSet create(String[] values); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FixedLengthTokenizer.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FixedLengthTokenizer.java index 4da782ee46..f2d83769d3 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FixedLengthTokenizer.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FixedLengthTokenizer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ * @author peter.zozom * @author Dave Syer * @author Lucas Ward + * @author Michael Minella */ public class FixedLengthTokenizer extends AbstractLineTokenizer { @@ -49,7 +50,7 @@ public class FixedLengthTokenizer extends AbstractLineTokenizer { * * @param ranges the column ranges expected in the input */ - public void setColumns(Range[] ranges) { + public void setColumns(Range... ranges) { this.ranges = Arrays.asList(ranges).toArray(new Range[ranges.length]); calculateMaxRange(ranges); } @@ -90,7 +91,7 @@ private void calculateMaxRange(Range[] ranges) { * Yields the tokens resulting from the splitting of the supplied * line. * - * @param line the line to be tokenised (can be null) + * @param line the line to be tokenized (can be null) * * @return the resulting tokens (empty if the line is null) * @throws IncorrectLineLengthException if line length is greater than or @@ -98,18 +99,18 @@ private void calculateMaxRange(Range[] ranges) { */ @Override protected List doTokenize(String line) { - List tokens = new ArrayList(ranges.length); + List tokens = new ArrayList<>(ranges.length); int lineLength; String token; lineLength = line.length(); if (lineLength < maxRange && isStrict()) { - throw new IncorrectLineLengthException("Line is shorter than max range " + maxRange, maxRange, lineLength); + throw new IncorrectLineLengthException("Line is shorter than max range " + maxRange, maxRange, lineLength, line); } if (!open && lineLength > maxRange && isStrict()) { - throw new IncorrectLineLengthException("Line is longer than max range " + maxRange, maxRange, lineLength); + throw new IncorrectLineLengthException("Line is longer than max range " + maxRange, maxRange, lineLength, line); } for (int i = 0; i < ranges.length; i++) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FlatFileFormatException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FlatFileFormatException.java index a8e3e36614..1f5a084267 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FlatFileFormatException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FlatFileFormatException.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,14 +18,29 @@ /** - * Exception indicating that some type of error has occured while + * Exception indicating that some type of error has occurred while * attempting to parse a line of input into tokens. * * @author Lucas Ward + * @author Michael Minella * */ +@SuppressWarnings("serial") public class FlatFileFormatException extends RuntimeException { + private String input; + + /** + * Create a new {@link FlatFileFormatException} based on a message. + * + * @param message the message for this exception + * @param input {@link String} containing the input for that caused this + * exception to be thrown. + */ + public FlatFileFormatException(String message, String input) { + super(message); + this.input = input; + } /** * Create a new {@link FlatFileFormatException} based on a message. * @@ -44,4 +59,11 @@ public FlatFileFormatException(String message) { public FlatFileFormatException(String message, Throwable cause) { super(message, cause); } + + /** + * Retrieve the input that caused this exception. + * + * @return String containing the input. + */ + public String getInput() { return input; } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FormatterLineAggregator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FormatterLineAggregator.java index 063df94dc6..c3129f82db 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FormatterLineAggregator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/FormatterLineAggregator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,7 +23,7 @@ /** * A {@link LineAggregator} implementation which produces a String by - * aggregating the provided item via the {@link Formatter} syntax.
      + * aggregating the provided item via the {@link Formatter} syntax.
      * * @see Formatter * @@ -60,7 +60,9 @@ public void setMaximumLength(int maximumLength) { /** * Set the format string used to aggregate items. - * + * + * @param format {@link String} containing the format to use. + * * @see Formatter */ public void setFormat(String format) { @@ -78,7 +80,7 @@ public void setLocale(Locale locale) { @Override protected String doAggregate(Object[] fields) { - Assert.notNull(format); + Assert.notNull(format, "A format is required"); String value = String.format(locale, format, fields); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/IncorrectLineLengthException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/IncorrectLineLengthException.java index baff68f3e4..201ca632c4 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/IncorrectLineLengthException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/IncorrectLineLengthException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,29 +20,79 @@ * is expected. * * @author Lucas Ward + * @author Michael Minella * @since 1.1 */ +@SuppressWarnings("serial") public class IncorrectLineLengthException extends FlatFileFormatException { private int actualLength; private int expectedLength; - + + /** + * @param message the message for this exception. + * @param expectedLength int containing the length that was expected. + * @param actualLength int containing the actual length. + * @param input the {@link String} that contained the contents that caused + * the exception to be thrown. + * + * @since 2.2.6 + */ + public IncorrectLineLengthException(String message, int expectedLength, int actualLength, String input) { + super(message, input); + this.expectedLength = expectedLength; + this.actualLength = actualLength; + } + + /** + * @param message the message for this exception. + * @param expectedLength int containing the length that was expected. + * @param actualLength int containing the actual length. + */ public IncorrectLineLengthException(String message, int expectedLength, int actualLength) { super(message); this.expectedLength = expectedLength; this.actualLength = actualLength; } - + + /** + * @param expectedLength int containing the length that was expected. + * @param actualLength int containing the actual length. + * @param input the {@link String} that contained the contents that caused + * the exception to be thrown. + + * @since 2.2.6 + */ + public IncorrectLineLengthException(int expectedLength, int actualLength, String input) { + super("Incorrect line length in record: expected " + expectedLength + " actual " + actualLength, input); + this.actualLength = actualLength; + this.expectedLength = expectedLength; + } + + /** + * @param expectedLength int containing the length that was expected. + * @param actualLength int containing the actual length. + */ public IncorrectLineLengthException(int expectedLength, int actualLength) { super("Incorrect line length in record: expected " + expectedLength + " actual " + actualLength); this.actualLength = actualLength; this.expectedLength = expectedLength; } - + + /** + * Retrieves the actual length that was recorded for this exception. + * + * @return int containing the actual length. + */ public int getActualLength() { return actualLength; } - + + /** + * Retrieves the expected length that was recorded for this exception. + * + * @return int containing the expected length. + */ public int getExpectedLength() { return expectedLength; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/IncorrectTokenCountException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/IncorrectTokenCountException.java index f7cf3abd68..960b1b7bd8 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/IncorrectTokenCountException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/IncorrectTokenCountException.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,19 +20,36 @@ * while parsing a file. * * @author Lucas Ward + * @author "Michael Minella" * @since 1.1 */ +@SuppressWarnings("serial") public class IncorrectTokenCountException extends FlatFileFormatException { private int actualCount; private int expectedCount; - + private String input; + + public IncorrectTokenCountException(String message, int expectedCount, int actualCount, String input) { + super(message); + this.expectedCount = expectedCount; + this.actualCount = actualCount; + this.input = input; + } + public IncorrectTokenCountException(String message, int expectedCount, int actualCount) { super(message); this.expectedCount = expectedCount; this.actualCount = actualCount; } - + + public IncorrectTokenCountException(int expectedCount, int actualCount, String input) { + super("Incorrect number of tokens found in record: expected " + expectedCount + " actual " + actualCount); + this.expectedCount = expectedCount; + this.actualCount = actualCount; + this.input = input; + } + public IncorrectTokenCountException(int expectedCount, int actualCount) { super("Incorrect number of tokens found in record: expected " + expectedCount + " actual " + actualCount); this.actualCount = actualCount; @@ -46,4 +63,10 @@ public int getActualCount() { public int getExpectedCount() { return expectedCount; } + + /** + * @return the line that caused the exception + * @since 2.2.6 + */ + public String getInput() { return input; } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/LineAggregator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/LineAggregator.java index 1510086973..5fac28302b 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/LineAggregator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/LineAggregator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/LineTokenizer.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/LineTokenizer.java index f8fa258063..b87cb4959a 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/LineTokenizer.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/LineTokenizer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,12 +16,14 @@ package org.springframework.batch.item.file.transform; +import org.springframework.lang.Nullable; /** * Interface that is used by framework to split string obtained typically from a * file into tokens. * * @author tomas.slanina + * @author Mahmoud Ben Hassine * */ public interface LineTokenizer { @@ -34,5 +36,5 @@ public interface LineTokenizer { * * @return the resulting tokens */ - FieldSet tokenize(String line); + FieldSet tokenize(@Nullable String line); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/PassThroughFieldExtractor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/PassThroughFieldExtractor.java index 1ce70368fa..9773975429 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/PassThroughFieldExtractor.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/PassThroughFieldExtractor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/PassThroughLineAggregator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/PassThroughLineAggregator.java index df698a3658..038a76e5f2 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/PassThroughLineAggregator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/PassThroughLineAggregator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/PatternMatchingCompositeLineTokenizer.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/PatternMatchingCompositeLineTokenizer.java index e12fac113f..35075f9041 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/PatternMatchingCompositeLineTokenizer.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/PatternMatchingCompositeLineTokenizer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,11 +20,12 @@ import org.springframework.batch.support.PatternMatcher; import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** * A {@link LineTokenizer} implementation that stores a mapping of String - * patterns to delegate {@link LineTokenizer}s. Each line tokenizied will be + * patterns to delegate {@link LineTokenizer}s. Each line tokenized will be * checked to see if it matches a pattern. If the line matches a key in the map * of delegates, then the corresponding delegate {@link LineTokenizer} will be * used. Patterns are sorted starting with the most specific, and the first @@ -46,7 +47,7 @@ public class PatternMatchingCompositeLineTokenizer implements LineTokenizer, Ini * java.lang.String) */ @Override - public FieldSet tokenize(String line) { + public FieldSet tokenize(@Nullable String line) { return tokenizers.match(line).tokenize(line); } @@ -63,6 +64,6 @@ public void afterPropertiesSet() throws Exception { public void setTokenizers(Map tokenizers) { Assert.isTrue(!tokenizers.isEmpty(), "The 'tokenizers' property must be non-empty"); - this.tokenizers = new PatternMatcher(tokenizers); + this.tokenizers = new PatternMatcher<>(tokenizers); } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/Range.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/Range.java index 94ac6a0792..11e4c48bc4 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/Range.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/Range.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/RangeArrayPropertyEditor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/RangeArrayPropertyEditor.java index 0398b47ab3..37eec49341 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/RangeArrayPropertyEditor.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/RangeArrayPropertyEditor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,20 +16,20 @@ package org.springframework.batch.item.file.transform; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + import java.beans.PropertyEditorSupport; import java.util.Arrays; import java.util.Comparator; -import org.springframework.util.Assert; -import org.springframework.util.StringUtils; - /** * Property editor implementation which parses string and creates array of - * ranges. Ranges can be provided in any order.
      Input string should be + * ranges. Ranges can be provided in any order.
      Input string should be * provided in following format: 'range1, range2, range3,...' where range is * specified as: *

        - *
      • 'X-Y', where X is minimum value and Y is maximum value (condition X<=Y + *
      • 'X-Y', where X is minimum value and Y is maximum value (condition X<=Y * is verified)
      • *
      • or 'Z', where Z is minimum and maximum is calculated as (minimum of * adjacent range - 1). Maximum of the last range is never calculated. Range @@ -37,8 +37,8 @@ *
      * Minimum and maximum values can be from interval <1, Integer.MAX_VALUE-1> *

      - * Examples:
      - * '1, 15, 25, 38, 55-60' is equal to '1-14, 15-24, 25-37, 38-54, 55-60'
      + * Examples:
      + * '1, 15, 25, 38, 55-60' is equal to '1-14, 15-24, 25-37, 38-54, 55-60'
      * '36, 14, 1-10, 15, 49-57' is equal to '36-48, 14-14, 1-10, 15-35, 49-57' *

      * Property editor also allows to validate whether ranges are disjoint. Validation @@ -55,7 +55,7 @@ public class RangeArrayPropertyEditor extends PropertyEditorSupport { * Set force disjoint ranges. If set to TRUE, ranges are validated to be disjoint. * For example: defining ranges '1-10, 5-15' will cause IllegalArgumentException in * case of forceDisjointRanges=TRUE. - * @param forceDisjointRanges + * @param forceDisjointRanges true to force disjoint ranges. */ public void setForceDisjointRanges(boolean forceDisjointRanges) { this.forceDisjointRanges = forceDisjointRanges; @@ -98,7 +98,7 @@ public void setAsText(String text) throws IllegalArgumentException { public String getAsText() { Range[] ranges = (Range[])getValue(); - StringBuffer sb = new StringBuffer(); + StringBuilder sb = new StringBuilder(); for (int i = 0; i < ranges.length; i++) { if(i>0) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/RecursiveCollectionLineAggregator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/RecursiveCollectionLineAggregator.java index 4b75752e94..b4c5289946 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/RecursiveCollectionLineAggregator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/RecursiveCollectionLineAggregator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,12 +30,12 @@ public class RecursiveCollectionLineAggregator implements LineAggregator delegate = new PassThroughLineAggregator(); + private LineAggregator delegate = new PassThroughLineAggregator<>(); /** * Public setter for the {@link LineAggregator} to use on single items, that * are not Strings. This can be used to strategise the conversion of - * collection and array elements to a String.
      + * collection and array elements to a String.
      * * @param delegate the line aggregator to set. Defaults to a pass through. */ @@ -47,13 +47,13 @@ public void setDelegate(LineAggregator delegate) { * (non-Javadoc) * @see org.springframework.batch.item.file.transform.LineAggregator#aggregate(java.lang.Object) */ - @Override + @Override public String aggregate(Collection items) { StringBuilder builder = new StringBuilder(); for (T value : items) { - builder.append(delegate.aggregate(value) + LINE_SEPARATOR); + builder.append(delegate.aggregate(value)).append(LINE_SEPARATOR); } - return builder.delete(builder.length()-LINE_SEPARATOR.length(),builder.length()).toString(); + return builder.delete(builder.length() - LINE_SEPARATOR.length(), builder.length()).toString(); } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/RegexLineTokenizer.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/RegexLineTokenizer.java index 29ca7e6d6c..76e37d07c3 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/RegexLineTokenizer.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/RegexLineTokenizer.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -58,7 +58,7 @@ protected List doTokenize(String line) { boolean matchFound = matcher.find(); if (matchFound) { - List tokens = new ArrayList(matcher.groupCount()); + List tokens = new ArrayList<>(matcher.groupCount()); for (int i = 1; i <= matcher.groupCount(); i++) { tokens.add(matcher.group(i)); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/package-info.java new file mode 100644 index 0000000000..4c5085445f --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of io file support transform concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.item.file.transform; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/package.html deleted file mode 100644 index 79853a581b..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/transform/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of io file support transform concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/function/FunctionItemProcessor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/function/FunctionItemProcessor.java new file mode 100644 index 0000000000..7de337a053 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/function/FunctionItemProcessor.java @@ -0,0 +1,47 @@ +/* + * Copyright 2017-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.function; + +import java.util.function.Function; + +import org.springframework.batch.item.ItemProcessor; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * An {@link ItemProcessor} implementation that delegates to a {@link Function} + * + * @author Michael Minella + * @since 4.0 + */ +public class FunctionItemProcessor implements ItemProcessor{ + + private final Function function; + + /** + * @param function the delegate. Must not be null + */ + public FunctionItemProcessor(Function function) { + Assert.notNull(function, "A function is required"); + this.function = function; + } + + @Nullable + @Override + public O process(I item) throws Exception { + return this.function.apply(item); + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/function/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/function/package-info.java new file mode 100644 index 0000000000..f71c35ba6b --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/function/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Adapters for {@code java.util.function} components. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.function; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsItemReader.java index 77b33c7a60..9414b4e723 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,23 +16,24 @@ package org.springframework.batch.item.jms; -import javax.jms.Message; - import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.item.ItemReader; import org.springframework.beans.factory.InitializingBean; import org.springframework.jms.core.JmsOperations; import org.springframework.jms.core.JmsTemplate; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; +import javax.jms.Message; + /** * An {@link ItemReader} for JMS using a {@link JmsTemplate}. The template * should have a default destination, which will be used to provide items in - * {@link #read()}.
      - *
      + * {@link #read()}.
      + *
      * - * The implementation is thread safe after its properties are set (normal + * The implementation is thread-safe after its properties are set (normal * singleton behavior). * * @author Dave Syer @@ -47,7 +48,7 @@ public class JmsItemReader implements ItemReader, InitializingBean { protected JmsOperations jmsTemplate; /** - * Setter for jms template. + * Setter for JMS template. * * @param jmsTemplate a {@link JmsOperations} instance */ @@ -76,7 +77,8 @@ public void setItemType(Class itemType) { this.itemType = itemType; } - @Override + @Nullable + @Override @SuppressWarnings("unchecked") public T read() { if (itemType != null && itemType.isAssignableFrom(Message.class)) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsItemWriter.java index db3308d602..2ea054447a 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,8 +16,6 @@ package org.springframework.batch.item.jms; -import java.util.List; - import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.item.ItemWriter; @@ -25,13 +23,15 @@ import org.springframework.jms.core.JmsTemplate; import org.springframework.util.Assert; +import java.util.List; + /** * An {@link ItemWriter} for JMS using a {@link JmsTemplate}. The template * should have a default destination, which will be used to send items in - * {@link #write(List)}.
      - *
      + * {@link #write(List)}.
      + *
      * - * The implementation is thread safe after its properties are set (normal + * The implementation is thread-safe after its properties are set (normal * singleton behavior). * * @author Dave Syer @@ -61,7 +61,7 @@ public void setJmsTemplate(JmsOperations jmsTemplate) { } /** - * Send the items one-by-one to the default destination of the jms template. + * Send the items one-by-one to the default destination of the JMS template. * * @see org.springframework.batch.item.ItemWriter#write(java.util.List) */ diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsMethodArgumentsKeyGenerator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsMethodArgumentsKeyGenerator.java index 28916c93cf..86040011c2 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsMethodArgumentsKeyGenerator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsMethodArgumentsKeyGenerator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsMethodInvocationRecoverer.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsMethodInvocationRecoverer.java index 5b6be54185..55d8416bff 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsMethodInvocationRecoverer.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsMethodInvocationRecoverer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,12 +17,15 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + +import org.springframework.lang.Nullable; import org.springframework.retry.interceptor.MethodInvocationRecoverer; import org.springframework.jms.JmsException; import org.springframework.jms.core.JmsOperations; /** * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public class JmsMethodInvocationRecoverer implements MethodInvocationRecoverer { @@ -42,12 +45,13 @@ public void setJmsTemplate(JmsOperations jmsTemplate) { /** * Send one message per item in the arguments list using the default destination of - * the jms template. If the recovery is successful null is returned. + * the jms template. If the recovery is successful {@code null} is returned. * * @see org.springframework.retry.interceptor.MethodInvocationRecoverer#recover(Object[], * Throwable) */ @Override + @Nullable public T recover(Object[] items, Throwable cause) { try { for (Object item : items) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsNewMethodArgumentsIdentifier.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsNewMethodArgumentsIdentifier.java index 1600654b94..ad564d6e03 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsNewMethodArgumentsIdentifier.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/JmsNewMethodArgumentsIdentifier.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/builder/JmsItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/builder/JmsItemReaderBuilder.java new file mode 100644 index 0000000000..019ecf4e54 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/builder/JmsItemReaderBuilder.java @@ -0,0 +1,81 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.jms.builder; + +import javax.jms.Message; + +import org.springframework.batch.item.jms.JmsItemReader; +import org.springframework.jms.core.JmsOperations; +import org.springframework.util.Assert; + +/** + * Creates a fully qualified JmsItemReader. + * + * @author Glenn Renfro + * + * @since 4.0 + */ +public class JmsItemReaderBuilder { + + protected Class itemType; + + protected JmsOperations jmsTemplate; + + /** + * Establish the JMS template that will be used by the JmsItemReader. + * + * @param jmsTemplate a {@link JmsOperations} instance + * @return this instance for method chaining. + * @see JmsItemReader#setJmsTemplate(JmsOperations) + */ + public JmsItemReaderBuilder jmsTemplate(JmsOperations jmsTemplate) { + this.jmsTemplate = jmsTemplate; + + return this; + } + + /** + * Set the expected type of incoming message payloads. Set this to {@link Message} to + * receive the raw underlying message. + * + * @param itemType the java class of the items to be delivered. Typically the same as + * the class parameter + * @return this instance for method chaining. + * + * @throws IllegalStateException if the message payload is of the wrong type. + * @see JmsItemReader#setItemType(Class) + */ + public JmsItemReaderBuilder itemType(Class itemType) { + this.itemType = itemType; + + return this; + } + + /** + * Returns a fully constructed {@link JmsItemReader}. + * + * @return a new {@link JmsItemReader} + */ + public JmsItemReader build() { + Assert.notNull(this.jmsTemplate, "jmsTemplate is required."); + JmsItemReader jmsItemReader = new JmsItemReader<>(); + + jmsItemReader.setItemType(this.itemType); + jmsItemReader.setJmsTemplate(this.jmsTemplate); + return jmsItemReader; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/builder/JmsItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/builder/JmsItemWriterBuilder.java new file mode 100644 index 0000000000..82e1d29c28 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/builder/JmsItemWriterBuilder.java @@ -0,0 +1,59 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.jms.builder; + +import org.springframework.batch.item.jms.JmsItemWriter; +import org.springframework.jms.core.JmsOperations; +import org.springframework.util.Assert; + +/** + * Creates a fully qualified JmsItemWriter. + * + * @author Glenn Renfro + * + * @since 4.0 + */ +public class JmsItemWriterBuilder { + + private JmsOperations jmsTemplate; + + /** + * Establish the JMS template that will be used by the {@link JmsItemWriter}. + * + * @param jmsTemplate a {@link JmsOperations} instance + * @return this instance for method chaining. + * @see JmsItemWriter#setJmsTemplate(JmsOperations) + */ + public JmsItemWriterBuilder jmsTemplate(JmsOperations jmsTemplate) { + this.jmsTemplate = jmsTemplate; + + return this; + } + + /** + * Returns a fully constructed {@link JmsItemWriter}. + * + * @return a new {@link JmsItemWriter} + */ + public JmsItemWriter build() { + Assert.notNull(this.jmsTemplate, "jmsTemplate is required."); + JmsItemWriter jmsItemWriter = new JmsItemWriter<>(); + + jmsItemWriter.setJmsTemplate(this.jmsTemplate); + return jmsItemWriter; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/builder/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/builder/package-info.java new file mode 100644 index 0000000000..ba5a0aa85a --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/builder/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Builders for JMS item reader and writer. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.jms.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/package-info.java new file mode 100644 index 0000000000..2e0e156aa6 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/jms/package-info.java @@ -0,0 +1,10 @@ +/** + * JMS based reader/writer and related components. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.jms; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/GsonJsonObjectMarshaller.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/GsonJsonObjectMarshaller.java new file mode 100644 index 0000000000..443075b688 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/GsonJsonObjectMarshaller.java @@ -0,0 +1,45 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import com.google.gson.Gson; + +/** + * A json object marshaller that uses Google Gson + * to marshal an object into a json representation. + * + * @param type of objects to marshal + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public class GsonJsonObjectMarshaller implements JsonObjectMarshaller { + + private Gson gson = new Gson(); + + /** + * Set the {@link Gson} object to use. + * @param gson object to use + */ + public void setGson(Gson gson) { + this.gson = gson; + } + + @Override + public String marshal(T item) { + return gson.toJson(item); + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/GsonJsonObjectReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/GsonJsonObjectReader.java new file mode 100644 index 0000000000..44f2c4e0bf --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/GsonJsonObjectReader.java @@ -0,0 +1,99 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; + +import com.google.gson.Gson; +import com.google.gson.JsonIOException; +import com.google.gson.JsonSyntaxException; +import com.google.gson.stream.JsonReader; +import com.google.gson.stream.JsonToken; + +import org.springframework.batch.item.ParseException; +import org.springframework.core.io.Resource; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * Implementation of {@link JsonObjectReader} based on + * Google Gson. + * + * @param type of the target object + * + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public class GsonJsonObjectReader implements JsonObjectReader { + + private Class itemType; + + private JsonReader jsonReader; + + private Gson mapper = new Gson(); + + private InputStream inputStream; + + /** + * Create a new {@link GsonJsonObjectReader} instance. + * @param itemType the target item type + */ + public GsonJsonObjectReader(Class itemType) { + this.itemType = itemType; + } + + /** + * Set the object mapper to use to map Json objects to domain objects. + * @param mapper the object mapper to use + */ + public void setMapper(Gson mapper) { + Assert.notNull(mapper, "The mapper must not be null"); + this.mapper = mapper; + } + + @Override + public void open(Resource resource) throws Exception { + Assert.notNull(resource, "The resource must not be null"); + this.inputStream = resource.getInputStream(); + this.jsonReader = this.mapper.newJsonReader(new InputStreamReader(this.inputStream)); + Assert.state(this.jsonReader.peek() == JsonToken.BEGIN_ARRAY, + "The Json input stream must start with an array of Json objects"); + this.jsonReader.beginArray(); + } + + @Nullable + @Override + public T read() throws Exception { + try { + if (this.jsonReader.hasNext()) { + return this.mapper.fromJson(this.jsonReader, this.itemType); + } + } catch (IOException |JsonIOException | JsonSyntaxException e) { + throw new ParseException("Unable to read next JSON object", e); + } + return null; + } + + @Override + public void close() throws Exception { + this.inputStream.close(); + this.jsonReader.close(); + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JacksonJsonObjectMarshaller.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JacksonJsonObjectMarshaller.java new file mode 100644 index 0000000000..5e4b9159b5 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JacksonJsonObjectMarshaller.java @@ -0,0 +1,52 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.ObjectMapper; + +import org.springframework.batch.item.ItemStreamException; + +/** + * A json object marshaller that uses Jackson + * to marshal an object into a json representation. + * + * @param type of objects to marshal + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public class JacksonJsonObjectMarshaller implements JsonObjectMarshaller { + + private ObjectMapper objectMapper = new ObjectMapper(); + + /** + * Set the {@link ObjectMapper} to use. + * @param objectMapper to use + */ + public void setObjectMapper(ObjectMapper objectMapper) { + this.objectMapper = objectMapper; + } + + @Override + public String marshal(T item) { + try { + return objectMapper.writeValueAsString(item); + } catch (JsonProcessingException e) { + throw new ItemStreamException("Unable to marshal object " + item + " to Json", e); + } + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JacksonJsonObjectReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JacksonJsonObjectReader.java new file mode 100644 index 0000000000..18e853e8da --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JacksonJsonObjectReader.java @@ -0,0 +1,95 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import java.io.IOException; +import java.io.InputStream; + +import com.fasterxml.jackson.core.JsonParser; +import com.fasterxml.jackson.core.JsonToken; +import com.fasterxml.jackson.databind.ObjectMapper; + +import org.springframework.batch.item.ParseException; +import org.springframework.core.io.Resource; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * Implementation of {@link JsonObjectReader} based on + * Jackson. + * + * @param type of the target object + * + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public class JacksonJsonObjectReader implements JsonObjectReader { + + private Class itemType; + + private JsonParser jsonParser; + + private ObjectMapper mapper = new ObjectMapper(); + + private InputStream inputStream; + + /** + * Create a new {@link JacksonJsonObjectReader} instance. + * @param itemType the target item type + */ + public JacksonJsonObjectReader(Class itemType) { + this.itemType = itemType; + } + + /** + * Set the object mapper to use to map Json objects to domain objects. + * @param mapper the object mapper to use + */ + public void setMapper(ObjectMapper mapper) { + Assert.notNull(mapper, "The mapper must not be null"); + this.mapper = mapper; + } + + @Override + public void open(Resource resource) throws Exception { + Assert.notNull(resource, "The resource must not be null"); + this.inputStream = resource.getInputStream(); + this.jsonParser = this.mapper.getFactory().createParser(this.inputStream); + Assert.state(this.jsonParser.nextToken() == JsonToken.START_ARRAY, + "The Json input stream must start with an array of Json objects"); + } + + @Nullable + @Override + public T read() throws Exception { + try { + if (this.jsonParser.nextToken() == JsonToken.START_OBJECT) { + return this.mapper.readValue(this.jsonParser, this.itemType); + } + } catch (IOException e) { + throw new ParseException("Unable to read next JSON object", e); + } + return null; + } + + @Override + public void close() throws Exception { + this.inputStream.close(); + this.jsonParser.close(); + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JsonFileItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JsonFileItemWriter.java new file mode 100644 index 0000000000..7724e23944 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JsonFileItemWriter.java @@ -0,0 +1,111 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import java.util.Iterator; +import java.util.List; + +import org.springframework.batch.item.support.AbstractFileItemWriter; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; +import org.springframework.util.ClassUtils; + +/** + * Item writer that writes data in json format to an output file. The location + * of the output file is defined by a {@link Resource} and must represent a + * writable file. Items are transformed to json format using a + * {@link JsonObjectMarshaller}. Items will be enclosed in a json array as follows: + * + *

      + * + * [ + * {json object}, + * {json object}, + * {json object} + * ] + * + *

      + * + * The implementation is not thread-safe. + * + * @see GsonJsonObjectMarshaller + * @see JacksonJsonObjectMarshaller + * @param type of object to write as json representation + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public class JsonFileItemWriter extends AbstractFileItemWriter { + + private static final char JSON_OBJECT_SEPARATOR = ','; + private static final char JSON_ARRAY_START = '['; + private static final char JSON_ARRAY_STOP = ']'; + + private JsonObjectMarshaller jsonObjectMarshaller; + + /** + * Create a new {@link JsonFileItemWriter} instance. + * @param resource to write json data to + * @param jsonObjectMarshaller used to marshal object into json representation + */ + public JsonFileItemWriter(Resource resource, JsonObjectMarshaller jsonObjectMarshaller) { + Assert.notNull(resource, "resource must not be null"); + Assert.notNull(jsonObjectMarshaller, "json object marshaller must not be null"); + setResource(resource); + setJsonObjectMarshaller(jsonObjectMarshaller); + setHeaderCallback(writer -> writer.write(JSON_ARRAY_START)); + setFooterCallback(writer -> writer.write(this.lineSeparator + JSON_ARRAY_STOP + this.lineSeparator)); + setExecutionContextName(ClassUtils.getShortName(JsonFileItemWriter.class)); + } + + /** + * Assert that mandatory properties (jsonObjectMarshaller) are set. + * + * @see org.springframework.beans.factory.InitializingBean#afterPropertiesSet() + */ + @Override + public void afterPropertiesSet() throws Exception { + if (this.append) { + this.shouldDeleteIfExists = false; + } + } + + /** + * Set the {@link JsonObjectMarshaller} to use to marshal object to json. + * @param jsonObjectMarshaller the marshaller to use + */ + public void setJsonObjectMarshaller(JsonObjectMarshaller jsonObjectMarshaller) { + this.jsonObjectMarshaller = jsonObjectMarshaller; + } + + @Override + public String doWrite(List items) { + StringBuilder lines = new StringBuilder(); + Iterator iterator = items.iterator(); + if (!items.isEmpty() && state.getLinesWritten() > 0) { + lines.append(JSON_OBJECT_SEPARATOR).append(this.lineSeparator); + } + while (iterator.hasNext()) { + T item = iterator.next(); + lines.append(' ').append(this.jsonObjectMarshaller.marshal(item)); + if (iterator.hasNext()) { + lines.append(JSON_OBJECT_SEPARATOR).append(this.lineSeparator); + } + } + return lines.toString(); + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JsonItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JsonItemReader.java new file mode 100644 index 0000000000..1ef19dc680 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JsonItemReader.java @@ -0,0 +1,128 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import org.springframework.batch.item.ItemStreamReader; +import org.springframework.batch.item.file.ResourceAwareItemReaderItemStream; +import org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader; +import org.springframework.core.io.Resource; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * {@link ItemStreamReader} implementation that reads Json objects from a + * {@link Resource} having the following format: + *

      + * + * [ + * { + * // JSON object + * }, + * { + * // JSON object + * } + * ] + * + *

      + * + * The implementation is not thread-safe. + * + * @param the type of json objects to read + * + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public class JsonItemReader extends AbstractItemCountingItemStreamItemReader implements + ResourceAwareItemReaderItemStream { + + private static final Log LOGGER = LogFactory.getLog(JsonItemReader.class); + + private Resource resource; + + private JsonObjectReader jsonObjectReader; + + private boolean strict = true; + + /** + * Create a new {@link JsonItemReader} instance. + * @param resource the input json resource + * @param jsonObjectReader the json object reader to use + */ + public JsonItemReader(Resource resource, JsonObjectReader jsonObjectReader) { + Assert.notNull(resource, "The resource must not be null."); + Assert.notNull(jsonObjectReader, "The json object reader must not be null."); + this.resource = resource; + this.jsonObjectReader = jsonObjectReader; + } + + /** + * Set the {@link JsonObjectReader} to use to read and map Json fragments to domain objects. + * @param jsonObjectReader the json object reader to use + */ + public void setJsonObjectReader(JsonObjectReader jsonObjectReader) { + this.jsonObjectReader = jsonObjectReader; + } + + /** + * In strict mode the reader will throw an exception on + * {@link #open(org.springframework.batch.item.ExecutionContext)} if the + * input resource does not exist. + * @param strict true by default + */ + public void setStrict(boolean strict) { + this.strict = strict; + } + + @Override + public void setResource(Resource resource) { + this.resource = resource; + } + + @Nullable + @Override + protected T doRead() throws Exception { + return jsonObjectReader.read(); + } + + @Override + protected void doOpen() throws Exception { + if (!this.resource.exists()) { + if (this.strict) { + throw new IllegalStateException("Input resource must exist (reader is in 'strict' mode)"); + } + LOGGER.warn("Input resource does not exist " + this.resource.getDescription()); + return; + } + if (!this.resource.isReadable()) { + if (this.strict) { + throw new IllegalStateException("Input resource must be readable (reader is in 'strict' mode)"); + } + LOGGER.warn("Input resource is not readable " + this.resource.getDescription()); + return; + } + this.jsonObjectReader.open(this.resource); + } + + @Override + protected void doClose() throws Exception { + this.jsonObjectReader.close(); + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JsonObjectMarshaller.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JsonObjectMarshaller.java new file mode 100644 index 0000000000..457926cf56 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JsonObjectMarshaller.java @@ -0,0 +1,36 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +/** + * Strategy interface to marshal an object into a json representation. + * Implementations are required to return a valid json object. + * + * @param type of objects to marshal + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public interface JsonObjectMarshaller { + + /** + * Marshal an object into a json representation. + * @param object to marshal + * @return json representation fo the object + */ + String marshal(T object); + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JsonObjectReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JsonObjectReader.java new file mode 100644 index 0000000000..a83b2ea93d --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/JsonObjectReader.java @@ -0,0 +1,57 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import org.springframework.core.io.Resource; +import org.springframework.lang.Nullable; + +/** + * Strategy interface for Json readers. Implementations are expected to use + * a streaming API in order to read Json objects one at a time. + * + * @param type of the target object + * + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public interface JsonObjectReader { + + /** + * Open the Json resource for reading. + * @param resource the input resource + * @throws Exception if unable to open the resource + */ + default void open(Resource resource) throws Exception { + + } + + /** + * Read the next object in the Json resource if any. + * @return the next object or {@code null} if the resource is exhausted + * @throws Exception if unable to read the next object + */ + @Nullable + T read() throws Exception; + + /** + * Close the input resource. + * @throws Exception if unable to close the input resource + */ + default void close() throws Exception { + + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/builder/JsonFileItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/builder/JsonFileItemWriterBuilder.java new file mode 100644 index 0000000000..39dae114ca --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/builder/JsonFileItemWriterBuilder.java @@ -0,0 +1,261 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json.builder; + +import org.springframework.batch.item.file.FlatFileFooterCallback; +import org.springframework.batch.item.file.FlatFileHeaderCallback; +import org.springframework.batch.item.json.JsonFileItemWriter; +import org.springframework.batch.item.json.JsonObjectMarshaller; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; + +/** + * Builder for {@link JsonFileItemWriter}. + * + * @param type of objects to write as Json output. + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public class JsonFileItemWriterBuilder { + + private Resource resource; + private JsonObjectMarshaller jsonObjectMarshaller; + private FlatFileHeaderCallback headerCallback; + private FlatFileFooterCallback footerCallback; + + private String name; + private String encoding = JsonFileItemWriter.DEFAULT_CHARSET; + private String lineSeparator = JsonFileItemWriter.DEFAULT_LINE_SEPARATOR; + + private boolean append = false; + private boolean forceSync = false; + private boolean saveState = true; + private boolean shouldDeleteIfExists = true; + private boolean shouldDeleteIfEmpty = false; + private boolean transactional = JsonFileItemWriter.DEFAULT_TRANSACTIONAL; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public JsonFileItemWriterBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public JsonFileItemWriterBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * A flag indicating that changes should be force-synced to disk on flush. Defaults + * to false. + * + * @param forceSync value to set the flag to + * @return The current instance of the builder. + * @see JsonFileItemWriter#setForceSync(boolean) + */ + public JsonFileItemWriterBuilder forceSync(boolean forceSync) { + this.forceSync = forceSync; + + return this; + } + + /** + * String used to separate lines in output. Defaults to the System property + * line.separator. + * + * @param lineSeparator value to use for a line separator + * @return The current instance of the builder. + * @see JsonFileItemWriter#setLineSeparator(String) + */ + public JsonFileItemWriterBuilder lineSeparator(String lineSeparator) { + this.lineSeparator = lineSeparator; + + return this; + } + + /** + * Set the {@link JsonObjectMarshaller} to use to marshal objects to json. + * + * @param jsonObjectMarshaller to use + * @return The current instance of the builder. + * @see JsonFileItemWriter#setJsonObjectMarshaller(JsonObjectMarshaller) + */ + public JsonFileItemWriterBuilder jsonObjectMarshaller(JsonObjectMarshaller jsonObjectMarshaller) { + this.jsonObjectMarshaller = jsonObjectMarshaller; + + return this; + } + + /** + * The {@link Resource} to be used as output. + * + * @param resource the output of the writer. + * @return The current instance of the builder. + * @see JsonFileItemWriter#setResource(Resource) + */ + public JsonFileItemWriterBuilder resource(Resource resource) { + this.resource = resource; + + return this; + } + + /** + * Encoding used for output. + * + * @param encoding encoding type. + * @return The current instance of the builder. + * @see JsonFileItemWriter#setEncoding(String) + */ + public JsonFileItemWriterBuilder encoding(String encoding) { + this.encoding = encoding; + + return this; + } + + /** + * If set to true, once the step is complete, if the resource previously provided is + * empty, it will be deleted. + * + * @param shouldDelete defaults to false + * @return The current instance of the builder + * @see JsonFileItemWriter#setShouldDeleteIfEmpty(boolean) + */ + public JsonFileItemWriterBuilder shouldDeleteIfEmpty(boolean shouldDelete) { + this.shouldDeleteIfEmpty = shouldDelete; + + return this; + } + + /** + * If set to true, upon the start of the step, if the resource already exists, it will + * be deleted and recreated. + * + * @param shouldDelete defaults to true + * @return The current instance of the builder + * @see JsonFileItemWriter#setShouldDeleteIfExists(boolean) + */ + public JsonFileItemWriterBuilder shouldDeleteIfExists(boolean shouldDelete) { + this.shouldDeleteIfExists = shouldDelete; + + return this; + } + + /** + * If set to true and the file exists, the output will be appended to the existing + * file. + * + * @param append defaults to false + * @return The current instance of the builder + * @see JsonFileItemWriter#setAppendAllowed(boolean) + */ + public JsonFileItemWriterBuilder append(boolean append) { + this.append = append; + + return this; + } + + /** + * A callback for header processing. + * + * @param callback {@link FlatFileHeaderCallback} implementation + * @return The current instance of the builder + * @see JsonFileItemWriter#setHeaderCallback(FlatFileHeaderCallback) + */ + public JsonFileItemWriterBuilder headerCallback(FlatFileHeaderCallback callback) { + this.headerCallback = callback; + + return this; + } + + /** + * A callback for footer processing. + * + * @param callback {@link FlatFileFooterCallback} implementation + * @return The current instance of the builder + * @see JsonFileItemWriter#setFooterCallback(FlatFileFooterCallback) + */ + public JsonFileItemWriterBuilder footerCallback(FlatFileFooterCallback callback) { + this.footerCallback = callback; + + return this; + } + + /** + * If set to true, the flushing of the buffer is delayed while a transaction is active. + * + * @param transactional defaults to true + * @return The current instance of the builder + * @see JsonFileItemWriter#setTransactional(boolean) + */ + public JsonFileItemWriterBuilder transactional(boolean transactional) { + this.transactional = transactional; + + return this; + } + + /** + * Validate the configuration and build a new {@link JsonFileItemWriter}. + * + * @return a new instance of the {@link JsonFileItemWriter} + */ + public JsonFileItemWriter build() { + Assert.notNull(this.resource, "A resource is required."); + Assert.notNull(this.jsonObjectMarshaller, "A json object marshaller is required."); + + if (this.saveState) { + Assert.hasText(this.name, "A name is required when saveState is true"); + } + + JsonFileItemWriter jsonFileItemWriter = new JsonFileItemWriter<>(this.resource, this.jsonObjectMarshaller); + + jsonFileItemWriter.setName(this.name); + jsonFileItemWriter.setAppendAllowed(this.append); + jsonFileItemWriter.setEncoding(this.encoding); + if (this.headerCallback != null) { + jsonFileItemWriter.setHeaderCallback(this.headerCallback); + } + if (this.footerCallback != null) { + jsonFileItemWriter.setFooterCallback(this.footerCallback); + } + jsonFileItemWriter.setForceSync(this.forceSync); + jsonFileItemWriter.setLineSeparator(this.lineSeparator); + jsonFileItemWriter.setSaveState(this.saveState); + jsonFileItemWriter.setShouldDeleteIfEmpty(this.shouldDeleteIfEmpty); + jsonFileItemWriter.setShouldDeleteIfExists(this.shouldDeleteIfExists); + jsonFileItemWriter.setTransactional(this.transactional); + + return jsonFileItemWriter; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/builder/JsonItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/builder/JsonItemReaderBuilder.java new file mode 100644 index 0000000000..92bb485a9f --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/builder/JsonItemReaderBuilder.java @@ -0,0 +1,157 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json.builder; + +import org.springframework.batch.item.json.JsonItemReader; +import org.springframework.batch.item.json.JsonObjectReader; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * A builder for {@link JsonItemReader}. + * + * @param type of the target item + * + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public class JsonItemReaderBuilder { + + private JsonObjectReader jsonObjectReader; + + private Resource resource; + + private String name; + + private boolean strict = true; + + private boolean saveState = true; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + /** + * Set the {@link JsonObjectReader} to use to read and map Json objects to domain objects. + * @param jsonObjectReader to use + * @return The current instance of the builder. + * @see JsonItemReader#setJsonObjectReader(JsonObjectReader) + */ + public JsonItemReaderBuilder jsonObjectReader(JsonObjectReader jsonObjectReader) { + this.jsonObjectReader = jsonObjectReader; + + return this; + } + + /** + * The {@link Resource} to be used as input. + * @param resource the input to the reader. + * @return The current instance of the builder. + * @see JsonItemReader#setResource(Resource) + */ + public JsonItemReaderBuilder resource(Resource resource) { + this.resource = resource; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public JsonItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Setting this value to true indicates that it is an error if the input + * does not exist and an exception will be thrown. Defaults to true. + * @param strict indicates the input resource must exist + * @return The current instance of the builder. + * @see JsonItemReader#setStrict(boolean) + */ + public JsonItemReaderBuilder strict(boolean strict) { + this.strict = strict; + + return this; + } + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public JsonItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * Configure the max number of items to be read. + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public JsonItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * @param currentItemCount current index + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public JsonItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * Validate the configuration and build a new {@link JsonItemReader}. + * @return a new instance of the {@link JsonItemReader} + */ + public JsonItemReader build() { + Assert.notNull(this.jsonObjectReader, "A json object reader is required."); + Assert.notNull(this.resource, "A resource is required."); + if (this.saveState) { + Assert.state(StringUtils.hasText(this.name), "A name is required when saveState is set to true."); + } + + JsonItemReader reader = new JsonItemReader<>(this.resource, this.jsonObjectReader); + reader.setName(this.name); + reader.setStrict(this.strict); + reader.setSaveState(this.saveState); + reader.setMaxItemCount(this.maxItemCount); + reader.setCurrentItemCount(this.currentItemCount); + + return reader; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/builder/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/builder/package-info.java new file mode 100644 index 0000000000..0b03185ebc --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/builder/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Builders for JSON item reader and writer. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.json.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/package-info.java new file mode 100644 index 0000000000..667ea71817 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/json/package-info.java @@ -0,0 +1,11 @@ +/** + *

      + * Infrastructure implementations of JSON input and output. + *

      + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.json; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/KafkaItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/KafkaItemReader.java new file mode 100644 index 0000000000..a022b27eac --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/KafkaItemReader.java @@ -0,0 +1,190 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.kafka; + +import java.time.Duration; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashMap; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.Properties; + +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.common.TopicPartition; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.support.AbstractItemStreamItemReader; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + *

      + * An {@link org.springframework.batch.item.ItemReader} implementation for Apache Kafka. + * Uses a {@link KafkaConsumer} to read data from a given topic. + * Multiple partitions within the same topic can be assigned to this reader. + *

      + * + *

      + * Since {@link KafkaConsumer} is not thread-safe, this reader is not thead-safe. + *

      + * + * @author Mathieu Ouellet + * @author Mahmoud Ben Hassine + * @since 4.2 + */ +public class KafkaItemReader extends AbstractItemStreamItemReader { + + private static final String TOPIC_PARTITION_OFFSETS = "topic.partition.offsets"; + + private static final long DEFAULT_POLL_TIMEOUT = 30L; + + private List topicPartitions; + + private Map partitionOffsets; + + private KafkaConsumer kafkaConsumer; + + private Properties consumerProperties; + + private Iterator> consumerRecords; + + private Duration pollTimeout = Duration.ofSeconds(DEFAULT_POLL_TIMEOUT); + + private boolean saveState = true; + + /** + * Create a new {@link KafkaItemReader}. + *

      {@code consumerProperties} must contain the following keys: + * 'bootstrap.servers', 'group.id', 'key.deserializer' and 'value.deserializer'

      . + * @param consumerProperties properties of the consumer + * @param topicName name of the topic to read data from + * @param partitions list of partitions to read data from + */ + public KafkaItemReader(Properties consumerProperties, String topicName, Integer... partitions) { + this(consumerProperties, topicName, Arrays.asList(partitions)); + } + + /** + * Create a new {@link KafkaItemReader}. + *

      {@code consumerProperties} must contain the following keys: + * 'bootstrap.servers', 'group.id', 'key.deserializer' and 'value.deserializer'

      . + * @param consumerProperties properties of the consumer + * @param topicName name of the topic to read data from + * @param partitions list of partitions to read data from + */ + public KafkaItemReader(Properties consumerProperties, String topicName, List partitions) { + Assert.notNull(consumerProperties, "Consumer properties must not be null"); + Assert.isTrue(consumerProperties.containsKey(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG), + ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG + " property must be provided"); + Assert.isTrue(consumerProperties.containsKey(ConsumerConfig.GROUP_ID_CONFIG), + ConsumerConfig.GROUP_ID_CONFIG + " property must be provided"); + Assert.isTrue(consumerProperties.containsKey(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG), + ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG + " property must be provided"); + Assert.isTrue(consumerProperties.containsKey(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG), + ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG + " property must be provided"); + this.consumerProperties = consumerProperties; + Assert.hasLength(topicName, "Topic name must not be null or empty"); + Assert.isTrue(!partitions.isEmpty(), "At least one partition must be provided"); + this.topicPartitions = new ArrayList<>(); + for (Integer partition : partitions) { + this.topicPartitions.add(new TopicPartition(topicName, partition)); + } + } + + /** + * Set a timeout for the consumer topic polling duration. Default to 30 seconds. + * @param pollTimeout for the consumer poll operation + */ + public void setPollTimeout(Duration pollTimeout) { + Assert.notNull(pollTimeout, "pollTimeout must not be null"); + Assert.isTrue(!pollTimeout.isZero(), "pollTimeout must not be zero"); + Assert.isTrue(!pollTimeout.isNegative(), "pollTimeout must not be negative"); + this.pollTimeout = pollTimeout; + } + + /** + * Set the flag that determines whether to save internal data for + * {@link ExecutionContext}. Only switch this to false if you don't want to + * save any state from this stream, and you don't need it to be restartable. + * Always set it to false if the reader is being used in a concurrent + * environment. + * @param saveState flag value (default true). + */ + public void setSaveState(boolean saveState) { + this.saveState = saveState; + } + + /** + * The flag that determines whether to save internal state for restarts. + * @return true if the flag was set + */ + public boolean isSaveState() { + return this.saveState; + } + + @Override + public void open(ExecutionContext executionContext) { + this.kafkaConsumer = new KafkaConsumer<>(this.consumerProperties); + this.partitionOffsets = new HashMap<>(); + for (TopicPartition topicPartition : this.topicPartitions) { + this.partitionOffsets.put(topicPartition, 0L); + } + if (this.saveState && executionContext.containsKey(TOPIC_PARTITION_OFFSETS)) { + Map offsets = (Map) executionContext.get(TOPIC_PARTITION_OFFSETS); + for (Map.Entry entry : offsets.entrySet()) { + this.partitionOffsets.put(entry.getKey(), entry.getValue() == 0 ? 0 : entry.getValue() + 1); + } + } + this.kafkaConsumer.assign(this.topicPartitions); + this.partitionOffsets.forEach(this.kafkaConsumer::seek); + } + + @Nullable + @Override + public V read() { + if (this.consumerRecords == null || !this.consumerRecords.hasNext()) { + this.consumerRecords = this.kafkaConsumer.poll(this.pollTimeout).iterator(); + } + if (this.consumerRecords.hasNext()) { + ConsumerRecord record = this.consumerRecords.next(); + this.partitionOffsets.put(new TopicPartition(record.topic(), record.partition()), record.offset()); + return record.value(); + } + else { + return null; + } + } + + @Override + public void update(ExecutionContext executionContext) { + if (this.saveState) { + executionContext.put(TOPIC_PARTITION_OFFSETS, new HashMap<>(this.partitionOffsets)); + } + this.kafkaConsumer.commitSync(); + } + + @Override + public void close() { + if (this.kafkaConsumer != null) { + this.kafkaConsumer.close(); + } + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/KafkaItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/KafkaItemWriter.java new file mode 100644 index 0000000000..d924dd5105 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/KafkaItemWriter.java @@ -0,0 +1,61 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.kafka; + +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.KeyValueItemWriter; +import org.springframework.kafka.core.KafkaTemplate; +import org.springframework.util.Assert; + +/** + *

      + * An {@link ItemWriter} implementation for Apache Kafka using a + * {@link KafkaTemplate} with default topic configured. + *

      + * + * @author Mathieu Ouellet + * @since 4.2 + * + */ +public class KafkaItemWriter extends KeyValueItemWriter { + + private KafkaTemplate kafkaTemplate; + + @Override + protected void writeKeyValue(K key, T value) { + if (this.delete) { + this.kafkaTemplate.sendDefault(key, null); + } + else { + this.kafkaTemplate.sendDefault(key, value); + } + } + + @Override + protected void init() { + Assert.notNull(this.kafkaTemplate, "KafkaTemplate must not be null."); + Assert.notNull(this.kafkaTemplate.getDefaultTopic(), "KafkaTemplate must have the default topic set."); + } + + /** + * Set the {@link KafkaTemplate} to use. + * @param kafkaTemplate to use + */ + public void setKafkaTemplate(KafkaTemplate kafkaTemplate) { + this.kafkaTemplate = kafkaTemplate; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/builder/KafkaItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/builder/KafkaItemReaderBuilder.java new file mode 100644 index 0000000000..901ce2bbda --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/builder/KafkaItemReaderBuilder.java @@ -0,0 +1,153 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.kafka.builder; + +import java.time.Duration; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.Properties; + +import org.apache.kafka.clients.consumer.ConsumerConfig; + +import org.springframework.batch.item.kafka.KafkaItemReader; +import org.springframework.util.Assert; + +/** + * A builder implementation for the {@link KafkaItemReader}. + * + * @author Mathieu Ouellet + * @author Mahmoud Ben Hassine + * @since 4.2 + * @see KafkaItemReader + */ +public class KafkaItemReaderBuilder { + + private Properties consumerProperties; + + private String topic; + + private List partitions = new ArrayList<>(); + + private Duration pollTimeout = Duration.ofSeconds(30L); + + private boolean saveState = true; + + private String name; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public KafkaItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + return this; + } + + /** + * The name used to calculate the key within the {@link org.springframework.batch.item.ExecutionContext}. + * Required if {@link #saveState(boolean)} is set to true. + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public KafkaItemReaderBuilder name(String name) { + this.name = name; + return this; + } + + /** + * Configure the underlying consumer properties. + *

      {@code consumerProperties} must contain the following keys: + * 'bootstrap.servers', 'group.id', 'key.deserializer' and 'value.deserializer'

      . + * @param consumerProperties properties of the consumer + * @return The current instance of the builder. + */ + public KafkaItemReaderBuilder consumerProperties(Properties consumerProperties) { + this.consumerProperties = consumerProperties; + return this; + } + + /** + * A list of partitions to manually assign to the consumer. + * @param partitions list of partitions to assign to the consumer + * @return The current instance of the builder. + */ + public KafkaItemReaderBuilder partitions(Integer... partitions) { + return partitions(Arrays.asList(partitions)); + } + + /** + * A list of partitions to manually assign to the consumer. + * @param partitions list of partitions to assign to the consumer + * @return The current instance of the builder. + */ + public KafkaItemReaderBuilder partitions(List partitions) { + this.partitions = partitions; + return this; + } + + /** + * A topic name to manually assign to the consumer. + * @param topic name to assign to the consumer + * @return The current instance of the builder. + */ + public KafkaItemReaderBuilder topic(String topic) { + this.topic = topic; + return this; + } + + /** + * Set the pollTimeout for the poll() operations. Default to 30 seconds. + * @param pollTimeout timeout for the poll operation + * @return The current instance of the builder. + * @see KafkaItemReader#setPollTimeout(Duration) + */ + public KafkaItemReaderBuilder pollTimeout(Duration pollTimeout) { + this.pollTimeout = pollTimeout; + return this; + } + + public KafkaItemReader build() { + if (this.saveState) { + Assert.hasText(this.name, "A name is required when saveState is set to true"); + } + Assert.notNull(consumerProperties, "Consumer properties must not be null"); + Assert.isTrue(consumerProperties.containsKey(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG), + ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG + " property must be provided"); + Assert.isTrue(consumerProperties.containsKey(ConsumerConfig.GROUP_ID_CONFIG), + ConsumerConfig.GROUP_ID_CONFIG + " property must be provided"); + Assert.isTrue(consumerProperties.containsKey(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG), + ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG + " property must be provided"); + Assert.isTrue(consumerProperties.containsKey(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG), + ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG + " property must be provided"); + Assert.hasLength(topic, "Topic name must not be null or empty"); + Assert.notNull(pollTimeout, "pollTimeout must not be null"); + Assert.isTrue(!pollTimeout.isZero(), "pollTimeout must not be zero"); + Assert.isTrue(!pollTimeout.isNegative(), "pollTimeout must not be negative"); + Assert.isTrue(!partitions.isEmpty(), "At least one partition must be provided"); + + KafkaItemReader reader = new KafkaItemReader<>(this.consumerProperties, this.topic, this.partitions); + reader.setPollTimeout(this.pollTimeout); + reader.setSaveState(this.saveState); + reader.setName(this.name); + return reader; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/builder/KafkaItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/builder/KafkaItemWriterBuilder.java new file mode 100644 index 0000000000..30c83e315e --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/builder/KafkaItemWriterBuilder.java @@ -0,0 +1,88 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.kafka.builder; + +import org.springframework.batch.item.kafka.KafkaItemWriter; +import org.springframework.core.convert.converter.Converter; +import org.springframework.kafka.core.KafkaTemplate; +import org.springframework.util.Assert; + +/** + * A builder implementation for the {@link KafkaItemWriter} + * + * @author Mathieu Ouellet + * @since 4.2 + */ +public class KafkaItemWriterBuilder { + + private KafkaTemplate kafkaTemplate; + + private Converter itemKeyMapper; + + private boolean delete; + + /** + * Establish the KafkaTemplate to be used by the KafkaItemWriter. + * @param kafkaTemplate the template to be used + * @return this instance for method chaining + * @see KafkaItemWriter#setKafkaTemplate(KafkaTemplate) + */ + public KafkaItemWriterBuilder kafkaTemplate(KafkaTemplate kafkaTemplate) { + this.kafkaTemplate = kafkaTemplate; + return this; + } + + /** + * Set the {@link Converter} to use to derive the key from the item. + * @param itemKeyMapper the Converter to use. + * @return The current instance of the builder. + * @see KafkaItemWriter#setItemKeyMapper(Converter) + */ + public KafkaItemWriterBuilder itemKeyMapper(Converter itemKeyMapper) { + this.itemKeyMapper = itemKeyMapper; + return this; + } + + /** + * Indicate if the items being passed to the writer are all to be sent as delete events to the topic. A delete + * event is made of a key with a null value. If set to false (default), the items will be sent with provided value + * and key converter by the itemKeyMapper. If set to true, the items will be sent with the key converter from the + * value by the itemKeyMapper and a null value. + * @param delete removal indicator. + * @return The current instance of the builder. + * @see KafkaItemWriter#setDelete(boolean) + */ + public KafkaItemWriterBuilder delete(boolean delete) { + this.delete = delete; + return this; + } + + /** + * Validates and builds a {@link KafkaItemWriter}. + * @return a {@link KafkaItemWriter} + */ + public KafkaItemWriter build() { + Assert.notNull(this.kafkaTemplate, "kafkaTemplate is required."); + Assert.notNull(this.itemKeyMapper, "itemKeyMapper is required."); + + KafkaItemWriter writer = new KafkaItemWriter<>(); + writer.setKafkaTemplate(this.kafkaTemplate); + writer.setItemKeyMapper(this.itemKeyMapper); + writer.setDelete(this.delete); + return writer; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/builder/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/builder/package-info.java new file mode 100644 index 0000000000..45fb008395 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/builder/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Builders for Apache Kafka item reader and writer. + * + * @author Mathieu Ouellet + */ +@NonNullApi +package org.springframework.batch.item.kafka.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/package-info.java new file mode 100644 index 0000000000..c0e084822b --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/kafka/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Apache Kafka related readers and writers + * + * @author Mathieu Ouellet + */ +@NonNullApi +package org.springframework.batch.item.kafka; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/LdifReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/LdifReader.java new file mode 100644 index 0000000000..6572dd9fae --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/LdifReader.java @@ -0,0 +1,175 @@ +/* + * Copyright 2005-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.ldif; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.batch.item.file.ResourceAwareItemReaderItemStream; +import org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.core.io.Resource; +import org.springframework.lang.Nullable; +import org.springframework.ldap.core.LdapAttributes; +import org.springframework.ldap.ldif.parser.LdifParser; +import org.springframework.util.Assert; +import org.springframework.util.ClassUtils; + +/** + * The {@link LdifReader LdifReader} is an adaptation of the {@link org.springframework.batch.item.file.FlatFileItemReader FlatFileItemReader} + * built around an {@link LdifParser LdifParser}. + *

      + * Unlike the {@link org.springframework.batch.item.file.FlatFileItemReader FlatFileItemReader}, the {@link LdifReader LdifReader} + * does not require a mapper. Instead, this version of the {@link LdifReader LdifReader} simply returns an {@link LdapAttributes LdapAttributes} + * object which can be consumed and manipulated as necessary by {@link org.springframework.batch.item.ItemProcessor ItemProcessor} or any + * output service. Alternatively, the {@link RecordMapper RecordMapper} interface can be implemented and set in a + * {@link MappingLdifReader MappingLdifReader} to map records to objects for return. + *

      + * {@link LdifReader LdifReader} usage is mimics that of the {@link org.springframework.batch.item.file.FlatFileItemReader FlatFileItemReader} + * for all intensive purposes. Adjustments have been made to process records instead of lines, however. As such, the + * {@link #recordsToSkip recordsToSkip} attribute indicates the number of records from the top of the file that should not be processed. + * Implementations of the {@link RecordCallbackHandler RecordCallbackHandler} interface can be used to execute operations on those skipped records. + *

      + * As with the {@link org.springframework.batch.item.file.FlatFileItemReader FlatFileItemReader}, the {@link #strict strict} option differentiates + * between whether or not to require the resource to exist before processing. In the case of a value set to false, a warning is logged instead of + * an exception being thrown. + * + * @author Keith Barlow + * + */ +public class LdifReader extends AbstractItemCountingItemStreamItemReader + implements ResourceAwareItemReaderItemStream, InitializingBean { + + private static final Logger LOG = LoggerFactory.getLogger(LdifReader.class); + + private Resource resource; + + private LdifParser ldifParser; + + private int recordCount = 0; + + private int recordsToSkip = 0; + + private boolean strict = true; + + private RecordCallbackHandler skippedRecordsCallback; + + public LdifReader() { + setName(ClassUtils.getShortName(LdifReader.class)); + } + + /** + * In strict mode the reader will throw an exception on + * {@link #open(org.springframework.batch.item.ExecutionContext)} if the + * input resource does not exist. + * @param strict true by default + */ + public void setStrict(boolean strict) { + this.strict = strict; + } + + /** + * {@link RecordCallbackHandler RecordCallbackHandler} implementations can be used to take action on skipped records. + * + * @param skippedRecordsCallback will be called for each one of the initial + * skipped lines before any items are read. + */ + public void setSkippedRecordsCallback(RecordCallbackHandler skippedRecordsCallback) { + this.skippedRecordsCallback = skippedRecordsCallback; + } + + /** + * Public setter for the number of lines to skip at the start of a file. Can + * be used if the file contains a header without useful (column name) + * information, and without a comment delimiter at the beginning of the + * lines. + * + * @param recordsToSkip the number of lines to skip + */ + public void setRecordsToSkip(int recordsToSkip) { + this.recordsToSkip = recordsToSkip; + } + + @Override + protected void doClose() throws Exception { + if (ldifParser != null) { + ldifParser.close(); + } + this.recordCount = 0; + } + + @Override + protected void doOpen() throws Exception { + if (resource == null) + throw new IllegalStateException("A resource has not been set."); + + if (!resource.exists()) { + if (strict) { + throw new IllegalStateException("Input resource must exist (reader is in 'strict' mode): "+resource); + } else { + LOG.warn("Input resource does not exist " + resource.getDescription()); + return; + } + } + + ldifParser.open(); + + for (int i = 0; i < recordsToSkip; i++) { + LdapAttributes record = ldifParser.getRecord(); + if (skippedRecordsCallback != null) { + skippedRecordsCallback.handleRecord(record); + } + } + } + + @Nullable + @Override + protected LdapAttributes doRead() throws Exception { + LdapAttributes attributes = null; + + try { + if (ldifParser != null) { + while (attributes == null && ldifParser.hasMoreRecords()) { + attributes = ldifParser.getRecord(); + } + recordCount++; + } + + return attributes; + + } catch(Exception ex){ + LOG.error("Parsing error at record " + recordCount + " in resource=" + + resource.getDescription() + ", input=[" + attributes + "]", ex); + throw ex; + } + } + + /** + * Establishes the resource that will be used as the input for the LdifReader. + * + * @param resource the resource that will be read. + */ + public void setResource(Resource resource) { + this.resource = resource; + this.ldifParser = new LdifParser(resource); + } + + public void afterPropertiesSet() throws Exception { + Assert.notNull(resource, "A resource is required to parse."); + Assert.notNull(ldifParser, "A parser is required"); + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/MappingLdifReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/MappingLdifReader.java new file mode 100644 index 0000000000..776bd5809e --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/MappingLdifReader.java @@ -0,0 +1,174 @@ +/* + * Copyright 2005-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.ldif; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.batch.item.file.ResourceAwareItemReaderItemStream; +import org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.core.io.Resource; +import org.springframework.lang.Nullable; +import org.springframework.ldap.core.LdapAttributes; +import org.springframework.ldap.ldif.parser.LdifParser; +import org.springframework.util.Assert; +import org.springframework.util.ClassUtils; + +/** + * The {@link MappingLdifReader MappingLdifReader} is an adaptation of the {@link org.springframework.batch.item.file.FlatFileItemReader FlatFileItemReader} + * built around an {@link LdifParser LdifParser}. It differs from the standard {@link LdifReader LdifReader} in its ability to map + * {@link LdapAttributes LdapAttributes} objects to POJOs. + *

      + * The {@link MappingLdifReader MappingLdifReader} requires an {@link RecordMapper RecordMapper} implementation. If mapping + * is not required, the {@link LdifReader LdifReader} should be used instead. It simply returns an {@link LdapAttributes LdapAttributes} + * object which can be consumed and manipulated as necessary by {@link org.springframework.batch.item.ItemProcessor ItemProcessor} or any + * output service. + *

      + * As with the {@link org.springframework.batch.item.file.FlatFileItemReader FlatFileItemReader}, the {@link #strict strict} option + * differentiates between whether or not to require the resource to exist before processing. In the case of a value set to false, a warning + * is logged instead of an exception being thrown. + * + * @author Keith Barlow + * + */ +public class MappingLdifReader extends AbstractItemCountingItemStreamItemReader + implements ResourceAwareItemReaderItemStream, InitializingBean { + + private static final Logger LOG = LoggerFactory.getLogger(MappingLdifReader.class); + + private Resource resource; + + private LdifParser ldifParser; + + private int recordCount = 0; + + private int recordsToSkip = 0; + + private boolean strict = true; + + private RecordCallbackHandler skippedRecordsCallback; + + private RecordMapper recordMapper; + + public MappingLdifReader() { + setName(ClassUtils.getShortName(MappingLdifReader.class)); + } + + /** + * In strict mode the reader will throw an exception on + * {@link #open(org.springframework.batch.item.ExecutionContext)} if the + * input resource does not exist. + * @param strict false by default + */ + public void setStrict(boolean strict) { + this.strict = strict; + } + + /** + * {@link RecordCallbackHandler RecordCallbackHandler} implementations can be used to take action on skipped records. + * + * @param skippedRecordsCallback will be called for each one of the initial + * skipped lines before any items are read. + */ + public void setSkippedRecordsCallback(RecordCallbackHandler skippedRecordsCallback) { + this.skippedRecordsCallback = skippedRecordsCallback; + } + + /** + * Public setter for the number of lines to skip at the start of a file. Can + * be used if the file contains a header without useful (column name) + * information, and without a comment delimiter at the beginning of the + * lines. + * + * @param recordsToSkip the number of lines to skip + */ + public void setRecordsToSkip(int recordsToSkip) { + this.recordsToSkip = recordsToSkip; + } + + /** + * Setter for object mapper. This property is required to be set. + * @param recordMapper maps record to an object + */ + public void setRecordMapper(RecordMapper recordMapper) { + this.recordMapper = recordMapper; + } + + @Override + protected void doClose() throws Exception { + if (ldifParser != null) { + ldifParser.close(); + } + this.recordCount = 0; + } + + @Override + protected void doOpen() throws Exception { + if (resource == null) + throw new IllegalStateException("A resource has not been set."); + + if (!resource.exists()) { + if (strict) { + throw new IllegalStateException("Input resource must exist (reader is in 'strict' mode): "+resource); + } else { + LOG.warn("Input resource does not exist " + resource.getDescription()); + return; + } + } + + ldifParser.open(); + + for (int i = 0; i < recordsToSkip; i++) { + LdapAttributes record = ldifParser.getRecord(); + if (skippedRecordsCallback != null) { + skippedRecordsCallback.handleRecord(record); + } + } + } + + @Nullable + @Override + protected T doRead() throws Exception { + LdapAttributes attributes = null; + + try { + if (ldifParser != null) { + while (attributes == null && ldifParser.hasMoreRecords()) { + attributes = ldifParser.getRecord(); + } + recordCount++; + return recordMapper.mapRecord(attributes); + } + + return null; + } catch(Exception ex){ + LOG.error("Parsing error at record " + recordCount + " in resource=" + + resource.getDescription() + ", input=[" + attributes + "]", ex); + throw ex; + } + } + + public void setResource(Resource resource) { + this.resource = resource; + this.ldifParser = new LdifParser(resource); + } + + public void afterPropertiesSet() throws Exception { + Assert.notNull(resource, "A resource is required to parse."); + Assert.notNull(ldifParser, "A parser is required"); + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/RecordCallbackHandler.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/RecordCallbackHandler.java new file mode 100644 index 0000000000..3b6fdcf205 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/RecordCallbackHandler.java @@ -0,0 +1,36 @@ +/* + * Copyright 2005-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.ldif; + +import org.springframework.ldap.core.LdapAttributes; + +/** + * This interface can be used to operate on skipped records during open in the {@link LdifReader LdifReader} and the + * {@link MappingLdifReader MappingLdifReader}. + * + * @author Keith Barlow + * + */ +public interface RecordCallbackHandler { + + /** + * Execute operations on the supplied record. + * + * @param attributes represents the record + */ + void handleRecord(LdapAttributes attributes); + +} \ No newline at end of file diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/RecordMapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/RecordMapper.java new file mode 100644 index 0000000000..c60d53617e --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/RecordMapper.java @@ -0,0 +1,42 @@ +/* + * Copyright 2005-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.ldif; + +import org.springframework.lang.Nullable; +import org.springframework.ldap.core.LdapAttributes; + +/** + * This interface should be implemented to map {@link LdapAttributes LdapAttributes} objects to POJOs. The resulting + * implementations can be used in the {@link MappingLdifReader MappingLdifReader}. + * + * @author Keith Barlow + * @author Mahmoud Ben Hassine + * + * @param type the record will be mapped to + */ +public interface RecordMapper { + + /** + * Maps an {@link LdapAttributes LdapAttributes} object to the specified type. + * + * @param attributes attributes + * @return object of type T or {@code null} if unable to map the record to + * an object. + */ + @Nullable + T mapRecord(LdapAttributes attributes); + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/builder/LdifReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/builder/LdifReaderBuilder.java new file mode 100644 index 0000000000..e0172506df --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/builder/LdifReaderBuilder.java @@ -0,0 +1,186 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.ldif.builder; + +import org.springframework.batch.item.ldif.LdifReader; +import org.springframework.batch.item.ldif.RecordCallbackHandler; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; + +/** + * Creates a fully qualified LdifReader. + * + * @author Glenn Renfro + * + * @since 4.0 + */ +public class LdifReaderBuilder { + private Resource resource; + + private int recordsToSkip = 0; + + private boolean strict = true; + + private RecordCallbackHandler skippedRecordsCallback; + + private boolean saveState = true; + + private String name; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public LdifReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public LdifReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public LdifReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public LdifReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * In strict mode the reader will throw an exception on + * {@link LdifReader#open(org.springframework.batch.item.ExecutionContext)} if the + * input resource does not exist. + * + * @param strict true by default + * @return this instance for method chaining. + * @see LdifReader#setStrict(boolean) + */ + public LdifReaderBuilder strict(boolean strict) { + this.strict = strict; + + return this; + } + + /** + * {@link RecordCallbackHandler RecordCallbackHandler} implementations can be used to + * take action on skipped records. + * + * @param skippedRecordsCallback will be called for each one of the initial skipped + * lines before any items are read. + * @return this instance for method chaining. + * @see LdifReader#setSkippedRecordsCallback(RecordCallbackHandler) + */ + public LdifReaderBuilder skippedRecordsCallback(RecordCallbackHandler skippedRecordsCallback) { + this.skippedRecordsCallback = skippedRecordsCallback; + + return this; + } + + /** + * Public setter for the number of lines to skip at the start of a file. Can be used + * if the file contains a header without useful (column name) information, and without + * a comment delimiter at the beginning of the lines. + * + * @param recordsToSkip the number of lines to skip + * @return this instance for method chaining. + * @see LdifReader#setRecordsToSkip(int) + */ + public LdifReaderBuilder recordsToSkip(int recordsToSkip) { + this.recordsToSkip = recordsToSkip; + + return this; + } + + /** + * Establishes the resource that will be used as the input for the LdifReader. + * + * @param resource the resource that will be read. + * @return this instance for method chaining. + * @see LdifReader#setResource(Resource) + */ + public LdifReaderBuilder resource(Resource resource) { + this.resource = resource; + + return this; + } + + /** + * Returns a fully constructed {@link LdifReader}. + * + * @return a new {@link org.springframework.batch.item.ldif.LdifReader} + */ + public LdifReader build() { + Assert.notNull(this.resource, "Resource is required."); + if (this.saveState) { + Assert.hasText(this.name, "A name is required when saveState is set to true"); + } + LdifReader reader = new LdifReader(); + reader.setResource(this.resource); + reader.setRecordsToSkip(this.recordsToSkip); + reader.setSaveState(this.saveState); + reader.setName(this.name); + reader.setCurrentItemCount(this.currentItemCount); + reader.setMaxItemCount(this.maxItemCount); + if (this.skippedRecordsCallback != null) { + reader.setSkippedRecordsCallback(this.skippedRecordsCallback); + } + reader.setStrict(this.strict); + + return reader; + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/builder/MappingLdifReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/builder/MappingLdifReaderBuilder.java new file mode 100644 index 0000000000..66405d308b --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/builder/MappingLdifReaderBuilder.java @@ -0,0 +1,202 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.ldif.builder; + +import org.springframework.batch.item.ldif.MappingLdifReader; +import org.springframework.batch.item.ldif.RecordCallbackHandler; +import org.springframework.batch.item.ldif.RecordMapper; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; + +/** + * Creates a fully qualified MappingLdifReader. + * + * @author Glenn Renfro + * + * @since 4.0 + */ +public class MappingLdifReaderBuilder { + private Resource resource; + + private int recordsToSkip = 0; + + private boolean strict = true; + + private RecordCallbackHandler skippedRecordsCallback; + + private RecordMapper recordMapper; + + private boolean saveState = true; + + private String name; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public MappingLdifReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public MappingLdifReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public MappingLdifReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public MappingLdifReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * In strict mode the reader will throw an exception on + * {@link MappingLdifReader#open(org.springframework.batch.item.ExecutionContext)} if + * the input resource does not exist. + * + * @param strict true by default + * @return this instance for method chaining. + * @see MappingLdifReader#setStrict(boolean) + */ + public MappingLdifReaderBuilder strict(boolean strict) { + this.strict = strict; + + return this; + } + + /** + * {@link RecordCallbackHandler RecordCallbackHandler} implementations can be used to + * take action on skipped records. + * + * @param skippedRecordsCallback will be called for each one of the initial skipped + * lines before any items are read. + * @return this instance for method chaining. + * @see MappingLdifReader#setSkippedRecordsCallback(RecordCallbackHandler) + */ + public MappingLdifReaderBuilder skippedRecordsCallback(RecordCallbackHandler skippedRecordsCallback) { + this.skippedRecordsCallback = skippedRecordsCallback; + + return this; + } + + /** + * Public setter for the number of lines to skip at the start of a file. Can be used + * if the file contains a header without useful (column name) information, and without + * a comment delimiter at the beginning of the lines. + * + * @param recordsToSkip the number of lines to skip + * @return this instance for method chaining. + * @see MappingLdifReader#setRecordsToSkip(int) + */ + public MappingLdifReaderBuilder recordsToSkip(int recordsToSkip) { + this.recordsToSkip = recordsToSkip; + + return this; + } + + /** + * Establishes the resource that will be used as the input for the MappingLdifReader. + * + * @param resource the resource that will be read. + * @return this instance for method chaining. + * @see MappingLdifReader#setResource(Resource) + */ + public MappingLdifReaderBuilder resource(Resource resource) { + this.resource = resource; + + return this; + } + + /** + * Setter for object mapper. This property is required to be set. + * + * @param recordMapper maps record to an object + * @return this instance for method chaining. + */ + public MappingLdifReaderBuilder recordMapper(RecordMapper recordMapper) { + this.recordMapper = recordMapper; + + return this; + } + + /** + * Returns a fully constructed {@link MappingLdifReader}. + * + * @return a new {@link org.springframework.batch.item.ldif.MappingLdifReader} + */ + public MappingLdifReader build() { + Assert.notNull(this.resource, "Resource is required."); + Assert.notNull(this.recordMapper, "RecordMapper is required."); + if (this.saveState) { + Assert.hasText(this.name, "A name is required when saveState is set to true"); + } + MappingLdifReader reader = new MappingLdifReader<>(); + reader.setResource(this.resource); + reader.setRecordsToSkip(this.recordsToSkip); + reader.setSaveState(saveState); + reader.setCurrentItemCount(this.currentItemCount); + reader.setMaxItemCount(this.maxItemCount); + reader.setRecordMapper(this.recordMapper); + reader.setName(this.name); + if (this.skippedRecordsCallback != null) { + reader.setSkippedRecordsCallback(this.skippedRecordsCallback); + } + reader.setStrict(this.strict); + + return reader; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/builder/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/builder/package-info.java new file mode 100644 index 0000000000..2bcd72c032 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/builder/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Builders for LDIF related components. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.ldif.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/package-info.java new file mode 100644 index 0000000000..de9ae6f8fd --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/ldif/package-info.java @@ -0,0 +1,10 @@ +/** + *

      This package contains the classes required for using the LdifParser in Spring LDAP.

      + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.ldif; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/DefaultMailErrorHandler.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/DefaultMailErrorHandler.java index 11a09063d6..004765f73e 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/DefaultMailErrorHandler.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/DefaultMailErrorHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/MailErrorHandler.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/MailErrorHandler.java index a6a09dcf86..d3eed538d9 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/MailErrorHandler.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/MailErrorHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/SimpleMailMessageItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/SimpleMailMessageItemWriter.java index 0aa0d08a85..08130d752c 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/SimpleMailMessageItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/SimpleMailMessageItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -63,7 +63,7 @@ public class SimpleMailMessageItemWriter implements ItemWriter items) throws MailException mailSender.send(items.toArray(new SimpleMailMessage[items.size()])); } catch (MailSendException e) { - @SuppressWarnings("unchecked") Map failedMessages = e.getFailedMessages(); for (Entry entry : failedMessages.entrySet()) { mailErrorHandler.handle((SimpleMailMessage) entry.getKey(), entry.getValue()); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/builder/SimpleMailMessageItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/builder/SimpleMailMessageItemWriterBuilder.java new file mode 100644 index 0000000000..46f12aec73 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/builder/SimpleMailMessageItemWriterBuilder.java @@ -0,0 +1,82 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.mail.builder; + +import java.util.List; + +import org.springframework.batch.item.mail.DefaultMailErrorHandler; +import org.springframework.batch.item.mail.MailErrorHandler; +import org.springframework.batch.item.mail.SimpleMailMessageItemWriter; +import org.springframework.mail.MailSender; +import org.springframework.util.Assert; + +/** + * Creates a fully qualified SimpleMailMessageItemWriter. + * + * @author Glenn Renfro + * + * @since 4.0 + */ + +public class SimpleMailMessageItemWriterBuilder { + + private MailSender mailSender; + + private MailErrorHandler mailErrorHandler = new DefaultMailErrorHandler(); + + /** + * A {@link MailSender} to be used to send messages in + * {@link SimpleMailMessageItemWriter#write(List)}. + * + * @param mailSender strategy for sending simple mails. + * @return this instance for method chaining. + * @see SimpleMailMessageItemWriter#setMailSender(MailSender) + */ + public SimpleMailMessageItemWriterBuilder mailSender(MailSender mailSender) { + this.mailSender = mailSender; + return this; + } + + /** + * The handler for failed messages. Defaults to a {@link DefaultMailErrorHandler}. + * + * @param mailErrorHandler the mail error handler to set. + * @return this instance for method chaining. + * @see SimpleMailMessageItemWriter#setMailErrorHandler(MailErrorHandler) + */ + public SimpleMailMessageItemWriterBuilder mailErrorHandler(MailErrorHandler mailErrorHandler) { + this.mailErrorHandler = mailErrorHandler; + return this; + } + + /** + * Returns a fully constructed {@link SimpleMailMessageItemWriter}. + * + * @return a new {@link SimpleMailMessageItemWriter} + */ + public SimpleMailMessageItemWriter build() { + Assert.notNull(this.mailSender, "A mailSender is required"); + + SimpleMailMessageItemWriter writer = new SimpleMailMessageItemWriter(); + writer.setMailSender(this.mailSender); + if (mailErrorHandler != null) { + writer.setMailErrorHandler(this.mailErrorHandler); + } + + return writer; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/builder/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/builder/package-info.java new file mode 100644 index 0000000000..42a7de1483 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/builder/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Builders for JavaMail related components. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.mail.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/javamail/MimeMessageItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/javamail/MimeMessageItemWriter.java index 6c8b72e72b..6a9097f4d1 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/javamail/MimeMessageItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/javamail/MimeMessageItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,12 +15,6 @@ */ package org.springframework.batch.item.mail.javamail; -import java.util.List; -import java.util.Map; -import java.util.Map.Entry; - -import javax.mail.internet.MimeMessage; - import org.springframework.batch.item.ItemWriter; import org.springframework.batch.item.mail.DefaultMailErrorHandler; import org.springframework.batch.item.mail.MailErrorHandler; @@ -31,6 +25,11 @@ import org.springframework.mail.javamail.MimeMailMessage; import org.springframework.util.Assert; +import javax.mail.internet.MimeMessage; +import java.util.List; +import java.util.Map; +import java.util.Map.Entry; + /** *

      * A simple {@link ItemWriter} that can send mail messages. If it fails there is @@ -68,7 +67,7 @@ public class MimeMessageItemWriter implements ItemWriter { /** * A {@link JavaMailSender} to be used to send messages in {@link #write(List)}. * - * @param mailSender + * @param mailSender service for doing the work of sending a MIME message */ public void setJavaMailSender(JavaMailSender mailSender) { this.mailSender = mailSender; @@ -105,7 +104,6 @@ public void write(List items) throws MailException { mailSender.send(items.toArray(new MimeMessage[items.size()])); } catch (MailSendException e) { - @SuppressWarnings("unchecked") Map failedMessages = e.getFailedMessages(); for (Entry entry : failedMessages.entrySet()) { mailErrorHandler.handle(new MimeMailMessage((MimeMessage)entry.getKey()), entry.getValue()); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/javamail/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/javamail/package-info.java new file mode 100644 index 0000000000..c1811df0f2 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/javamail/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * JavaMail related components. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.mail.javamail; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/package-info.java new file mode 100644 index 0000000000..413f4b3833 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/mail/package-info.java @@ -0,0 +1,10 @@ +/** + * Java Mail based components. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.mail; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/package-info.java new file mode 100644 index 0000000000..ae555480b6 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure interfaces and primary dependencies for item concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.item; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/package.html deleted file mode 100644 index 9a571c3890..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure interfaces and primary dependencies for item concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractFileItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractFileItemWriter.java new file mode 100644 index 0000000000..3ca3ba9f9e --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractFileItemWriter.java @@ -0,0 +1,638 @@ +/* + * Copyright 2006-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support; + +import java.io.BufferedWriter; +import java.io.File; +import java.io.FileOutputStream; +import java.io.IOException; +import java.io.Writer; +import java.nio.channels.Channels; +import java.nio.channels.FileChannel; +import java.nio.charset.UnsupportedCharsetException; +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStream; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.WriteFailedException; +import org.springframework.batch.item.WriterNotOpenException; +import org.springframework.batch.item.file.FlatFileFooterCallback; +import org.springframework.batch.item.file.FlatFileHeaderCallback; +import org.springframework.batch.item.file.ResourceAwareItemWriterItemStream; +import org.springframework.batch.item.util.FileUtils; +import org.springframework.batch.support.transaction.TransactionAwareBufferedWriter; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; + +/** + * Base class for item writers that write data to a file or stream. + * This class provides common features like restart, force sync, append etc. + * The location of the output file is defined by a {@link Resource} which must + * represent a writable file.
      + * + * Uses buffered writer to improve performance.
      + * + * The implementation is not thread-safe. + * + * @author Waseem Malik + * @author Tomas Slanina + * @author Robert Kasanicky + * @author Dave Syer + * @author Michael Minella + * @author Mahmoud Ben Hassine + * + * @since 4.1 + */ +public abstract class AbstractFileItemWriter extends AbstractItemStreamItemWriter + implements ResourceAwareItemWriterItemStream, InitializingBean { + + public static final boolean DEFAULT_TRANSACTIONAL = true; + + protected static final Log logger = LogFactory.getLog(AbstractFileItemWriter.class); + + public static final String DEFAULT_LINE_SEPARATOR = System.getProperty("line.separator"); + + // default encoding for writing to output files - set to UTF-8. + public static final String DEFAULT_CHARSET = "UTF-8"; + + private static final String WRITTEN_STATISTICS_NAME = "written"; + + private static final String RESTART_DATA_NAME = "current.count"; + + private Resource resource; + + protected OutputState state = null; + + private boolean saveState = true; + + private boolean forceSync = false; + + protected boolean shouldDeleteIfExists = true; + + private boolean shouldDeleteIfEmpty = false; + + private String encoding = DEFAULT_CHARSET; + + private FlatFileHeaderCallback headerCallback; + + private FlatFileFooterCallback footerCallback; + + protected String lineSeparator = DEFAULT_LINE_SEPARATOR; + + private boolean transactional = DEFAULT_TRANSACTIONAL; + + protected boolean append = false; + + /** + * Flag to indicate that changes should be force-synced to disk on flush. + * Defaults to false, which means that even with a local disk changes could + * be lost if the OS crashes in between a write and a cache flush. Setting + * to true may result in slower performance for usage patterns involving many + * frequent writes. + * + * @param forceSync the flag value to set + */ + public void setForceSync(boolean forceSync) { + this.forceSync = forceSync; + } + + /** + * Public setter for the line separator. Defaults to the System property + * line.separator. + * @param lineSeparator the line separator to set + */ + public void setLineSeparator(String lineSeparator) { + this.lineSeparator = lineSeparator; + } + + /** + * Setter for resource. Represents a file that can be written. + * + * @param resource the resource to be written to + */ + @Override + public void setResource(Resource resource) { + this.resource = resource; + } + + /** + * Sets encoding for output template. + * + * @param newEncoding {@link String} containing the encoding to be used for + * the writer. + */ + public void setEncoding(String newEncoding) { + this.encoding = newEncoding; + } + + /** + * Flag to indicate that the target file should be deleted if it already + * exists, otherwise it will be created. Defaults to true, so no appending + * except on restart. If set to false and {@link #setAppendAllowed(boolean) + * appendAllowed} is also false then there will be an exception when the + * stream is opened to prevent existing data being potentially corrupted. + * + * @param shouldDeleteIfExists the flag value to set + */ + public void setShouldDeleteIfExists(boolean shouldDeleteIfExists) { + this.shouldDeleteIfExists = shouldDeleteIfExists; + } + + /** + * Flag to indicate that the target file should be appended if it already + * exists. If this flag is set then the flag + * {@link #setShouldDeleteIfExists(boolean) shouldDeleteIfExists} is + * automatically set to false, so that flag should not be set explicitly. + * Defaults value is false. + * + * @param append the flag value to set + */ + public void setAppendAllowed(boolean append) { + this.append = append; + } + + /** + * Flag to indicate that the target file should be deleted if no lines have + * been written (other than header and footer) on close. Defaults to false. + * + * @param shouldDeleteIfEmpty the flag value to set + */ + public void setShouldDeleteIfEmpty(boolean shouldDeleteIfEmpty) { + this.shouldDeleteIfEmpty = shouldDeleteIfEmpty; + } + + /** + * Set the flag indicating whether or not state should be saved in the + * provided {@link ExecutionContext} during the {@link ItemStream} call to + * update. Setting this to false means that it will always start at the + * beginning on a restart. + * + * @param saveState if true, state will be persisted + */ + public void setSaveState(boolean saveState) { + this.saveState = saveState; + } + + /** + * headerCallback will be called before writing the first item to file. + * Newline will be automatically appended after the header is written. + * + * @param headerCallback {@link FlatFileHeaderCallback} to generate the header + * + */ + public void setHeaderCallback(FlatFileHeaderCallback headerCallback) { + this.headerCallback = headerCallback; + } + + /** + * footerCallback will be called after writing the last item to file, but + * before the file is closed. + * + * @param footerCallback {@link FlatFileFooterCallback} to generate the footer + * + */ + public void setFooterCallback(FlatFileFooterCallback footerCallback) { + this.footerCallback = footerCallback; + } + + /** + * Flag to indicate that writing to the buffer should be delayed if a + * transaction is active. Defaults to true. + * + * @param transactional true if writing to buffer should be delayed. + * + */ + public void setTransactional(boolean transactional) { + this.transactional = transactional; + } + + /** + * Writes out a string followed by a "new line", where the format of the new + * line separator is determined by the underlying operating system. + * + * @param items list of items to be written to output stream + * @throws Exception if an error occurs while writing items to the output stream + */ + @Override + public void write(List items) throws Exception { + if (!getOutputState().isInitialized()) { + throw new WriterNotOpenException("Writer must be open before it can be written to"); + } + + if (logger.isDebugEnabled()) { + logger.debug("Writing to file with " + items.size() + " items."); + } + + OutputState state = getOutputState(); + + String lines = doWrite(items); + try { + state.write(lines); + } + catch (IOException e) { + throw new WriteFailedException("Could not write data. The file may be corrupt.", e); + } + state.setLinesWritten(state.getLinesWritten() + items.size()); + } + + /** + * Write out a string of items followed by a "new line", where the format of the new + * line separator is determined by the underlying operating system. + * @param items to be written + * @return written lines + */ + protected abstract String doWrite(List items); + + /** + * @see ItemStream#close() + */ + @Override + public void close() { + super.close(); + if (state != null) { + try { + if (footerCallback != null && state.outputBufferedWriter != null) { + footerCallback.writeFooter(state.outputBufferedWriter); + state.outputBufferedWriter.flush(); + } + } + catch (IOException e) { + throw new ItemStreamException("Failed to write footer before closing", e); + } + finally { + state.close(); + if (state.linesWritten == 0 && shouldDeleteIfEmpty) { + try { + resource.getFile().delete(); + } + catch (IOException e) { + throw new ItemStreamException("Failed to delete empty file on close", e); + } + } + state = null; + } + } + } + + /** + * Initialize the reader. This method may be called multiple times before + * close is called. + * + * @see ItemStream#open(ExecutionContext) + */ + @Override + public void open(ExecutionContext executionContext) throws ItemStreamException { + super.open(executionContext); + + Assert.notNull(resource, "The resource must be set"); + + if (!getOutputState().isInitialized()) { + doOpen(executionContext); + } + } + + private void doOpen(ExecutionContext executionContext) throws ItemStreamException { + OutputState outputState = getOutputState(); + if (executionContext.containsKey(getExecutionContextKey(RESTART_DATA_NAME))) { + outputState.restoreFrom(executionContext); + } + try { + outputState.initializeBufferedWriter(); + } + catch (IOException ioe) { + throw new ItemStreamException("Failed to initialize writer", ioe); + } + if (outputState.lastMarkedByteOffsetPosition == 0 && !outputState.appending) { + if (headerCallback != null) { + try { + headerCallback.writeHeader(outputState.outputBufferedWriter); + outputState.write(lineSeparator); + } + catch (IOException e) { + throw new ItemStreamException("Could not write headers. The file may be corrupt.", e); + } + } + } + } + + /** + * @see ItemStream#update(ExecutionContext) + */ + @Override + public void update(ExecutionContext executionContext) { + super.update(executionContext); + if (state == null) { + throw new ItemStreamException("ItemStream not open or already closed."); + } + + Assert.notNull(executionContext, "ExecutionContext must not be null"); + + if (saveState) { + + try { + executionContext.putLong(getExecutionContextKey(RESTART_DATA_NAME), state.position()); + } + catch (IOException e) { + throw new ItemStreamException("ItemStream does not return current position properly", e); + } + + executionContext.putLong(getExecutionContextKey(WRITTEN_STATISTICS_NAME), state.linesWritten); + } + } + + // Returns object representing state. + protected OutputState getOutputState() { + if (state == null) { + File file; + try { + file = resource.getFile(); + } + catch (IOException e) { + throw new ItemStreamException("Could not convert resource to file: [" + resource + "]", e); + } + Assert.state(!file.exists() || file.canWrite(), "Resource is not writable: [" + resource + "]"); + state = new OutputState(); + state.setDeleteIfExists(shouldDeleteIfExists); + state.setAppendAllowed(append); + state.setEncoding(encoding); + } + return state; + } + + /** + * Encapsulates the runtime state of the writer. All state changing + * operations on the writer go through this class. + */ + protected class OutputState { + + private FileOutputStream os; + + // The bufferedWriter over the file channel that is actually written + Writer outputBufferedWriter; + + FileChannel fileChannel; + + // this represents the charset encoding (if any is needed) for the + // output file + String encoding = DEFAULT_CHARSET; + + boolean restarted = false; + + long lastMarkedByteOffsetPosition = 0; + + long linesWritten = 0; + + boolean shouldDeleteIfExists = true; + + boolean initialized = false; + + private boolean append = false; + + private boolean appending = false; + + /** + * Return the byte offset position of the cursor in the output file as a + * long integer. + * @return the byte offset position of the cursor in the output file + * @throws IOException If unable to get the offset position + */ + public long position() throws IOException { + long pos = 0; + + if (fileChannel == null) { + return 0; + } + + outputBufferedWriter.flush(); + pos = fileChannel.position(); + if (transactional) { + pos += ((TransactionAwareBufferedWriter) outputBufferedWriter).getBufferSize(); + } + + return pos; + + } + + /** + * @param append if true, append to previously created file + */ + public void setAppendAllowed(boolean append) { + this.append = append; + } + + /** + * @param executionContext state from which to restore writing from + */ + public void restoreFrom(ExecutionContext executionContext) { + lastMarkedByteOffsetPosition = executionContext.getLong(getExecutionContextKey(RESTART_DATA_NAME)); + linesWritten = executionContext.getLong(getExecutionContextKey(WRITTEN_STATISTICS_NAME)); + if (shouldDeleteIfEmpty && linesWritten == 0) { + // previous execution deleted the output file because no items were written + restarted = false; + lastMarkedByteOffsetPosition = 0; + } else { + restarted = true; + } + } + + /** + * @param shouldDeleteIfExists indicator + */ + public void setDeleteIfExists(boolean shouldDeleteIfExists) { + this.shouldDeleteIfExists = shouldDeleteIfExists; + } + + /** + * @param encoding file encoding + */ + public void setEncoding(String encoding) { + this.encoding = encoding; + } + + public long getLinesWritten() { + return linesWritten; + } + + public void setLinesWritten(long linesWritten) { + this.linesWritten = linesWritten; + } + + /** + * Close the open resource and reset counters. + */ + public void close() { + + initialized = false; + restarted = false; + try { + if (outputBufferedWriter != null) { + outputBufferedWriter.close(); + } + } + catch (IOException ioe) { + throw new ItemStreamException("Unable to close the the ItemWriter", ioe); + } + finally { + if (!transactional) { + closeStream(); + } + } + } + + private void closeStream() { + try { + if (fileChannel != null) { + fileChannel.close(); + } + } + catch (IOException ioe) { + throw new ItemStreamException("Unable to close the the ItemWriter", ioe); + } + finally { + try { + if (os != null) { + os.close(); + } + } + catch (IOException ioe) { + throw new ItemStreamException("Unable to close the the ItemWriter", ioe); + } + } + } + + /** + * @param line String to be written to the file + * @throws IOException If unable to write the String to the file + */ + public void write(String line) throws IOException { + if (!initialized) { + initializeBufferedWriter(); + } + + outputBufferedWriter.write(line); + outputBufferedWriter.flush(); + } + + /** + * Truncate the output at the last known good point. + * + * @throws IOException if unable to work with file + */ + public void truncate() throws IOException { + fileChannel.truncate(lastMarkedByteOffsetPosition); + fileChannel.position(lastMarkedByteOffsetPosition); + } + + /** + * Creates the buffered writer for the output file channel based on + * configuration information. + * @throws IOException if unable to initialize buffer + */ + private void initializeBufferedWriter() throws IOException { + + File file = resource.getFile(); + FileUtils.setUpOutputFile(file, restarted, append, shouldDeleteIfExists); + + os = new FileOutputStream(file.getAbsolutePath(), true); + fileChannel = os.getChannel(); + + outputBufferedWriter = getBufferedWriter(fileChannel, encoding); + outputBufferedWriter.flush(); + + if (append) { + // Bug in IO library? This doesn't work... + // lastMarkedByteOffsetPosition = fileChannel.position(); + if (file.length() > 0) { + appending = true; + // Don't write the headers again + } + } + + Assert.state(outputBufferedWriter != null, + "Unable to initialize buffered writer"); + // in case of restarting reset position to last committed point + if (restarted) { + checkFileSize(); + truncate(); + } + + initialized = true; + } + + public boolean isInitialized() { + return initialized; + } + + /** + * Returns the buffered writer opened to the beginning of the file + * specified by the absolute path name contained in absoluteFileName. + */ + private Writer getBufferedWriter(FileChannel fileChannel, String encoding) { + try { + final FileChannel channel = fileChannel; + if (transactional) { + TransactionAwareBufferedWriter writer = new TransactionAwareBufferedWriter(channel, () -> closeStream()); + + writer.setEncoding(encoding); + writer.setForceSync(forceSync); + return writer; + } + else { + Writer writer = new BufferedWriter(Channels.newWriter(fileChannel, encoding)) { + @Override + public void flush() throws IOException { + super.flush(); + if (forceSync) { + channel.force(false); + } + } + }; + + return writer; + } + } + catch (UnsupportedCharsetException ucse) { + throw new ItemStreamException("Bad encoding configuration for output file " + fileChannel, ucse); + } + } + + /** + * Checks (on setState) to make sure that the current output file's size + * is not smaller than the last saved commit point. If it is, then the + * file has been damaged in some way and whole task must be started over + * again from the beginning. + * @throws IOException if there is an IO problem + */ + private void checkFileSize() throws IOException { + long size = -1; + + outputBufferedWriter.flush(); + size = fileChannel.size(); + + if (size < lastMarkedByteOffsetPosition) { + throw new ItemStreamException("Current file size is smaller than size at last commit"); + } + } + + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractItemCountingItemStreamItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractItemCountingItemStreamItemReader.java index 5c3e5c950f..3c2d6963a1 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractItemCountingItemStreamItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractItemCountingItemStreamItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ import org.springframework.batch.item.ItemStreamException; import org.springframework.batch.item.ParseException; import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -29,9 +30,11 @@ * item count in the {@link ExecutionContext} (therefore requires item ordering * to be preserved between runs). * - * Subclasses are inherently *not* thread-safe. + * Subclasses are inherently not thread-safe. * * @author Robert Kasanicky + * @author Glenn Renfro + * @author Mahmoud Ben Hassine */ public abstract class AbstractItemCountingItemStreamItemReader extends AbstractItemStreamItemReader { @@ -48,18 +51,21 @@ public abstract class AbstractItemCountingItemStreamItemReader extends Abstra /** * Read next item from input. * - * @return item - * @throws Exception + * @return an item or {@code null} if the data source is exhausted + * @throws Exception Allows subclasses to throw checked exceptions for interpretation by the framework */ + @Nullable protected abstract T doRead() throws Exception; /** * Open resources necessary to start reading input. + * @throws Exception Allows subclasses to throw checked exceptions for interpretation by the framework */ protected abstract void doOpen() throws Exception; /** * Close the resources opened in {@link #doOpen()}. + * @throws Exception Allows subclasses to throw checked exceptions for interpretation by the framework */ protected abstract void doClose() throws Exception; @@ -67,6 +73,9 @@ public abstract class AbstractItemCountingItemStreamItemReader extends Abstra * Move to the given item index. Subclasses should override this method if * there is a more efficient way of moving to given index than re-reading * the input using {@link #doRead()}. + * + * @param itemIndex index of item (0 based) to jump to. + * @throws Exception Allows subclasses to throw checked exceptions for interpretation by the framework */ protected void jumpToItem(int itemIndex) throws Exception { for (int i = 0; i < itemIndex; i++) { @@ -74,6 +83,7 @@ protected void jumpToItem(int itemIndex) throws Exception { } } + @Nullable @Override public T read() throws Exception, UnexpectedInputException, ParseException { if (currentItemCount >= maxItemCount) { @@ -114,9 +124,10 @@ public void setCurrentItemCount(int count) { * * @see #setName(String) * - * @param count the value of the maximum item count + * @param count the value of the maximum item count. count must be greater than zero. */ public void setMaxItemCount(int count) { + Assert.isTrue(count > 0, "count must be greater than zero"); this.maxItemCount = count; } @@ -149,21 +160,25 @@ public void open(ExecutionContext executionContext) throws ItemStreamException { maxItemCount = executionContext.getInt(getExecutionContextKey(READ_COUNT_MAX)); } + int itemCount = 0; if (executionContext.containsKey(getExecutionContextKey(READ_COUNT))) { - int itemCount = executionContext.getInt(getExecutionContextKey(READ_COUNT)); - - if (itemCount < maxItemCount) { - try { - jumpToItem(itemCount); - } - catch (Exception e) { - throw new ItemStreamException("Could not move to stored position on restart", e); - } - } - currentItemCount = itemCount; + itemCount = executionContext.getInt(getExecutionContextKey(READ_COUNT)); + } + else if(currentItemCount > 0) { + itemCount = currentItemCount; + } + if (itemCount > 0 && itemCount < maxItemCount) { + try { + jumpToItem(itemCount); + } + catch (Exception e) { + throw new ItemStreamException("Could not move to stored position on restart", e); + } } + currentItemCount = itemCount; + } @Override @@ -179,16 +194,6 @@ public void update(ExecutionContext executionContext) throws ItemStreamException } - /** - * The name of the component which will be used as a stem for keys in the - * {@link ExecutionContext}. Subclasses should provide a default value, e.g. - * the short form of the class name. - * - * @param name the name for the component - */ - public void setName(String name) { - this.setExecutionContextName(name); - } /** * Set the flag that determines whether to save internal data for diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractItemStreamItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractItemStreamItemReader.java index 606f9c73be..b3567a2a9a 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractItemStreamItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractItemStreamItemReader.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,6 +17,7 @@ package org.springframework.batch.item.support; import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemStreamReader; import org.springframework.batch.item.ItemStreamSupport; @@ -25,6 +26,6 @@ * @author Dave Syer * */ -public abstract class AbstractItemStreamItemReader extends ItemStreamSupport implements ItemReader { +public abstract class AbstractItemStreamItemReader extends ItemStreamSupport implements ItemStreamReader { } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractItemStreamItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractItemStreamItemWriter.java index 1d9e1e53d5..07a3538f07 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractItemStreamItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/AbstractItemStreamItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,6 +17,7 @@ package org.springframework.batch.item.support; import org.springframework.batch.item.ItemStreamSupport; +import org.springframework.batch.item.ItemStreamWriter; import org.springframework.batch.item.ItemWriter; @@ -25,6 +26,6 @@ * @author Dave Syer * */ -public abstract class AbstractItemStreamItemWriter extends ItemStreamSupport implements ItemWriter { +public abstract class AbstractItemStreamItemWriter extends ItemStreamSupport implements ItemStreamWriter { } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ClassifierCompositeItemProcessor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ClassifierCompositeItemProcessor.java new file mode 100644 index 0000000000..4827984858 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ClassifierCompositeItemProcessor.java @@ -0,0 +1,66 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support; + +import org.springframework.batch.item.ItemProcessor; +import org.springframework.classify.Classifier; +import org.springframework.classify.ClassifierSupport; +import org.springframework.lang.Nullable; + +/** + * Calls one of a collection of ItemProcessors, based on a router + * pattern implemented through the provided {@link Classifier}. + * + * Note the user is responsible for injecting a {@link Classifier} + * that returns an ItemProcessor that conforms to the declared input and output types. + * + * @author Jimmy Praet + * @since 3.0 + */ +public class ClassifierCompositeItemProcessor implements ItemProcessor { + + private Classifier> classifier = + new ClassifierSupport<> (null); + + /** + * Establishes the classifier that will determine which {@link ItemProcessor} to use. + * @param classifier the {@link Classifier} to set + */ + public void setClassifier(Classifier> classifier) { + this.classifier = classifier; + } + + /** + * Delegates to injected {@link ItemProcessor} instances according to the + * classification by the {@link Classifier}. + */ + @Nullable + @Override + public O process(I item) throws Exception { + return processItem(classifier.classify(item), item); + } + + /* + * Helper method to work around wildcard capture compiler error: see https://docs.oracle.com/javase/tutorial/java/generics/capture.html + * The method process(capture#4-of ?) in the type ItemProcessor is not applicable for the arguments (I) + */ + @SuppressWarnings("unchecked") + private O processItem(ItemProcessor processor, I input) throws Exception { + return processor.process((T) input); + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ClassifierCompositeItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ClassifierCompositeItemWriter.java index b8ab7e1463..9eafc2aada 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ClassifierCompositeItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ClassifierCompositeItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,13 +17,14 @@ package org.springframework.batch.item.support; import java.util.ArrayList; -import java.util.HashMap; +import java.util.LinkedHashMap; import java.util.List; import java.util.Map; import org.springframework.classify.Classifier; import org.springframework.classify.ClassifierSupport; import org.springframework.batch.item.ItemWriter; +import org.springframework.util.Assert; /** * Calls one of a collection of ItemWriters for each item, based on a router @@ -32,16 +33,18 @@ * The implementation is thread-safe if all delegates are thread-safe. * * @author Dave Syer + * @author Glenn Renfro * @since 2.0 */ public class ClassifierCompositeItemWriter implements ItemWriter { - private Classifier> classifier = new ClassifierSupport>(null); + private Classifier> classifier = new ClassifierSupport<>(null); /** * @param classifier the classifier to set */ public void setClassifier(Classifier> classifier) { + Assert.notNull(classifier, "A classifier is required."); this.classifier = classifier; } @@ -52,12 +55,12 @@ public void setClassifier(Classifier> classifier) { @Override public void write(List items) throws Exception { - Map, List> map = new HashMap, List>(); + Map, List> map = new LinkedHashMap<>(); for (T item : items) { ItemWriter key = classifier.classify(item); if (!map.containsKey(key)) { - map.put(key, new ArrayList()); + map.put(key, new ArrayList<>()); } map.get(key).add(item); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/CompositeItemProcessor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/CompositeItemProcessor.java index c1f9d57d4e..28705db22c 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/CompositeItemProcessor.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/CompositeItemProcessor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,17 +16,18 @@ package org.springframework.batch.item.support; -import java.util.List; - import org.springframework.batch.item.ItemProcessor; import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; +import java.util.List; + /** * Composite {@link ItemProcessor} that passes the item through a sequence of * injected ItemTransformers (return value of previous - * transformation is the entry value of the next).
      - *
      + * transformation is the entry value of the next).
      + *
      * * Note the user is responsible for injecting a chain of {@link ItemProcessor}s * that conforms to declared input and output types. @@ -37,6 +38,7 @@ public class CompositeItemProcessor implements ItemProcessor, Initia private List> delegates; + @Nullable @Override @SuppressWarnings("unchecked") public O process(I item) throws Exception { @@ -53,7 +55,7 @@ public O process(I item) throws Exception { } /* - * Helper method to work around wildcard capture compiler error: see http://docs.oracle.com/javase/tutorial/java/generics/capture.html + * Helper method to work around wildcard capture compiler error: see https://docs.oracle.com/javase/tutorial/java/generics/capture.html * The method process(capture#1-of ?) in the type ItemProcessor is not applicable for the arguments (Object) */ @SuppressWarnings("unchecked") @@ -67,8 +69,14 @@ public void afterPropertiesSet() throws Exception { Assert.notEmpty(delegates, "The 'delegates' may not be empty"); } + /** + * Establishes the {@link ItemProcessor} delegates that will work on the item to be + * processed. + * @param delegates list of {@link ItemProcessor} delegates that will work on the + * item. + */ public void setDelegates(List> delegates) { this.delegates = delegates; - } + } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/CompositeItemStream.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/CompositeItemStream.java index e56dfbb97b..8f397f501b 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/CompositeItemStream.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/CompositeItemStream.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,21 +31,22 @@ */ public class CompositeItemStream implements ItemStream { - private List streams = new ArrayList(); + private List streams = new ArrayList<>(); /** - * Public setter for the listeners. + * Public setter for the {@link ItemStream}s. * - * @param listeners + * @param streams array of {@link ItemStream}. */ - public void setStreams(ItemStream[] listeners) { - this.streams = Arrays.asList(listeners); + public void setStreams(ItemStream[] streams) { + this.streams = Arrays.asList(streams); } /** * Register a {@link ItemStream} as one of the interesting providers under * the provided key. - * + * + * @param stream an instance of {@link ItemStream} to be added to the list of streams. */ public void register(ItemStream stream) { synchronized (streams) { @@ -77,7 +78,10 @@ public void update(ExecutionContext executionContext) { /** * Broadcast the call to close. - * @throws ItemStreamException + + * @throws ItemStreamException thrown if one of the {@link ItemStream}s in + * the list fails to close. This is a sequential operation so all itemStreams + * in the list after the one that failed to close will remain open. */ @Override public void close() throws ItemStreamException { @@ -88,7 +92,10 @@ public void close() throws ItemStreamException { /** * Broadcast the call to open. - * @throws ItemStreamException + * + * @throws ItemStreamException thrown if one of the {@link ItemStream}s in + * the list fails to open. This is a sequential operation so all itemStreams + * in the list after the one that failed to open will not be opened. */ @Override public void open(ExecutionContext executionContext) throws ItemStreamException { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/CompositeItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/CompositeItemWriter.java index 13db29cc2a..8630a958c3 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/CompositeItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/CompositeItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,8 +16,6 @@ package org.springframework.batch.item.support; -import java.util.List; - import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemStream; import org.springframework.batch.item.ItemStreamException; @@ -26,9 +24,11 @@ import org.springframework.beans.factory.InitializingBean; import org.springframework.util.Assert; +import java.util.List; + /** - * Calls a collection of {@link ItemWriter}s in fixed-order sequence.
      - *
      + * Calls a collection of {@link ItemWriter}s in fixed-order sequence.
      + *
      * * The implementation is thread-safe if all delegates are thread-safe. * @@ -41,6 +41,14 @@ public class CompositeItemWriter implements ItemStreamWriter, Initializing private boolean ignoreItemStream = false; + /** + * Establishes the policy whether to call the open, close, or update methods for the + * item writer delegates associated with the CompositeItemWriter. + * + * @param ignoreItemStream if false the delegates' open, close, or update methods will + * be called when the corresponding methods on the CompositeItemWriter are called. If + * true the delegates' open, close, nor update methods will not be called (default is false). + */ public void setIgnoreItemStream(boolean ignoreItemStream) { this.ignoreItemStream = ignoreItemStream; } @@ -58,6 +66,12 @@ public void afterPropertiesSet() throws Exception { Assert.notEmpty(delegates, "The 'delegates' may not be empty"); } + /** + * The list of item writers to use as delegates. Items are written to each of the + * delegates. + * + * @param delegates the list of delegates to use. The delegates list must not be null nor be empty. + */ public void setDelegates(List> delegates) { this.delegates = delegates; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/IteratorItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/IteratorItemReader.java index 9cbcbc159d..54d1536e30 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/IteratorItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/IteratorItemReader.java @@ -1,74 +1,75 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.item.support; - -import java.util.Iterator; - -import org.springframework.batch.item.ItemReader; -import org.springframework.batch.item.ParseException; -import org.springframework.batch.item.UnexpectedInputException; -import org.springframework.util.Assert; - -/** - * An {@link ItemReader} that pulls data from a {@link Iterator} or - * {@link Iterable} using the constructors. - * - * @author Juliusz Brzostek - * @author Dave Syer - */ -public class IteratorItemReader implements ItemReader { - - /** - * Internal iterator - */ - private final Iterator iterator; - - /** - * Construct a new reader from this iterable (could be a collection), by - * extracting an instance of {@link Iterator} from it. - * - * @param iterable in instance of {@link Iterable} - * - * @see Iterable#iterator() - */ - public IteratorItemReader(Iterable iterable) { - Assert.notNull(iterable, "Iterable argument cannot be null!"); - this.iterator = iterable.iterator(); - } - - /** - * Construct a new reader from this iterator directly. - * @param iterator an instance of {@link Iterator} - */ - public IteratorItemReader(Iterator iterator) { - Assert.notNull(iterator, "Iterator argument cannot be null!"); - this.iterator = iterator; - } - - /** - * Implementation of {@link ItemReader#read()} that just iterates over the - * iterator provided. - */ - @Override - public T read() throws Exception, UnexpectedInputException, ParseException { - if (iterator.hasNext()) - return iterator.next(); - else - return null; // end of data - } - -} +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support; + +import java.util.Iterator; + +import org.springframework.batch.item.ItemReader; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * An {@link ItemReader} that pulls data from a {@link Iterator} or + * {@link Iterable} using the constructors. + * + * @author Juliusz Brzostek + * @author Dave Syer + * @author Mahmoud Ben Hassine + */ +public class IteratorItemReader implements ItemReader { + + /** + * Internal iterator + */ + private final Iterator iterator; + + /** + * Construct a new reader from this iterable (could be a collection), by + * extracting an instance of {@link Iterator} from it. + * + * @param iterable in instance of {@link Iterable} + * + * @see Iterable#iterator() + */ + public IteratorItemReader(Iterable iterable) { + Assert.notNull(iterable, "Iterable argument cannot be null!"); + this.iterator = iterable.iterator(); + } + + /** + * Construct a new reader from this iterator directly. + * @param iterator an instance of {@link Iterator} + */ + public IteratorItemReader(Iterator iterator) { + Assert.notNull(iterator, "Iterator argument cannot be null!"); + this.iterator = iterator; + } + + /** + * Implementation of {@link ItemReader#read()} that just iterates over the + * iterator provided. + */ + @Nullable + @Override + public T read() { + if (iterator.hasNext()) + return iterator.next(); + else + return null; // end of data + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ListItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ListItemReader.java index 108ff8ab8f..2def56dc6e 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ListItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ListItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,6 +21,7 @@ import org.springframework.aop.support.AopUtils; import org.springframework.batch.item.ItemReader; +import org.springframework.lang.Nullable; /** * An {@link ItemReader} that pulls data from a list. Useful for testing. @@ -39,11 +40,12 @@ public ListItemReader(List list) { this.list = list; } else { - this.list = new ArrayList(list); + this.list = new ArrayList<>(list); } } - @Override + @Nullable + @Override public T read() { if (!list.isEmpty()) { return list.remove(0); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ListItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ListItemWriter.java new file mode 100644 index 0000000000..1e1a9c1eca --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ListItemWriter.java @@ -0,0 +1,38 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.support; + +import org.springframework.batch.item.ItemWriter; + +import java.util.ArrayList; +import java.util.List; + +/** + * @author mminella + */ +public class ListItemWriter implements ItemWriter { + + private List writtenItems = new ArrayList<>(); + + @Override + public void write(List items) throws Exception { + writtenItems.addAll(items); + } + + public List getWrittenItems() { + return this.writtenItems; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/PassThroughItemProcessor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/PassThroughItemProcessor.java index e5825cbf13..9c93526115 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/PassThroughItemProcessor.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/PassThroughItemProcessor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,6 +17,7 @@ package org.springframework.batch.item.support; import org.springframework.batch.item.ItemProcessor; +import org.springframework.lang.Nullable; /** * Simple {@link ItemProcessor} that does nothing - simply passes its argument @@ -35,7 +36,8 @@ public class PassThroughItemProcessor implements ItemProcessor { * @return the item * @see ItemProcessor#process(Object) */ - @Override + @Nullable + @Override public T process(T item) throws Exception { return item; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ScriptItemProcessor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ScriptItemProcessor.java new file mode 100644 index 0000000000..17f3b54a30 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/ScriptItemProcessor.java @@ -0,0 +1,140 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.support; + +import org.springframework.lang.Nullable; +import org.springframework.scripting.support.StaticScriptSource; +import org.springframework.util.StringUtils; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.core.io.Resource; +import org.springframework.scripting.ScriptEvaluator; +import org.springframework.scripting.ScriptSource; +import org.springframework.scripting.support.ResourceScriptSource; +import org.springframework.scripting.support.StandardScriptEvaluator; +import org.springframework.util.Assert; + +import java.util.HashMap; +import java.util.Map; + +/** + *

      + * {@link org.springframework.batch.item.ItemProcessor} implementation that passes the current + * item to process to the provided script. Exposes the current item for processing via the + * {@link org.springframework.batch.item.support.ScriptItemProcessor#ITEM_BINDING_VARIABLE_NAME} + * key name ("item"). A custom key name can be set by invoking: + * {@link org.springframework.batch.item.support.ScriptItemProcessor#setItemBindingVariableName} + * with the desired key name. The thread safety of this {@link org.springframework.batch.item.ItemProcessor} + * depends on the implementation of the {@link org.springframework.scripting.ScriptEvaluator} used. + *

      + * + * + * @author Chris Schaefer + * @since 3.0 + */ +public class ScriptItemProcessor implements ItemProcessor, InitializingBean { + private static final String ITEM_BINDING_VARIABLE_NAME = "item"; + + private String language; + private ScriptSource script; + private ScriptSource scriptSource; + private ScriptEvaluator scriptEvaluator; + private String itemBindingVariableName = ITEM_BINDING_VARIABLE_NAME; + + @Nullable + @Override + @SuppressWarnings("unchecked") + public O process(I item) throws Exception { + Map arguments = new HashMap<>(); + arguments.put(itemBindingVariableName, item); + + return (O) scriptEvaluator.evaluate(getScriptSource(), arguments); + } + + /** + *

      + * Sets the {@link org.springframework.core.io.Resource} location of the script to use. + * The script language will be deduced from the filename extension. + *

      + * + * @param resource the {@link org.springframework.core.io.Resource} location of the script to use. + */ + public void setScript(Resource resource) { + Assert.notNull(resource, "The script resource cannot be null"); + + this.script = new ResourceScriptSource(resource); + } + + /** + *

      + * Sets the provided {@link String} as the script source code to use. + *

      + * + * @param scriptSource the {@link String} form of the script source code to use. + * @param language the language of the script. + */ + public void setScriptSource(String scriptSource, String language) { + Assert.hasText(language, "Language must contain the script language"); + Assert.hasText(scriptSource, "Script source must contain the script source to evaluate"); + + this.language = language; + this.scriptSource = new StaticScriptSource(scriptSource); + } + + /** + *

      + * Provides the ability to change the key name that scripts use to obtain the current + * item to process if the variable represented by: + * {@link org.springframework.batch.item.support.ScriptItemProcessor#ITEM_BINDING_VARIABLE_NAME} + * is not suitable ("item"). + *

      + * + * @param itemBindingVariableName the desired binding variable name + */ + public void setItemBindingVariableName(String itemBindingVariableName) { + this.itemBindingVariableName = itemBindingVariableName; + } + + @Override + public void afterPropertiesSet() throws Exception { + scriptEvaluator = new StandardScriptEvaluator(); + + Assert.state(scriptSource != null || script != null, + "Either the script source or script file must be provided"); + + Assert.state(scriptSource == null || script == null, + "Either a script source or script file must be provided, not both"); + + if (scriptSource != null) { + Assert.isTrue(!StringUtils.isEmpty(language), + "Language must be provided when using script source"); + + ((StandardScriptEvaluator) scriptEvaluator).setLanguage(language); + } + } + + private ScriptSource getScriptSource() { + if (script != null) { + return script; + } + + if (scriptSource != null) { + return scriptSource; + } + + throw new IllegalStateException("Either a script source or script needs to be provided."); + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/SingleItemPeekableItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/SingleItemPeekableItemReader.java index c01e1933bc..2b29de94e4 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/SingleItemPeekableItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/SingleItemPeekableItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2010 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,6 +25,7 @@ import org.springframework.batch.item.ParseException; import org.springframework.batch.item.PeekableItemReader; import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.lang.Nullable; /** *

      @@ -34,7 +35,7 @@ *

      * *

      - * Intentionally not thread safe: it wouldn't be possible to honour the peek in + * Intentionally not thread-safe: it wouldn't be possible to honour the peek in * multiple threads because only one of the threads that peeked would get that * item in the next call to read. *

      @@ -66,7 +67,8 @@ public void setDelegate(ItemReader delegate) { * * @see ItemReader#read() */ - @Override + @Nullable + @Override public T read() throws Exception, UnexpectedInputException, ParseException { if (next != null) { T item = next; @@ -86,7 +88,8 @@ public T read() throws Exception, UnexpectedInputException, ParseException { * * @see PeekableItemReader#peek() */ - @Override + @Nullable + @Override public T peek() throws Exception, UnexpectedInputException, ParseException { if (next == null) { updateDelegate(executionContext); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/SynchronizedItemStreamReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/SynchronizedItemStreamReader.java new file mode 100644 index 0000000000..50d200dd5e --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/SynchronizedItemStreamReader.java @@ -0,0 +1,76 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.support; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamReader; +import org.springframework.batch.item.NonTransientResourceException; +import org.springframework.batch.item.ParseException; +import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * + * This is a simple ItemStreamReader decorator with a synchronized ItemReader.read() + * method - which makes a non-thread-safe ItemReader thread-safe. + * + * However, if reprocessing an item is problematic then using this will make a job not + * restartable. + * + * Here are some links about the motivation behind this class: + * - https://projects.spring.io/spring-batch/faq.html#threading-reader} + * - https://stackoverflow.com/a/20002493/2910265} + * + * @author Matthew Ouyang + * @since 3.0.4 + * + * @param type of object being read + */ +public class SynchronizedItemStreamReader implements ItemStreamReader, InitializingBean { + + private ItemStreamReader delegate; + + public void setDelegate(ItemStreamReader delegate) { + this.delegate = delegate; + } + + /** + * This delegates to the read method of the delegate + */ + @Nullable + public synchronized T read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException { + return this.delegate.read(); + } + + public void close() { + this.delegate.close(); + } + + public void open(ExecutionContext executionContext) { + this.delegate.open(executionContext); + } + + public void update(ExecutionContext executionContext) { + this.delegate.update(executionContext); + } + + @Override + public void afterPropertiesSet() throws Exception { + Assert.notNull(this.delegate, "A delegate item reader is required"); + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/ClassifierCompositeItemProcessorBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/ClassifierCompositeItemProcessorBuilder.java new file mode 100644 index 0000000000..698a4b557d --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/ClassifierCompositeItemProcessorBuilder.java @@ -0,0 +1,59 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.support.ClassifierCompositeItemProcessor; +import org.springframework.classify.Classifier; +import org.springframework.util.Assert; + +/** + * Creates a fully qualified {@link ClassifierCompositeItemProcessor}. + * + * @author Glenn Renfro + * + * @since 4.0 + */ +public class ClassifierCompositeItemProcessorBuilder { + + private Classifier> classifier; + + /** + * Establishes the classifier that will determine which {@link ItemProcessor} to use. + * @param classifier the classifier to set + * @return this instance for method chaining + * @see ClassifierCompositeItemProcessor#setClassifier(Classifier) + */ + public ClassifierCompositeItemProcessorBuilder classifier(Classifier> classifier) { + this.classifier = classifier; + + return this; + } + + /** + * Returns a fully constructed {@link ClassifierCompositeItemProcessor}. + * + * @return a new {@link ClassifierCompositeItemProcessor} + */ + public ClassifierCompositeItemProcessor build() { + Assert.notNull(classifier, "A classifier is required."); + + ClassifierCompositeItemProcessor processor = new ClassifierCompositeItemProcessor<>(); + processor.setClassifier(this.classifier); + return processor; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/ClassifierCompositeItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/ClassifierCompositeItemWriterBuilder.java new file mode 100644 index 0000000000..2f5360d75f --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/ClassifierCompositeItemWriterBuilder.java @@ -0,0 +1,63 @@ +/* + * Copyright 2017-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.ClassifierCompositeItemWriter; +import org.springframework.classify.Classifier; +import org.springframework.util.Assert; + +/** + * Creates a fully qualified ClassifierCompositeItemWriter. + * + * @author Glenn Renfro + * @author Mahmoud Ben Hassine + * + * @since 4.0 + */ +public class ClassifierCompositeItemWriterBuilder { + + private Classifier> classifier; + + /** + * Establish the classifier to be used for the selection of which {@link ItemWriter} + * to use. + * + * @param classifier the classifier to set + * @return this instance for method chaining + * @see org.springframework.batch.item.support.ClassifierCompositeItemWriter#setClassifier(Classifier) + */ + public ClassifierCompositeItemWriterBuilder classifier(Classifier> classifier) { + this.classifier = classifier; + + return this; + } + + /** + * Returns a fully constructed {@link ClassifierCompositeItemWriter}. + * + * @return a new {@link ClassifierCompositeItemWriter} + */ + public ClassifierCompositeItemWriter build() { + Assert.notNull(classifier, "A classifier is required."); + + ClassifierCompositeItemWriter writer = new ClassifierCompositeItemWriter<>(); + writer.setClassifier(this.classifier); + return writer; + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/CompositeItemProcessorBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/CompositeItemProcessorBuilder.java new file mode 100644 index 0000000000..0d8a00958a --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/CompositeItemProcessorBuilder.java @@ -0,0 +1,71 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import java.util.Arrays; +import java.util.List; + +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.support.CompositeItemProcessor; +import org.springframework.util.Assert; + +/** + * Creates a fully qualified {@link CompositeItemProcessorBuilder}. + * + * @author Glenn Renfro + * @author Drummond Dawson + * @since 4.0 + */ +public class CompositeItemProcessorBuilder { + private List> delegates; + + /** + * Establishes the {@link ItemProcessor} delegates that will work on the item to be processed. + * @param delegates list of {@link ItemProcessor} delegates that will work on the item. + * @return this instance for method chaining. + * @see CompositeItemProcessor#setDelegates(List) + */ + public CompositeItemProcessorBuilder delegates(List> delegates) { + this.delegates = delegates; + + return this; + } + + /** + * Establishes the {@link ItemProcessor} delegates that will work on the item to be processed. + * @param delegates the {@link ItemProcessor} delegates that will work on the item. + * @return this instance for method chaining. + * @see CompositeItemProcessorBuilder#delegates(List) + */ + public CompositeItemProcessorBuilder delegates(ItemProcessor... delegates) { + return delegates(Arrays.asList(delegates)); + } + + /** + * Returns a fully constructed {@link CompositeItemProcessor}. + * + * @return a new {@link CompositeItemProcessor} + */ + public CompositeItemProcessor build() { + Assert.notNull(delegates, "A list of delegates is required."); + Assert.notEmpty(delegates, "The delegates list must have one or more delegates."); + + CompositeItemProcessor processor = new CompositeItemProcessor<>(); + processor.setDelegates(this.delegates); + return processor; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/CompositeItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/CompositeItemWriterBuilder.java new file mode 100644 index 0000000000..b2ef41c571 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/CompositeItemWriterBuilder.java @@ -0,0 +1,101 @@ +/* + * Copyright 2017-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import java.util.Arrays; +import java.util.List; + +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.CompositeItemWriter; +import org.springframework.util.Assert; + +/** + * Creates a fully qualified CompositeItemWriter. + * + * @author Glenn Renfro + * @author Drummond Dawson + * @author Mahmoud Ben Hassine + * @since 4.0 + */ +public class CompositeItemWriterBuilder { + private List> delegates; + + private boolean ignoreItemStream = false; + + /** + * Establishes the policy whether to call the open, close, or update methods for the + * item writer delegates associated with the CompositeItemWriter. + * + * @param ignoreItemStream if false the delegates' open, close, or update methods will + * be called when the corresponding methods on the CompositeItemWriter are called. If + * true the delegates' open, close, nor update methods will not be called (default is false). + * @return this instance for method chaining. + * + * @see CompositeItemWriter#setIgnoreItemStream(boolean) + */ + public CompositeItemWriterBuilder ignoreItemStream(boolean ignoreItemStream) { + this.ignoreItemStream = ignoreItemStream; + + return this; + } + + /** + * The list of item writers to use as delegates. Items are written to each of the + * delegates. + * + * @param delegates the list of delegates to use. The delegates list must not be null + * nor be empty. + * @return this instance for method chaining. + * + * @see CompositeItemWriter#setDelegates(List) + */ + public CompositeItemWriterBuilder delegates(List> delegates) { + this.delegates = delegates; + + return this; + } + + /** + * The item writers to use as delegates. Items are written to each of the + * delegates. + * + * @param delegates the delegates to use. + * @return this instance for method chaining. + * + * @see CompositeItemWriter#setDelegates(List) + */ + @SafeVarargs + @SuppressWarnings("varargs") + public final CompositeItemWriterBuilder delegates(ItemWriter... delegates) { + return delegates(Arrays.asList(delegates)); + } + + /** + * Returns a fully constructed {@link CompositeItemWriter}. + * + * @return a new {@link CompositeItemWriter} + */ + public CompositeItemWriter build() { + Assert.notNull(delegates, "A list of delegates is required."); + Assert.notEmpty(delegates, "The delegates list must have one or more delegates."); + + CompositeItemWriter writer = new CompositeItemWriter<>(); + writer.setDelegates(this.delegates); + writer.setIgnoreItemStream(this.ignoreItemStream); + return writer; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/ScriptItemProcessorBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/ScriptItemProcessorBuilder.java new file mode 100644 index 0000000000..1653cd55c7 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/ScriptItemProcessorBuilder.java @@ -0,0 +1,129 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import org.springframework.batch.item.support.ScriptItemProcessor; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * Creates a fully qualified ScriptItemProcessor. + * + * @author Glenn Renfro + * + * @since 4.0 + */ +public class ScriptItemProcessorBuilder { + + private String language; + + private Resource scriptResource; + + private String scriptSource; + + private String itemBindingVariableName; + + /** + * Sets the {@link org.springframework.core.io.Resource} location of the script to + * use. The script language will be deduced from the filename extension. + * + * @param resource the {@link org.springframework.core.io.Resource} location of the + * script to use. + * @return this instance for method chaining + * @see ScriptItemProcessor#setScript(Resource) + * + */ + public ScriptItemProcessorBuilder scriptResource(Resource resource) { + this.scriptResource = resource; + + return this; + } + + /** + * Establishes the language of the script. + * + * @param language the language of the script. + * @return this instance for method chaining + * @see ScriptItemProcessor#setScriptSource(String, String) + */ + public ScriptItemProcessorBuilder language(String language) { + this.language = language; + + return this; + } + + /** + * Sets the provided {@link String} as the script source code to use. Language must + * not be null nor empty when using script. + * + * @param scriptSource the {@link String} form of the script source code to use. + * @return this instance for method chaining + * @see ScriptItemProcessor#setScriptSource(String, String) + */ + public ScriptItemProcessorBuilder scriptSource(String scriptSource) { + this.scriptSource = scriptSource; + + return this; + } + + /** + * Provides the ability to change the key name that scripts use to obtain the current + * item to process if the variable represented by: + * {@link ScriptItemProcessor#ITEM_BINDING_VARIABLE_NAME} + * is not suitable ("item"). + * + * @param itemBindingVariableName the desired binding variable name + * @return this instance for method chaining + * @see ScriptItemProcessor#setItemBindingVariableName(String) + */ + public ScriptItemProcessorBuilder itemBindingVariableName(String itemBindingVariableName) { + this.itemBindingVariableName = itemBindingVariableName; + + return this; + } + + /** + * Returns a fully constructed {@link ScriptItemProcessor}. + * + * @return a new {@link ScriptItemProcessor} + */ + public ScriptItemProcessor build() { + if (this.scriptResource == null && !StringUtils.hasText(this.scriptSource)) { + throw new IllegalArgumentException("scriptResource or scriptSource is required."); + } + + if (StringUtils.hasText(this.scriptSource)) { + Assert.hasText(this.language, "language is required when using scriptSource."); + } + + ScriptItemProcessor processor = new ScriptItemProcessor<>(); + if (StringUtils.hasText(this.itemBindingVariableName)) { + processor.setItemBindingVariableName(this.itemBindingVariableName); + } + + if (this.scriptResource != null) { + processor.setScript(this.scriptResource); + } + + if (this.scriptSource != null) { + processor.setScriptSource(this.scriptSource, this.language); + } + + return processor; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/SingleItemPeekableItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/SingleItemPeekableItemReaderBuilder.java new file mode 100644 index 0000000000..6b95d05313 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/SingleItemPeekableItemReaderBuilder.java @@ -0,0 +1,61 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.support.SingleItemPeekableItemReader; +import org.springframework.util.Assert; + +/** + * Creates a fully qualified SingleItemPeekeableItemReader. + * + * @author Glenn Renfro + * + * @since 4.0 + */ +public class SingleItemPeekableItemReaderBuilder { + + private ItemReader delegate; + + /** + * The item reader to use as a delegate. Items are read from the delegate and passed + * to the caller in + * {@link org.springframework.batch.item.support.SingleItemPeekableItemReader#read()}. + * + * @param delegate the delegate to set + * @return this instance for method chaining + * @see SingleItemPeekableItemReader#setDelegate(ItemReader) + */ + public SingleItemPeekableItemReaderBuilder delegate(ItemReader delegate) { + this.delegate = delegate; + + return this; + } + + /** + * Returns a fully constructed {@link SingleItemPeekableItemReader}. + * + * @return a new {@link SingleItemPeekableItemReader} + */ + public SingleItemPeekableItemReader build() { + Assert.notNull(this.delegate, "A delegate is required"); + + SingleItemPeekableItemReader reader = new SingleItemPeekableItemReader<>(); + reader.setDelegate(this.delegate); + return reader; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/SynchronizedItemStreamReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/SynchronizedItemStreamReaderBuilder.java new file mode 100644 index 0000000000..560b0c06d4 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/SynchronizedItemStreamReaderBuilder.java @@ -0,0 +1,61 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import org.springframework.batch.item.ItemStreamReader; +import org.springframework.batch.item.support.SynchronizedItemStreamReader; +import org.springframework.util.Assert; + +/** + * Creates a fully qualified SynchronizedItemStreamReader. + * + * @author Glenn Renfro + * + * @since 4.0 + */ +public class SynchronizedItemStreamReaderBuilder { + + private ItemStreamReader delegate; + + /** + * The item stream reader to use as a delegate. Items are read from the delegate and + * passed to the caller in + * {@link org.springframework.batch.item.support.SynchronizedItemStreamReader#read()}. + * + * @param delegate the delegate to set + * @return this instance for method chaining + * @see SynchronizedItemStreamReader#setDelegate(ItemStreamReader) + */ + public SynchronizedItemStreamReaderBuilder delegate(ItemStreamReader delegate) { + this.delegate = delegate; + + return this; + } + + /** + * Returns a fully constructed {@link SynchronizedItemStreamReader}. + * + * @return a new {@link SynchronizedItemStreamReader} + */ + public SynchronizedItemStreamReader build() { + Assert.notNull(this.delegate, "A delegate is required"); + + SynchronizedItemStreamReader reader = new SynchronizedItemStreamReader<>(); + reader.setDelegate(this.delegate); + return reader; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/package-info.java new file mode 100644 index 0000000000..672042949c --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/builder/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Builders for support classes. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.support.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/package-info.java new file mode 100644 index 0000000000..3b67e5685a --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Internal support package + *

      + */ +@NonNullApi +package org.springframework.batch.item.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/package.html deleted file mode 100644 index ca70983dfe..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/support/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Internal support package -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/util/ExecutionContextUserSupport.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/util/ExecutionContextUserSupport.java index 336d3c900e..1b3365f4c6 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/util/ExecutionContextUserSupport.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/util/ExecutionContextUserSupport.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -54,10 +54,13 @@ public void setName(String name) { /** * Prefix the argument with {@link #getName()} to create a unique key that can be safely used to identify data * stored in {@link ExecutionContext}. + * + * @param suffix {@link String} to be used to generate the key. + * @return the key that was generated based on the name and the suffix. */ - public String getKey(String s) { + public String getKey(String suffix) { Assert.hasText(name, "Name must be assigned for the sake of defining the execution context keys prefix."); - return name + "." + s; + return name + "." + suffix; } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/util/FileUtils.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/util/FileUtils.java index 4a16dd1dd5..7d78a2444c 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/util/FileUtils.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/util/FileUtils.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -43,15 +43,10 @@ private FileUtils() { * @param append true signals input file may already exist (but doesn't have to) * @param overwriteOutputFile If set to true, output file will be overwritten (this flag is ignored when processing * is restart) - * - * @throws IllegalArgumentException when file is null - * @throws ItemStreamException when starting output file processing, file exists and flag "overwriteOutputFile" is - * set to false - * @throws ItemStreamException when unable to create file or file is not writable */ public static void setUpOutputFile(File file, boolean restarted, boolean append, boolean overwriteOutputFile) { - Assert.notNull(file); + Assert.notNull(file, "An output file is required"); try { if (!restarted) { @@ -74,6 +69,9 @@ public static void setUpOutputFile(File file, boolean restarted, boolean append, } else { if (!file.exists()) { + if (file.getParent() != null) { + new File(file.getParent()).mkdirs(); + } if (!createNewFile(file)) { throw new ItemStreamException("Output file was not created: [" + file.getAbsolutePath() + "]"); @@ -92,8 +90,23 @@ public static void setUpOutputFile(File file, boolean restarted, boolean append, } /** + * Set up output file for batch processing. This method implements common logic for handling output files when + * starting or restarting file I/O. When starting output file processing, creates/overwrites new file. When + * restarting output file processing, checks whether file is writable. + * + * @param file file to be set up + * @param restarted true signals that we are restarting output file processing + * @param overwriteOutputFile If set to true, output file will be overwritten (this flag is ignored when processing + * is restart) + * + * @throws IllegalArgumentException when file is null + * @throws ItemStreamException when starting output file processing, file exists and flag "overwriteOutputFile" is + * set to false + * @throws ItemStreamException when unable to create file or file is not writable + * * @deprecated use the version with explicit append parameter instead. Here append=false is assumed. */ + @Deprecated public static void setUpOutputFile(File file, boolean restarted, boolean overwriteOutputFile) { setUpOutputFile(file, restarted, false, overwriteOutputFile); } @@ -102,6 +115,10 @@ public static void setUpOutputFile(File file, boolean restarted, boolean overwri * Create a new file if it doesn't already exist. * * @param file the file to create on the filesystem + * @return true if file was created else false. + * + * @throws IOException is thrown if error occurs during creation and file + * does not exist. */ public static boolean createNewFile(File file) throws IOException { @@ -113,7 +130,7 @@ public static boolean createNewFile(File file) throws IOException { return file.createNewFile() && file.exists(); } catch (IOException e) { - // On some filesystems you can get an exception here even though the + // On some file systems you can get an exception here even though the // files was successfully created if (file.exists()) { return true; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/util/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/util/package-info.java new file mode 100644 index 0000000000..2e35f78b8e --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/util/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Infrastructure utility classes. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.util; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/BeanValidatingItemProcessor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/BeanValidatingItemProcessor.java new file mode 100644 index 0000000000..8056a04876 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/BeanValidatingItemProcessor.java @@ -0,0 +1,65 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.validator; + +import javax.validation.Validator; + +import org.springframework.util.Assert; +import org.springframework.validation.beanvalidation.LocalValidatorFactoryBean; +import org.springframework.validation.beanvalidation.SpringValidatorAdapter; + +/** + * A {@link ValidatingItemProcessor} that uses the Bean Validation API (JSR-303) + * to validate items. + * + * @param type of items to validate + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public class BeanValidatingItemProcessor extends ValidatingItemProcessor { + + private Validator validator; + + /** + * Create a new instance of {@link BeanValidatingItemProcessor} with the + * default configuration. + */ + public BeanValidatingItemProcessor() { + LocalValidatorFactoryBean localValidatorFactoryBean = new LocalValidatorFactoryBean(); + localValidatorFactoryBean.afterPropertiesSet(); + this.validator = localValidatorFactoryBean.getValidator(); + } + + /** + * Create a new instance of {@link BeanValidatingItemProcessor}. + * @param localValidatorFactoryBean used to configure the Bean Validation validator + */ + public BeanValidatingItemProcessor(LocalValidatorFactoryBean localValidatorFactoryBean) { + Assert.notNull(localValidatorFactoryBean, "localValidatorFactoryBean must not be null"); + this.validator = localValidatorFactoryBean.getValidator(); + } + + @Override + public void afterPropertiesSet() throws Exception { + SpringValidatorAdapter springValidatorAdapter = new SpringValidatorAdapter(this.validator); + SpringValidator springValidator = new SpringValidator<>(); + springValidator.setValidator(springValidatorAdapter); + springValidator.afterPropertiesSet(); + setValidator(springValidator); + super.afterPropertiesSet(); + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/SpringValidator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/SpringValidator.java index 888098014c..42f3ca728e 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/SpringValidator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/SpringValidator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -59,7 +59,7 @@ public void validate(T item) throws ValidationException { * @return string of field errors followed by global errors. */ private String errorsToString(Errors errors) { - StringBuffer builder = new StringBuffer(); + StringBuilder builder = new StringBuilder(); appendCollection(errors.getFieldErrors(), builder); appendCollection(errors.getGlobalErrors(), builder); @@ -71,7 +71,7 @@ private String errorsToString(Errors errors) { * Append the string representation of elements of the collection (separated * by new lines) to the given StringBuilder. */ - private void appendCollection(Collection collection, StringBuffer builder) { + private void appendCollection(Collection collection, StringBuilder builder) { for (Object value : collection) { builder.append("\n"); builder.append(value.toString()); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/ValidatingItemProcessor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/ValidatingItemProcessor.java index f419295acb..156d362e00 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/ValidatingItemProcessor.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/ValidatingItemProcessor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,6 +17,7 @@ import org.springframework.batch.item.ItemProcessor; import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -43,6 +44,8 @@ public ValidatingItemProcessor() { /** * Creates a ValidatingItemProcessor based on the given Validator. + * + * @param validator the {@link Validator} instance to be used. */ public ValidatingItemProcessor(Validator validator) { this.validator = validator; @@ -51,7 +54,7 @@ public ValidatingItemProcessor(Validator validator) { /** * Set the validator used to validate each item. * - * @param validator + * @param validator the {@link Validator} instance to be used. */ public void setValidator(Validator validator) { this.validator = validator; @@ -60,7 +63,9 @@ public void setValidator(Validator validator) { /** * Should the processor filter invalid records instead of skipping them? * - * @param filter + * @param filter if set to {@code true}, items that fail validation are filtered + * ({@code null} is returned). Otherwise, a {@link ValidationException} will be + * thrown. */ public void setFilter(boolean filter) { this.filter = filter; @@ -72,7 +77,8 @@ public void setFilter(boolean filter) { * @return the input item * @throws ValidationException if validation fails */ - @Override + @Nullable + @Override public T process(T item) throws ValidationException { try { validator.validate(item); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/ValidationException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/ValidationException.java index d58fe913b7..e883d38fc6 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/ValidationException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/ValidationException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,6 +23,7 @@ * * @author Ben Hale */ +@SuppressWarnings("serial") public class ValidationException extends ItemReaderException { /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/Validator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/Validator.java index 167fa0cc73..91a0ac2e1a 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/Validator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/Validator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/package-info.java new file mode 100644 index 0000000000..c56d78dab1 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of item validator concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.item.validator; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/package.html deleted file mode 100644 index 26c82e9c14..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/validator/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of item validator concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxEventItemReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxEventItemReader.java index a2ec0f80a4..ee7598cfc5 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxEventItemReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxEventItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,7 +16,10 @@ package org.springframework.batch.item.xml; +import java.io.IOException; import java.io.InputStream; +import java.util.ArrayList; +import java.util.List; import java.util.NoSuchElementException; import javax.xml.namespace.QName; @@ -36,9 +39,11 @@ import org.springframework.batch.item.xml.stax.FragmentEventReader; import org.springframework.beans.factory.InitializingBean; import org.springframework.core.io.Resource; +import org.springframework.lang.Nullable; import org.springframework.oxm.Unmarshaller; import org.springframework.util.Assert; import org.springframework.util.ClassUtils; +import org.springframework.util.StringUtils; /** * Item reader for reading XML input based on StAX. @@ -47,9 +52,10 @@ * wrapped with StartDocument and EndDocument events so that the fragments can be further processed like standalone XML * documents. * - * The implementation is *not* thread-safe. + * The implementation is not thread-safe. * * @author Robert Kasanicky + * @author Mahmoud Ben Hassine */ public class StaxEventItemReader extends AbstractItemCountingItemStreamItemReader implements ResourceAwareItemReaderItemStream, InitializingBean { @@ -66,13 +72,13 @@ public class StaxEventItemReader extends AbstractItemCountingItemStreamItemRe private InputStream inputStream; - private String fragmentRootElementName; + private List fragmentRootElementNames; private boolean noInput; private boolean strict = true; - private String fragmentRootElementNameSpace; + private XMLInputFactory xmlInputFactory = StaxUtils.createXmlInputFactory(); public StaxEventItemReader() { setName(ClassUtils.getShortName(StaxEventItemReader.class)); @@ -81,7 +87,7 @@ public StaxEventItemReader() { /** * In strict mode the reader will throw an exception on * {@link #open(org.springframework.batch.item.ExecutionContext)} if the input resource does not exist. - * @param strict false by default + * @param strict true by default */ public void setStrict(boolean strict) { this.strict = strict; @@ -103,7 +109,26 @@ public void setUnmarshaller(Unmarshaller unmarshaller) { * @param fragmentRootElementName name of the root element of the fragment */ public void setFragmentRootElementName(String fragmentRootElementName) { - this.fragmentRootElementName = fragmentRootElementName; + setFragmentRootElementNames(new String[] {fragmentRootElementName}); + } + + /** + * @param fragmentRootElementNames list of the names of the root element of the fragment + */ + public void setFragmentRootElementNames(String[] fragmentRootElementNames) { + this.fragmentRootElementNames = new ArrayList<>(); + for (String fragmentRootElementName : fragmentRootElementNames) { + this.fragmentRootElementNames.add(parseFragmentRootElementName(fragmentRootElementName)); + } + } + + /** + * Set the {@link XMLInputFactory}. + * @param xmlInputFactory to use + */ + public void setXmlInputFactory(XMLInputFactory xmlInputFactory) { + Assert.notNull(xmlInputFactory, "XMLInputFactory must not be null"); + this.xmlInputFactory = xmlInputFactory; } /** @@ -117,11 +142,10 @@ public void setFragmentRootElementName(String fragmentRootElementName) { @Override public void afterPropertiesSet() throws Exception { Assert.notNull(unmarshaller, "The Unmarshaller must not be null."); - Assert.hasLength(fragmentRootElementName, "The FragmentRootElementName must not be null"); - if (fragmentRootElementName.contains("{")) { - fragmentRootElementNameSpace = fragmentRootElementName.replaceAll("\\{(.*)\\}.*", "$1"); - fragmentRootElementName = fragmentRootElementName.replaceAll("\\{.*\\}(.*)", "$1"); - } + Assert.notEmpty(fragmentRootElementNames, "The FragmentRootElementNames must not be empty"); + for (QName fragmentRootElementName : fragmentRootElementNames) { + Assert.hasText(fragmentRootElementName.getLocalPart(), "The FragmentRootElementNames must not contain empty elements"); + } } /** @@ -129,6 +153,8 @@ public void afterPropertiesSet() throws Exception { * * This implementation simply looks for the next corresponding element, it does not care about element nesting. You * will need to override this method to correctly handle composite fragments. + * + * @param reader the {@link XMLEventReader} to be used to find next fragment. * * @return true if next fragment was found, false otherwise. * @@ -145,11 +171,8 @@ protected boolean moveCursorToNextFragment(XMLEventReader reader) throws NonTran return false; } QName startElementName = ((StartElement) reader.peek()).getName(); - if (startElementName.getLocalPart().equals(fragmentRootElementName)) { - if (fragmentRootElementNameSpace == null - || startElementName.getNamespaceURI().equals(fragmentRootElementNameSpace)) { - return true; - } + if (isFragmentRootElementName(startElementName)) { + return true; } reader.nextEvent(); @@ -198,7 +221,7 @@ protected void doOpen() throws Exception { } inputStream = resource.getInputStream(); - eventReader = XMLInputFactory.newInstance().createXMLEventReader(inputStream); + eventReader = xmlInputFactory.createXMLEventReader(inputStream); fragmentReader = new DefaultFragmentEventReader(eventReader); noInput = false; @@ -207,8 +230,9 @@ protected void doOpen() throws Exception { /** * Move to next fragment and map it to item. */ + @Nullable @Override - protected T doRead() throws Exception { + protected T doRead() throws IOException, XMLStreamException { if (noInput) { return null; @@ -249,8 +273,8 @@ protected T doRead() throws Exception { protected void jumpToItem(int itemIndex) throws Exception { for (int i = 0; i < itemIndex; i++) { try { - readToStartFragment(); - readToEndFragment(); + QName fragmentName = readToStartFragment(); + readToEndFragment(fragmentName); } catch (NoSuchElementException e) { if (itemIndex == (i + 1)) { // we can presume a NoSuchElementException on the last item means the EOF was reached on the last run @@ -264,16 +288,16 @@ protected void jumpToItem(int itemIndex) throws Exception { } /* - * Read until the first StartElement tag that matches the provided fragmentRootElementName. Because there may be any + * Read until the first StartElement tag that matches any of the provided fragmentRootElementNames. Because there may be any * number of tags in between where the reader is now and the fragment start, this is done in a loop until the * element type and name match. */ - private void readToStartFragment() throws XMLStreamException { + private QName readToStartFragment() throws XMLStreamException { while (true) { XMLEvent nextEvent = eventReader.nextEvent(); if (nextEvent.isStartElement() - && ((StartElement) nextEvent).getName().getLocalPart().equals(fragmentRootElementName)) { - return; + && isFragmentRootElementName(((StartElement) nextEvent).getName())) { + return ((StartElement) nextEvent).getName(); } } } @@ -283,13 +307,36 @@ private void readToStartFragment() throws XMLStreamException { * number of tags in between where the reader is now and the fragment end tag, this is done in a loop until the * element type and name match */ - private void readToEndFragment() throws XMLStreamException { + private void readToEndFragment(QName fragmentRootElementName) throws XMLStreamException { while (true) { XMLEvent nextEvent = eventReader.nextEvent(); if (nextEvent.isEndElement() - && ((EndElement) nextEvent).getName().getLocalPart().equals(fragmentRootElementName)) { + && fragmentRootElementName.equals(((EndElement) nextEvent).getName())) { return; } } } + + private boolean isFragmentRootElementName(QName name) { + for (QName fragmentRootElementName : fragmentRootElementNames) { + if (fragmentRootElementName.getLocalPart().equals(name.getLocalPart())) { + if (!StringUtils.hasText(fragmentRootElementName.getNamespaceURI()) + || fragmentRootElementName.getNamespaceURI().equals(name.getNamespaceURI())) { + return true; + } + } + } + return false; + } + + private QName parseFragmentRootElementName(String fragmentRootElementName) { + String name = fragmentRootElementName; + String nameSpace = null; + if (fragmentRootElementName.contains("{")) { + nameSpace = fragmentRootElementName.replaceAll("\\{(.*)\\}.*", "$1"); + name = fragmentRootElementName.replaceAll("\\{.*\\}(.*)", "$1"); + } + return new QName(nameSpace, name, ""); + } + } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxEventItemWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxEventItemWriter.java index 9b4468fae1..b5f9f8947b 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxEventItemWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxEventItemWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2013 the original author or authors. + * Copyright 2006-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,9 +24,10 @@ import java.io.UnsupportedEncodingException; import java.io.Writer; import java.nio.channels.FileChannel; +import java.util.Collections; import java.util.List; import java.util.Map; - +import javax.xml.namespace.QName; import javax.xml.stream.FactoryConfigurationError; import javax.xml.stream.XMLEventFactory; import javax.xml.stream.XMLEventWriter; @@ -36,14 +37,18 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemStreamException; import org.springframework.batch.item.ItemWriter; import org.springframework.batch.item.WriteFailedException; +import org.springframework.batch.item.WriterNotOpenException; import org.springframework.batch.item.file.ResourceAwareItemWriterItemStream; import org.springframework.batch.item.support.AbstractItemStreamItemWriter; import org.springframework.batch.item.util.FileUtils; import org.springframework.batch.item.xml.stax.NoStartEndDocumentStreamWriter; +import org.springframework.batch.item.xml.stax.UnclosedElementCollectingEventWriter; +import org.springframework.batch.item.xml.stax.UnopenedElementClosingEventWriter; import org.springframework.batch.support.transaction.TransactionAwareBufferedWriter; import org.springframework.beans.factory.InitializingBean; import org.springframework.core.io.Resource; @@ -62,7 +67,7 @@ * This item writer also provides restart, statistics and transaction features * by implementing corresponding interfaces. * - * The implementation is *not* thread-safe. + * The implementation is not thread-safe. * * @author Peter Zozom * @author Robert Kasanicky @@ -75,17 +80,20 @@ public class StaxEventItemWriter extends AbstractItemStreamItemWriter impl private static final Log log = LogFactory.getLog(StaxEventItemWriter.class); // default encoding - private static final String DEFAULT_ENCODING = "UTF-8"; + public static final String DEFAULT_ENCODING = "UTF-8"; // default encoding - private static final String DEFAULT_XML_VERSION = "1.0"; + public static final String DEFAULT_XML_VERSION = "1.0"; // default root tag name - private static final String DEFAULT_ROOT_TAG_NAME = "root"; + public static final String DEFAULT_ROOT_TAG_NAME = "root"; // restart data property name private static final String RESTART_DATA_NAME = "position"; + // unclosed header callback elements property name + private static final String UNCLOSED_HEADER_CALLBACK_ELEMENTS_NAME = "unclosedHeaderCallbackElements"; + // restart data property name private static final String WRITE_STATISTICS_NAME = "record.count"; @@ -143,6 +151,13 @@ public class StaxEventItemWriter extends AbstractItemStreamItemWriter impl private boolean forceSync; private boolean shouldDeleteIfEmpty = false; + + private boolean restarted = false; + + private boolean initialized = false; + + // List holding the QName of elements that were opened in the header callback, but not closed + private List unclosedHeaderCallbackElements = Collections.emptyList(); public StaxEventItemWriter() { setExecutionContextName(ClassUtils.getShortName(StaxEventItemWriter.class)); @@ -169,6 +184,8 @@ public void setMarshaller(Marshaller marshaller) { /** * headerCallback is called before writing any items. + * + * @param headerCallback the {@link StaxWriterCallback} to be called prior to writing items. */ public void setHeaderCallback(StaxWriterCallback headerCallback) { this.headerCallback = headerCallback; @@ -176,7 +193,9 @@ public void setHeaderCallback(StaxWriterCallback headerCallback) { /** * footerCallback is called after writing all items but before closing the - * file + * file. + * + *@param footerCallback the {@link StaxWriterCallback} to be called after writing items. */ public void setFooterCallback(StaxWriterCallback footerCallback) { this.footerCallback = footerCallback; @@ -235,7 +254,7 @@ public void setEncoding(String encoding) { /** * Get XML version. - * + * * @return the XML version used */ public String getVersion() { @@ -321,7 +340,8 @@ public void setRootElementAttributes(Map rootElementAttributes) * Set "overwrite" flag for the output file. Flag is ignored when output * file processing is restarted. * - * @param overwriteOutput + * @param overwriteOutput If set to true, output file will be overwritten + * (this flag is ignored when processing is restart). */ public void setOverwriteOutput(boolean overwriteOutput) { this.overwriteOutput = overwriteOutput; @@ -332,12 +352,12 @@ public void setSaveState(boolean saveState) { } /** - * @throws Exception + * @throws Exception thrown if error occurs * @see org.springframework.beans.factory.InitializingBean#afterPropertiesSet() */ @Override public void afterPropertiesSet() throws Exception { - Assert.notNull(marshaller); + Assert.notNull(marshaller, "A Marshaller is required"); if (rootTagName.contains("{")) { rootTagNamespace = rootTagName.replaceAll("\\{(.*)\\}.*", "$1"); rootTagName = rootTagName.replaceAll("\\{.*\\}(.*)", "$1"); @@ -350,9 +370,12 @@ public void afterPropertiesSet() throws Exception { /** * Open the output source + * + * @param executionContext the batch context. * * @see org.springframework.batch.item.ItemStream#open(ExecutionContext) */ + @SuppressWarnings("unchecked") @Override public void open(ExecutionContext executionContext) { super.open(executionContext); @@ -360,13 +383,17 @@ public void open(ExecutionContext executionContext) { Assert.notNull(resource, "The resource must be set"); long startAtPosition = 0; - boolean restarted = false; - + // if restart data is provided, restart from provided offset // otherwise start from beginning if (executionContext.containsKey(getExecutionContextKey(RESTART_DATA_NAME))) { startAtPosition = executionContext.getLong(getExecutionContextKey(RESTART_DATA_NAME)); currentRecordCount = executionContext.getLong(getExecutionContextKey(WRITE_STATISTICS_NAME)); + if (executionContext.containsKey(getExecutionContextKey(UNCLOSED_HEADER_CALLBACK_ELEMENTS_NAME))) { + unclosedHeaderCallbackElements = (List) executionContext + .get(getExecutionContextKey(UNCLOSED_HEADER_CALLBACK_ELEMENTS_NAME)); + } + restarted = true; if (shouldDeleteIfEmpty && currentRecordCount == 0) { // previous execution deleted the output file because no items were written @@ -375,14 +402,19 @@ public void open(ExecutionContext executionContext) { } else { restarted = true; } + } else { + currentRecordCount = 0; + restarted = false; } - open(startAtPosition, restarted); + open(startAtPosition); if (startAtPosition == 0) { try { if (headerCallback != null) { - headerCallback.write(delegateEventWriter); + UnclosedElementCollectingEventWriter headerCallbackWriter = new UnclosedElementCollectingEventWriter(delegateEventWriter); + headerCallback.write(headerCallbackWriter); + unclosedHeaderCallbackElements = headerCallbackWriter.getUnclosedElements(); } } catch (IOException e) { @@ -390,12 +422,14 @@ public void open(ExecutionContext executionContext) { } } + this.initialized = true; + } /** * Helper method for opening output source at given file position */ - private void open(long position, boolean restarted) { + private void open(long position) { File file; FileOutputStream os = null; @@ -420,14 +454,14 @@ private void open(long position, boolean restarted) { // If the current XMLOutputFactory implementation is supplied by // Woodstox >= 3.2.9 we want to disable its // automatic end element feature (see: - // http://jira.codehaus.org/browse/WSTX-165) per - // http://jira.springframework.org/browse/BATCH-761). + // https://jira.codehaus.org/browse/WSTX-165) per + // https://jira.spring.io/browse/BATCH-761). outputFactory.setProperty("com.ctc.wstx.automaticEndElements", Boolean.FALSE); } if (outputFactory.isPropertySupported("com.ctc.wstx.outputValidateStructure")) { // On restart we don't write the root element so we have to disable // structural validation (see: - // http://jira.springframework.org/browse/BATCH-1681). + // https://jira.spring.io/browse/BATCH-1681). outputFactory.setProperty("com.ctc.wstx.outputValidateStructure", Boolean.FALSE); } @@ -472,10 +506,13 @@ public void run() { /** * Subclasses can override to customize the writer. - * @param outputFactory - * @param writer + * + * @param outputFactory the factory to be used to create an {@link XMLEventWriter}. + * @param writer the {@link Writer} to be used by the {@link XMLEventWriter} for + * writing to character streams. * @return an xml writer - * @throws XMLStreamException + * + * @throws XMLStreamException thrown if error occured creating {@link XMLEventWriter}. */ protected XMLEventWriter createXmlEventWriter(XMLOutputFactory outputFactory, Writer writer) throws XMLStreamException { @@ -484,8 +521,10 @@ protected XMLEventWriter createXmlEventWriter(XMLOutputFactory outputFactory, Wr /** * Subclasses can override to customize the factory. + * * @return a factory for the xml output - * @throws FactoryConfigurationError + * + * @throws FactoryConfigurationError throw if an instance of this factory cannot be loaded. */ protected XMLOutputFactory createXmlOutputFactory() throws FactoryConfigurationError { return XMLOutputFactory.newInstance(); @@ -493,8 +532,10 @@ protected XMLOutputFactory createXmlOutputFactory() throws FactoryConfigurationE /** * Subclasses can override to customize the event factory. + * * @return a factory for the xml events - * @throws FactoryConfigurationError + * + * @throws FactoryConfigurationError thrown if an instance of this factory cannot be loaded. */ protected XMLEventFactory createXmlEventFactory() throws FactoryConfigurationError { XMLEventFactory factory = XMLEventFactory.newInstance(); @@ -502,11 +543,11 @@ protected XMLEventFactory createXmlEventFactory() throws FactoryConfigurationErr } /** - * Subclasses can override to customize the stax result. + * Subclasses can override to customize the STAX result. + * * @return a result for writing to - * @throws Exception */ - protected Result createStaxResult() throws Exception { + protected Result createStaxResult() { return StaxUtils.getResult(eventWriter); } @@ -518,7 +559,9 @@ protected Result createStaxResult() throws Exception { * * * @param writer XML event writer - * @throws XMLStreamException + * + * @throws XMLStreamException thrown if error occurs while setting the + * prefix or default name space. */ protected void initNamespaceContext(XMLEventWriter writer) throws XMLStreamException { if (StringUtils.hasText(getRootTagNamespace())) { @@ -536,7 +579,9 @@ protected void initNamespaceContext(XMLEventWriter writer) throws XMLStreamExcep if (key.contains(":")) { prefix = key.substring(key.indexOf(":") + 1); } - log.debug("registering prefix: " +prefix + "=" + entry.getValue()); + if (log.isDebugEnabled()) { + log.debug("registering prefix: " +prefix + "=" + entry.getValue()); + } writer.setPrefix(prefix, entry.getValue()); } } @@ -553,7 +598,8 @@ protected void initNamespaceContext(XMLEventWriter writer) throws XMLStreamExcep * version and root tag name can be retrieved with corresponding getters. * * @param writer XML event writer - * @throws XMLStreamException + * + * @throws XMLStreamException thrown if error occurs. */ protected void startDocument(XMLEventWriter writer) throws XMLStreamException { @@ -605,7 +651,8 @@ protected void startDocument(XMLEventWriter writer) throws XMLStreamException { * Writes the EndDocument tag manually. * * @param writer XML event writer - * @throws XMLStreamException + * + * @throws XMLStreamException thrown if error occurs. */ protected void endDocument(XMLEventWriter writer) throws XMLStreamException { @@ -640,7 +687,12 @@ public void close() { try { if (footerCallback != null) { - footerCallback.write(delegateEventWriter); + XMLEventWriter footerCallbackWriter = delegateEventWriter; + if (restarted && !unclosedHeaderCallbackElements.isEmpty()) { + footerCallbackWriter = new UnopenedElementClosingEventWriter( + delegateEventWriter, bufferedWriter, unclosedHeaderCallbackElements); + } + footerCallback.write(footerCallbackWriter); } delegateEventWriter.flush(); endDocument(delegateEventWriter); @@ -681,6 +733,8 @@ public void close() { } } } + + this.initialized = false; } private void closeStream() { @@ -696,11 +750,16 @@ private void closeStream() { * Write the value objects and flush them to the file. * * @param items the value object - * @throws IOException - * @throws XmlMappingException + * + * @throws IOException thrown if general error occurs. + * @throws XmlMappingException thrown if error occurs during XML Mapping. */ @Override - public void write(List items) throws XmlMappingException, Exception { + public void write(List items) throws XmlMappingException, IOException { + + if(!this.initialized) { + throw new WriterNotOpenException("Writer must be open before it can be written to"); + } currentRecordCount += items.size(); @@ -716,17 +775,15 @@ public void write(List items) throws XmlMappingException, Exception channel.force(false); } } - catch (XMLStreamException e) { + catch (XMLStreamException | IOException e) { throw new WriteFailedException("Failed to flush the events", e); } - catch (IOException e) { - throw new WriteFailedException("Failed to flush the events", e); - } - } /** * Get the restart data. + * + * @param executionContext the batch context. * * @see org.springframework.batch.item.ItemStream#update(ExecutionContext) */ @@ -737,6 +794,10 @@ public void update(ExecutionContext executionContext) { Assert.notNull(executionContext, "ExecutionContext must not be null"); executionContext.putLong(getExecutionContextKey(RESTART_DATA_NAME), getPosition()); executionContext.putLong(getExecutionContextKey(WRITE_STATISTICS_NAME), currentRecordCount); + if (!unclosedHeaderCallbackElements.isEmpty()) { + executionContext.put(getExecutionContextKey(UNCLOSED_HEADER_CALLBACK_ELEMENTS_NAME), + unclosedHeaderCallbackElements); + } } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxUtils.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxUtils.java index 2d039f9b5d..d6e8162473 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxUtils.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxUtils.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,131 +16,39 @@ package org.springframework.batch.item.xml; -import java.lang.reflect.Constructor; -import java.lang.reflect.Method; - import javax.xml.stream.XMLEventReader; import javax.xml.stream.XMLEventWriter; +import javax.xml.stream.XMLInputFactory; +import javax.xml.stream.XMLStreamException; import javax.xml.transform.Result; import javax.xml.transform.Source; +import javax.xml.transform.stax.StAXResult; +import javax.xml.transform.stax.StAXSource; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.springframework.util.Assert; -import org.springframework.util.ClassUtils; /** - * This class provides a little bit of indirection to avoid ugly conditional object creation. It is unfortunately - * a bit redundant assuming a Spring 3.0 environment, but is necessary to work with Spring WS 1.5.x. - *

      - * The returned object determines whether the environment has Spring OXM as included in the Spring 3.x series of relies - * or whether it has Spring OXM from Spring WS 1.5x and factories a StaxSource instance appropriately. - *

      - * As the only class state maintained is to cache java reflection metadata, which is thread safe, this class is thread-safe. + * StAX utility methods. + *
      + * This class is thread-safe. * * @author Josh Long + * @author Mahmoud Ben Hassine * */ -@SuppressWarnings("restriction") public abstract class StaxUtils { - private static final Log logger = LogFactory.getLog(StaxUtils.class); - - private static ClassLoader defaultClassLoader = ClassUtils.getDefaultClassLoader(); - - // regular object. - private static String staxSourceClassNameOnSpringWs15 = "org.springframework.xml.transform.StaxSource"; - private static String staxResultClassNameOnSpringOxm15 = "org.springframework.xml.transform.StaxResult"; - - // in Spring 3, StaxUtils is package private, so use static utility StaxUtils#createStaxSource / StaxUtils#createStaxResult - private static String staxSourceClassNameOnSpringOxm30 = "org.springframework.util.xml.StaxUtils"; - - private static boolean hasSpringWs15StaxSupport = ClassUtils.isPresent(staxSourceClassNameOnSpringWs15, defaultClassLoader); - - private static boolean hasSpring30StaxSupport = ClassUtils.isPresent(staxSourceClassNameOnSpringOxm30, defaultClassLoader); - - private static Method staxUtilsSourceMethodOnSpring30, staxUtilsResultMethodOnSpring30; - - @SuppressWarnings("rawtypes") - private static Constructor staxSourceClassCtorOnSpringWs15, staxResultClassCtorOnSpringWs15; - - static { - try { - - // cache the factory method / constructor so that we spend as little time in reflection as possible - if (hasSpring30StaxSupport) { - Class clzz = ClassUtils.forName(staxSourceClassNameOnSpringOxm30, defaultClassLoader); - - // javax.xml.transform.Source - staxUtilsSourceMethodOnSpring30 = ClassUtils.getStaticMethod(clzz, "createStaxSource", new Class[]{ XMLEventReader.class}); - - // javax.xml.transform.Result - staxUtilsResultMethodOnSpring30 = ClassUtils.getStaticMethod(clzz, "createStaxResult", new Class[]{XMLEventWriter.class}); - } else if (hasSpringWs15StaxSupport) { - - // javax.xml.transform.Source - Class staxSourceClassOnSpringWs15 = ClassUtils.forName(staxSourceClassNameOnSpringWs15, defaultClassLoader); - staxSourceClassCtorOnSpringWs15 = staxSourceClassOnSpringWs15.getConstructor(XMLEventReader.class); - - // javax.xml.transform.Result - Class staxResultClassOnSpringWs15 = ClassUtils.forName(staxResultClassNameOnSpringOxm15, defaultClassLoader); - staxResultClassCtorOnSpringWs15 = staxResultClassOnSpringWs15.getConstructor(XMLEventWriter.class); - } else { - - logger.debug("'StaxSource' was not detected in Spring 3.0's OXM support or Spring WS 1.5's OXM support. " + - "This is a problem if you intend to use the " +StaxEventItemWriter.class.getName() + " or " + - StaxEventItemReader.class.getName()+". Please add the appropriate dependencies."); - - } - } catch (Exception ex) { - logger.error("Could not precache required class and method metadata in " + StaxUtils.class.getName()); - } + public static Source getSource(XMLEventReader r) throws XMLStreamException { + return new StAXSource(r); } - public static Source getSource(XMLEventReader r) throws Exception { - if (hasSpring30StaxSupport) { - // org.springframework.util.xml.StaxUtils.createStaxSource(r) - Object result = staxUtilsSourceMethodOnSpring30.invoke(null,r); - Assert.isInstanceOf(Source.class, result, "the result should be assignable to " + Source.class.getName()); - return (Source) result; - } else if (hasSpringWs15StaxSupport) { - Object result = staxSourceClassCtorOnSpringWs15.newInstance(r); - Assert.isInstanceOf(Source.class, result, "the result should be assignable to " + Source.class.getName()); - return (Source) result; - } - // maybe you don't have either environment? - return null; + public static Result getResult(XMLEventWriter w) { + return new StAXResult(w); } - public static Result getResult(XMLEventWriter w) throws Exception { - if (hasSpring30StaxSupport) { - Object result = staxUtilsResultMethodOnSpring30.invoke(null,w); - Assert.isInstanceOf(Result.class, result, "the result should be assignable to " + Result.class.getName()); - return (Result) result; - } else if (hasSpringWs15StaxSupport) { - Object result = staxResultClassCtorOnSpringWs15.newInstance(w); - Assert.isInstanceOf(Result.class, result, "the result should be assignable to " + Result.class.getName()); - return (Result) result; - } - // maybe you don't have either environment? - return null; + public static XMLInputFactory createXmlInputFactory() { + XMLInputFactory xmlInputFactory = XMLInputFactory.newInstance(); + xmlInputFactory.setProperty(XMLInputFactory.SUPPORT_DTD, false); + xmlInputFactory.setProperty(XMLInputFactory.IS_SUPPORTING_EXTERNAL_ENTITIES, false); + return xmlInputFactory; } - - public static XMLEventWriter getXmlEventWriter(Result r) throws Exception { - Method m = r.getClass().getDeclaredMethod("getXMLEventWriter", new Class[]{}); - boolean accessible = m.isAccessible(); - m.setAccessible(true); - Object result = m.invoke(r); - m.setAccessible(accessible); - return (XMLEventWriter) result; - } - - public static XMLEventReader getXmlEventReader(Source s) throws Exception { - Method m = s.getClass().getDeclaredMethod("getXMLEventReader", new Class[]{}); - boolean accessible = m.isAccessible(); - m.setAccessible(true); - Object result = m.invoke(s); - m.setAccessible(accessible); - return (XMLEventReader) result; - } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxWriterCallback.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxWriterCallback.java index 496e5a057e..57915f8756 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxWriterCallback.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/StaxWriterCallback.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,6 +31,10 @@ public interface StaxWriterCallback { /** * Write contents using the supplied {@link XMLEventWriter}. It is not * required to flush the writer inside this method. + * + * @param writer the {@link XMLEventWriter} to be used to write the contents. + * + * @throws IOException thrown if an error occurs during writing. */ void write(XMLEventWriter writer) throws IOException; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/builder/StaxEventItemReaderBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/builder/StaxEventItemReaderBuilder.java new file mode 100644 index 0000000000..d5e09beaf5 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/builder/StaxEventItemReaderBuilder.java @@ -0,0 +1,228 @@ +/* + * Copyright 2017-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.xml.builder; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import javax.xml.stream.XMLInputFactory; + +import org.springframework.batch.item.xml.StaxEventItemReader; +import org.springframework.batch.item.xml.StaxUtils; +import org.springframework.core.io.Resource; +import org.springframework.oxm.Unmarshaller; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * A fluent builder for the {@link StaxEventItemReader} + * + * @author Michael Minella + * @author Glenn Renfro + * @since 4.0 + */ +public class StaxEventItemReaderBuilder { + + private boolean strict = true; + + private Resource resource; + + private Unmarshaller unmarshaller; + + private List fragmentRootElements = new ArrayList<>(); + + private boolean saveState = true; + + private String name; + + private int maxItemCount = Integer.MAX_VALUE; + + private int currentItemCount; + + private XMLInputFactory xmlInputFactory = StaxUtils.createXmlInputFactory(); + + /** + * Configure if the state of the {@link org.springframework.batch.item.ItemStreamSupport} + * should be persisted within the {@link org.springframework.batch.item.ExecutionContext} + * for restart purposes. + * + * @param saveState defaults to true + * @return The current instance of the builder. + */ + public StaxEventItemReaderBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link #saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see org.springframework.batch.item.ItemStreamSupport#setName(String) + */ + public StaxEventItemReaderBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * Configure the max number of items to be read. + * + * @param maxItemCount the max items to be read + * @return The current instance of the builder. + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setMaxItemCount(int) + */ + public StaxEventItemReaderBuilder maxItemCount(int maxItemCount) { + this.maxItemCount = maxItemCount; + + return this; + } + + /** + * Index for the current item. Used on restarts to indicate where to start from. + * + * @param currentItemCount current index + * @return this instance for method chaining + * @see org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader#setCurrentItemCount(int) + */ + public StaxEventItemReaderBuilder currentItemCount(int currentItemCount) { + this.currentItemCount = currentItemCount; + + return this; + } + + /** + * The {@link Resource} to be used as input. + * + * @param resource the input to the reader. + * @return The current instance of the builder. + * @see StaxEventItemReader#setResource(Resource) + */ + public StaxEventItemReaderBuilder resource(Resource resource) { + this.resource = resource; + + return this; + } + + /** + * An implementation of the {@link Unmarshaller} from Spring's OXM module. + * + * @param unmarshaller component responsible for unmarshalling XML chunks + * @return The current instance of the builder. + * @see StaxEventItemReader#setUnmarshaller + */ + public StaxEventItemReaderBuilder unmarshaller(Unmarshaller unmarshaller) { + this.unmarshaller = unmarshaller; + + return this; + } + + /** + * Adds the list of fragments to be used as the root of each chunk to the + * configuration. + * + * @param fragmentRootElements the XML root elements to be used to identify XML + * chunks. + * @return The current instance of the builder. + * @see StaxEventItemReader#setFragmentRootElementNames(String[]) + */ + public StaxEventItemReaderBuilder addFragmentRootElements(String... fragmentRootElements) { + this.fragmentRootElements.addAll(Arrays.asList(fragmentRootElements)); + + return this; + } + + /** + * Adds the list of fragments to be used as the root of each chunk to the + * configuration. + * + * @param fragmentRootElements the XML root elements to be used to identify XML + * chunks. + * @return The current instance of the builder. + * @see StaxEventItemReader#setFragmentRootElementNames(String[]) + */ + public StaxEventItemReaderBuilder addFragmentRootElements(List fragmentRootElements) { + this.fragmentRootElements.addAll(fragmentRootElements); + + return this; + } + + /** + * Setting this value to true indicates that it is an error if the input does not + * exist and an exception will be thrown. Defaults to true. + * + * @param strict indicates the input file must exist + * @return The current instance of the builder + * @see StaxEventItemReader#setStrict(boolean) + */ + public StaxEventItemReaderBuilder strict(boolean strict) { + this.strict = strict; + + return this; + } + + /** + * Set the {@link XMLInputFactory}. + * + * @param xmlInputFactory to use + * @return The current instance of the builder + * @see StaxEventItemReader#setXmlInputFactory(XMLInputFactory) + */ + public StaxEventItemReaderBuilder xmlInputFactory(XMLInputFactory xmlInputFactory) { + this.xmlInputFactory = xmlInputFactory; + + return this; + } + + /** + * Validates the configuration and builds a new {@link StaxEventItemReader} + * + * @return a new instance of the {@link StaxEventItemReader} + */ + public StaxEventItemReader build() { + Assert.notNull(this.resource, "A resource is required."); + + StaxEventItemReader reader = new StaxEventItemReader<>(); + + if (this.saveState) { + Assert.state(StringUtils.hasText(this.name), "A name is required when saveState is set to true."); + } + else { + reader.setName(this.name); + } + + Assert.notEmpty(this.fragmentRootElements, "At least one fragment root element is required"); + + reader.setSaveState(this.saveState); + reader.setResource(this.resource); + reader.setFragmentRootElementNames( + this.fragmentRootElements.toArray(new String[this.fragmentRootElements.size()])); + + reader.setStrict(this.strict); + reader.setUnmarshaller(this.unmarshaller); + reader.setCurrentItemCount(this.currentItemCount); + reader.setMaxItemCount(this.maxItemCount); + reader.setXmlInputFactory(this.xmlInputFactory); + + return reader; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/builder/StaxEventItemWriterBuilder.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/builder/StaxEventItemWriterBuilder.java new file mode 100644 index 0000000000..d155298b53 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/builder/StaxEventItemWriterBuilder.java @@ -0,0 +1,285 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.xml.builder; + +import java.util.Map; + +import org.springframework.batch.item.xml.StaxEventItemWriter; +import org.springframework.batch.item.xml.StaxWriterCallback; +import org.springframework.core.io.Resource; +import org.springframework.oxm.Marshaller; +import org.springframework.util.Assert; + +/** + * A builder for the {@link StaxEventItemWriter}. + * + * @author Michael Minella + * @since 4.0 + * @see StaxEventItemWriter + */ +public class StaxEventItemWriterBuilder { + + private Resource resource; + + private Marshaller marshaller; + + private StaxWriterCallback headerCallback; + + private StaxWriterCallback footerCallback; + + private boolean transactional = true; + + private boolean forceSync = false; + + private boolean shouldDeleteIfEmpty = false; + + private String encoding = StaxEventItemWriter.DEFAULT_ENCODING; + + private String version = StaxEventItemWriter.DEFAULT_XML_VERSION; + + private String rootTagName = StaxEventItemWriter.DEFAULT_ROOT_TAG_NAME; + + private Map rootElementAttributes; + + private boolean overwriteOutput = true; + + private boolean saveState = true; + + private String name; + + /** + * The name used to calculate the key within the + * {@link org.springframework.batch.item.ExecutionContext}. Required if + * {@link StaxEventItemWriterBuilder#saveState(boolean)} is set to true. + * + * @param name name of the reader instance + * @return The current instance of the builder. + * @see StaxEventItemWriter#setName(String) + */ + public StaxEventItemWriterBuilder name(String name) { + this.name = name; + + return this; + } + + /** + * The {@link Resource} to be used as output. + * + * @param resource the output from the writer + * @return the current instance of the builder. + * @see StaxEventItemWriter#setResource(Resource) + */ + public StaxEventItemWriterBuilder resource(Resource resource) { + this.resource = resource; + + return this; + } + + /** + * The {@link Marshaller} implementation responsible for the serialization of the + * items to XML. This field is required. + * + * @param marshaller the component used to generate XML + * @return the current instance of the builder. + * @see StaxEventItemWriter#setMarshaller(Marshaller) + */ + public StaxEventItemWriterBuilder marshaller(Marshaller marshaller) { + this.marshaller = marshaller; + + return this; + } + + /** + * A {@link StaxWriterCallback} to provide any header elements + * + * @param headerCallback a {@link StaxWriterCallback} + * @return the current instance of the builder. + * @see StaxEventItemWriter#setHeaderCallback(StaxWriterCallback) + */ + public StaxEventItemWriterBuilder headerCallback(StaxWriterCallback headerCallback) { + this.headerCallback = headerCallback; + + return this; + } + + /** + * A {@link StaxWriterCallback} to provide any footer elements + * + * @param footerCallback a {@link StaxWriterCallback} + * @return the current instance of the builder. + * @see StaxEventItemWriter#setFooterCallback(StaxWriterCallback) + */ + public StaxEventItemWriterBuilder footerCallback(StaxWriterCallback footerCallback) { + this.footerCallback = footerCallback; + + return this; + } + + /** + * The resulting writer is participating in a transaction and writes should be delayed + * as late as possible. + * + * @param transactional indicates that the writer is transactional. Defaults to false. + * @return the current instance of the builder + * @see StaxEventItemWriter#setTransactional(boolean) + */ + public StaxEventItemWriterBuilder transactional(boolean transactional) { + this.transactional = transactional; + + return this; + } + + /** + * Flag to indicate that changes should be force-synced to disk on flush. + * + * @param forceSync indicates if force sync should occur. Defaults to false. + * @return the current instance of the builder + * @see StaxEventItemWriter#setForceSync(boolean) + */ + public StaxEventItemWriterBuilder forceSync(boolean forceSync) { + this.forceSync = forceSync; + + return this; + } + + /** + * Flag to indicate that the output file should be deleted if no results were written + * to it. Defaults to false. + * + * @param shouldDelete indicator + * @return the current instance of the builder + * @see StaxEventItemWriter#setShouldDeleteIfEmpty(boolean) + */ + public StaxEventItemWriterBuilder shouldDeleteIfEmpty(boolean shouldDelete) { + this.shouldDeleteIfEmpty = shouldDelete; + + return this; + } + + /** + * Encoding for the file. Defaults to UTF-8. + * + * @param encoding String encoding algorithm + * @return the current instance of the builder + * @see StaxEventItemWriter#setEncoding(String) + */ + public StaxEventItemWriterBuilder encoding(String encoding) { + this.encoding = encoding; + + return this; + } + + /** + * Version of XML to be generated. Must be supported by the {@link Marshaller} + * provided. + * + * @param version XML version + * @return the current instance of the builder + * @see StaxEventItemWriter#version + */ + public StaxEventItemWriterBuilder version(String version) { + this.version = version; + + return this; + } + + /** + * The name of the root tag for the output document. + * + * @param rootTagName tag name + * @return the current instance of the builder + * @see StaxEventItemWriter#setRootTagName(String) + */ + public StaxEventItemWriterBuilder rootTagName(String rootTagName) { + this.rootTagName = rootTagName; + + return this; + } + + /** + * A Map of attributes to be included in the document's root element. + * + * @param rootElementAttributes map fo attributes + * @return the current instance of the builder. + * @see StaxEventItemWriter#setRootElementAttributes(Map) + */ + public StaxEventItemWriterBuilder rootElementAttributes(Map rootElementAttributes) { + this.rootElementAttributes = rootElementAttributes; + + return this; + } + + /** + * Indicates if an existing file should be overwritten if found. Defaults to true. + * + * @param overwriteOutput indicator + * @return the current instance of the builder. + * @see StaxEventItemWriter#setOverwriteOutput(boolean) + */ + public StaxEventItemWriterBuilder overwriteOutput(boolean overwriteOutput) { + this.overwriteOutput = overwriteOutput; + + return this; + } + + /** + * Indicates if the state of the writer should be saved in the + * {@link org.springframework.batch.item.ExecutionContext}. Setting this to false + * will impact restartability. Defaults to true. + * + * @param saveState indicator + * @return the current instance of the builder + * @see StaxEventItemWriter#setSaveState(boolean) + */ + public StaxEventItemWriterBuilder saveState(boolean saveState) { + this.saveState = saveState; + + return this; + } + + /** + * Returns a configured {@link StaxEventItemWriter} + * + * @return a StaxEventItemWriter + */ + public StaxEventItemWriter build() { + Assert.notNull(this.marshaller, "A marshaller is required"); + + if(this.saveState) { + Assert.notNull(this.name, "A name is required"); + } + + StaxEventItemWriter writer = new StaxEventItemWriter<>(); + + writer.setEncoding(this.encoding); + writer.setFooterCallback(this.footerCallback); + writer.setForceSync(this.forceSync); + writer.setHeaderCallback(this.headerCallback); + writer.setMarshaller(this.marshaller); + writer.setOverwriteOutput(this.overwriteOutput); + writer.setResource(this.resource); + writer.setRootElementAttributes(this.rootElementAttributes); + writer.setRootTagName(this.rootTagName); + writer.setSaveState(this.saveState); + writer.setShouldDeleteIfEmpty(this.shouldDeleteIfEmpty); + writer.setTransactional(this.transactional); + writer.setVersion(this.version); + writer.setName(this.name); + + return writer; + } + + } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/builder/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/builder/package-info.java new file mode 100644 index 0000000000..8f60fbf801 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/builder/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Builders for Stax event item reader and writer. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.xml.builder; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/package-info.java new file mode 100644 index 0000000000..6b81565cb7 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of xml input and output. + *

      + */ +@NonNullApi +package org.springframework.batch.item.xml; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/package.html deleted file mode 100644 index 1367ff7b44..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of xml input and output. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/AbstractEventReaderWrapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/AbstractEventReaderWrapper.java index 066cdba5b9..b652314092 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/AbstractEventReaderWrapper.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/AbstractEventReaderWrapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/AbstractEventWriterWrapper.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/AbstractEventWriterWrapper.java index 4648b1c8ae..6b945c8b16 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/AbstractEventWriterWrapper.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/AbstractEventWriterWrapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/DefaultFragmentEventReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/DefaultFragmentEventReader.java index ab9911ba69..97e7be168c 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/DefaultFragmentEventReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/DefaultFragmentEventReader.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/FragmentEventReader.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/FragmentEventReader.java index 030bc99a84..1047683227 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/FragmentEventReader.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/FragmentEventReader.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/NoStartEndDocumentStreamWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/NoStartEndDocumentStreamWriter.java index dea26eb7e9..929bcc2989 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/NoStartEndDocumentStreamWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/NoStartEndDocumentStreamWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/UnclosedElementCollectingEventWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/UnclosedElementCollectingEventWriter.java new file mode 100644 index 0000000000..50deb82001 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/UnclosedElementCollectingEventWriter.java @@ -0,0 +1,57 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.xml.stax; + +import java.util.LinkedList; +import java.util.List; + +import javax.xml.namespace.QName; +import javax.xml.stream.XMLEventWriter; +import javax.xml.stream.XMLStreamException; +import javax.xml.stream.events.XMLEvent; + +/** + * Delegating XMLEventWriter, which collects the QNames of elements that were opened but not closed. + * + * @author Jimmy Praet + * @since 3.0 + */ +public class UnclosedElementCollectingEventWriter extends AbstractEventWriterWrapper { + + private LinkedList unclosedElements = new LinkedList<>(); + + public UnclosedElementCollectingEventWriter(XMLEventWriter wrappedEventWriter) { + super(wrappedEventWriter); + } + + /* (non-Javadoc) + * @see org.springframework.batch.item.xml.stax.AbstractEventWriterWrapper#add(javax.xml.stream.events.XMLEvent) + */ + @Override + public void add(XMLEvent event) throws XMLStreamException { + if (event.isStartElement()) { + unclosedElements.addLast(event.asStartElement().getName()); + } else if (event.isEndElement()) { + unclosedElements.removeLast(); + } + super.add(event); + } + + public List getUnclosedElements() { + return unclosedElements; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/UnopenedElementClosingEventWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/UnopenedElementClosingEventWriter.java new file mode 100644 index 0000000000..768c9f20d0 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/UnopenedElementClosingEventWriter.java @@ -0,0 +1,82 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.xml.stax; + +import java.io.IOException; +import java.io.Writer; +import java.util.LinkedList; +import java.util.List; + +import javax.xml.namespace.QName; +import javax.xml.stream.XMLEventWriter; +import javax.xml.stream.XMLStreamException; +import javax.xml.stream.events.XMLEvent; + +import org.springframework.dao.DataAccessResourceFailureException; +import org.springframework.util.StringUtils; + +/** + * Delegating XMLEventWriter, which writes EndElement events that match a given collection of QNames directly + * to the underlying java.io.Writer instead of to the delegate XMLEventWriter. + * + * @author Jimmy Praet + * @since 3.0 + */ +public class UnopenedElementClosingEventWriter extends AbstractEventWriterWrapper { + + private LinkedList unopenedElements; + + private Writer ioWriter; + + public UnopenedElementClosingEventWriter(XMLEventWriter wrappedEventWriter, Writer ioWriter, List unopenedElements) { + super(wrappedEventWriter); + this.unopenedElements = new LinkedList<>(unopenedElements); + this.ioWriter = ioWriter; + } + + /* (non-Javadoc) + * @see org.springframework.batch.item.xml.stax.AbstractEventWriterWrapper#add(javax.xml.stream.events.XMLEvent) + */ + @Override + public void add(XMLEvent event) throws XMLStreamException { + if (isUnopenedElementCloseEvent(event)) { + QName element = unopenedElements.removeLast(); + String nsPrefix = !StringUtils.hasText(element.getPrefix()) ? "" : element.getPrefix() + ":"; + try { + super.flush(); + ioWriter.write(""); + ioWriter.flush(); + } + catch (IOException ioe) { + throw new DataAccessResourceFailureException("Unable to close tag: " + element, ioe); + } + } else { + super.add(event); + } + } + + private boolean isUnopenedElementCloseEvent(XMLEvent event) { + if (unopenedElements.isEmpty()) { + return false; + } else if (!event.isEndElement()) { + return false; + } else { + return unopenedElements.getLast().equals(event.asEndElement().getName()); + } + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/package-info.java new file mode 100644 index 0000000000..b1f51036c8 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/xml/stax/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Item reader and writer based on Stax. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.item.xml.stax; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/CheckpointSupport.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/CheckpointSupport.java new file mode 100644 index 0000000000..572f211b59 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/CheckpointSupport.java @@ -0,0 +1,132 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.jsr.item; + +import java.io.Serializable; + +import javax.batch.api.chunk.ItemReader; +import javax.batch.api.chunk.ItemWriter; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.ItemStreamSupport; +import org.springframework.util.Assert; +import org.springframework.util.SerializationUtils; + +/** + * Provides support for JSR-352 checkpointing. Checkpoint objects are copied prior + * to being added to the {@link ExecutionContext} for persistence by the framework. + * If the checkpoint object cannot be copied and further changes occur to the same + * instance, side effects may occur. In cases like this, it is recommended that a + * copy of the object being acted upon in the reader/writer is returned via the + * {@link ItemReader#checkpointInfo()} or {@link ItemWriter#checkpointInfo()} calls. + * + * @author Michael Minella + * @since 3.0 + */ +public abstract class CheckpointSupport extends ItemStreamSupport{ + + private final Log logger = LogFactory.getLog(this.getClass()); + + private final String checkpointKey; + + /** + * @param checkpointKey key to store the checkpoint object with in the {@link ExecutionContext} + */ + public CheckpointSupport(String checkpointKey) { + Assert.hasText(checkpointKey, "checkpointKey is required"); + this.checkpointKey = checkpointKey; + } + + /* (non-Javadoc) + * @see org.springframework.batch.item.ItemStreamSupport#open(org.springframework.batch.item.ExecutionContext) + */ + @Override + public void open(ExecutionContext executionContext) + throws ItemStreamException { + try { + String executionContextKey = getExecutionContextKey(checkpointKey); + Serializable checkpoint = (Serializable) executionContext.get(executionContextKey); + doOpen(checkpoint); + } catch (Exception e) { + throw new ItemStreamException(e); + } + } + + /** + * Used to open a batch artifact with previously saved checkpoint information. + * + * @param checkpoint previously saved checkpoint object + * @throws Exception thrown by the implementation + */ + protected abstract void doOpen(Serializable checkpoint) throws Exception; + + /* (non-Javadoc) + * @see org.springframework.batch.item.ItemStreamSupport#update(org.springframework.batch.item.ExecutionContext) + */ + @Override + public void update(ExecutionContext executionContext) + throws ItemStreamException { + try { + executionContext.put(getExecutionContextKey(checkpointKey), deepCopy(doCheckpoint())); + } catch (Exception e) { + throw new ItemStreamException(e); + } + } + + /** + * Used to provide a {@link Serializable} representing the current state of the + * batch artifact. + * + * @return the current state of the batch artifact + * @throws Exception thrown by the implementation + */ + protected abstract Serializable doCheckpoint() throws Exception; + + /* (non-Javadoc) + * @see org.springframework.batch.item.ItemStreamSupport#close() + */ + @Override + public void close() throws ItemStreamException { + try { + doClose(); + } catch (Exception e) { + throw new ItemStreamException(e); + } + } + + /** + * Used to close the underlying batch artifact + * + * @throws Exception thrown by the underlying implementation + */ + protected abstract void doClose() throws Exception; + + private Object deepCopy(Serializable orig) { + Object obj = orig; + + try { + obj = SerializationUtils.deserialize(SerializationUtils.serialize(orig)); + } catch (Exception e) { + logger.warn("Unable to copy checkpoint object. Updating the instance passed may cause side effects"); + } + + return obj; + } + +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/ItemProcessorAdapter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/ItemProcessorAdapter.java new file mode 100644 index 0000000000..cc4070aaab --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/ItemProcessorAdapter.java @@ -0,0 +1,38 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.jsr.item; + +import javax.batch.api.chunk.ItemProcessor; + +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +public class ItemProcessorAdapter implements org.springframework.batch.item.ItemProcessor { + + private ItemProcessor delegate; + + public ItemProcessorAdapter(ItemProcessor processor) { + Assert.notNull(processor, "An ItemProcessor implementation is required"); + this.delegate = processor; + } + + @Nullable + @SuppressWarnings("unchecked") + @Override + public O process(I item) throws Exception { + return (O) delegate.processItem(item); + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/ItemReaderAdapter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/ItemReaderAdapter.java new file mode 100644 index 0000000000..5825205e45 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/ItemReaderAdapter.java @@ -0,0 +1,82 @@ +/* + * Copyright 2013-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.jsr.item; + +import java.io.Serializable; + +import javax.batch.api.chunk.ItemReader; + +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; +import org.springframework.util.ClassUtils; + +/** + * Adapter that wraps an {@link ItemReader} for use by Spring Batch. All calls are delegated as appropriate + * to the corresponding method on the delegate. + * + * @author Michael Minella + * @since 3.0 + */ +public class ItemReaderAdapter extends CheckpointSupport implements org.springframework.batch.item.ItemReader { + + private static final String CHECKPOINT_KEY = "reader.checkpoint"; + + private ItemReader delegate; + + /** + * @param reader the {@link ItemReader} implementation to delegate to + */ + public ItemReaderAdapter(ItemReader reader) { + super(CHECKPOINT_KEY); + Assert.notNull(reader, "An ItemReader implementation is required"); + this.delegate = reader; + setExecutionContextName(ClassUtils.getShortName(delegate.getClass())); + } + + /* (non-Javadoc) + * @see org.springframework.batch.item.ItemReader#read() + */ + @Nullable + @SuppressWarnings("unchecked") + @Override + public T read() throws Exception { + return (T) delegate.readItem(); + } + + /* (non-Javadoc) + * @see org.springframework.batch.jsr.item.CheckpointSupport#doClose() + */ + @Override + protected void doClose() throws Exception{ + delegate.close(); + } + + /* (non-Javadoc) + * @see org.springframework.batch.jsr.item.CheckpointSupport#doCheckpoint() + */ + @Override + protected Serializable doCheckpoint() throws Exception { + return delegate.checkpointInfo(); + } + + /* (non-Javadoc) + * @see org.springframework.batch.jsr.item.CheckpointSupport#doOpen(java.io.Serializable) + */ + @Override + protected void doOpen(Serializable checkpoint) throws Exception { + delegate.open(checkpoint); + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/ItemWriterAdapter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/ItemWriterAdapter.java new file mode 100644 index 0000000000..69a73604a3 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/ItemWriterAdapter.java @@ -0,0 +1,82 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.jsr.item; + +import java.io.Serializable; +import java.util.List; + +import javax.batch.api.chunk.ItemWriter; + +import org.springframework.util.Assert; +import org.springframework.util.ClassUtils; + +/** + * Adapter that wraps an {@link ItemWriter} for use by Spring Batch. All calls are delegated as appropriate + * to the corresponding method on the delegate. + * + * @author Michael Minella + * @since 3.0 + */ +public class ItemWriterAdapter extends CheckpointSupport implements org.springframework.batch.item.ItemWriter { + + private static final String CHECKPOINT_KEY = "writer.checkpoint"; + + private ItemWriter delegate; + + /** + * @param writer a {@link ItemWriter} to delegate calls to + */ + public ItemWriterAdapter(ItemWriter writer) { + super(CHECKPOINT_KEY); + Assert.notNull(writer, "An ItemWriter implementation is required"); + this.delegate = writer; + super.setExecutionContextName(ClassUtils.getShortName(delegate.getClass())); + } + + /* (non-Javadoc) + * @see org.springframework.batch.item.ItemWriter#write(java.util.List) + */ + @SuppressWarnings("unchecked") + @Override + public void write(List items) throws Exception { + delegate.writeItems((List) items); + } + + /* (non-Javadoc) + * @see org.springframework.batch.jsr.item.CheckpointSupport#doOpen(java.io.Serializable) + */ + @Override + protected void doOpen(Serializable checkpoint) throws Exception { + delegate.open(checkpoint); + } + + /* (non-Javadoc) + * @see org.springframework.batch.jsr.item.CheckpointSupport#doCheckpoint() + */ + @Override + protected Serializable doCheckpoint() throws Exception { + Serializable checkpointInfo = delegate.checkpointInfo(); + return checkpointInfo; + } + + /* (non-Javadoc) + * @see org.springframework.batch.jsr.item.CheckpointSupport#doClose() + */ + @Override + protected void doClose() throws Exception{ + delegate.close(); + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/package-info.java new file mode 100644 index 0000000000..c4cad41a1f --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/item/package-info.java @@ -0,0 +1,10 @@ +/** + * Components for adapting JSR item based components to Spring Batch. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.jsr.item; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/repeat/CheckpointAlgorithmAdapter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/repeat/CheckpointAlgorithmAdapter.java new file mode 100644 index 0000000000..9357021348 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/repeat/CheckpointAlgorithmAdapter.java @@ -0,0 +1,101 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.jsr.repeat; + +import javax.batch.api.chunk.CheckpointAlgorithm; +import javax.batch.operations.BatchRuntimeException; + +import org.springframework.batch.repeat.CompletionPolicy; +import org.springframework.batch.repeat.RepeatContext; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.util.Assert; + +/** + * Wrapper for the {@link CheckpointAlgorithm} to be used via the rest + * of the framework. + * + * @author Michael Minella + * @see CheckpointAlgorithm + * @see CompletionPolicy + */ +public class CheckpointAlgorithmAdapter implements CompletionPolicy { + + private CheckpointAlgorithm policy; + private boolean isComplete = false; + + public CheckpointAlgorithmAdapter(CheckpointAlgorithm policy) { + Assert.notNull(policy, "A CheckpointAlgorithm is required"); + + this.policy = policy; + } + + /* (non-Javadoc) + * @see org.springframework.batch.repeat.CompletionPolicy#isComplete(org.springframework.batch.repeat.RepeatContext, org.springframework.batch.repeat.RepeatStatus) + */ + @Override + public boolean isComplete(RepeatContext context, RepeatStatus result) { + try { + isComplete = policy.isReadyToCheckpoint(); + return isComplete; + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + /* (non-Javadoc) + * @see org.springframework.batch.repeat.CompletionPolicy#isComplete(org.springframework.batch.repeat.RepeatContext) + */ + @Override + public boolean isComplete(RepeatContext context) { + try { + isComplete = policy.isReadyToCheckpoint(); + return isComplete; + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } + + /* (non-Javadoc) + * @see org.springframework.batch.repeat.CompletionPolicy#start(org.springframework.batch.repeat.RepeatContext) + */ + @Override + public RepeatContext start(RepeatContext parent) { + try { + policy.beginCheckpoint(); + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + + return parent; + } + + /** + * If {@link CheckpointAlgorithm#isReadyToCheckpoint()} is true + * we will call {@link CheckpointAlgorithm#endCheckpoint()} + * + * @param context a {@link RepeatContext} + */ + @Override + public void update(RepeatContext context) { + try { + if(isComplete) { + policy.endCheckpoint(); + } + } catch (Exception e) { + throw new BatchRuntimeException(e); + } + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/repeat/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/repeat/package-info.java new file mode 100644 index 0000000000..3ae51bc5c4 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/jsr/repeat/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * APIs for JSR-352 repeat support. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.jsr.repeat; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/package-info.java new file mode 100644 index 0000000000..8293c2f96d --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of . concerns. + *

      + */ +@NonNullApi +package org.springframework.batch; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/package.html deleted file mode 100644 index 5137c3e68e..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of . concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/poller/DirectPoller.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/poller/DirectPoller.java index ee945fd698..3e93b015c2 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/poller/DirectPoller.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/poller/DirectPoller.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -48,7 +48,7 @@ public DirectPoller(long interval) { */ @Override public Future poll(Callable callable) throws Exception { - return new DirectPollingFuture(interval, callable); + return new DirectPollingFuture<>(interval, callable); } private static class DirectPollingFuture implements Future { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/poller/Poller.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/poller/Poller.java index 8d3f5bc34d..55e9df8b01 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/poller/Poller.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/poller/Poller.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2010 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,10 +35,11 @@ * } * }); * - * Result result = future.get(1000L, TimeUnit.MILLSECONDS); + * Result result = future.get(1000L, TimeUnit.MILLISECONDS); * * * @author Dave Syer + * @author Mahmoud Ben Hassine * */ public interface Poller { @@ -50,7 +51,7 @@ public interface Poller { * * @param callable a {@link Callable} to use to retrieve a result * @return a future which itself can be used to get the result - * + * @throws java.lang.Exception allows for checked exceptions */ Future poll(Callable callable) throws Exception; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/poller/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/poller/package-info.java new file mode 100644 index 0000000000..5318e22460 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/poller/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * APIs for polling support. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.poller; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/CompletionPolicy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/CompletionPolicy.java index f39895e4b3..2095c59d6f 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/CompletionPolicy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/CompletionPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatCallback.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatCallback.java index cb72f80c08..085a83fa3a 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatCallback.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatCallback.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatContext.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatContext.java index 4ece5f36e8..93d7c9ac81 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatContext.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatContext.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -54,6 +54,8 @@ public interface RepeatContext extends AttributeAccessor { /** * Public accessor for the complete flag. + * + * @return indicator if the repeat is complete */ boolean isCompleteOnly(); @@ -66,6 +68,8 @@ public interface RepeatContext extends AttributeAccessor { /** * Public accessor for the termination flag. If this flag is set then the * complete flag will also be. + * + * @return indicates if the repeat should terminate */ boolean isTerminateOnly(); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatException.java index ff148168b5..6fc4a564da 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,6 +18,7 @@ import org.springframework.core.NestedRuntimeException; +@SuppressWarnings("serial") public class RepeatException extends NestedRuntimeException { public RepeatException(String msg) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatListener.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatListener.java index 3c42f329ca..18ace59161 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatListener.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -59,7 +59,7 @@ public interface RepeatListener { /** * Called when a repeat callback fails by throwing an exception. There will * be one call to this method for each exception thrown during a repeat - * operation (e.g. a chunk).
      + * operation (e.g. a chunk).
      * * There is no need to re-throw the exception here - that will be done by * the enclosing framework. diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatOperations.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatOperations.java index a392de23f1..77ab5e4037 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatOperations.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatOperations.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatStatus.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatStatus.java index 8d5273843a..fe8ffd5bca 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatStatus.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/RepeatStatus.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/callback/NestedRepeatCallback.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/callback/NestedRepeatCallback.java index 7c3d523a26..773fb10a60 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/callback/NestedRepeatCallback.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/callback/NestedRepeatCallback.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/callback/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/callback/package-info.java new file mode 100644 index 0000000000..878daec20c --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/callback/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of repeat callback concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.repeat.callback; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/callback/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/callback/package.html deleted file mode 100644 index a25a6cf386..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/callback/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of repeat callback concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/RepeatContextCounter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/RepeatContextCounter.java index c062b7bc34..cc02a64535 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/RepeatContextCounter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/RepeatContextCounter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/RepeatContextSupport.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/RepeatContextSupport.java index 5125a8c10a..9f2fb5dcbc 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/RepeatContextSupport.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/RepeatContextSupport.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,13 +35,13 @@ public class RepeatContextSupport extends SynchronizedAttributeAccessor implemen private volatile boolean terminateOnly; - private Map> callbacks = new HashMap>(); + private Map> callbacks = new HashMap<>(); /** * Constructor for {@link RepeatContextSupport}. The parent can be null, but * should be set to the enclosing repeat context if there is one, e.g. if * this context is an inner loop. - * @param parent + * @param parent {@link RepeatContext} to be used as the parent context. */ public RepeatContextSupport(RepeatContext parent) { super(); @@ -128,7 +128,7 @@ public void registerDestructionCallback(String name, Runnable callback) { synchronized (callbacks) { Set set = callbacks.get(name); if (set == null) { - set = new HashSet(); + set = new HashSet<>(); callbacks.put(name, set); } set.add(callback); @@ -143,12 +143,12 @@ public void registerDestructionCallback(String name, Runnable callback) { @Override public void close() { - List errors = new ArrayList(); + List errors = new ArrayList<>(); Set>> copy; synchronized (callbacks) { - copy = new HashSet>>(callbacks.entrySet()); + copy = new HashSet<>(callbacks.entrySet()); } for (Map.Entry> entry : copy) { @@ -180,7 +180,7 @@ public void close() { return; } - throw (RuntimeException) errors.get(0); + throw errors.get(0); } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/SynchronizedAttributeAccessor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/SynchronizedAttributeAccessor.java index 97e09abc31..a3444f8bd6 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/SynchronizedAttributeAccessor.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/SynchronizedAttributeAccessor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -151,7 +151,7 @@ public Object setAttributeIfAbsent(String name, Object value) { */ @Override public String toString() { - StringBuffer buffer = new StringBuffer("SynchronizedAttributeAccessor: ["); + StringBuilder buffer = new StringBuilder("SynchronizedAttributeAccessor: ["); synchronized (support) { String[] names = attributeNames(); for (int i = 0; i < names.length; i++) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/package-info.java new file mode 100644 index 0000000000..e8d0f11f93 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of repeat context concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.repeat.context; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/package.html deleted file mode 100644 index 7bb43facdf..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/context/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of repeat context concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/CompositeExceptionHandler.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/CompositeExceptionHandler.java index 7f3748ce0b..0b1e2b917a 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/CompositeExceptionHandler.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/CompositeExceptionHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,7 +21,7 @@ import org.springframework.batch.repeat.RepeatContext; /** - * Composiste {@link ExceptionHandler} that loops though a list of delegates. + * Composite {@link ExceptionHandler} that loops though a list of delegates. * * @author Dave Syer * diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/DefaultExceptionHandler.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/DefaultExceptionHandler.java index 56806dfb3c..57089aa39a 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/DefaultExceptionHandler.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/DefaultExceptionHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/ExceptionHandler.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/ExceptionHandler.java index 86f96916ab..814f38a812 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/ExceptionHandler.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/ExceptionHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/LogOrRethrowExceptionHandler.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/LogOrRethrowExceptionHandler.java index 6fae383be3..17af7bb688 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/LogOrRethrowExceptionHandler.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/LogOrRethrowExceptionHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -70,7 +70,7 @@ public static enum Level { protected final Log logger = LogFactory.getLog(LogOrRethrowExceptionHandler.class); - private Classifier exceptionClassifier = new ClassifierSupport(Level.RETHROW); + private Classifier exceptionClassifier = new ClassifierSupport<>(Level.RETHROW); /** * Setter for the {@link Classifier} used by this handler. The default is to @@ -86,7 +86,8 @@ public void setExceptionClassifier(Classifier exceptionClassif * Classify the throwables and decide whether to rethrow based on the * result. The context is not used. * - * @throws Throwable + * @throws Throwable thrown if {@link LogOrRethrowExceptionHandler#exceptionClassifier} + * is classified as {@link Level#RETHROW}. * * @see ExceptionHandler#handleException(RepeatContext, Throwable) */ diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/RethrowOnThresholdExceptionHandler.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/RethrowOnThresholdExceptionHandler.java index 027baf6e00..91a8dec126 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/RethrowOnThresholdExceptionHandler.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/RethrowOnThresholdExceptionHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -43,12 +43,7 @@ public class RethrowOnThresholdExceptionHandler implements ExceptionHandler { protected final Log logger = LogFactory.getLog(RethrowOnThresholdExceptionHandler.class); - private Classifier exceptionClassifier = new Classifier() { - @Override - public RethrowOnThresholdExceptionHandler.IntegerHolder classify(Throwable classifiable) { - return ZERO; - } - }; + private Classifier exceptionClassifier = (Classifier) classifiable -> ZERO; private boolean useParent = false; @@ -78,11 +73,11 @@ public RethrowOnThresholdExceptionHandler() { * @param thresholds the threshold value map. */ public void setThresholds(Map, Integer> thresholds) { - Map, IntegerHolder> typeMap = new HashMap, IntegerHolder>(); + Map, IntegerHolder> typeMap = new HashMap<>(); for (Entry, Integer> entry : thresholds.entrySet()) { typeMap.put(entry.getKey(), new IntegerHolder(entry.getValue())); } - exceptionClassifier = new SubclassClassifier(typeMap, ZERO); + exceptionClassifier = new SubclassClassifier<>(typeMap, ZERO); } /** @@ -90,7 +85,7 @@ public void setThresholds(Map, Integer> thresholds) { * result. The context is used to accumulate the number of exceptions of the * same type according to the classifier. * - * @throws Throwable + * @throws Throwable is thrown if number of exceptions exceeds threshold. * @see ExceptionHandler#handleException(RepeatContext, Throwable) */ @Override @@ -123,7 +118,7 @@ private static class IntegerHolder { private final int value; /** - * @param value + * @param value value within holder */ public IntegerHolder(int value) { this.value = value; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/SimpleLimitExceptionHandler.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/SimpleLimitExceptionHandler.java index 4c9be23241..bdc95d2609 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/SimpleLimitExceptionHandler.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/SimpleLimitExceptionHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -57,7 +57,7 @@ public void afterPropertiesSet() throws Exception { if (limit <= 0) { return; } - Map, Integer> thresholds = new HashMap, Integer>(); + Map, Integer> thresholds = new HashMap<>(); for (Class type : exceptionClasses) { thresholds.put(type, limit); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/package-info.java new file mode 100644 index 0000000000..e32d20693c --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of repeat exception handler concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.repeat.exception; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/package.html deleted file mode 100644 index edbc051c89..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/exception/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of repeat exception handler concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/interceptor/RepeatOperationsInterceptor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/interceptor/RepeatOperationsInterceptor.java index c6de1aeadb..2b3ea81035 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/interceptor/RepeatOperationsInterceptor.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/interceptor/RepeatOperationsInterceptor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -47,12 +47,12 @@ public class RepeatOperationsInterceptor implements MethodInterceptor { /** * Setter for the {@link RepeatOperations}. * - * @param batchTempate + * @param batchTemplate template to be used * @throws IllegalArgumentException if the argument is null. */ - public void setRepeatOperations(RepeatOperations batchTempate) { - Assert.notNull(batchTempate, "'repeatOperations' cannot be null."); - this.repeatOperations = batchTempate; + public void setRepeatOperations(RepeatOperations batchTemplate) { + Assert.notNull(batchTemplate, "'repeatOperations' cannot be null."); + this.repeatOperations = batchTemplate; } /** @@ -144,6 +144,7 @@ private boolean isComplete(Object result) { * @author Dave Syer * */ + @SuppressWarnings("serial") private static class RepeatOperationsInterceptorException extends RepeatException { /** * @param message diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/interceptor/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/interceptor/package-info.java new file mode 100644 index 0000000000..d67cb6a468 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/interceptor/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of repeat aop concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.repeat.interceptor; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/interceptor/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/interceptor/package.html deleted file mode 100644 index bb348cede9..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/interceptor/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of repeat aop concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/CompositeRepeatListener.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/CompositeRepeatListener.java index a0d9991273..35f2f70359 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/CompositeRepeatListener.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/CompositeRepeatListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,17 +24,19 @@ import org.springframework.batch.repeat.RepeatListener; /** + * Allows a user to register one or more RepeatListeners to be notified on batch events. + * * @author Dave Syer * */ public class CompositeRepeatListener implements RepeatListener { - private List listeners = new ArrayList(); + private List listeners = new ArrayList<>(); /** * Public setter for the listeners. * - * @param listeners + * @param listeners array of RepeatListeners to be used by the CompositeRepeatListener. */ public void setListeners(RepeatListener[] listeners) { this.listeners = Arrays.asList(listeners); @@ -43,7 +45,7 @@ public void setListeners(RepeatListener[] listeners) { /** * Register additional listener. * - * @param listener + * @param listener the RepeatListener to be added to the list of listeners to be notified. */ public void register(RepeatListener listener) { if (!listeners.contains(listener)) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/RepeatListenerSupport.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/RepeatListenerSupport.java index 05a1212935..ce1ffffc7e 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/RepeatListenerSupport.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/RepeatListenerSupport.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/package-info.java new file mode 100644 index 0000000000..db3be41dae --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of repeat interceptor concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.repeat.listener; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/package.html deleted file mode 100644 index 8a95d83825..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/listener/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of repeat interceptor concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/package-info.java new file mode 100644 index 0000000000..e247cdbeeb --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of repeat concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.repeat; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/package.html deleted file mode 100644 index 531b1f8c45..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of repeat concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/CompletionPolicySupport.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/CompletionPolicySupport.java index 016b9dbb6c..52e5bc9de3 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/CompletionPolicySupport.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/CompletionPolicySupport.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/CompositeCompletionPolicy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/CompositeCompletionPolicy.java index 4ce9b680d9..1f27c6bd1a 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/CompositeCompletionPolicy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/CompositeCompletionPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,7 +27,7 @@ /** * Composite policy that loops through a list of delegate policies and answers - * calls by a concensus. + * calls by a consensus. * * @author Dave Syer * @@ -39,7 +39,8 @@ public class CompositeCompletionPolicy implements CompletionPolicy { /** * Setter for the policies. * - * @param policies + * @param policies an array of completion policies to be used to determine + * {@link #isComplete(RepeatContext)} by consensus. */ public void setPolicies(CompletionPolicy[] policies) { this.policies = Arrays.asList(policies).toArray(new CompletionPolicy[policies.length]); @@ -87,7 +88,7 @@ public boolean isComplete(RepeatContext context) { */ @Override public RepeatContext start(RepeatContext context) { - List list = new ArrayList(); + List list = new ArrayList<>(); for (int i = 0; i < policies.length; i++) { list.add(policies[i].start(context)); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/CountingCompletionPolicy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/CountingCompletionPolicy.java index 485099cb95..8b4ec15a31 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/CountingCompletionPolicy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/CountingCompletionPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/DefaultResultCompletionPolicy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/DefaultResultCompletionPolicy.java index dec9ce70d5..74d3c0626d 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/DefaultResultCompletionPolicy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/DefaultResultCompletionPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/SimpleCompletionPolicy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/SimpleCompletionPolicy.java index c7fdb8cbac..15b1ca3bfd 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/SimpleCompletionPolicy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/SimpleCompletionPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/TimeoutTerminationPolicy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/TimeoutTerminationPolicy.java index 830c8a973a..1f7a47eb33 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/TimeoutTerminationPolicy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/TimeoutTerminationPolicy.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,7 +22,7 @@ /** * Termination policy that times out after a fixed period. Allows graceful exit * from a batch if the latest result comes in after the timeout expires (i.e. - * does not throw a timeout exception).
      + * does not throw a timeout exception).
      * * N.B. It may often be the case that the batch governed by this policy will be * transactional, and the transaction might have its own timeout. In this case @@ -35,7 +35,7 @@ public class TimeoutTerminationPolicy extends CompletionPolicySupport { /** - * Default timeout value in millisecs (the value equivalent to 30 seconds). + * Default timeout value in milliseconds (the value equivalent to 30 seconds). */ public static final long DEFAULT_TIMEOUT = 30000L; @@ -52,7 +52,7 @@ public TimeoutTerminationPolicy() { * Construct a {@link TimeoutTerminationPolicy} with the specified timeout * value (in milliseconds). * - * @param timeout + * @param timeout duration of the timeout. */ public TimeoutTerminationPolicy(long timeout) { super(); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/package-info.java new file mode 100644 index 0000000000..6cdf5440d6 --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/package-info.java @@ -0,0 +1,9 @@ +/** + *

      + * Infrastructure implementations of repeat policy concerns. + *

      + */ +@NonNullApi +package org.springframework.batch.repeat.policy; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/package.html deleted file mode 100644 index e30f24f4e7..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/policy/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

      -Infrastructure implementations of repeat policy concerns. -

      - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatInternalState.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatInternalState.java index 0b50d910d5..077d1999cb 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatInternalState.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatInternalState.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatInternalStateSupport.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatInternalStateSupport.java index cde8c00cfb..c5fd6f3f99 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatInternalStateSupport.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatInternalStateSupport.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,7 +23,7 @@ public class RepeatInternalStateSupport implements RepeatInternalState { // Accumulation of failed results. - private final Set throwables = new HashSet(); + private final Set throwables = new HashSet<>(); /* (non-Javadoc) * @see org.springframework.batch.repeat.support.BatchInternalState#getThrowables() diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatSynchronizationManager.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatSynchronizationManager.java index ab4ac1f590..7946d4a153 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatSynchronizationManager.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatSynchronizationManager.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,7 +35,7 @@ */ public final class RepeatSynchronizationManager { - private static final ThreadLocal contextHolder = new ThreadLocal(); + private static final ThreadLocal contextHolder = new ThreadLocal<>(); private RepeatSynchronizationManager() { } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatTemplate.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatTemplate.java index 5412304a94..1820f9e431 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatTemplate.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/RepeatTemplate.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,6 +23,7 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.batch.repeat.CompletionPolicy; import org.springframework.batch.repeat.RepeatCallback; import org.springframework.batch.repeat.RepeatContext; @@ -40,13 +41,13 @@ * {@link RepeatOperations}. Provides a framework including interceptors and * policies. Subclasses just need to provide a method that gets the next result * and one that waits for all the results to be returned from concurrent - * processes or threads.
      + * processes or threads.
      * * N.B. the template accumulates thrown exceptions during the iteration, and * they are all processed together when the main loop ends (i.e. finished * processing the items). Clients that do not want to stop execution when an * exception is thrown can use a specific {@link CompletionPolicy} that does not - * finish when exceptions are received. This is not the default behaviour.
      + * finish when exceptions are received. This is not the default behaviour.
      * * Clients that want to take some business action when an exception is thrown by * the {@link RepeatCallback} can consider using a custom {@link RepeatListener} @@ -76,7 +77,7 @@ public class RepeatTemplate implements RepeatOperations { * Set the listeners for this template, registering them for callbacks at * appropriate times in the iteration. * - * @param listeners + * @param listeners listeners to be used */ public void setListeners(RepeatListener[] listeners) { this.listeners = Arrays.asList(listeners).toArray(new RepeatListener[listeners.length]); @@ -85,12 +86,12 @@ public void setListeners(RepeatListener[] listeners) { /** * Register an additional listener. * - * @param listener + * @param listener a single listener to be added to the list */ public void registerListener(RepeatListener listener) { - List list = new ArrayList(Arrays.asList(listeners)); + List list = new ArrayList<>(Arrays.asList(listeners)); list.add(listener); - listeners = (RepeatListener[]) list.toArray(new RepeatListener[list.size()]); + listeners = list.toArray(new RepeatListener[list.size()]); } /** @@ -121,7 +122,7 @@ public void setExceptionHandler(ExceptionHandler exceptionHandler) { * @throws IllegalArgumentException if the argument is null */ public void setCompletionPolicy(CompletionPolicy terminationPolicy) { - Assert.notNull(terminationPolicy); + Assert.notNull(terminationPolicy, "CompletionPolicy is required"); this.completionPolicy = terminationPolicy; } @@ -159,7 +160,7 @@ public RepeatStatus iterate(RepeatCallback callback) { * * @param callback the callback to process each element of the loop. * - * @return the aggregate of {@link ContinuationPolicy#canContinue(Object)} + * @return the aggregate of {@link RepeatTemplate#canContinue(RepeatStatus)} * for all the results from the callback. * */ @@ -172,8 +173,7 @@ private RepeatStatus executeInternal(final RepeatCallback callback) { // processing takes place. boolean running = !isMarkedComplete(context); - for (int i = 0; i < listeners.length; i++) { - RepeatListener interceptor = listeners[i]; + for (RepeatListener interceptor : listeners) { interceptor.open(context); running = running && !isMarkedComplete(context); if (!running) @@ -188,7 +188,7 @@ private RepeatStatus executeInternal(final RepeatCallback callback) { Collection throwables = state.getThrowables(); // Keep a separate list of exceptions we handled that need to be // rethrown - Collection deferred = new ArrayList(); + Collection deferred = new ArrayList<>(); try { @@ -248,9 +248,11 @@ private RepeatStatus executeInternal(final RepeatCallback callback) { try { if (!deferred.isEmpty()) { - Throwable throwable = (Throwable) deferred.iterator().next(); - logger.debug("Handling fatal exception explicitly (rethrowing first of " + deferred.size() + "): " - + throwable.getClass().getName() + ": " + throwable.getMessage()); + Throwable throwable = deferred.iterator().next(); + if (logger.isDebugEnabled()) { + logger.debug("Handling fatal exception explicitly (rethrowing first of " + deferred.size() + "): " + + throwable.getClass().getName() + ": " + throwable.getMessage()); + } rethrow(throwable); } @@ -285,12 +287,16 @@ private void doHandle(Throwable throwable, RepeatContext context, Collection { * at any given time. */ public ResultHolderResultQueue(int throttleLimit) { - results = new PriorityBlockingQueue(throttleLimit, new ResultHolderComparator()); + results = new PriorityBlockingQueue<>(throttleLimit, new ResultHolderComparator()); waits = new Semaphore(throttleLimit); } @@ -100,8 +100,8 @@ public void put(ResultHolder holder) throws IllegalArgumentException { } /** - * Get the next result as soon as it becomes available.
      - *
      + * Get the next result as soon as it becomes available.
      + *
      * Release result immediately if: *
        *
      • There is a result that is continuable.
      • diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/ResultQueue.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/ResultQueue.java index c3c83f292e..c89dfbdee6 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/ResultQueue.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/ResultQueue.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,12 +35,12 @@ interface ResultQueue { /** - * In a master-slave pattern, the master calls this method paired with + * In a manager-worker pattern, the manager calls this method paired with * {@link #take()} to manage the flow of items. Normally a task is submitted - * for processing in another thread, at which point the master uses this + * for processing in another thread, at which point the manager uses this * method to keep track of the number of expected results. It has the * personality of an counter increment, rather than a work queue, which is - * usually managed elsewhere, e.g. by a {@link TaskExecutor}.

        + * usually managed elsewhere, e.g. by a {@link TaskExecutor}.

        * Implementations may choose to block here, if they need to limit the * number or rate of tasks being submitted. * @@ -50,7 +50,7 @@ interface ResultQueue { /** * Once it is expecting a result, clients call this method to satisfy the - * expectation. In a master-worker pattern, the workers call this method to + * expectation. In a manager-worker pattern, the workers call this method to * deposit the result of a finished task on the queue for collection. * * @param result the result for later collection. @@ -72,7 +72,7 @@ interface ResultQueue { T take() throws NoSuchElementException, InterruptedException; /** - * Used by master thread to verify that there are results available from + * Used by manager thread to verify that there are results available from * {@link #take()} without possibly having to block and wait. * * @return true if there are no results available @@ -80,7 +80,7 @@ interface ResultQueue { boolean isEmpty(); /** - * Check if any results are expected. Usually used by master thread to drain + * Check if any results are expected. Usually used by manager thread to drain * queue when it is finished. * * @return true if more results are expected, but possibly not yet diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplate.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplate.java index 927eceb616..8737033d90 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplate.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplate.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,21 +27,21 @@ /** * Provides {@link RepeatOperations} support including interceptors that can be - * used to modify or monitor the behaviour at run time.
        + * used to modify or monitor the behaviour at run time.
        * * This implementation is sufficient to be used to configure transactional * behaviour for each item by making the {@link RepeatCallback} transactional, * or for the whole batch by making the execute method transactional (but only - * then if the task executor is synchronous).
        + * then if the task executor is synchronous).
        * - * This class is thread safe if its collaborators are thread safe (interceptors, + * This class is thread-safe if its collaborators are thread-safe (interceptors, * terminationPolicy, callback). Normally this will be the case, but clients * need to be aware that if the task executor is asynchronous, then the other * collaborators should be also. In particular the {@link RepeatCallback} that - * is wrapped in the execute method must be thread safe - often it is based on - * some form of data source, which itself should be both thread safe and + * is wrapped in the execute method must be thread-safe - often it is based on + * some form of data source, which itself should be both thread-safe and * transactional (multiple threads could be accessing it at any given time, and - * each thread would have its own transaction).
        + * each thread would have its own transaction).
        * * @author Dave Syer * @@ -83,7 +83,7 @@ public void setThrottleLimit(int throttleLimit) { * @throws IllegalArgumentException if the argument is null */ public void setTaskExecutor(TaskExecutor taskExecutor) { - Assert.notNull(taskExecutor); + Assert.notNull(taskExecutor, "A TaskExecutor is required"); this.taskExecutor = taskExecutor; } @@ -99,7 +99,7 @@ public void setTaskExecutor(TaskExecutor taskExecutor) { protected RepeatStatus getNextResult(RepeatContext context, RepeatCallback callback, RepeatInternalState state) throws Throwable { - ExecutingRunnable runnable = null; + ExecutingRunnable runnable; ResultQueue queue = ((ResultQueueInternalState) state).getResultQueue(); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/ThrottleLimitResultQueue.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/ThrottleLimitResultQueue.java index b469d111f9..dd764c4435 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/ThrottleLimitResultQueue.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/ThrottleLimitResultQueue.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -44,7 +44,7 @@ public class ThrottleLimitResultQueue implements ResultQueue { * at any given time. */ public ThrottleLimitResultQueue(int throttleLimit) { - results = new LinkedBlockingQueue(); + results = new LinkedBlockingQueue<>(); waits = new Semaphore(throttleLimit); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/package-info.java new file mode 100644 index 0000000000..6d690403cc --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/package-info.java @@ -0,0 +1,9 @@ +/** + *

        + * Infrastructure implementations of repeat support concerns. + *

        + */ +@NonNullApi +package org.springframework.batch.repeat.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/package.html deleted file mode 100644 index 6e3f0eef50..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/repeat/support/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

        -Infrastructure implementations of repeat support concerns. -

        - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/AnnotationMethodResolver.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/AnnotationMethodResolver.java index eb8ad37e7e..33f4201143 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/AnnotationMethodResolver.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/AnnotationMethodResolver.java @@ -1,11 +1,11 @@ /* - * Copyright 2002-2008 the original author or authors. + * Copyright 2002-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,12 +24,13 @@ import org.springframework.aop.support.AopUtils; import org.springframework.core.annotation.AnnotationUtils; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.ObjectUtils; import org.springframework.util.ReflectionUtils; /** - * MethodResolver implementation that finds a single Method on the + * {@link MethodResolver} implementation that finds a single Method on the * given Class that contains the specified annotation type. * * @author Mark Fisher @@ -40,7 +41,9 @@ public class AnnotationMethodResolver implements MethodResolver { /** - * Create a MethodResolver for the specified Method-level annotation type + * Create a {@link MethodResolver} for the specified Method-level annotation type. + * + * @param annotationType establish the annotation to be used. */ public AnnotationMethodResolver(Class annotationType) { Assert.notNull(annotationType, "annotationType must not be null"); @@ -64,7 +67,8 @@ public AnnotationMethodResolver(Class annotationType) { * @throws IllegalArgumentException if more than one Method has the * specified annotation */ - @Override + @Nullable + @Override public Method findMethod(Object candidate) { Assert.notNull(candidate, "candidate object must not be null"); Class targetClass = AopUtils.getTargetClass(candidate); @@ -86,10 +90,11 @@ public Method findMethod(Object candidate) { * @throws IllegalArgumentException if more than one Method has the * specified annotation */ - @Override + @Nullable + @Override public Method findMethod(final Class clazz) { Assert.notNull(clazz, "class must not be null"); - final AtomicReference annotatedMethod = new AtomicReference(); + final AtomicReference annotatedMethod = new AtomicReference<>(); ReflectionUtils.doWithMethods(clazz, new ReflectionUtils.MethodCallback() { @Override public void doWith(Method method) throws IllegalArgumentException, IllegalAccessException { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/DatabaseType.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/DatabaseType.java index 756e56f5ab..4e002e8b12 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/DatabaseType.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/DatabaseType.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,39 +16,43 @@ package org.springframework.batch.support; -import java.util.HashMap; -import java.util.Map; - -import javax.sql.DataSource; - import org.springframework.jdbc.support.JdbcUtils; import org.springframework.jdbc.support.MetaDataAccessException; +import org.springframework.util.StringUtils; + +import javax.sql.DataSource; +import java.util.HashMap; +import java.util.Map; /** * Enum representing a database type, such as DB2 or oracle. The type also * contains a product name, which is expected to be the same as the product name * provided by the database driver's metadata. - * + * * @author Lucas Ward * @since 2.0 */ public enum DatabaseType { - DERBY("Apache Derby"), - DB2("DB2"), - DB2ZOS("DB2ZOS"), + DERBY("Apache Derby"), + DB2("DB2"), + DB2VSE("DB2VSE"), + DB2ZOS("DB2ZOS"), + DB2AS400("DB2AS400"), HSQL("HSQL Database Engine"), SQLSERVER("Microsoft SQL Server"), MYSQL("MySQL"), ORACLE("Oracle"), POSTGRES("PostgreSQL"), - SYBASE("Sybase"), H2("H2"); - + SYBASE("Sybase"), + H2("H2"), + SQLITE("SQLite"); + private static final Map nameMap; - + static{ - nameMap = new HashMap(); + nameMap = new HashMap<>(); for(DatabaseType type: values()){ nameMap.put(type.getProductName(), type); } @@ -56,48 +60,59 @@ public enum DatabaseType { //A description is necessary due to the nature of database descriptions //in metadata. private final String productName; - + private DatabaseType(String productName) { this.productName = productName; } - + public String getProductName() { return productName; } - + /** * Static method to obtain a DatabaseType from the provided product name. - * - * @param productName - * @return DatabaseType for given product name. + * + * @param productName {@link String} containing the product name. + * @return the {@link DatabaseType} for given product name. + * * @throws IllegalArgumentException if none is found. */ public static DatabaseType fromProductName(String productName){ + if(productName.equals("MariaDB")) + productName = "MySQL"; if(!nameMap.containsKey(productName)){ - throw new IllegalArgumentException("DatabaseType not found for product name: [" + + throw new IllegalArgumentException("DatabaseType not found for product name: [" + productName + "]"); } else{ return nameMap.get(productName); } } - + /** * Convenience method that pulls a database product name from the DataSource's metadata. - * - * @param dataSource - * @return DatabaseType - * @throws MetaDataAccessException + * + * @param dataSource {@link DataSource} to the database to be used. + * @return {@link DatabaseType} for the {@link DataSource} specified. + * + * @throws MetaDataAccessException thrown if error occured during Metadata lookup. */ public static DatabaseType fromMetaData(DataSource dataSource) throws MetaDataAccessException { String databaseProductName = JdbcUtils.extractDatabaseMetaData(dataSource, "getDatabaseProductName").toString(); - if ("DB2".equals(databaseProductName)) { + if (StringUtils.hasText(databaseProductName) && databaseProductName.startsWith("DB2")) { String databaseProductVersion = JdbcUtils.extractDatabaseMetaData(dataSource, "getDatabaseProductVersion").toString(); - if (!databaseProductVersion.startsWith("SQL")) { + if (databaseProductVersion.startsWith("ARI")) { + databaseProductName = "DB2VSE"; + } + else if (databaseProductVersion.startsWith("DSN")) { databaseProductName = "DB2ZOS"; } + else if (databaseProductName.indexOf("AS") != -1 && (databaseProductVersion.startsWith("QSQ") || + databaseProductVersion.substring(databaseProductVersion.indexOf('V')).matches("V\\dR\\d[mM]\\d"))) { + databaseProductName = "DB2AS400"; + } else { databaseProductName = JdbcUtils.commonDatabaseName(databaseProductName); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/DefaultPropertyEditorRegistrar.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/DefaultPropertyEditorRegistrar.java index 8ec345cf4d..42389382de 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/DefaultPropertyEditorRegistrar.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/DefaultPropertyEditorRegistrar.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,7 +29,7 @@ * A re-usable {@link PropertyEditorRegistrar} that can be used wherever one * needs to register custom {@link PropertyEditor} instances with a * {@link PropertyEditorRegistry} (like a bean wrapper, or a type converter). It - * is not thread safe, but useful where one is confident that binding or + * is not thread safe, but useful where one is confident that binding or * initialisation can only be single threaded (e.g in a standalone application * with no threads). * @@ -63,7 +63,7 @@ public void registerCustomEditors(PropertyEditorRegistry registry) { * @see CustomEditorConfigurer#setCustomEditors(Map) */ public void setCustomEditors(Map customEditors) { - this.customEditors = new HashMap, PropertyEditor>(); + this.customEditors = new HashMap<>(); for (Entry entry : customEditors.entrySet()) { Object key = entry.getKey(); Class requiredType = null; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/IntArrayPropertyEditor.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/IntArrayPropertyEditor.java index 1f268d3afe..140b75b7be 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/IntArrayPropertyEditor.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/IntArrayPropertyEditor.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/LastModifiedResourceComparator.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/LastModifiedResourceComparator.java index 593f199cab..658e5432e6 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/LastModifiedResourceComparator.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/LastModifiedResourceComparator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/MethodInvoker.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/MethodInvoker.java index 46f3992afa..fb14ec07d6 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/MethodInvoker.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/MethodInvoker.java @@ -1,11 +1,11 @@ /* - * Copyright 2002-2008 the original author or authors. + * Copyright 2002-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,14 +16,18 @@ package org.springframework.batch.support; +import org.springframework.lang.Nullable; + /** * A strategy interface for invoking a method. * Typically used by adapters. * * @author Mark Fisher + * @author Mahmoud Ben Hassine */ public interface MethodInvoker { + @Nullable Object invokeMethod(Object ... args); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/MethodInvokerUtils.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/MethodInvokerUtils.java index fac868f408..9a50018015 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/MethodInvokerUtils.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/MethodInvokerUtils.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -52,12 +52,12 @@ public static MethodInvoker getMethodInvokerByName(Object object, String methodN Method method = ClassUtils.getMethodIfAvailable(object.getClass(), methodName, paramTypes); if (method == null) { String errorMsg = "no method found with name [" + methodName + "] on class [" - + object.getClass().getSimpleName() + "] compatable with the signature [" + + object.getClass().getSimpleName() + "] compatible with the signature [" + getParamTypesString(paramTypes) + "]."; Assert.isTrue(!paramsRequired, errorMsg); // if no method was found for the given parameters, and the // parameters aren't required, then try with no params - method = ClassUtils.getMethodIfAvailable(object.getClass(), methodName, new Class[] {}); + method = ClassUtils.getMethodIfAvailable(object.getClass(), methodName); Assert.notNull(method, errorMsg); } return new SimpleMethodInvoker(object, method); @@ -66,11 +66,11 @@ public static MethodInvoker getMethodInvokerByName(Object object, String methodN /** * Create a String representation of the array of parameter types. * - * @param paramTypes - * @return String + * @param paramTypes types of the parameters to be used + * @return String a String representation of those types */ public static String getParamTypesString(Class... paramTypes) { - StringBuffer paramTypesList = new StringBuffer("("); + StringBuilder paramTypesList = new StringBuilder("("); for (int i = 0; i < paramTypes.length; i++) { paramTypesList.append(paramTypes[i].getSimpleName()); if (i + 1 < paramTypes.length) { @@ -116,22 +116,19 @@ public static MethodInvoker getMethodInvokerByAnnotation(final Class targetClass = (target instanceof Advised) ? ((Advised) target).getTargetSource() .getTargetClass() : target.getClass(); if (mi != null) { - ReflectionUtils.doWithMethods(targetClass, new ReflectionUtils.MethodCallback() { - @Override - public void doWith(Method method) throws IllegalArgumentException, IllegalAccessException { - Annotation annotation = AnnotationUtils.findAnnotation(method, annotationType); - if (annotation != null) { - Class[] paramTypes = method.getParameterTypes(); - if (paramTypes.length > 0) { - String errorMsg = "The method [" + method.getName() + "] on target class [" - + targetClass.getSimpleName() + "] is incompatable with the signature [" - + getParamTypesString(expectedParamTypes) + "] expected for the annotation [" - + annotationType.getSimpleName() + "]."; + ReflectionUtils.doWithMethods(targetClass, method -> { + Annotation annotation = AnnotationUtils.findAnnotation(method, annotationType); + if (annotation != null) { + Class[] paramTypes = method.getParameterTypes(); + if (paramTypes.length > 0) { + String errorMsg = "The method [" + method.getName() + "] on target class [" + + targetClass.getSimpleName() + "] is incompatible with the signature [" + + getParamTypesString(expectedParamTypes) + "] expected for the annotation [" + + annotationType.getSimpleName() + "]."; - Assert.isTrue(paramTypes.length == expectedParamTypes.length, errorMsg); - for (int i = 0; i < paramTypes.length; i++) { - Assert.isTrue(expectedParamTypes[i].isAssignableFrom(paramTypes[i]), errorMsg); - } + Assert.isTrue(paramTypes.length == expectedParamTypes.length, errorMsg); + for (int i = 0; i < paramTypes.length; i++) { + Assert.isTrue(expectedParamTypes[i].isAssignableFrom(paramTypes[i]), errorMsg); } } } @@ -162,17 +159,14 @@ public static MethodInvoker getMethodInvokerByAnnotation(final Class annotatedMethod = new AtomicReference(); - ReflectionUtils.doWithMethods(targetClass, new ReflectionUtils.MethodCallback() { - @Override - public void doWith(Method method) throws IllegalArgumentException, IllegalAccessException { - Annotation annotation = AnnotationUtils.findAnnotation(method, annotationType); - if (annotation != null) { - Assert.isNull(annotatedMethod.get(), "found more than one method on target class [" - + targetClass.getSimpleName() + "] with the annotation type [" - + annotationType.getSimpleName() + "]."); - annotatedMethod.set(method); - } + final AtomicReference annotatedMethod = new AtomicReference<>(); + ReflectionUtils.doWithMethods(targetClass, method -> { + Annotation annotation = AnnotationUtils.findAnnotation(method, annotationType); + if (annotation != null) { + Assert.isNull(annotatedMethod.get(), "found more than one method on target class [" + + targetClass.getSimpleName() + "] with the annotation type [" + + annotationType.getSimpleName() + "]."); + annotatedMethod.set(method); } }); Method method = annotatedMethod.get(); @@ -188,24 +182,23 @@ public void doWith(Method method) throws IllegalArgumentException, IllegalAccess * Create a {@link MethodInvoker} for the delegate from a single public * method. * - * @param target an object to search for an appropriate method - * @return a MethodInvoker that calls a method on the delegate + * @param target an object to search for an appropriate method. + * @param the class. + * @param the type. + * @return a {@link MethodInvoker} that calls a method on the delegate. */ public static MethodInvoker getMethodInvokerForSingleArgument(Object target) { - final AtomicReference methodHolder = new AtomicReference(); - ReflectionUtils.doWithMethods(target.getClass(), new ReflectionUtils.MethodCallback() { - @Override - public void doWith(Method method) throws IllegalArgumentException, IllegalAccessException { - if (method.getParameterTypes() == null || method.getParameterTypes().length != 1) { - return; - } - if (method.getReturnType().equals(Void.TYPE) || ReflectionUtils.isEqualsMethod(method)) { - return; - } - Assert.state(methodHolder.get() == null, - "More than one non-void public method detected with single argument."); - methodHolder.set(method); + final AtomicReference methodHolder = new AtomicReference<>(); + ReflectionUtils.doWithMethods(target.getClass(), method -> { + if (method.getParameterTypes() == null || method.getParameterTypes().length != 1) { + return; + } + if (method.getReturnType().equals(Void.TYPE) || ReflectionUtils.isEqualsMethod(method)) { + return; } + Assert.state(methodHolder.get() == null, + "More than one non-void public method detected with single argument."); + methodHolder.set(method); }); Method method = methodHolder.get(); return new SimpleMethodInvoker(target, method); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/MethodResolver.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/MethodResolver.java index 3cab3454f0..b8b9706974 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/MethodResolver.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/MethodResolver.java @@ -1,11 +1,11 @@ /* - * Copyright 2002-2008 the original author or authors. + * Copyright 2002-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,10 +18,13 @@ import java.lang.reflect.Method; +import org.springframework.lang.Nullable; + /** * Strategy interface for detecting a single Method on a Class. * * @author Mark Fisher + * @author Mahmoud Ben Hassine */ public interface MethodResolver { @@ -38,6 +41,7 @@ public interface MethodResolver { * @throws IllegalArgumentException if more than one Method defined on the * given candidate's Class matches this resolver's criteria */ + @Nullable Method findMethod(Object candidate) throws IllegalArgumentException; /** @@ -52,6 +56,7 @@ public interface MethodResolver { * @throws IllegalArgumentException if more than one Method defined on the * given Class matches this resolver's criteria */ + @Nullable Method findMethod(Class clazz); } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/PatternMatcher.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/PatternMatcher.java index 3fbfefbd73..06f6bd1687 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/PatternMatcher.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/PatternMatcher.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,8 +30,8 @@ */ public class PatternMatcher { - private Map map = new HashMap(); - private List sorted = new ArrayList(); + private Map map = new HashMap<>(); + private List sorted = new ArrayList<>(); /** * Initialize a new {@link PatternMatcher} with a map of patterns to values @@ -41,7 +41,7 @@ public PatternMatcher(Map map) { super(); this.map = map; // Sort keys to start with the most specific - sorted = new ArrayList(map.keySet()); + sorted = new ArrayList<>(map.keySet()); Collections.sort(sorted, new Comparator() { @Override public int compare(String o1, String o2) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/PropertiesConverter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/PropertiesConverter.java index d2b7b13f3c..e7c1028b8a 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/PropertiesConverter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/PropertiesConverter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -49,7 +49,7 @@ public final class PropertiesConverter { // prevents the class from being instantiated private PropertiesConverter() { - }; + } /** * Parse a String to a Properties object. If string is null, an empty @@ -96,7 +96,7 @@ public static Properties stringToProperties(String stringToParse) { * an empty properties object is passed in, a blank string is returned, * otherwise it's string representation is returned. * - * @param propertiesToParse + * @param propertiesToParse contains the properties be converted. * @return String representation of properties object */ public static String propertiesToString(Properties propertiesToParse) { diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/ReflectionUtils.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/ReflectionUtils.java new file mode 100644 index 0000000000..4b0c64f14f --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/ReflectionUtils.java @@ -0,0 +1,60 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.support; + +import java.lang.annotation.Annotation; +import java.lang.reflect.Method; +import java.util.HashSet; +import java.util.Set; + +import org.springframework.core.annotation.AnnotationUtils; + +/** + * Provides reflection based utilities for Spring Batch that are not available + * via Spring Core + * + * @author Michael Minella + * @since 2.2.6 + */ +public class ReflectionUtils { + + private ReflectionUtils() {} + + /** + * Returns a {@link java.util.Set} of {@link java.lang.reflect.Method} instances that + * are annotated with the annotation provided. + * + * @param clazz The class to search for a method with the given annotation type + * @param annotationType The type of annotation to look for + * @return a set of {@link java.lang.reflect.Method} instances if any are found, an empty set if not. + */ + @SuppressWarnings("rawtypes") + public static final Set findMethod(Class clazz, Class annotationType) { + + Method [] declaredMethods = org.springframework.util.ReflectionUtils.getAllDeclaredMethods(clazz); + Set results = new HashSet<>(); + + for (Method curMethod : declaredMethods) { + Annotation annotation = AnnotationUtils.findAnnotation(curMethod, annotationType); + + if(annotation != null) { + results.add(curMethod); + } + } + + return results; + } +} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/SerializationUtils.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/SerializationUtils.java deleted file mode 100644 index ac4f5ac815..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/SerializationUtils.java +++ /dev/null @@ -1,82 +0,0 @@ -/* - * Copyright 2006-2010 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.support; - -import java.io.ByteArrayInputStream; -import java.io.ByteArrayOutputStream; -import java.io.IOException; -import java.io.ObjectInputStream; -import java.io.ObjectOutputStream; -import java.io.OptionalDataException; - - -/** - * Static utility to help with serialization. - * - * @author Dave Syer - * - */ -public class SerializationUtils { - - /** - * Serialize the object provided. - * - * @param object the object to serialize - * @return an array of bytes representing the object in a portable fashion - */ - public static byte[] serialize(Object object) { - - if (object==null) { - return null; - } - - ByteArrayOutputStream stream = new ByteArrayOutputStream(); - try { - new ObjectOutputStream(stream).writeObject(object); - } catch (IOException e) { - throw new IllegalArgumentException("Could not serialize object of type: "+object.getClass(), e); - } - - return stream.toByteArray(); - - } - - /** - * @param bytes a serialized object created - * @return the result of deserializing the bytes - */ - public static Object deserialize(byte[] bytes) { - - if (bytes==null) { - return null; - } - - try { - return new ObjectInputStream(new ByteArrayInputStream(bytes)).readObject(); - } - catch (OptionalDataException e) { - throw new IllegalArgumentException("Could not deserialize object: eof="+e.eof+ " at length="+e.length, e); - } - catch (IOException e) { - throw new IllegalArgumentException("Could not deserialize object", e); - } - catch (ClassNotFoundException e) { - throw new IllegalStateException("Could not deserialize object type", e); - } - - } - -} diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/SimpleMethodInvoker.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/SimpleMethodInvoker.java index 0ad66fda27..fe16883a87 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/SimpleMethodInvoker.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/SimpleMethodInvoker.java @@ -1,11 +1,11 @@ /* - * Copyright 2002-2008 the original author or authors. + * Copyright 2002-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,7 +21,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,6 +35,7 @@ import java.util.Arrays; import org.springframework.aop.framework.Advised; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.ClassUtils; @@ -65,7 +66,7 @@ public SimpleMethodInvoker(Object object, String methodName, Class... paramTy this.method = ClassUtils.getMethodIfAvailable(object.getClass(), methodName, paramTypes); if (this.method == null) { // try with no params - this.method = ClassUtils.getMethodIfAvailable(object.getClass(), methodName, new Class[] {}); + this.method = ClassUtils.getMethodIfAvailable(object.getClass(), methodName); } if (this.method == null) { throw new IllegalArgumentException("No methods found for name: [" + methodName + "] in class: [" @@ -81,7 +82,8 @@ public SimpleMethodInvoker(Object object, String methodName, Class... paramTy * org.springframework.batch.core.configuration.util.MethodInvoker#invokeMethod * (java.lang.Object[]) */ - @Override + @Nullable + @Override public Object invokeMethod(Object... args) { Class[] parameterTypes = method.getParameterTypes(); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/SystemPropertyInitializer.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/SystemPropertyInitializer.java index 5f9d1070ac..8e0ea2e429 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/SystemPropertyInitializer.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/SystemPropertyInitializer.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/annotation/Classifier.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/annotation/Classifier.java index d28ca042b5..6e7aa6c145 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/annotation/Classifier.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/annotation/Classifier.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/package-info.java new file mode 100644 index 0000000000..01ab17a99f --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/package-info.java @@ -0,0 +1,9 @@ +/** + *

        + * Infrastructure implementations of support concerns. + *

        + */ +@NonNullApi +package org.springframework.batch.support; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/package.html deleted file mode 100644 index 3da319dce4..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

        -Infrastructure implementations of support concerns. -

        - - diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/FlushFailedException.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/FlushFailedException.java index 5d2c930998..4dcac0986d 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/FlushFailedException.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/FlushFailedException.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ * @author Lucas Ward * @author Ben Hale */ +@SuppressWarnings("serial") public class FlushFailedException extends RuntimeException { /** diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/ResourcelessTransactionManager.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/ResourcelessTransactionManager.java index 9c72784420..66d10d2b9e 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/ResourcelessTransactionManager.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/ResourcelessTransactionManager.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,7 +16,8 @@ package org.springframework.batch.support.transaction; -import java.util.Stack; +import java.util.ArrayList; +import java.util.List; import org.springframework.transaction.TransactionDefinition; import org.springframework.transaction.TransactionException; @@ -24,6 +25,7 @@ import org.springframework.transaction.support.DefaultTransactionStatus; import org.springframework.transaction.support.TransactionSynchronizationManager; +@SuppressWarnings("serial") public class ResourcelessTransactionManager extends AbstractPlatformTransactionManager { @Override @@ -33,36 +35,39 @@ protected void doBegin(Object transaction, TransactionDefinition definition) thr @Override protected void doCommit(DefaultTransactionStatus status) throws TransactionException { - logger.debug("Committing resourceless transaction on [" + status.getTransaction() + "]"); + if (logger.isDebugEnabled()) { + logger.debug("Committing resourceless transaction on [" + status.getTransaction() + "]"); + } } @Override protected Object doGetTransaction() throws TransactionException { Object transaction = new ResourcelessTransaction(); - Stack resources; + List resources; if (!TransactionSynchronizationManager.hasResource(this)) { - resources = new Stack(); + resources = new ArrayList<>(); TransactionSynchronizationManager.bindResource(this, resources); } else { @SuppressWarnings("unchecked") - Stack stack = (Stack) TransactionSynchronizationManager.getResource(this); + List stack = (List) TransactionSynchronizationManager.getResource(this); resources = stack; } - resources.push(transaction); + resources.add(transaction); return transaction; } @Override protected void doRollback(DefaultTransactionStatus status) throws TransactionException { - logger.debug("Rolling back resourceless transaction on [" + status.getTransaction() + "]"); + if (logger.isDebugEnabled()) { + logger.debug("Rolling back resourceless transaction on [" + status.getTransaction() + "]"); + } } @Override protected boolean isExistingTransaction(Object transaction) throws TransactionException { if (TransactionSynchronizationManager.hasResource(this)) { - @SuppressWarnings("unchecked") - Stack stack = (Stack) TransactionSynchronizationManager.getResource(this); + List stack = (List) TransactionSynchronizationManager.getResource(this); return stack.size() > 1; } return ((ResourcelessTransaction) transaction).isActive(); @@ -74,9 +79,7 @@ protected void doSetRollbackOnly(DefaultTransactionStatus status) throws Transac @Override protected void doCleanupAfterCompletion(Object transaction) { - @SuppressWarnings("unchecked") - Stack list = (Stack) TransactionSynchronizationManager.getResource(this); - Stack resources = list; + List resources = (List) TransactionSynchronizationManager.getResource(this); resources.clear(); TransactionSynchronizationManager.unbindResource(this); ((ResourcelessTransaction) transaction).clear(); diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/TransactionAwareBufferedWriter.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/TransactionAwareBufferedWriter.java index 5bd5d978a3..13ae6d4932 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/TransactionAwareBufferedWriter.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/TransactionAwareBufferedWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,20 +30,16 @@ * buffer if a transaction is active. If a transaction is detected on the call * to {@link #write(String)} the parameter is buffered and passed on to the * underlying writer only when the transaction is committed. - * + * * @author Dave Syer * @author Michael Minella - * + * */ public class TransactionAwareBufferedWriter extends Writer { - private static final String BUFFER_KEY_PREFIX = TransactionAwareBufferedWriter.class.getName() + ".BUFFER_KEY"; - - private static final String CLOSE_KEY_PREFIX = TransactionAwareBufferedWriter.class.getName() + ".CLOSE_KEY"; - - private final String bufferKey; + private final Object bufferKey; - private final String closeKey; + private final Object closeKey; private FileChannel channel; @@ -60,16 +56,16 @@ public class TransactionAwareBufferedWriter extends Writer { * Create a new instance with the underlying file channel provided, and a callback * to execute on close. The callback should clean up related resources like * output streams or channels. - * - * @param channel channel used to do the actuall file IO + * + * @param channel channel used to do the actual file IO * @param closeCallback callback to execute on close */ public TransactionAwareBufferedWriter(FileChannel channel, Runnable closeCallback) { super(); this.channel = channel; this.closeCallback = closeCallback; - this.bufferKey = BUFFER_KEY_PREFIX + "." + hashCode(); - this.closeKey = CLOSE_KEY_PREFIX + "." + hashCode(); + this.bufferKey = new Object(); + this.closeKey = new Object(); } public void setEncoding(String encoding) { @@ -82,7 +78,7 @@ public void setEncoding(String encoding) { * be lost if the OS crashes in between a write and a cache flush. Setting * to true may result in slower performance for usage patterns involving * many frequent writes. - * + * * @param forceSync the flag value to set */ public void setForceSync(boolean forceSync) { @@ -92,11 +88,11 @@ public void setForceSync(boolean forceSync) { /** * @return */ - private StringBuffer getCurrentBuffer() { + private StringBuilder getCurrentBuffer() { if (!TransactionSynchronizationManager.hasResource(bufferKey)) { - TransactionSynchronizationManager.bindResource(bufferKey, new StringBuffer()); + TransactionSynchronizationManager.bindResource(bufferKey, new StringBuilder()); TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() { @Override @@ -117,7 +113,7 @@ public void beforeCommit(boolean readOnly) { } private void complete() throws IOException { - StringBuffer buffer = (StringBuffer) TransactionSynchronizationManager.getResource(bufferKey); + StringBuilder buffer = (StringBuilder) TransactionSynchronizationManager.getResource(bufferKey); if (buffer != null) { String string = buffer.toString(); byte[] bytes = string.getBytes(encoding); @@ -149,14 +145,14 @@ private void clear() { } - return (StringBuffer) TransactionSynchronizationManager.getResource(bufferKey); + return (StringBuilder) TransactionSynchronizationManager.getResource(bufferKey); } /** * Convenience method for clients to determine if there is any unflushed * data. - * + * * @return the current size (in bytes) of unflushed buffered data */ public long getBufferSize() { @@ -179,7 +175,7 @@ private boolean transactionActive() { /* * (non-Javadoc) - * + * * @see java.io.Writer#close() */ @Override @@ -195,7 +191,7 @@ public void close() throws IOException { /* * (non-Javadoc) - * + * * @see java.io.Writer#flush() */ @Override @@ -207,7 +203,7 @@ public void flush() throws IOException { /* * (non-Javadoc) - * + * * @see java.io.Writer#write(char[], int, int) */ @Override @@ -226,7 +222,7 @@ public void write(char[] cbuf, int off, int len) throws IOException { return; } - StringBuffer buffer = getCurrentBuffer(); + StringBuilder buffer = getCurrentBuffer(); buffer.append(cbuf, off, len); } } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/TransactionAwareProxyFactory.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/TransactionAwareProxyFactory.java index 7b18d6ad55..1c5d872ec5 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/TransactionAwareProxyFactory.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/TransactionAwareProxyFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -159,39 +159,39 @@ private T createInstance() { } public static Map createTransactionalMap() { - return (Map) new TransactionAwareProxyFactory>(new ConcurrentHashMap()).createInstance(); + return new TransactionAwareProxyFactory<>(new ConcurrentHashMap()).createInstance(); } public static Map createTransactionalMap(Map map) { - return (Map) new TransactionAwareProxyFactory>(new ConcurrentHashMap(map)).createInstance(); + return new TransactionAwareProxyFactory<>(new ConcurrentHashMap<>(map)).createInstance(); } public static ConcurrentMap createAppendOnlyTransactionalMap() { - return new TransactionAwareProxyFactory>(new ConcurrentHashMap(), true).createInstance(); + return new TransactionAwareProxyFactory<>(new ConcurrentHashMap(), true).createInstance(); } public static Set createAppendOnlyTransactionalSet() { - return (Set) new TransactionAwareProxyFactory>(new CopyOnWriteArraySet(), true).createInstance(); + return new TransactionAwareProxyFactory<>(new CopyOnWriteArraySet(), true).createInstance(); } public static Set createTransactionalSet() { - return (Set) new TransactionAwareProxyFactory>(new CopyOnWriteArraySet()).createInstance(); + return new TransactionAwareProxyFactory<>(new CopyOnWriteArraySet()).createInstance(); } public static Set createTransactionalSet(Set set) { - return (Set) new TransactionAwareProxyFactory>(new CopyOnWriteArraySet(set)).createInstance(); + return new TransactionAwareProxyFactory<>(new CopyOnWriteArraySet<>(set)).createInstance(); } public static List createAppendOnlyTransactionalList() { - return (List) new TransactionAwareProxyFactory>(new CopyOnWriteArrayList(), true).createInstance(); + return new TransactionAwareProxyFactory<>(new CopyOnWriteArrayList(), true).createInstance(); } public static List createTransactionalList() { - return (List) new TransactionAwareProxyFactory>(new CopyOnWriteArrayList()).createInstance(); + return new TransactionAwareProxyFactory<>(new CopyOnWriteArrayList()).createInstance(); } public static List createTransactionalList(List list) { - return (List) new TransactionAwareProxyFactory>(new CopyOnWriteArrayList(list)).createInstance(); + return new TransactionAwareProxyFactory<>(new CopyOnWriteArrayList<>(list)).createInstance(); } private class TargetSynchronization extends TransactionSynchronizationAdapter { @@ -252,7 +252,7 @@ public Object invoke(MethodInvocation invocation) throws Throwable { return invocation.proceed(); } if (result instanceof Collection) { - HashSet set = new HashSet((Collection) result); + HashSet set = new HashSet<>((Collection) result); set.addAll((Collection) invocation.proceed()); result = set; } diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/package-info.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/package-info.java new file mode 100644 index 0000000000..19e932270a --- /dev/null +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/package-info.java @@ -0,0 +1,9 @@ +/** + *

        + * Infrastructure implementations of support transaction concerns. + *

        + */ +@NonNullApi +package org.springframework.batch.support.transaction; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/package.html b/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/package.html deleted file mode 100644 index 77cef996ac..0000000000 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/support/transaction/package.html +++ /dev/null @@ -1,7 +0,0 @@ - - -

        -Infrastructure implementations of support transaction concerns. -

        - - diff --git a/spring-batch-infrastructure/src/site/apt/changelog.apt b/spring-batch-infrastructure/src/site/apt/changelog.apt deleted file mode 100644 index 7c41b85a29..0000000000 --- a/spring-batch-infrastructure/src/site/apt/changelog.apt +++ /dev/null @@ -1,7 +0,0 @@ -Changelog: Spring Batch Infrastructure - -* 1.0-M2 - -** 2007/07/12 - - * No-one uses this file: we should just switch to auto-generated changelogs? diff --git a/spring-batch-infrastructure/src/site/apt/index.apt b/spring-batch-infrastructure/src/site/apt/index.apt deleted file mode 100644 index 1da232bae9..0000000000 --- a/spring-batch-infrastructure/src/site/apt/index.apt +++ /dev/null @@ -1,57 +0,0 @@ - ------ - Spring Batch Infrastructure - ------ - Dave Syer - ------ - December 2006 - -Introduction - - This module provides framework code for the Spring Batch project. - The core interfaces are <<>>, <<>>, <<>>, - <<>> and <<>>. - - The reader and writer interfaces are implemented in some generic - ways to help users with I/O in a transactional envionment. Spring - Batch provides some file-based (flat file and xml) implementations - and some database implementations. - - The main implementations of the repeat and retry interfaces are - <<>> and <<>>. Example usage: - -+--- -RepeatTemplate template = new RepeatTemplate(); - -template.setCompletionPolicy(new FixedChunkSizeCompletionPolicy(2)); - -template.iterate(new RepeatCallback() { - - public RepeatStatus doInIteration(RepeatContext context) { - // Do stuff in batch... - return RepeatStatus.CONTINUABLE; // Return RepeatStatus.FINISHED to signal exhausted data - } - -}); -+--- - - The callback is executed repeatedly, until the completion policy - determines that the batch should end (in the example, twice). - - The framework provides <<>> for automatic retry of - a business operation. This is independent of the batching support, - but will often be used in conjunction with it. Example usage: - -+--- -RetryTemplate template = new RetryTemplate(); - -template.setRetryPolicy(new TimeoutRetryPolicy(30000L)); - -Object result = template.execute(new RetryCallback() { - - public Object doWithRetry(RetryContext context) { - // Do stuff that might fail, e.g. webservice operation - return result; - } - -}); -+--- diff --git a/spring-batch-infrastructure/src/site/site.xml b/spring-batch-infrastructure/src/site/site.xml deleted file mode 100644 index 372e1dac34..0000000000 --- a/spring-batch-infrastructure/src/site/site.xml +++ /dev/null @@ -1,23 +0,0 @@ - - - - - Spring Batch: ${project.name} - index.html - - - - - - - - - - - - - - - - - diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/AbstractItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/AbstractItemReaderTests.java index e53a5cc36a..42ab5322f1 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/AbstractItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/AbstractItemReaderTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/AbstractItemStreamItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/AbstractItemStreamItemReaderTests.java index dac63a0daf..8a4b5d7d28 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/AbstractItemStreamItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/AbstractItemStreamItemReaderTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item; import static org.junit.Assert.*; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ExecutionContextTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ExecutionContextTests.java index fd7ef8f9e2..a8f32a0c7e 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ExecutionContextTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ExecutionContextTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,11 +25,12 @@ import org.junit.Before; import org.junit.Test; -import org.springframework.batch.support.SerializationUtils; +import org.springframework.util.SerializationUtils; /** * @author Lucas Ward - * + * @author Mahmoud Ben Hassine + * */ public class ExecutionContextTests { @@ -124,18 +125,6 @@ public void testEquals() { assertTrue(tempContext.equals(context)); } - @Test - public void testSerializationCheck() { - // adding a non serializable object should cause an error. - try { - context.put("1", new Object()); - fail(); - } - catch (IllegalArgumentException ex) { - // expected - } - } - /** * Putting null value is equivalent to removing the entry for the given key. */ @@ -181,7 +170,7 @@ public void testCopyConstructor() throws Exception { } @Test - public void testCopyConstructorNullnNput() throws Exception { + public void testCopyConstructorNullInput() throws Exception { ExecutionContext context = new ExecutionContext((ExecutionContext) null); assertTrue(context.isEmpty()); } @@ -189,6 +178,7 @@ public void testCopyConstructorNullnNput() throws Exception { /** * Value object for testing serialization */ + @SuppressWarnings("serial") private static class TestSerializable implements Serializable { int value; @@ -203,15 +193,19 @@ public int hashCode() { @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + if (getClass() != obj.getClass()) { return false; + } TestSerializable other = (TestSerializable) obj; - if (value != other.value) + if (value != other.value) { return false; + } return true; } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ItemReaderTests.java index 1c5faf4c71..670ea0b937 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ItemReaderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,12 +19,14 @@ import static org.junit.Assert.assertEquals; import org.junit.Test; +import org.springframework.lang.Nullable; public class ItemReaderTests { ItemReader provider = new ItemReader() { - @Override + @Nullable + @Override public String read() { return "foo"; } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ItemRecoveryHandlerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ItemRecoveryHandlerTests.java index 706a197ddb..8e1f0f2cbb 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ItemRecoveryHandlerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ItemRecoveryHandlerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ItemStreamExceptionTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ItemStreamExceptionTests.java index 4c019e0a47..92220cf0f9 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ItemStreamExceptionTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/ItemStreamExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/UnexpectedInputExceptionTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/UnexpectedInputExceptionTests.java index 981ea38dfd..75c32f6768 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/UnexpectedInputExceptionTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/UnexpectedInputExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/AbstractDelegatorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/AbstractDelegatorTests.java index 3951989cf9..44ee06b121 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/AbstractDelegatorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/AbstractDelegatorTests.java @@ -1,20 +1,37 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.adapter; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.fail; - import java.util.ArrayList; import java.util.List; import org.junit.Before; +import org.junit.Ignore; import org.junit.Test; + import org.springframework.batch.item.adapter.AbstractMethodInvokingDelegator.InvocationTargetThrowableWrapper; -import org.springframework.util.Assert; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; /** * Tests for {@link AbstractMethodInvokingDelegator} - * + * * @author Robert Kasanicky */ public class AbstractDelegatorTests { @@ -59,7 +76,7 @@ public void testDelegationWithArgument() throws Exception { // using the arguments setter should work equally well foo.setName("foo"); - Assert.state(!foo.getName().equals(NEW_FOO_NAME)); + assertTrue(!foo.getName().equals(NEW_FOO_NAME)); delegator.setArguments(new Object[] { NEW_FOO_NAME }); delegator.afterPropertiesSet(); delegator.invokeDelegateMethod(); @@ -84,6 +101,7 @@ public void testDelegationWithCheckedNullArgument() throws Exception { * results */ @Test + @Ignore //FIXME public void testDelegationWithMultipleArguments() throws Exception { FooService fooService = new FooService(); delegator.setTargetObject(fooService); @@ -94,7 +112,7 @@ public void testDelegationWithMultipleArguments() throws Exception { final int FOO_VALUE = 12345; delegator.invokeDelegateMethodWithArguments(new Object[] { FOO_NAME, FOO_VALUE }); - Foo foo = (Foo) fooService.getProcessedFooNameValuePairs().get(0); + Foo foo = fooService.getProcessedFooNameValuePairs().get(0); assertEquals(FOO_NAME, foo.getName()); assertEquals(FOO_VALUE, foo.getValue()); } @@ -269,7 +287,7 @@ public void failUgly() throws Throwable { private static class FooService { - private List processedFooNameValuePairs = new ArrayList(); + private List processedFooNameValuePairs = new ArrayList<>(); @SuppressWarnings("unused") public void processNameValuePair(String name, int value) { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/HippyMethodInvokerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/HippyMethodInvokerTests.java index ccdadd2f69..0f62c039fa 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/HippyMethodInvokerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/HippyMethodInvokerTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.adapter; import static org.junit.Assert.assertEquals; @@ -75,7 +90,7 @@ public Class foo(Set arg) { } } - TreeSet arg = new TreeSet(); + TreeSet arg = new TreeSet<>(); OverloadingPojo target = new OverloadingPojo(); assertEquals(target.foo(arg), Set.class); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/ItemProcessorAdapterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/ItemProcessorAdapterTests.java index be3379c228..dfa1d40903 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/ItemProcessorAdapterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/ItemProcessorAdapterTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.adapter; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/ItemReaderAdapterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/ItemReaderAdapterTests.java index b512d6e5a2..983277ff21 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/ItemReaderAdapterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/ItemReaderAdapterTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.adapter; import static org.junit.Assert.*; @@ -33,7 +48,7 @@ public class ItemReaderAdapterTests { */ @Test public void testNext() throws Exception { - List returnedItems = new ArrayList(); + List returnedItems = new ArrayList<>(); Object item; while ((item = provider.read()) != null) { returnedItems.add(item); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/ItemWriterAdapterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/ItemWriterAdapterTests.java index ccc68398e4..107d52b0dc 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/ItemWriterAdapterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/ItemWriterAdapterTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.adapter; import static org.junit.Assert.assertEquals; @@ -37,7 +52,7 @@ public class ItemWriterAdapterTests { @Test public void testProcess() throws Exception { Foo foo; - List foos = new ArrayList(); + List foos = new ArrayList<>(); while ((foo = fooService.generateFoo()) != null) { foos.add(foo); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/PropertyExtractingDelegatingItemProccessorIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/PropertyExtractingDelegatingItemProccessorIntegrationTests.java deleted file mode 100644 index 8551ef0185..0000000000 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/PropertyExtractingDelegatingItemProccessorIntegrationTests.java +++ /dev/null @@ -1,56 +0,0 @@ -package org.springframework.batch.item.adapter; - -import static org.junit.Assert.*; -import org.junit.runner.RunWith; -import org.junit.Test; - -import java.util.Collections; -import java.util.List; - -import org.springframework.batch.item.sample.Foo; -import org.springframework.batch.item.sample.FooService; -import org.springframework.test.context.ContextConfiguration; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; -import org.springframework.beans.factory.annotation.Autowired; - -/** - * Tests for {@link PropertyExtractingDelegatingItemWriter} - * - * @author Robert Kasanicky - */ -@RunWith(SpringJUnit4ClassRunner.class) -@ContextConfiguration(locations = "pe-delegating-item-writer.xml") -public class PropertyExtractingDelegatingItemProccessorIntegrationTests { - - @Autowired - private PropertyExtractingDelegatingItemWriter processor; - - @Autowired - private FooService fooService; - - /* - * Regular usage scenario - input object should be passed to the service the injected invoker points to. - */ - @Test - public void testProcess() throws Exception { - Foo foo; - while ((foo = fooService.generateFoo()) != null) { - processor.write(Collections.singletonList(foo)); - } - - List input = fooService.getGeneratedFoos(); - List processed = fooService.getProcessedFooNameValuePairs(); - assertEquals(input.size(), processed.size()); - assertFalse(fooService.getProcessedFooNameValuePairs().isEmpty()); - - for (int i = 0; i < input.size(); i++) { - Foo inputFoo = input.get(i); - Foo outputFoo = processed.get(i); - assertEquals(inputFoo.getName(), outputFoo.getName()); - assertEquals(inputFoo.getValue(), outputFoo.getValue()); - assertEquals(0, outputFoo.getId()); - } - - } - -} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/PropertyExtractingDelegatingItemProcessorIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/PropertyExtractingDelegatingItemProcessorIntegrationTests.java new file mode 100644 index 0000000000..10f60daf49 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/adapter/PropertyExtractingDelegatingItemProcessorIntegrationTests.java @@ -0,0 +1,72 @@ +/* + * Copyright 2008-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.adapter; + +import static org.junit.Assert.*; +import org.junit.runner.RunWith; +import org.junit.Test; + +import java.util.Collections; +import java.util.List; + +import org.springframework.batch.item.sample.Foo; +import org.springframework.batch.item.sample.FooService; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.beans.factory.annotation.Autowired; + +/** + * Tests for {@link PropertyExtractingDelegatingItemWriter} + * + * @author Robert Kasanicky + * @author Mahmoud Ben Hassine + */ +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(locations = "pe-delegating-item-writer.xml") +public class PropertyExtractingDelegatingItemProcessorIntegrationTests { + + @Autowired + private PropertyExtractingDelegatingItemWriter processor; + + @Autowired + private FooService fooService; + + /* + * Regular usage scenario - input object should be passed to the service the injected invoker points to. + */ + @Test + public void testProcess() throws Exception { + Foo foo; + while ((foo = fooService.generateFoo()) != null) { + processor.write(Collections.singletonList(foo)); + } + + List input = fooService.getGeneratedFoos(); + List processed = fooService.getProcessedFooNameValuePairs(); + assertEquals(input.size(), processed.size()); + assertFalse(fooService.getProcessedFooNameValuePairs().isEmpty()); + + for (int i = 0; i < input.size(); i++) { + Foo inputFoo = input.get(i); + Foo outputFoo = processed.get(i); + assertEquals(inputFoo.getName(), outputFoo.getName()); + assertEquals(inputFoo.getValue(), outputFoo.getValue()); + assertEquals(0, outputFoo.getId()); + } + + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/AmqpItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/AmqpItemReaderTests.java index fbd493772c..819ee26825 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/AmqpItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/AmqpItemReaderTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -46,7 +46,7 @@ public void testNoItemType() { final AmqpTemplate amqpTemplate = mock(AmqpTemplate.class); when(amqpTemplate.receiveAndConvert()).thenReturn("foo"); - final AmqpItemReader amqpItemReader = new AmqpItemReader(amqpTemplate); + final AmqpItemReader amqpItemReader = new AmqpItemReader<>(amqpTemplate); assertEquals("foo", amqpItemReader.read()); } @@ -55,7 +55,7 @@ public void testNonMessageItemType() { final AmqpTemplate amqpTemplate = mock(AmqpTemplate.class); when(amqpTemplate.receiveAndConvert()).thenReturn("foo"); - final AmqpItemReader amqpItemReader = new AmqpItemReader(amqpTemplate); + final AmqpItemReader amqpItemReader = new AmqpItemReader<>(amqpTemplate); amqpItemReader.setItemType(String.class); assertEquals("foo", amqpItemReader.read()); @@ -69,7 +69,7 @@ public void testMessageItemType() { when(amqpTemplate.receive()).thenReturn(message); - final AmqpItemReader amqpItemReader = new AmqpItemReader(amqpTemplate); + final AmqpItemReader amqpItemReader = new AmqpItemReader<>(amqpTemplate); amqpItemReader.setItemType(Message.class); assertEquals(message, amqpItemReader.read()); @@ -82,7 +82,7 @@ public void testTypeMismatch() { when(amqpTemplate.receiveAndConvert()).thenReturn("foo"); - final AmqpItemReader amqpItemReader = new AmqpItemReader(amqpTemplate); + final AmqpItemReader amqpItemReader = new AmqpItemReader<>(amqpTemplate); amqpItemReader.setItemType(Integer.class); try { @@ -98,7 +98,7 @@ public void testTypeMismatch() { public void testNullItemType() { final AmqpTemplate amqpTemplate = mock(AmqpTemplate.class); - final AmqpItemReader amqpItemReader = new AmqpItemReader(amqpTemplate); + final AmqpItemReader amqpItemReader = new AmqpItemReader<>(amqpTemplate); amqpItemReader.setItemType(null); } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/AmqpItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/AmqpItemWriterTests.java index f80157e274..d5cd610785 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/AmqpItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/AmqpItemWriterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -45,7 +45,7 @@ public void voidTestWrite() throws Exception { amqpTemplate.convertAndSend("bar"); - AmqpItemWriter amqpItemWriter = new AmqpItemWriter(amqpTemplate); + AmqpItemWriter amqpItemWriter = new AmqpItemWriter<>(amqpTemplate); amqpItemWriter.write(Arrays.asList("foo", "bar")); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/builder/AmqpItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/builder/AmqpItemReaderBuilderTests.java new file mode 100644 index 0000000000..b887ccbda0 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/builder/AmqpItemReaderBuilderTests.java @@ -0,0 +1,88 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.amqp.builder; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.amqp.core.AmqpTemplate; +import org.springframework.amqp.core.Message; +import org.springframework.batch.item.amqp.AmqpItemReader; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +/** + * @author Glenn Renfro + */ +public class AmqpItemReaderBuilderTests { + + @Mock + AmqpTemplate amqpTemplate; + + @Before + public void setup() { + MockitoAnnotations.initMocks(this); + } + + @Test + public void testNoItemType() { + when(this.amqpTemplate.receiveAndConvert()).thenReturn("foo"); + + final AmqpItemReader amqpItemReader = new AmqpItemReaderBuilder() + .amqpTemplate(this.amqpTemplate).build(); + assertEquals("foo", amqpItemReader.read()); + } + + @Test + public void testNonMessageItemType() { + when(this.amqpTemplate.receiveAndConvert()).thenReturn("foo"); + + final AmqpItemReader amqpItemReader = new AmqpItemReaderBuilder() + .amqpTemplate(this.amqpTemplate).itemType(String.class).build(); + + assertEquals("foo", amqpItemReader.read()); + } + + @Test + public void testMessageItemType() { + final Message message = mock(Message.class); + + when(this.amqpTemplate.receive()).thenReturn(message); + + final AmqpItemReader amqpItemReader = new AmqpItemReaderBuilder() + .amqpTemplate(this.amqpTemplate).itemType(Message.class).build(); + + assertEquals(message, amqpItemReader.read()); + } + + @Test + public void testNullAmqpTemplate() { + try { + new AmqpItemReaderBuilder().build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "amqpTemplate is required.", iae.getMessage()); + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/builder/AmqpItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/builder/AmqpItemWriterBuilderTests.java new file mode 100644 index 0000000000..050b51f36f --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/amqp/builder/AmqpItemWriterBuilderTests.java @@ -0,0 +1,59 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.amqp.builder; + +import java.util.Arrays; + +import org.junit.Test; + +import org.springframework.amqp.core.AmqpTemplate; +import org.springframework.amqp.core.Message; +import org.springframework.batch.item.amqp.AmqpItemWriter; + +import static org.aspectj.bridge.MessageUtil.fail; +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; + +/** + * @author Glenn Renfro + */ +public class AmqpItemWriterBuilderTests { + + @Test + public void testNullAmqpTemplate() { + try { + new AmqpItemWriterBuilder().build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "amqpTemplate is required.", iae.getMessage()); + } + } + + @Test + public void voidTestWrite() throws Exception { + AmqpTemplate amqpTemplate = mock(AmqpTemplate.class); + + AmqpItemWriter amqpItemWriter = + new AmqpItemWriterBuilder().amqpTemplate(amqpTemplate).build(); + amqpItemWriter.write(Arrays.asList("foo", "bar")); + verify(amqpTemplate).convertAndSend("foo"); + verify(amqpTemplate).convertAndSend("bar"); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/AvroItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/AvroItemReaderTests.java new file mode 100644 index 0000000000..2166b3d74a --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/AvroItemReaderTests.java @@ -0,0 +1,81 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.avro; + +import org.apache.avro.generic.GenericRecord; +import org.junit.Test; + +import org.springframework.batch.item.avro.example.User; +import org.springframework.batch.item.avro.support.AvroItemReaderTestSupport; +import org.springframework.core.io.ClassPathResource; + +/** + * @author David Turanski + */ +public class AvroItemReaderTests extends AvroItemReaderTestSupport { + + @Test + public void readGenericRecordsUsingResources() throws Exception { + + AvroItemReader itemReader = new AvroItemReader<>(dataResource, schemaResource); + itemReader.setName(itemReader.getClass().getSimpleName()); + itemReader.setEmbeddedSchema(false); + + verify(itemReader, genericAvroGeneratedUsers()); + } + + + @Test + public void readSpecificUsers() throws Exception { + + AvroItemReader itemReader = new AvroItemReader<>(dataResource, User.class); + itemReader.setEmbeddedSchema(false); + itemReader.setName(itemReader.getClass().getSimpleName()); + + verify(itemReader, avroGeneratedUsers()); + } + + @Test + public void readSpecificUsersWithEmbeddedSchema() throws Exception { + + AvroItemReader itemReader = new AvroItemReader<>(dataResourceWithSchema, User.class); + itemReader.setEmbeddedSchema(true); + itemReader.setName(itemReader.getClass().getSimpleName()); + + verify(itemReader, avroGeneratedUsers()); + } + + @Test + public void readPojosWithNoEmbeddedSchema() throws Exception { + + AvroItemReader itemReader = new AvroItemReader<>(plainOldUserDataResource, PlainOldUser.class); + itemReader.setEmbeddedSchema(false); + itemReader.setName(itemReader.getClass().getSimpleName()); + + verify(itemReader, plainOldUsers()); + } + + @Test(expected = IllegalStateException.class) + public void dataResourceDoesNotExist() { + new AvroItemReader(new ClassPathResource("doesnotexist"), schemaResource); + } + + @Test(expected = IllegalStateException.class) + public void schemaResourceDoesNotExist() { + new AvroItemReader(dataResource, new ClassPathResource("doesnotexist")); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/AvroItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/AvroItemWriterTests.java new file mode 100644 index 0000000000..d1f672f15d --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/AvroItemWriterTests.java @@ -0,0 +1,98 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.avro; + +import java.io.ByteArrayOutputStream; + +import org.apache.avro.generic.GenericRecord; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.avro.example.User; +import org.springframework.batch.item.avro.support.AvroItemWriterTestSupport; +import org.springframework.core.io.WritableResource; + +/** + * @author David Turanski + * @author Mahmoud Ben Hassine + */ +public class AvroItemWriterTests extends AvroItemWriterTestSupport { + + private ByteArrayOutputStream outputStream = new ByteArrayOutputStream(2048); + + private WritableResource output = new OutputStreamResource(outputStream); + + @Test + public void itemWriterForAvroGeneratedClass() throws Exception { + + AvroItemWriter avroItemWriter = new AvroItemWriter<>(this.output, this.schemaResource, User.class); + avroItemWriter.open(new ExecutionContext()); + avroItemWriter.write(this.avroGeneratedUsers()); + avroItemWriter.close(); + + verifyRecordsWithEmbeddedHeader(this.outputStream.toByteArray(), this.avroGeneratedUsers(), User.class); + } + + @Test + public void itemWriterForGenericRecords() throws Exception { + + AvroItemWriter avroItemWriter = + new AvroItemWriter<>(this.output, this.plainOldUserSchemaResource, GenericRecord.class); + + avroItemWriter.open(new ExecutionContext()); + avroItemWriter.write(this.genericPlainOldUsers()); + avroItemWriter.close(); + + verifyRecordsWithEmbeddedHeader(this.outputStream.toByteArray(), this.genericPlainOldUsers(), GenericRecord.class); + + } + + @Test + public void itemWriterForPojos() throws Exception { + + AvroItemWriter avroItemWriter = new AvroItemWriter<>(this.output, this.plainOldUserSchemaResource, PlainOldUser.class); + avroItemWriter.open(new ExecutionContext()); + avroItemWriter.write(this.plainOldUsers()); + avroItemWriter.close(); + + verifyRecordsWithEmbeddedHeader(this.outputStream.toByteArray(), this.plainOldUsers(), PlainOldUser.class); + + } + + @Test + public void itemWriterWithNoEmbeddedHeaders() throws Exception { + + AvroItemWriter avroItemWriter = new AvroItemWriter<>(this.output, PlainOldUser.class); + avroItemWriter.open(new ExecutionContext()); + avroItemWriter.write(this.plainOldUsers()); + avroItemWriter.close(); + + verifyRecords(this.outputStream.toByteArray(), this.plainOldUsers(), PlainOldUser.class, false); + + } + + @Test(expected = IllegalArgumentException.class) + public void shouldFailWitNoOutput() { + new AvroItemWriter<>(null, this.schemaResource, User.class).open(new ExecutionContext()); + + } + + @Test(expected = IllegalArgumentException.class) + public void shouldFailWitNoType() { + new AvroItemWriter<>(this.output, this.schemaResource, null).open(new ExecutionContext()); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/builder/AvroItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/builder/AvroItemReaderBuilderTests.java new file mode 100644 index 0000000000..034baa975b --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/builder/AvroItemReaderBuilderTests.java @@ -0,0 +1,85 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.avro.builder; + +import org.apache.avro.generic.GenericRecord; +import org.junit.Test; + +import org.springframework.batch.item.avro.AvroItemReader; +import org.springframework.batch.item.avro.example.User; +import org.springframework.batch.item.avro.support.AvroItemReaderTestSupport; + +/** + * @author David Turanski + */ +public class AvroItemReaderBuilderTests extends AvroItemReaderTestSupport { + + @Test + public void itemReaderWithSchemaResource() throws Exception { + + AvroItemReader avroItemReader = new AvroItemReaderBuilder().resource(dataResource) + .embeddedSchema(false).schema(schemaResource).build(); + + verify(avroItemReader, genericAvroGeneratedUsers()); + } + + @Test + public void itemReaderWithGeneratedData() throws Exception { + AvroItemReader avroItemReader = new AvroItemReaderBuilder() + .resource(dataResourceWithSchema).schema(schemaResource).build(); + verify(avroItemReader, genericAvroGeneratedUsers()); + } + + @Test + public void itemReaderWithSchemaString() throws Exception { + AvroItemReader avroItemReader = new AvroItemReaderBuilder() + .schema(schemaString(schemaResource)).resource(dataResourceWithSchema).build(); + + verify(avroItemReader, genericAvroGeneratedUsers()); + } + + @Test + public void itemReaderWithEmbeddedHeader() throws Exception { + AvroItemReader avroItemReader = new AvroItemReaderBuilder().resource(dataResourceWithSchema) + .type(User.class).build(); + verify(avroItemReader, avroGeneratedUsers()); + } + + @Test + public void itemReaderForSpecificType() throws Exception { + AvroItemReader avroItemReader = new AvroItemReaderBuilder().type(User.class) + .resource(dataResourceWithSchema).build(); + verify(avroItemReader, avroGeneratedUsers()); + } + + @Test(expected = IllegalArgumentException.class) + public void itemReaderWithNoSchemaStringShouldFail() { + new AvroItemReaderBuilder().schema("").resource(dataResource).build(); + + } + + @Test(expected = IllegalArgumentException.class) + public void itemReaderWithPartialConfigurationShouldFail() { + new AvroItemReaderBuilder().resource(dataResource).build(); + } + + @Test(expected = IllegalArgumentException.class) + public void itemReaderWithNoInputsShouldFail() { + new AvroItemReaderBuilder().schema(schemaResource).build(); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/builder/AvroItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/builder/AvroItemWriterBuilderTests.java new file mode 100644 index 0000000000..f158a2b433 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/builder/AvroItemWriterBuilderTests.java @@ -0,0 +1,124 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.avro.builder; + +import java.io.ByteArrayOutputStream; + +import org.apache.avro.generic.GenericRecord; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.avro.AvroItemWriter; +import org.springframework.batch.item.avro.example.User; +import org.springframework.batch.item.avro.support.AvroItemWriterTestSupport; +import org.springframework.core.io.WritableResource; + +/** + * @author David Turanski + */ +public class AvroItemWriterBuilderTests extends AvroItemWriterTestSupport { + + private ByteArrayOutputStream outputStream = new ByteArrayOutputStream(2048); + private WritableResource output = new OutputStreamResource(outputStream); + + @Test + public void itemWriterForAvroGeneratedClass() throws Exception { + + AvroItemWriter avroItemWriter = new AvroItemWriterBuilder() + .resource(output) + .schema(schemaResource) + .type(User.class) + .build(); + + avroItemWriter.open(new ExecutionContext()); + avroItemWriter.write(this.avroGeneratedUsers()); + avroItemWriter.close(); + + verifyRecordsWithEmbeddedHeader(outputStream.toByteArray(), this.avroGeneratedUsers(), User.class); + } + + + @Test + public void itemWriterForGenericRecords() throws Exception { + + AvroItemWriter avroItemWriter = new AvroItemWriterBuilder() + .type(GenericRecord.class) + .schema(plainOldUserSchemaResource) + .resource(output) + .build(); + + avroItemWriter.open(new ExecutionContext()); + avroItemWriter.write(this.genericPlainOldUsers()); + avroItemWriter.close(); + + verifyRecordsWithEmbeddedHeader(outputStream.toByteArray(), this.genericPlainOldUsers(), GenericRecord.class); + + } + + @Test + public void itemWriterForPojos() throws Exception { + + AvroItemWriter avroItemWriter = new AvroItemWriterBuilder() + .resource(output) + .schema(plainOldUserSchemaResource) + .type(PlainOldUser.class) + .build(); + + avroItemWriter.open(new ExecutionContext()); + avroItemWriter.write(this.plainOldUsers()); + avroItemWriter.close(); + + verifyRecordsWithEmbeddedHeader(outputStream.toByteArray(), this.plainOldUsers(), PlainOldUser.class); + + } + + @Test + public void itemWriterWithNoEmbeddedSchema() throws Exception { + + AvroItemWriter avroItemWriter = new AvroItemWriterBuilder() + .resource(output) + .type(PlainOldUser.class) + .build(); + avroItemWriter.open(new ExecutionContext()); + avroItemWriter.write(this.plainOldUsers()); + avroItemWriter.close(); + + verifyRecords(outputStream.toByteArray(), this.plainOldUsers(), PlainOldUser.class, false); + + } + + + @Test(expected = IllegalArgumentException.class) + public void shouldFailWitNoOutput() { + + new AvroItemWriterBuilder() + .type(GenericRecord.class) + .build(); + + } + + @Test(expected = IllegalArgumentException.class) + public void shouldFailWitNoType() { + + new AvroItemWriterBuilder() + .resource(output) + .schema(schemaResource) + .build(); + + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/example/AvroTestUtils.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/example/AvroTestUtils.java new file mode 100644 index 0000000000..be78295e8c --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/example/AvroTestUtils.java @@ -0,0 +1,80 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.avro.example; + +import java.io.File; +import java.io.FileOutputStream; +import org.apache.avro.Schema; +import org.apache.avro.file.DataFileWriter; +import org.apache.avro.io.DatumWriter; +import org.apache.avro.io.Encoder; +import org.apache.avro.io.EncoderFactory; +import org.apache.avro.specific.SpecificDatumWriter; +import org.springframework.core.io.ClassPathResource; +import org.springframework.core.io.Resource; + +/** + * Used to create test data. See http://avro.apache.org/docs/1.9.0/gettingstartedjava.html + * + * @author David Turanski + * @author Mahmoud Ben Hassine + */ +class AvroTestUtils { + + public static void main(String... args) { + try { + createTestDataWithNoEmbeddedSchema(); + createTestData(); + } catch (Exception e) { + e.printStackTrace(); + } + } + + static void createTestDataWithNoEmbeddedSchema() throws Exception { + + DatumWriter userDatumWriter = new SpecificDatumWriter<>(User.class); + + FileOutputStream fileOutputStream = new FileOutputStream("user-data-no-schema.avro"); + + Encoder encoder = EncoderFactory.get().binaryEncoder(fileOutputStream,null); + userDatumWriter.write(new User("David", 20, "blue"), encoder); + userDatumWriter.write(new User("Sue", 4, "red"), encoder); + userDatumWriter.write(new User("Alana", 13, "yellow"), encoder); + userDatumWriter.write(new User("Joe", 1, "pink"), encoder); + + encoder.flush(); + fileOutputStream.flush(); + fileOutputStream.close(); + } + + + static void createTestData() throws Exception { + + Resource schemaResource = new ClassPathResource("org/springframework/batch/item/avro/user-schema.json"); + + DatumWriter userDatumWriter = new SpecificDatumWriter<>(User.class); + DataFileWriter dataFileWriter = new DataFileWriter<>(userDatumWriter); + dataFileWriter.create(new Schema.Parser().parse(schemaResource.getInputStream()), new File("users.avro")); + dataFileWriter.append(new User("David", 20, "blue")); + dataFileWriter.append(new User("Sue", 4, "red")); + dataFileWriter.append(new User("Alana", 13, "yellow")); + dataFileWriter.append(new User("Joe", 1, "pink")); + dataFileWriter.close(); + } + + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/example/User.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/example/User.java new file mode 100644 index 0000000000..c4b13454c2 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/example/User.java @@ -0,0 +1,518 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Autogenerated by Avro + * + * DO NOT EDIT DIRECTLY + */ +package org.springframework.batch.item.avro.example; + +import org.apache.avro.generic.GenericArray; +import org.apache.avro.specific.SpecificData; +import org.apache.avro.util.Utf8; +import org.apache.avro.message.BinaryMessageEncoder; +import org.apache.avro.message.BinaryMessageDecoder; +import org.apache.avro.message.SchemaStore; + +@org.apache.avro.specific.AvroGenerated +public class User extends org.apache.avro.specific.SpecificRecordBase implements org.apache.avro.specific.SpecificRecord { + private static final long serialVersionUID = 1293362237195430714L; + public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"User\",\"namespace\":\"org.springframework.batch.item.avro.example\",\"fields\":[{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"favorite_number\",\"type\":[\"int\",\"null\"]},{\"name\":\"favorite_color\",\"type\":[\"string\",\"null\"]}]}"); + public static org.apache.avro.Schema getClassSchema() { return SCHEMA$; } + + private static SpecificData MODEL$ = new SpecificData(); + + private static final BinaryMessageEncoder ENCODER = + new BinaryMessageEncoder(MODEL$, SCHEMA$); + + private static final BinaryMessageDecoder DECODER = + new BinaryMessageDecoder(MODEL$, SCHEMA$); + + /** + * Return the BinaryMessageEncoder instance used by this class. + * @return the message encoder used by this class + */ + public static BinaryMessageEncoder getEncoder() { + return ENCODER; + } + + /** + * Return the BinaryMessageDecoder instance used by this class. + * @return the message decoder used by this class + */ + public static BinaryMessageDecoder getDecoder() { + return DECODER; + } + + /** + * Create a new BinaryMessageDecoder instance for this class that uses the specified {@link SchemaStore}. + * @param resolver a {@link SchemaStore} used to find schemas by fingerprint + * @return a BinaryMessageDecoder instance for this class backed by the given SchemaStore + */ + public static BinaryMessageDecoder createDecoder(SchemaStore resolver) { + return new BinaryMessageDecoder(MODEL$, SCHEMA$, resolver); + } + + /** + * Serializes this User to a ByteBuffer. + * @return a buffer holding the serialized data for this instance + * @throws java.io.IOException if this instance could not be serialized + */ + public java.nio.ByteBuffer toByteBuffer() throws java.io.IOException { + return ENCODER.encode(this); + } + + /** + * Deserializes a User from a ByteBuffer. + * @param b a byte buffer holding serialized data for an instance of this class + * @return a User instance decoded from the given buffer + * @throws java.io.IOException if the given bytes could not be deserialized into an instance of this class + */ + public static User fromByteBuffer( + java.nio.ByteBuffer b) throws java.io.IOException { + return DECODER.decode(b); + } + + private CharSequence name; + private Integer favorite_number; + private CharSequence favorite_color; + + /** + * Default constructor. Note that this does not initialize fields + * to their default values from the SCHEMA. If that is desired then + * one should use newBuilder(). + */ + public User() {} + + /** + * All-args constructor. + * @param name The new value for name + * @param favorite_number The new value for favorite_number + * @param favorite_color The new value for favorite_color + */ + public User(CharSequence name, Integer favorite_number, CharSequence favorite_color) { + this.name = name; + this.favorite_number = favorite_number; + this.favorite_color = favorite_color; + } + + public SpecificData getSpecificData() { return MODEL$; } + public org.apache.avro.Schema getSchema() { return SCHEMA$; } + // Used by DatumWriter. Applications should not call. + public Object get(int field$) { + switch (field$) { + case 0: return name; + case 1: return favorite_number; + case 2: return favorite_color; + default: throw new org.apache.avro.AvroRuntimeException("Bad index"); + } + } + + // Used by DatumReader. Applications should not call. + @SuppressWarnings(value="unchecked") + public void put(int field$, Object value$) { + switch (field$) { + case 0: name = (CharSequence)value$; break; + case 1: favorite_number = (Integer)value$; break; + case 2: favorite_color = (CharSequence)value$; break; + default: throw new org.apache.avro.AvroRuntimeException("Bad index"); + } + } + + /** + * Gets the value of the 'name' field. + * @return The value of the 'name' field. + */ + public CharSequence getName() { + return name; + } + + + /** + * Sets the value of the 'name' field. + * @param value the value to set. + */ + public void setName(CharSequence value) { + this.name = value; + } + + /** + * Gets the value of the 'favorite_number' field. + * @return The value of the 'favorite_number' field. + */ + public Integer getFavoriteNumber() { + return favorite_number; + } + + + /** + * Sets the value of the 'favorite_number' field. + * @param value the value to set. + */ + public void setFavoriteNumber(Integer value) { + this.favorite_number = value; + } + + /** + * Gets the value of the 'favorite_color' field. + * @return The value of the 'favorite_color' field. + */ + public CharSequence getFavoriteColor() { + return favorite_color; + } + + + /** + * Sets the value of the 'favorite_color' field. + * @param value the value to set. + */ + public void setFavoriteColor(CharSequence value) { + this.favorite_color = value; + } + + /** + * Creates a new User RecordBuilder. + * @return A new User RecordBuilder + */ + public static Builder newBuilder() { + return new Builder(); + } + + /** + * Creates a new User RecordBuilder by copying an existing Builder. + * @param other The existing builder to copy. + * @return A new User RecordBuilder + */ + public static Builder newBuilder(Builder other) { + if (other == null) { + return new Builder(); + } else { + return new Builder(other); + } + } + + /** + * Creates a new User RecordBuilder by copying an existing User instance. + * @param other The existing instance to copy. + * @return A new User RecordBuilder + */ + public static Builder newBuilder(User other) { + if (other == null) { + return new Builder(); + } else { + return new Builder(other); + } + } + + /** + * RecordBuilder for User instances. + */ + public static class Builder extends org.apache.avro.specific.SpecificRecordBuilderBase + implements org.apache.avro.data.RecordBuilder { + + private CharSequence name; + private Integer favorite_number; + private CharSequence favorite_color; + + /** Creates a new Builder */ + private Builder() { + super(SCHEMA$); + } + + /** + * Creates a Builder by copying an existing Builder. + * @param other The existing Builder to copy. + */ + private Builder(Builder other) { + super(other); + if (isValidValue(fields()[0], other.name)) { + this.name = data().deepCopy(fields()[0].schema(), other.name); + fieldSetFlags()[0] = other.fieldSetFlags()[0]; + } + if (isValidValue(fields()[1], other.favorite_number)) { + this.favorite_number = data().deepCopy(fields()[1].schema(), other.favorite_number); + fieldSetFlags()[1] = other.fieldSetFlags()[1]; + } + if (isValidValue(fields()[2], other.favorite_color)) { + this.favorite_color = data().deepCopy(fields()[2].schema(), other.favorite_color); + fieldSetFlags()[2] = other.fieldSetFlags()[2]; + } + } + + /** + * Creates a Builder by copying an existing User instance + * @param other The existing instance to copy. + */ + private Builder(User other) { + super(SCHEMA$); + if (isValidValue(fields()[0], other.name)) { + this.name = data().deepCopy(fields()[0].schema(), other.name); + fieldSetFlags()[0] = true; + } + if (isValidValue(fields()[1], other.favorite_number)) { + this.favorite_number = data().deepCopy(fields()[1].schema(), other.favorite_number); + fieldSetFlags()[1] = true; + } + if (isValidValue(fields()[2], other.favorite_color)) { + this.favorite_color = data().deepCopy(fields()[2].schema(), other.favorite_color); + fieldSetFlags()[2] = true; + } + } + + /** + * Gets the value of the 'name' field. + * @return The value. + */ + public CharSequence getName() { + return name; + } + + + /** + * Sets the value of the 'name' field. + * @param value The value of 'name'. + * @return This builder. + */ + public Builder setName(CharSequence value) { + validate(fields()[0], value); + this.name = value; + fieldSetFlags()[0] = true; + return this; + } + + /** + * Checks whether the 'name' field has been set. + * @return True if the 'name' field has been set, false otherwise. + */ + public boolean hasName() { + return fieldSetFlags()[0]; + } + + + /** + * Clears the value of the 'name' field. + * @return This builder. + */ + public Builder clearName() { + name = null; + fieldSetFlags()[0] = false; + return this; + } + + /** + * Gets the value of the 'favorite_number' field. + * @return The value. + */ + public Integer getFavoriteNumber() { + return favorite_number; + } + + + /** + * Sets the value of the 'favorite_number' field. + * @param value The value of 'favorite_number'. + * @return This builder. + */ + public Builder setFavoriteNumber(Integer value) { + validate(fields()[1], value); + this.favorite_number = value; + fieldSetFlags()[1] = true; + return this; + } + + /** + * Checks whether the 'favorite_number' field has been set. + * @return True if the 'favorite_number' field has been set, false otherwise. + */ + public boolean hasFavoriteNumber() { + return fieldSetFlags()[1]; + } + + + /** + * Clears the value of the 'favorite_number' field. + * @return This builder. + */ + public Builder clearFavoriteNumber() { + favorite_number = null; + fieldSetFlags()[1] = false; + return this; + } + + /** + * Gets the value of the 'favorite_color' field. + * @return The value. + */ + public CharSequence getFavoriteColor() { + return favorite_color; + } + + + /** + * Sets the value of the 'favorite_color' field. + * @param value The value of 'favorite_color'. + * @return This builder. + */ + public Builder setFavoriteColor(CharSequence value) { + validate(fields()[2], value); + this.favorite_color = value; + fieldSetFlags()[2] = true; + return this; + } + + /** + * Checks whether the 'favorite_color' field has been set. + * @return True if the 'favorite_color' field has been set, false otherwise. + */ + public boolean hasFavoriteColor() { + return fieldSetFlags()[2]; + } + + + /** + * Clears the value of the 'favorite_color' field. + * @return This builder. + */ + public Builder clearFavoriteColor() { + favorite_color = null; + fieldSetFlags()[2] = false; + return this; + } + + @Override + @SuppressWarnings("unchecked") + public User build() { + try { + User record = new User(); + record.name = fieldSetFlags()[0] ? this.name : (CharSequence) defaultValue(fields()[0]); + record.favorite_number = fieldSetFlags()[1] ? this.favorite_number : (Integer) defaultValue(fields()[1]); + record.favorite_color = fieldSetFlags()[2] ? this.favorite_color : (CharSequence) defaultValue(fields()[2]); + return record; + } catch (org.apache.avro.AvroMissingFieldException e) { + throw e; + } catch (Exception e) { + throw new org.apache.avro.AvroRuntimeException(e); + } + } + } + + @SuppressWarnings("unchecked") + private static final org.apache.avro.io.DatumWriter + WRITER$ = (org.apache.avro.io.DatumWriter)MODEL$.createDatumWriter(SCHEMA$); + + @Override public void writeExternal(java.io.ObjectOutput out) + throws java.io.IOException { + WRITER$.write(this, SpecificData.getEncoder(out)); + } + + @SuppressWarnings("unchecked") + private static final org.apache.avro.io.DatumReader + READER$ = (org.apache.avro.io.DatumReader)MODEL$.createDatumReader(SCHEMA$); + + @Override public void readExternal(java.io.ObjectInput in) + throws java.io.IOException { + READER$.read(this, SpecificData.getDecoder(in)); + } + + @Override protected boolean hasCustomCoders() { return true; } + + @Override public void customEncode(org.apache.avro.io.Encoder out) + throws java.io.IOException + { + out.writeString(this.name); + + if (this.favorite_number == null) { + out.writeIndex(1); + out.writeNull(); + } else { + out.writeIndex(0); + out.writeInt(this.favorite_number); + } + + if (this.favorite_color == null) { + out.writeIndex(1); + out.writeNull(); + } else { + out.writeIndex(0); + out.writeString(this.favorite_color); + } + + } + + @Override public void customDecode(org.apache.avro.io.ResolvingDecoder in) + throws java.io.IOException + { + org.apache.avro.Schema.Field[] fieldOrder = in.readFieldOrderIfDiff(); + if (fieldOrder == null) { + this.name = in.readString(this.name instanceof Utf8 ? (Utf8)this.name : null); + + if (in.readIndex() != 0) { + in.readNull(); + this.favorite_number = null; + } else { + this.favorite_number = in.readInt(); + } + + if (in.readIndex() != 0) { + in.readNull(); + this.favorite_color = null; + } else { + this.favorite_color = in.readString(this.favorite_color instanceof Utf8 ? (Utf8)this.favorite_color : null); + } + + } else { + for (int i = 0; i < 3; i++) { + switch (fieldOrder[i].pos()) { + case 0: + this.name = in.readString(this.name instanceof Utf8 ? (Utf8)this.name : null); + break; + + case 1: + if (in.readIndex() != 0) { + in.readNull(); + this.favorite_number = null; + } else { + this.favorite_number = in.readInt(); + } + break; + + case 2: + if (in.readIndex() != 0) { + in.readNull(); + this.favorite_color = null; + } else { + this.favorite_color = in.readString(this.favorite_color instanceof Utf8 ? (Utf8)this.favorite_color : null); + } + break; + + default: + throw new java.io.IOException("Corrupt ResolvingDecoder."); + } + } + } + } +} + + + + + + + + + + diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/support/AvroItemReaderTestSupport.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/support/AvroItemReaderTestSupport.java new file mode 100644 index 0000000000..62bb321ad2 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/support/AvroItemReaderTestSupport.java @@ -0,0 +1,47 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.avro.support; + +import java.util.ArrayList; +import java.util.List; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.avro.AvroItemReader; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * @author David Turanski + */ +public abstract class AvroItemReaderTestSupport extends AvroTestFixtures { + + protected void verify(AvroItemReader avroItemReader, List actual) throws Exception { + + avroItemReader.open(new ExecutionContext()); + List users = new ArrayList<>(); + + T user; + while ((user = avroItemReader.read()) != null) { + users.add(user); + } + + assertThat(users).hasSize(4); + assertThat(users).containsExactlyInAnyOrder(actual.get(0), actual.get(1), actual.get(2), actual.get(3)); + + avroItemReader.close(); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/support/AvroItemWriterTestSupport.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/support/AvroItemWriterTestSupport.java new file mode 100644 index 0000000000..fb9cf2ffcd --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/support/AvroItemWriterTestSupport.java @@ -0,0 +1,137 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.avro.support; + +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.net.URI; +import java.net.URL; +import java.util.ArrayList; +import java.util.List; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.avro.AvroItemReader; +import org.springframework.batch.item.avro.builder.AvroItemReaderBuilder; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.Resource; +import org.springframework.core.io.WritableResource; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * @author David Turanski + */ +public abstract class AvroItemWriterTestSupport extends AvroTestFixtures { + + /* + * This item reader configured for Specific Avro types. + */ + protected void verifyRecords(byte[] bytes, List actual, Class clazz, boolean embeddedSchema) throws Exception { + doVerify(bytes, clazz, actual, embeddedSchema); + } + + protected void verifyRecordsWithEmbeddedHeader(byte[] bytes, List actual, Class clazz) throws Exception { + doVerify(bytes, clazz, actual, true); + } + + + private void doVerify(byte[] bytes, Class clazz, List actual, boolean embeddedSchema) throws Exception { + AvroItemReader avroItemReader = new AvroItemReaderBuilder() + .type(clazz) + .resource(new ByteArrayResource(bytes)) + .embeddedSchema(embeddedSchema) + .build(); + + avroItemReader.open(new ExecutionContext()); + + List records = new ArrayList<>(); + T record; + while ((record = avroItemReader.read()) != null) { + records.add(record); + } + assertThat(records).hasSize(4); + assertThat(records).containsExactlyInAnyOrder(actual.get(0), actual.get(1), actual.get(2), actual.get(3)); + } + + + protected static class OutputStreamResource implements WritableResource { + + final private OutputStream outputStream; + + public OutputStreamResource(OutputStream outputStream) { + this.outputStream = outputStream; + } + + @Override + public OutputStream getOutputStream() throws IOException { + return this.outputStream; + } + + @Override + public boolean exists() { + return true; + } + + @Override + public URL getURL() throws IOException { + return null; + } + + @Override + public URI getURI() throws IOException { + return null; + } + + @Override + public File getFile() throws IOException { + return null; + } + + @Override + public long contentLength() throws IOException { + return 0; + } + + @Override + public long lastModified() throws IOException { + return 0; + } + + @Override + public Resource createRelative(String relativePath) throws IOException { + return null; + } + + @Override + public String getFilename() { + return null; + } + + @Override + public String getDescription() { + return "Output stream resource"; + } + + @Override + public InputStream getInputStream() throws IOException { + return null; + } + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/support/AvroTestFixtures.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/support/AvroTestFixtures.java new file mode 100644 index 0000000000..2d1a6a7f06 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/avro/support/AvroTestFixtures.java @@ -0,0 +1,185 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.avro.support; + +import java.io.FileOutputStream; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Paths; +import java.util.Arrays; +import java.util.List; +import java.util.Objects; +import java.util.stream.Collectors; + +import org.apache.avro.Schema; +import org.apache.avro.generic.GenericData; +import org.apache.avro.generic.GenericRecord; +import org.apache.avro.io.DatumWriter; +import org.apache.avro.io.Encoder; +import org.apache.avro.io.EncoderFactory; +import org.apache.avro.reflect.ReflectData; +import org.apache.avro.reflect.ReflectDatumWriter; +import org.springframework.batch.item.avro.example.User; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.ClassPathResource; +import org.springframework.core.io.Resource; + +/** + * @author David Turanski + * @author Mahmoud Ben Hassine + */ +public abstract class AvroTestFixtures { + + //@formatter:off + private final List avroGeneratedUsers = Arrays.asList( + new User("David", 20, "blue"), + new User("Sue", 4, "red"), + new User("Alana", 13, "yellow"), + new User("Joe", 1, "pink")); + + private List plainOldUsers = Arrays.asList( + new PlainOldUser("David", 20, "blue"), + new PlainOldUser("Sue", 4, "red"), + new PlainOldUser("Alana", 13, "yellow"), + new PlainOldUser("Joe", 1, "pink")); + //@formatter:on + + + protected Resource schemaResource = new ClassPathResource("org/springframework/batch/item/avro/user-schema.json"); + + + protected Resource plainOldUserSchemaResource = new ByteArrayResource(PlainOldUser.SCHEMA.toString().getBytes()); + + // Serialized data only + protected Resource dataResource = new ClassPathResource( + "org/springframework/batch/item/avro/user-data-no-schema.avro"); + + // Data written with DataFileWriter, includes embedded SCHEMA (more common) + protected Resource dataResourceWithSchema = new ClassPathResource( + "org/springframework/batch/item/avro/user-data.avro"); + + protected Resource plainOldUserDataResource + = new ClassPathResource("org/springframework/batch/item/avro/plain-old-user-data-no-schema.avro"); + + protected String schemaString(Resource resource) { + { + String content; + try { + content = new String(Files.readAllBytes(Paths.get(resource.getFile().getAbsolutePath()))); + } + catch (IOException e) { + throw new IllegalArgumentException(e.getMessage(), e); + } + return content; + } + } + + protected List avroGeneratedUsers() { + return this.avroGeneratedUsers; + } + + protected List genericAvroGeneratedUsers() { + return this.avroGeneratedUsers.stream().map(u-> { + GenericData.Record avroRecord; + avroRecord = new GenericData.Record(u.getSchema()); + avroRecord.put("name", u.getName()); + avroRecord.put("favorite_number", u.getFavoriteNumber()); + avroRecord.put("favorite_color",u.getFavoriteColor()); + return avroRecord; + } + ).collect(Collectors.toList()); + } + + protected List plainOldUsers() { + return this.plainOldUsers; + } + + protected List genericPlainOldUsers() { + return this.plainOldUsers.stream().map(PlainOldUser::toGenericRecord).collect(Collectors.toList()); + } + + protected static class PlainOldUser { + public static final Schema SCHEMA = ReflectData.get().getSchema(PlainOldUser.class); + private CharSequence name; + + private int favoriteNumber; + + private CharSequence favoriteColor; + + public PlainOldUser(){ + + } + + public PlainOldUser(CharSequence name, int favoriteNumber, CharSequence favoriteColor) { + this.name = name; + this.favoriteNumber = favoriteNumber; + this.favoriteColor = favoriteColor; + } + + public String getName() { + return name.toString(); + } + + public int getFavoriteNumber() { + return favoriteNumber; + } + + public String getFavoriteColor() { + return favoriteColor.toString(); + } + + public GenericRecord toGenericRecord() { + GenericData.Record avroRecord = new GenericData.Record(SCHEMA); + avroRecord.put("name", this.name); + avroRecord.put("favoriteNumber", this.favoriteNumber); + avroRecord.put("favoriteColor",this.favoriteColor); + return avroRecord; + } + + @Override + public boolean equals(Object o) { + if (this == o) return true; + if (o == null || getClass() != o.getClass()) return false; + PlainOldUser that = (PlainOldUser) o; + return favoriteNumber == that.favoriteNumber && + Objects.equals(name, that.name) && + Objects.equals(favoriteColor, that.favoriteColor); + } + + @Override + public int hashCode() { + return Objects.hash(name, favoriteNumber, favoriteColor); + } + } + + public static void createPlainOldUsersWithNoEmbeddedSchema() throws Exception { + + DatumWriter userDatumWriter = new ReflectDatumWriter<>(AvroTestFixtures.PlainOldUser.class); + + FileOutputStream fileOutputStream = new FileOutputStream("plain-old-user-data-no-schema.avro"); + + Encoder encoder = EncoderFactory.get().binaryEncoder(fileOutputStream,null); + userDatumWriter.write(new PlainOldUser("David", 20, "blue"), encoder); + userDatumWriter.write(new PlainOldUser("Sue", 4, "red"), encoder); + userDatumWriter.write(new PlainOldUser("Alana", 13, "yellow"), encoder); + userDatumWriter.write(new PlainOldUser("Joe", 1, "pink"), encoder); + + encoder.flush(); + fileOutputStream.flush(); + fileOutputStream.close(); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/GemfireItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/GemfireItemWriterTests.java index 657b64833a..b07bfde4cb 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/GemfireItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/GemfireItemWriterTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.data; import static org.junit.Assert.fail; @@ -15,27 +30,25 @@ import org.springframework.data.gemfire.GemfireTemplate; import org.springframework.core.convert.converter.Converter; -@SuppressWarnings({ "rawtypes", "serial", "unchecked" }) +@SuppressWarnings("serial") public class GemfireItemWriterTests { - private GemfireItemWriter writer; + private GemfireItemWriter writer; @Mock private GemfireTemplate template; - //private PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); - @Before public void setUp() throws Exception { MockitoAnnotations.initMocks(this); - writer = new GemfireItemWriter(); + writer = new GemfireItemWriter<>(); writer.setTemplate(template); - writer.setItemKeyMapper(new SpELItemKeyMapper("bar.val")); + writer.setItemKeyMapper(new SpELItemKeyMapper<>("bar.val")); writer.afterPropertiesSet(); } @Test public void testAfterPropertiesSet() throws Exception { - writer = new GemfireItemWriter(); + writer = new GemfireItemWriter<>(); try { writer.afterPropertiesSet(); @@ -50,7 +63,7 @@ public void testAfterPropertiesSet() throws Exception { } catch (IllegalArgumentException iae) { } - writer.setItemKeyMapper(new SpELItemKeyMapper("foo")); + writer.setItemKeyMapper(new SpELItemKeyMapper<>("foo")); writer.afterPropertiesSet(); } @@ -92,7 +105,7 @@ public void testWriteWithCustomItemKeyMapper() throws Exception { add(new Foo(new Bar("val2"))); } }; - writer = new GemfireItemWriter(); + writer = new GemfireItemWriter<>(); writer.setTemplate(template); writer.setItemKeyMapper(new Converter() { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/MongoItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/MongoItemReaderTests.java index 9552d44bfd..d80c67c2a2 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/MongoItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/MongoItemReaderTests.java @@ -1,12 +1,20 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.data; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.fail; -import static org.mockito.Matchers.eq; -import static org.mockito.Mockito.when; - import java.util.ArrayList; import java.util.HashMap; import java.util.Map; @@ -16,13 +24,23 @@ import org.mockito.ArgumentCaptor; import org.mockito.Mock; import org.mockito.MockitoAnnotations; + +import org.springframework.data.domain.PageRequest; import org.springframework.data.domain.Sort; +import org.springframework.data.domain.Sort.Order; import org.springframework.data.mongodb.core.MongoOperations; import org.springframework.data.mongodb.core.query.Query; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.when; + public class MongoItemReaderTests { - private MongoItemReader reader; + private MongoItemReader reader; @Mock private MongoOperations template; private Map sortOptions; @@ -30,9 +48,9 @@ public class MongoItemReaderTests { @Before public void setUp() throws Exception { MockitoAnnotations.initMocks(this); - reader = new MongoItemReader(); + reader = new MongoItemReader<>(); - sortOptions = new HashMap(); + sortOptions = new HashMap<>(); sortOptions.put("name", Sort.Direction.DESC); reader.setTemplate(template); @@ -44,8 +62,8 @@ public void setUp() throws Exception { } @Test - public void testAfterPropertiesSet() throws Exception{ - reader = new MongoItemReader(); + public void testAfterPropertiesSetForQueryString() throws Exception{ + reader = new MongoItemReader<>(); try { reader.afterPropertiesSet(); @@ -93,20 +111,33 @@ public void testAfterPropertiesSet() throws Exception{ reader.afterPropertiesSet(); } + + @Test + public void testAfterPropertiesSetForQueryObject() throws Exception{ + reader = new MongoItemReader<>(); + + reader.setTemplate(template); + reader.setTargetType(String.class); + + Query query1 = new Query().with(Sort.by(new Order(Sort.Direction.ASC, "_id"))); + reader.setQuery(query1); + + reader.afterPropertiesSet(); + } @Test public void testBasicQueryFirstPage() { ArgumentCaptor queryContainer = ArgumentCaptor.forClass(Query.class); - when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList()); + when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList<>()); assertFalse(reader.doPageRead().hasNext()); Query query = queryContainer.getValue(); assertEquals(50, query.getLimit()); assertEquals(0, query.getSkip()); - assertEquals("{ }", query.getQueryObject().toString()); - assertEquals("{ \"name\" : -1}", query.getSortObject().toString()); + assertEquals("{}", query.getQueryObject().toJson()); + assertEquals("{\"name\": -1}", query.getSortObject().toJson()); } @Test @@ -114,7 +145,7 @@ public void testBasicQuerySecondPage() { reader.page = 2; ArgumentCaptor queryContainer = ArgumentCaptor.forClass(Query.class); - when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList()); + when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList<>()); assertFalse(reader.doPageRead().hasNext()); @@ -122,9 +153,9 @@ public void testBasicQuerySecondPage() { assertEquals(50, query.getLimit()); assertEquals(100, query.getSkip()); - assertEquals("{ }", query.getQueryObject().toString()); - assertEquals("{ \"name\" : -1}", query.getSortObject().toString()); - assertNull(query.getFieldsObject()); + assertEquals("{}", query.getQueryObject().toJson()); + assertEquals("{\"name\": -1}", query.getSortObject().toJson()); + assertTrue(query.getFieldsObject().isEmpty()); } @Test @@ -132,16 +163,18 @@ public void testQueryWithFields() { reader.setFields("{name : 1, age : 1, _id: 0}"); ArgumentCaptor queryContainer = ArgumentCaptor.forClass(Query.class); - when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList()); + when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList<>()); assertFalse(reader.doPageRead().hasNext()); Query query = queryContainer.getValue(); assertEquals(50, query.getLimit()); assertEquals(0, query.getSkip()); - assertEquals("{ }", query.getQueryObject().toString()); - assertEquals("{ \"name\" : -1}", query.getSortObject().toString()); - assertEquals("{ \"name\" : 1 , \"age\" : 1 , \"_id\" : 0}", query.getFieldsObject().toString()); + assertEquals("{}", query.getQueryObject().toJson()); + assertEquals("{\"name\": -1}", query.getSortObject().toJson()); + assertEquals(1, query.getFieldsObject().get("name")); + assertEquals(1, query.getFieldsObject().get("age")); + assertEquals(0, query.getFieldsObject().get("_id")); } @Test @@ -149,35 +182,158 @@ public void testQueryWithHint() { reader.setHint("{ $natural : 1}"); ArgumentCaptor queryContainer = ArgumentCaptor.forClass(Query.class); - when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList()); + when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList<>()); assertFalse(reader.doPageRead().hasNext()); Query query = queryContainer.getValue(); assertEquals(50, query.getLimit()); assertEquals(0, query.getSkip()); - assertEquals("{ }", query.getQueryObject().toString()); - assertEquals("{ \"name\" : -1}", query.getSortObject().toString()); + assertEquals("{}", query.getQueryObject().toJson()); + assertEquals("{\"name\": -1}", query.getSortObject().toJson()); assertEquals("{ $natural : 1}", query.getHint()); } + @SuppressWarnings("serial") @Test public void testQueryWithParameters() { - reader.setParameterValues(new ArrayList(){{ + reader.setParameterValues(new ArrayList(){{ + add("foo"); + }}); + + reader.setQuery("{ name : ?0 }"); + ArgumentCaptor queryContainer = ArgumentCaptor.forClass(Query.class); + + when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList<>()); + + assertFalse(reader.doPageRead().hasNext()); + + Query query = queryContainer.getValue(); + assertEquals(50, query.getLimit()); + assertEquals(0, query.getSkip()); + assertEquals("{\"name\": \"foo\"}", query.getQueryObject().toJson()); + assertEquals("{\"name\": -1}", query.getSortObject().toJson()); + } + + @SuppressWarnings("serial") + @Test + public void testQueryWithCollection() { + reader.setParameterValues(new ArrayList(){{ add("foo"); }}); reader.setQuery("{ name : ?0 }"); + reader.setCollection("collection"); ArgumentCaptor queryContainer = ArgumentCaptor.forClass(Query.class); + ArgumentCaptor collectionContainer = ArgumentCaptor.forClass(String.class); - when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList()); + when(template.find(queryContainer.capture(), eq(String.class), collectionContainer.capture())).thenReturn(new ArrayList<>()); assertFalse(reader.doPageRead().hasNext()); Query query = queryContainer.getValue(); assertEquals(50, query.getLimit()); assertEquals(0, query.getSkip()); - assertEquals("{ \"name\" : \"foo\"}", query.getQueryObject().toString()); - assertEquals("{ \"name\" : -1}", query.getSortObject().toString()); + assertEquals("{\"name\": \"foo\"}", query.getQueryObject().toJson()); + assertEquals("{\"name\": -1}", query.getSortObject().toJson()); + assertEquals("collection", collectionContainer.getValue()); + } + + @Test + public void testQueryObject() throws Exception { + reader = new MongoItemReader<>(); + reader.setTemplate(template); + + Query query = new Query() + .with(Sort.by(new Order(Sort.Direction.ASC, "_id"))); + reader.setQuery(query); + reader.setTargetType(String.class); + + reader.afterPropertiesSet(); + + ArgumentCaptor queryContainer = ArgumentCaptor.forClass(Query.class); + when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList<>()); + + assertFalse(reader.doPageRead().hasNext()); + + Query actualQuery = queryContainer.getValue(); + assertFalse(reader.doPageRead().hasNext()); + assertEquals(10, actualQuery.getLimit()); + assertEquals(0, actualQuery.getSkip()); + } + + @Test + public void testQueryObjectWithIgnoredPageSize() throws Exception { + reader = new MongoItemReader<>(); + reader.setTemplate(template); + + Query query = new Query() + .with(Sort.by(new Order(Sort.Direction.ASC, "_id"))) + .with(PageRequest.of(0, 50)); + reader.setQuery(query); + reader.setTargetType(String.class); + + reader.afterPropertiesSet(); + + ArgumentCaptor queryContainer = ArgumentCaptor.forClass(Query.class); + when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList<>()); + + assertFalse(reader.doPageRead().hasNext()); + + Query actualQuery = queryContainer.getValue(); + assertFalse(reader.doPageRead().hasNext()); + assertEquals(10, actualQuery.getLimit()); + assertEquals(0, actualQuery.getSkip()); + } + + @Test + public void testQueryObjectWithPageSize() throws Exception { + reader = new MongoItemReader<>(); + reader.setTemplate(template); + + Query query = new Query() + .with(Sort.by(new Order(Sort.Direction.ASC, "_id"))) + .with(PageRequest.of(30, 50)); + reader.setQuery(query); + reader.setTargetType(String.class); + reader.setPageSize(100); + + reader.afterPropertiesSet(); + + ArgumentCaptor queryContainer = ArgumentCaptor.forClass(Query.class); + when(template.find(queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList<>()); + + assertFalse(reader.doPageRead().hasNext()); + + Query actualQuery = queryContainer.getValue(); + assertFalse(reader.doPageRead().hasNext()); + assertEquals(100, actualQuery.getLimit()); + assertEquals(0, actualQuery.getSkip()); + } + + @Test + public void testQueryObjectWithCollection() throws Exception { + reader = new MongoItemReader<>(); + reader.setTemplate(template); + + Query query = new Query() + .with(Sort.by(new Order(Sort.Direction.ASC, "_id"))); + reader.setQuery(query); + reader.setTargetType(String.class); + reader.setCollection("collection"); + + reader.afterPropertiesSet(); + + ArgumentCaptor queryContainer = ArgumentCaptor.forClass(Query.class); + ArgumentCaptor stringContainer = ArgumentCaptor.forClass(String.class); + when(template.find(queryContainer.capture(), eq(String.class), stringContainer.capture())).thenReturn(new ArrayList<>()); + + assertFalse(reader.doPageRead().hasNext()); + + Query actualQuery = queryContainer.getValue(); + assertFalse(reader.doPageRead().hasNext()); + assertEquals(10, actualQuery.getLimit()); + assertEquals(0, actualQuery.getSkip()); + assertEquals("collection", stringContainer.getValue()); } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/MongoItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/MongoItemWriterTests.java index ddfbca6fcc..05e5957e66 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/MongoItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/MongoItemWriterTests.java @@ -1,28 +1,47 @@ +/* + * Copyright 2013-2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.data; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.fail; -import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.verifyZeroInteractions; - import java.util.ArrayList; +import java.util.Collections; import java.util.List; import org.junit.Before; import org.junit.Test; import org.mockito.Mock; import org.mockito.MockitoAnnotations; + import org.springframework.batch.support.transaction.ResourcelessTransactionManager; import org.springframework.data.mongodb.core.MongoOperations; import org.springframework.transaction.PlatformTransactionManager; -import org.springframework.transaction.TransactionStatus; import org.springframework.transaction.support.TransactionCallback; import org.springframework.transaction.support.TransactionTemplate; -@SuppressWarnings({"rawtypes", "serial", "unchecked"}) +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.doAnswer; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.verifyZeroInteractions; + +@SuppressWarnings("serial") public class MongoItemWriterTests { - private MongoItemWriter writer; + private MongoItemWriter writer; @Mock private MongoOperations template; private PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); @@ -30,19 +49,19 @@ public class MongoItemWriterTests { @Before public void setUp() throws Exception { MockitoAnnotations.initMocks(this); - writer = new MongoItemWriter(); + writer = new MongoItemWriter<>(); writer.setTemplate(template); writer.afterPropertiesSet(); } @Test public void testAfterPropertiesSet() throws Exception { - writer = new MongoItemWriter(); + writer = new MongoItemWriter<>(); try { writer.afterPropertiesSet(); fail("Expected exception was not thrown"); - } catch (IllegalStateException iae) { + } catch (IllegalStateException ignore) { } writer.setTemplate(template); @@ -91,18 +110,14 @@ public void testWriteTransactionNoCollection() throws Exception { add(new Object()); }}; - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - - @Override - public Object doInTransaction(TransactionStatus status) { - try { - writer.write(items); - } catch (Exception e) { - fail("An exception was thrown while writing: " + e.getMessage()); - } - - return null; + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + writer.write(items); + } catch (Exception e) { + fail("An exception was thrown while writing: " + e.getMessage()); } + + return null; }); verify(template).save(items.get(0)); @@ -118,18 +133,14 @@ public void testWriteTransactionWithCollection() throws Exception { writer.setCollection("collection"); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - - @Override - public Object doInTransaction(TransactionStatus status) { - try { - writer.write(items); - } catch (Exception e) { - fail("An exception was thrown while writing: " + e.getMessage()); - } - - return null; + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + writer.write(items); + } catch (Exception e) { + fail("An exception was thrown while writing: " + e.getMessage()); } + + return null; }); verify(template).save(items.get(0), "collection"); @@ -146,17 +157,13 @@ public void testWriteTransactionFails() throws Exception { writer.setCollection("collection"); try { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - - @Override - public Object doInTransaction(TransactionStatus status) { - try { - writer.write(items); - } catch (Exception ignore) { - fail("unexpected exception thrown"); - } - throw new RuntimeException("force rollback"); + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + writer.write(items); + } catch (Exception ignore) { + fail("unexpected exception thrown"); } + throw new RuntimeException("force rollback"); }); } catch (RuntimeException re) { assertEquals(re.getMessage(), "force rollback"); @@ -170,7 +177,6 @@ public Object doInTransaction(TransactionStatus status) { /** * A pointless use case but validates that the flag is still honored. * - * @throws Exception */ @Test public void testWriteTransactionReadOnly() throws Exception { @@ -184,17 +190,13 @@ public void testWriteTransactionReadOnly() throws Exception { try { TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager); transactionTemplate.setReadOnly(true); - transactionTemplate.execute(new TransactionCallback() { - - @Override - public Object doInTransaction(TransactionStatus status) { - try { - writer.write(items); - } catch (Exception ignore) { - fail("unexpected exception thrown"); - } - return null; + transactionTemplate.execute((TransactionCallback) status -> { + try { + writer.write(items); + } catch (Exception ignore) { + fail("unexpected exception thrown"); } + return null; }); } catch (Throwable t) { fail("Unexpected exception was thrown"); @@ -232,4 +234,46 @@ public void testRemoveNoTransactionWithCollection() throws Exception { verify(template).remove(items.get(0), "collection"); verify(template).remove(items.get(1), "collection"); } + + // BATCH-2018 + @Test + public void testResourceKeyCollision() throws Exception { + final int limit = 5000; + @SuppressWarnings("unchecked") + List> writers = new ArrayList<>(limit); + final String[] results = new String[limit]; + for(int i = 0; i< limit; i++) { + final int index = i; + MongoOperations mongoOperations = mock(MongoOperations.class); + + doAnswer(invocation -> { + String val = (String) invocation.getArguments()[0]; + if(results[index] == null) { + results[index] = val; + } else { + results[index] += val; + } + return null; + }).when(mongoOperations).save(any(String.class)); + writers.add(i, new MongoItemWriter<>()); + writers.get(i).setTemplate(mongoOperations); + } + + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + for(int i=0; i< limit; i++) { + writers.get(i).write(Collections.singletonList(String.valueOf(i))); + } + } + catch (Exception e) { + throw new IllegalStateException("Unexpected Exception", e); + } + return null; + }); + + for(int i=0; i< limit; i++) { + assertEquals(String.valueOf(i), results[i]); + } + } + } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/Neo4jItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/Neo4jItemReaderTests.java index 17eb52d9b3..b8bdfcedee 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/Neo4jItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/Neo4jItemReaderTests.java @@ -1,13 +1,23 @@ +/* + * Copyright 2013-2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.data; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; -import static org.mockito.Matchers.isNull; -import static org.mockito.Mockito.when; - -import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collections; +import java.util.HashMap; import java.util.Map; import org.junit.Before; @@ -15,55 +25,64 @@ import org.mockito.ArgumentCaptor; import org.mockito.Mock; import org.mockito.MockitoAnnotations; -import org.springframework.data.neo4j.conversion.DefaultConverter; -import org.springframework.data.neo4j.conversion.EndResult; -import org.springframework.data.neo4j.conversion.Result; -import org.springframework.data.neo4j.conversion.ResultConverter; -import org.springframework.data.neo4j.template.Neo4jOperations; +import org.neo4j.ogm.session.Session; +import org.neo4j.ogm.session.SessionFactory; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.ArgumentMatchers.isNull; +import static org.mockito.Mockito.when; public class Neo4jItemReaderTests { - private Neo4jItemReader reader; @Mock - private Neo4jOperations template; + private Iterable result; @Mock - private Result result; + private SessionFactory sessionFactory; @Mock - private EndResult endResult; + private Session session; @Before public void setUp() throws Exception { - reader = new Neo4jItemReader(); - MockitoAnnotations.initMocks(this); + } + + private Neo4jItemReader buildSessionBasedReader() throws Exception { + Neo4jItemReader reader = new Neo4jItemReader<>(); - reader.setTemplate(template); + reader.setSessionFactory(this.sessionFactory); reader.setTargetType(String.class); reader.setStartStatement("n=node(*)"); reader.setReturnStatement("*"); reader.setOrderByStatement("n.age"); reader.setPageSize(50); reader.afterPropertiesSet(); + + return reader; } @Test public void testAfterPropertiesSet() throws Exception { - reader = new Neo4jItemReader(); + + Neo4jItemReader reader = new Neo4jItemReader<>(); try { reader.afterPropertiesSet(); - fail("Template was not set but exception was not thrown."); + fail("SessionFactory was not set but exception was not thrown."); } catch (IllegalStateException iae) { - assertEquals("A Neo4JOperations implementation is required", iae.getMessage()); + assertEquals("A SessionFactory is required", iae.getMessage()); } catch (Throwable t) { fail("Wrong exception was thrown:" + t); } - reader.setTemplate(template); + reader.setSessionFactory(this.sessionFactory); try { reader.afterPropertiesSet(); - fail("type was not set but exception was not thrown."); + fail("Target Type was not set but exception was not thrown."); } catch (IllegalStateException iae) { assertEquals("The type to be returned is required", iae.getMessage()); } catch (Throwable t) { @@ -106,59 +125,78 @@ public void testAfterPropertiesSet() throws Exception { reader.setOrderByStatement("n.age"); reader.afterPropertiesSet(); - } - - @Test - public void testNullResults() { - ArgumentCaptor query = ArgumentCaptor.forClass(String.class); - when(template.query(query.capture(), (Map) isNull())).thenReturn(null); + reader = new Neo4jItemReader<>(); + reader.setSessionFactory(this.sessionFactory); + reader.setTargetType(String.class); + reader.setStartStatement("n=node(*)"); + reader.setReturnStatement("n.name, n.phone"); + reader.setOrderByStatement("n.age"); - assertFalse(reader.doPageRead().hasNext()); - assertEquals("START n=node(*) RETURN * ORDER BY n.age SKIP 0 LIMIT 50", query.getValue()); + reader.afterPropertiesSet(); } + @SuppressWarnings("unchecked") @Test - public void testNoResults() { + public void testNullResultsWithSession() throws Exception { + + Neo4jItemReader itemReader = buildSessionBasedReader(); + ArgumentCaptor query = ArgumentCaptor.forClass(String.class); - when(template.query(query.capture(), (Map) isNull())).thenReturn(result); - when(result.to(String.class)).thenReturn(endResult); - when(endResult.iterator()).thenReturn(new ArrayList().iterator()); + when(this.sessionFactory.openSession()).thenReturn(this.session); + when(this.session.query(eq(String.class), query.capture(), isNull())).thenReturn(null); - assertFalse(reader.doPageRead().hasNext()); + assertFalse(itemReader.doPageRead().hasNext()); assertEquals("START n=node(*) RETURN * ORDER BY n.age SKIP 0 LIMIT 50", query.getValue()); } + @SuppressWarnings("unchecked") @Test - public void testResultsWithConverter() { - ResultConverter converter = new DefaultConverter(); - - reader.setResultConverter(converter); + public void testNoResultsWithSession() throws Exception { + Neo4jItemReader itemReader = buildSessionBasedReader(); ArgumentCaptor query = ArgumentCaptor.forClass(String.class); - when(template.query(query.capture(), (Map) isNull())).thenReturn(result); - when(result.to(String.class, converter)).thenReturn(endResult); - when(endResult.iterator()).thenReturn(new ArrayList(){{ - add(new String()); - }}.iterator()); + when(this.sessionFactory.openSession()).thenReturn(this.session); + when(this.session.query(eq(String.class), query.capture(), isNull())).thenReturn(result); + when(result.iterator()).thenReturn(Collections.emptyIterator()); - assertTrue(reader.doPageRead().hasNext()); + assertFalse(itemReader.doPageRead().hasNext()); assertEquals("START n=node(*) RETURN * ORDER BY n.age SKIP 0 LIMIT 50", query.getValue()); } + @SuppressWarnings("serial") @Test - public void testResultsWithMatchAndWhere() throws Exception { - reader.setMatchStatement("n -- m"); - reader.setWhereStatement("has(n.name)"); - reader.setReturnStatement("m"); - reader.afterPropertiesSet(); - when(template.query("START n=node(*) MATCH n -- m WHERE has(n.name) RETURN m ORDER BY n.age SKIP 0 LIMIT 50", null)).thenReturn(result); - when(result.to(String.class)).thenReturn(endResult); - when(endResult.iterator()).thenReturn(new ArrayList(){{ - add(new String()); - }}.iterator()); + public void testResultsWithMatchAndWhereWithSession() throws Exception { + Neo4jItemReader itemReader = buildSessionBasedReader(); + itemReader.setMatchStatement("n -- m"); + itemReader.setWhereStatement("has(n.name)"); + itemReader.setReturnStatement("m"); + itemReader.afterPropertiesSet(); + + when(this.sessionFactory.openSession()).thenReturn(this.session); + when(this.session.query(String.class, "START n=node(*) MATCH n -- m WHERE has(n.name) RETURN m ORDER BY n.age SKIP 0 LIMIT 50", null)).thenReturn(result); + when(result.iterator()).thenReturn(Arrays.asList("foo", "bar", "baz").iterator()); + + assertTrue(itemReader.doPageRead().hasNext()); + } - assertTrue(reader.doPageRead().hasNext()); + @SuppressWarnings("serial") + @Test + public void testResultsWithMatchAndWhereWithParametersWithSession() throws Exception { + Neo4jItemReader itemReader = buildSessionBasedReader(); + Map params = new HashMap<>(); + params.put("foo", "bar"); + itemReader.setParameterValues(params); + itemReader.setMatchStatement("n -- m"); + itemReader.setWhereStatement("has(n.name)"); + itemReader.setReturnStatement("m"); + itemReader.afterPropertiesSet(); + + when(this.sessionFactory.openSession()).thenReturn(this.session); + when(this.session.query(String.class, "START n=node(*) MATCH n -- m WHERE has(n.name) RETURN m ORDER BY n.age SKIP 0 LIMIT 50", params)).thenReturn(result); + when(result.iterator()).thenReturn(Arrays.asList("foo", "bar", "baz").iterator()); + + assertTrue(itemReader.doPageRead().hasNext()); } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/Neo4jItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/Neo4jItemWriterTests.java index 40ba364d67..85ffccadec 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/Neo4jItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/Neo4jItemWriterTests.java @@ -1,10 +1,20 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.data; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.fail; -import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.verifyZeroInteractions; - import java.util.ArrayList; import java.util.List; @@ -12,78 +22,128 @@ import org.junit.Test; import org.mockito.Mock; import org.mockito.MockitoAnnotations; -import org.springframework.data.neo4j.template.Neo4jOperations; +import org.neo4j.ogm.session.Session; +import org.neo4j.ogm.session.SessionFactory; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.verifyZeroInteractions; +import static org.mockito.Mockito.when; -@SuppressWarnings("rawtypes") public class Neo4jItemWriterTests { - private Neo4jItemWriter writer; + private Neo4jItemWriter writer; + + @Mock + private SessionFactory sessionFactory; @Mock - private Neo4jOperations template; + private Session session; @Before public void setUp() throws Exception { MockitoAnnotations.initMocks(this); - writer = new Neo4jItemWriter(); - - writer.setTemplate(template); } @Test public void testAfterPropertiesSet() throws Exception{ - writer = new Neo4jItemWriter(); + + writer = new Neo4jItemWriter<>(); try { writer.afterPropertiesSet(); - fail("Template was not set but exception was not thrown."); + fail("SessionFactory was not set but exception was not thrown."); } catch (IllegalStateException iae) { - assertEquals("A Neo4JOperations implementation is required", iae.getMessage()); + assertEquals("A SessionFactory is required", iae.getMessage()); } catch (Throwable t) { fail("Wrong exception was thrown."); } - writer.setTemplate(template); + writer.setSessionFactory(this.sessionFactory); + + writer.afterPropertiesSet(); + + writer = new Neo4jItemWriter<>(); + + writer.setSessionFactory(this.sessionFactory); writer.afterPropertiesSet(); } @Test - public void testWriteNull() throws Exception { + public void testWriteNullSession() throws Exception { + + writer = new Neo4jItemWriter<>(); + + writer.setSessionFactory(this.sessionFactory); + writer.afterPropertiesSet(); + writer.write(null); - verifyZeroInteractions(template); + verifyZeroInteractions(this.session); } @Test - public void testWriteNoItems() throws Exception { - writer.write(new ArrayList()); + public void testWriteNullWithSession() throws Exception { + writer = new Neo4jItemWriter<>(); + + writer.setSessionFactory(this.sessionFactory); + writer.afterPropertiesSet(); + + when(this.sessionFactory.openSession()).thenReturn(this.session); + writer.write(null); - verifyZeroInteractions(template); + verifyZeroInteractions(this.session); } @Test - public void testWriteItems() throws Exception { - List items = new ArrayList(); + public void testWriteNoItemsWithSession() throws Exception { + writer = new Neo4jItemWriter<>(); + + writer.setSessionFactory(this.sessionFactory); + writer.afterPropertiesSet(); + + when(this.sessionFactory.openSession()).thenReturn(this.session); + writer.write(new ArrayList<>()); + + verifyZeroInteractions(this.session); + } + + @Test + public void testWriteItemsWithSession() throws Exception { + writer = new Neo4jItemWriter<>(); + + writer.setSessionFactory(this.sessionFactory); + writer.afterPropertiesSet(); + + List items = new ArrayList<>(); items.add("foo"); items.add("bar"); + when(this.sessionFactory.openSession()).thenReturn(this.session); writer.write(items); - verify(template).save("foo"); - verify(template).save("bar"); + verify(this.session).save("foo"); + verify(this.session).save("bar"); } @Test - public void testDeleteItems() throws Exception { - List items = new ArrayList(); + public void testDeleteItemsWithSession() throws Exception { + writer = new Neo4jItemWriter<>(); + + writer.setSessionFactory(this.sessionFactory); + writer.afterPropertiesSet(); + + List items = new ArrayList<>(); items.add("foo"); items.add("bar"); writer.setDelete(true); + when(this.sessionFactory.openSession()).thenReturn(this.session); writer.write(items); - verify(template).delete("foo"); - verify(template).delete("bar"); + verify(this.session).delete("foo"); + verify(this.session).delete("bar"); } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/RepositoryItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/RepositoryItemReaderTests.java index 2469ae4ea4..5f0ba2ee4a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/RepositoryItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/RepositoryItemReaderTests.java @@ -1,12 +1,20 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.data; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; -import static org.mockito.Mockito.when; - import java.util.ArrayList; import java.util.HashMap; import java.util.Map; @@ -16,7 +24,10 @@ import org.mockito.ArgumentCaptor; import org.mockito.Mock; import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.adapter.DynamicMethodInvocationException; +import org.springframework.data.domain.Page; import org.springframework.data.domain.PageImpl; import org.springframework.data.domain.PageRequest; import org.springframework.data.domain.Pageable; @@ -24,20 +35,28 @@ import org.springframework.data.domain.Sort.Direction; import org.springframework.data.repository.PagingAndSortingRepository; -@SuppressWarnings("rawtypes") +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +@SuppressWarnings("serial") public class RepositoryItemReaderTests { - private RepositoryItemReader reader; + private RepositoryItemReader reader; @Mock - private PagingAndSortingRepository repository; + private PagingAndSortingRepository repository; private Map sorts; @Before public void setUp() throws Exception { MockitoAnnotations.initMocks(this); - sorts = new HashMap(); + sorts = new HashMap<>(); sorts.put("id", Direction.ASC); - reader = new RepositoryItemReader(); + reader = new RepositoryItemReader<>(); reader.setRepository(repository); reader.setPageSize(1); reader.setSort(sorts); @@ -47,13 +66,13 @@ public void setUp() throws Exception { @Test public void testAfterPropertiesSet() throws Exception { try { - new RepositoryItemReader().afterPropertiesSet(); + new RepositoryItemReader<>().afterPropertiesSet(); fail(); } catch (IllegalStateException e) { } try { - reader = new RepositoryItemReader(); + reader = new RepositoryItemReader<>(); reader.setRepository(repository); reader.afterPropertiesSet(); fail(); @@ -61,7 +80,7 @@ public void testAfterPropertiesSet() throws Exception { } try { - reader = new RepositoryItemReader(); + reader = new RepositoryItemReader<>(); reader.setRepository(repository); reader.setPageSize(-1); reader.afterPropertiesSet(); @@ -70,7 +89,7 @@ public void testAfterPropertiesSet() throws Exception { } try { - reader = new RepositoryItemReader(); + reader = new RepositoryItemReader<>(); reader.setRepository(repository); reader.setPageSize(1); reader.afterPropertiesSet(); @@ -78,7 +97,7 @@ public void testAfterPropertiesSet() throws Exception { } catch (IllegalStateException iae) { } - reader = new RepositoryItemReader(); + reader = new RepositoryItemReader<>(); reader.setRepository(repository); reader.setPageSize(1); reader.setSort(sorts); @@ -86,11 +105,10 @@ public void testAfterPropertiesSet() throws Exception { } @Test - @SuppressWarnings("unchecked") public void testDoReadFirstReadNoResults() throws Exception { ArgumentCaptor pageRequestContainer = ArgumentCaptor.forClass(PageRequest.class); - when(repository.findAll(pageRequestContainer.capture())).thenReturn(new PageImpl(new ArrayList())); + when(repository.findAll(pageRequestContainer.capture())).thenReturn(new PageImpl<>(new ArrayList<>())); assertNull(reader.doRead()); @@ -102,12 +120,12 @@ public void testDoReadFirstReadNoResults() throws Exception { } @Test - @SuppressWarnings({"serial", "unchecked"}) + @SuppressWarnings("serial") public void testDoReadFirstReadResults() throws Exception { ArgumentCaptor pageRequestContainer = ArgumentCaptor.forClass(PageRequest.class); final Object result = new Object(); - when(repository.findAll(pageRequestContainer.capture())).thenReturn(new PageImpl(new ArrayList(){{ + when(repository.findAll(pageRequestContainer.capture())).thenReturn(new PageImpl<>(new ArrayList(){{ add(result); }})); @@ -121,13 +139,13 @@ public void testDoReadFirstReadResults() throws Exception { } @Test - @SuppressWarnings({"serial", "unchecked"}) + @SuppressWarnings("serial") public void testDoReadFirstReadSecondPage() throws Exception { ArgumentCaptor pageRequestContainer = ArgumentCaptor.forClass(PageRequest.class); final Object result = new Object(); - when(repository.findAll(pageRequestContainer.capture())).thenReturn(new PageImpl(new ArrayList(){{ + when(repository.findAll(pageRequestContainer.capture())).thenReturn(new PageImpl<>(new ArrayList() {{ add(new Object()); - }})).thenReturn(new PageImpl(new ArrayList(){{ + }})).thenReturn(new PageImpl<>(new ArrayList(){{ add(result); }})); @@ -142,15 +160,15 @@ public void testDoReadFirstReadSecondPage() throws Exception { } @Test - @SuppressWarnings({"serial", "unchecked"}) + @SuppressWarnings("serial") public void testDoReadFirstReadExhausted() throws Exception { ArgumentCaptor pageRequestContainer = ArgumentCaptor.forClass(PageRequest.class); final Object result = new Object(); - when(repository.findAll(pageRequestContainer.capture())).thenReturn(new PageImpl(new ArrayList(){{ + when(repository.findAll(pageRequestContainer.capture())).thenReturn(new PageImpl<>(new ArrayList() {{ add(new Object()); - }})).thenReturn(new PageImpl(new ArrayList(){{ + }})).thenReturn(new PageImpl<>(new ArrayList(){{ add(result); - }})).thenReturn(new PageImpl(new ArrayList())); + }})).thenReturn(new PageImpl<>(new ArrayList<>())); assertFalse(reader.doRead() == result); assertEquals(result, reader.doRead()); @@ -164,11 +182,11 @@ public void testDoReadFirstReadExhausted() throws Exception { } @Test - @SuppressWarnings({"serial", "unchecked"}) + @SuppressWarnings("serial") public void testJumpToItem() throws Exception { reader.setPageSize(100); ArgumentCaptor pageRequestContainer = ArgumentCaptor.forClass(PageRequest.class); - when(repository.findAll(pageRequestContainer.capture())).thenReturn(new PageImpl(new ArrayList(){{ + when(repository.findAll(pageRequestContainer.capture())).thenReturn(new PageImpl<>(new ArrayList() {{ add(new Object()); }})); @@ -192,4 +210,122 @@ public void testInvalidMethodName() throws Exception { assertTrue(dmie.getCause() instanceof NoSuchMethodException); } } + + @Test + public void testDifferentTypes() throws Exception { + TestRepository differentRepository = mock(TestRepository.class); + RepositoryItemReader reader = new RepositoryItemReader<>(); + sorts = new HashMap<>(); + sorts.put("id", Direction.ASC); + reader.setRepository(differentRepository); + reader.setPageSize(1); + reader.setSort(sorts); + reader.setMethodName("findFirstNames"); + + ArgumentCaptor pageRequestContainer = ArgumentCaptor.forClass(PageRequest.class); + when(differentRepository.findFirstNames(pageRequestContainer.capture())).thenReturn(new PageImpl<>(new ArrayList(){{ + add("result"); + }})); + + assertEquals("result", reader.doRead()); + + Pageable pageRequest = pageRequestContainer.getValue(); + assertEquals(0, pageRequest.getOffset()); + assertEquals(0, pageRequest.getPageNumber()); + assertEquals(1, pageRequest.getPageSize()); + assertEquals("id: ASC", pageRequest.getSort().toString()); + } + + @Test + public void testSettingCurrentItemCountExplicitly() throws Exception { + reader.setCurrentItemCount(3); + reader.setPageSize(2); + + PageRequest request = PageRequest.of(1, 2, Sort.by(Direction.ASC, "id")); + when(repository.findAll(request)).thenReturn(new PageImpl<>(new ArrayList() {{ + add("3"); + add("4"); + }})); + + request = PageRequest.of(2, 2, Sort.by(Direction.ASC, "id")); + when(repository.findAll(request)).thenReturn(new PageImpl<>(new ArrayList(){{ + add("5"); + add("6"); + }})); + + reader.open(new ExecutionContext()); + + Object result = reader.read(); + + assertEquals("3", result); + assertEquals("4", reader.read()); + assertEquals("5", reader.read()); + assertEquals("6", reader.read()); + } + + @Test + public void testSettingCurrentItemCountRestart() throws Exception { + reader.setCurrentItemCount(3); + reader.setPageSize(2); + + PageRequest request = PageRequest.of(1, 2, Sort.by(Direction.ASC, "id")); + when(repository.findAll(request)).thenReturn(new PageImpl<>(new ArrayList(){{ + add("3"); + add("4"); + }})); + + request = PageRequest.of(2, 2, Sort.by(Direction.ASC, "id")); + when(repository.findAll(request)).thenReturn(new PageImpl<>(new ArrayList() {{ + add("5"); + add("6"); + }})); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + + Object result = reader.read(); + reader.update(executionContext); + reader.close(); + + assertEquals("3", result); + + reader.open(executionContext); + assertEquals("4", reader.read()); + assertEquals("5", reader.read()); + assertEquals("6", reader.read()); + } + + @Test + public void testResetOfPage() throws Exception { + reader.setPageSize(2); + + PageRequest request = PageRequest.of(0, 2, Sort.by(Direction.ASC, "id")); + when(repository.findAll(request)).thenReturn(new PageImpl<>(new ArrayList(){{ + add("1"); + add("2"); + }})); + + request = PageRequest.of(1, 2, Sort.by(Direction.ASC, "id")); + when(repository.findAll(request)).thenReturn(new PageImpl<>(new ArrayList() {{ + add("3"); + add("4"); + }})); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + + Object result = reader.read(); + reader.close(); + + assertEquals("1", result); + + reader.open(executionContext); + assertEquals("1", reader.read()); + assertEquals("2", reader.read()); + assertEquals("3", reader.read()); + } + + public interface TestRepository extends PagingAndSortingRepository, Long> { + Page findFirstNames(Pageable pageable); + } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/RepositoryItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/RepositoryItemWriterTests.java index c291533e71..681ac5cd90 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/RepositoryItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/RepositoryItemWriterTests.java @@ -1,10 +1,27 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.data; import static org.junit.Assert.fail; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.verifyZeroInteractions; +import java.io.Serializable; import java.util.ArrayList; +import java.util.Collections; import java.util.List; import org.junit.Before; @@ -13,18 +30,17 @@ import org.mockito.MockitoAnnotations; import org.springframework.data.repository.CrudRepository; -@SuppressWarnings("rawtypes") public class RepositoryItemWriterTests { @Mock - private CrudRepository repository; + private CrudRepository repository; - private RepositoryItemWriter writer; + private RepositoryItemWriter writer; @Before public void setUp() throws Exception { MockitoAnnotations.initMocks(this); - writer = new RepositoryItemWriter(); + writer = new RepositoryItemWriter<>(); writer.setMethodName("save"); writer.setRepository(repository); } @@ -46,17 +62,14 @@ public void testAfterPropertiesSet() throws Exception { public void testWriteNoItems() throws Exception { writer.write(null); - writer.write(new ArrayList()); + writer.write(new ArrayList<>()); verifyZeroInteractions(repository); } @Test - @SuppressWarnings({"serial", "unchecked"}) public void testWriteItems() throws Exception { - List items = new ArrayList() {{ - add("foo"); - }}; + List items = Collections.singletonList("foo"); writer.write(items); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/GemfireItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/GemfireItemWriterBuilderTests.java new file mode 100644 index 0000000000..cb2465b09a --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/GemfireItemWriterBuilderTests.java @@ -0,0 +1,121 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import java.util.Arrays; +import java.util.List; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.SpELItemKeyMapper; +import org.springframework.batch.item.data.GemfireItemWriter; +import org.springframework.data.gemfire.GemfireTemplate; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; + +/** + * @author Glenn Renfro + */ +public class GemfireItemWriterBuilderTests { + @Mock + private GemfireTemplate template; + + private SpELItemKeyMapper itemKeyMapper; + + private List items; + + @Before + public void setUp() { + MockitoAnnotations.initMocks(this); + this.items = Arrays.asList(new GemfireItemWriterBuilderTests.Foo(new GemfireItemWriterBuilderTests.Bar("val1")), + new GemfireItemWriterBuilderTests.Foo(new GemfireItemWriterBuilderTests.Bar("val2"))); + this.itemKeyMapper = new SpELItemKeyMapper<>("bar.val"); + } + + @Test + public void testBasicWrite() throws Exception { + GemfireItemWriter writer = new GemfireItemWriterBuilder() + .template(this.template).itemKeyMapper(this.itemKeyMapper).build(); + + writer.write(this.items); + + verify(this.template).put("val1", items.get(0)); + verify(this.template).put("val2", items.get(1)); + verify(this.template, never()).remove("val1"); + verify(this.template, never()).remove("val2"); + } + + @Test + public void testBasicDelete() throws Exception { + GemfireItemWriter writer = new GemfireItemWriterBuilder() + .template(this.template).delete(true).itemKeyMapper(this.itemKeyMapper).build(); + + writer.write(this.items); + + verify(this.template).remove("val1"); + verify(this.template).remove("val2"); + verify(this.template, never()).put("val1", items.get(0)); + verify(this.template, never()).put("val2", items.get(1)); + } + + @Test + public void testNullTemplate() { + try { + new GemfireItemWriterBuilder().itemKeyMapper(this.itemKeyMapper) + .build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", "template is required.", + iae.getMessage()); + } + } + + @Test + public void testNullItemKeyMapper() { + try { + new GemfireItemWriterBuilder().template(this.template).build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "itemKeyMapper is required.", iae.getMessage()); + } + } + + static class Foo { + public GemfireItemWriterBuilderTests.Bar bar; + + public Foo(GemfireItemWriterBuilderTests.Bar bar) { + this.bar = bar; + } + } + + static class Bar { + public String val; + + public Bar(String b1) { + this.val = b1; + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/MongoItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/MongoItemReaderBuilderTests.java new file mode 100644 index 0000000000..7aa228d62c --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/MongoItemReaderBuilderTests.java @@ -0,0 +1,216 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.Map; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.ArgumentCaptor; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.data.MongoItemReader; +import org.springframework.data.domain.Sort; +import org.springframework.data.mongodb.core.MongoOperations; +import org.springframework.data.mongodb.core.query.Query; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.when; + +/** + * @author Glenn Renfro + * @author Drummond Dawson + */ +public class MongoItemReaderBuilderTests { + @Mock + private MongoOperations template; + + private Map sortOptions; + + private ArgumentCaptor queryContainer; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + this.sortOptions = new HashMap<>(); + this.sortOptions.put("name", Sort.Direction.DESC); + this.queryContainer = ArgumentCaptor.forClass(Query.class); + } + + @Test + public void testBasic() throws Exception { + MongoItemReader reader = getBasicBuilder().build(); + + when(template.find(this.queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList<>()); + + assertNull("reader should not return result", reader.read()); + + Query query = this.queryContainer.getValue(); + assertEquals(50, query.getLimit()); + assertEquals(0, query.getSkip()); + assertEquals("{}", query.getQueryObject().toJson()); + assertEquals("{\"name\": -1}", query.getSortObject().toJson()); + } + + @Test + public void testFields() throws Exception { + MongoItemReader reader = getBasicBuilder() + .fields("{name : 1, age : 1, _id: 0}") + .build(); + + when(this.template.find(this.queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList<>()); + + assertNull("reader should not return result", reader.read()); + + Query query = this.queryContainer.getValue(); + assertEquals(1, query.getFieldsObject().get("name")); + assertEquals(1, query.getFieldsObject().get("age")); + assertEquals(0, query.getFieldsObject().get("_id")); + } + + @Test + public void testHint() throws Exception { + MongoItemReader reader = getBasicBuilder() + .hint("{ $natural : 1}") + .build(); + + when(this.template.find(this.queryContainer.capture(), eq(String.class))).thenReturn(new ArrayList<>()); + + assertNull("reader should not return result", reader.read()); + + Query query = this.queryContainer.getValue(); + assertEquals("{ $natural : 1}", query.getHint()); + } + + @Test + public void testCollection() throws Exception { + MongoItemReader reader = getBasicBuilder() + .parameterValues(Collections.singletonList("foo")) + .jsonQuery("{ name : ?0 }") + .collection("collection") + .build(); + + ArgumentCaptor collectionContainer = ArgumentCaptor.forClass(String.class); + + when(this.template.find(this.queryContainer.capture(), eq(String.class), collectionContainer.capture())) + .thenReturn(new ArrayList<>()); + + assertNull("reader should not return result", reader.read()); + + Query query = this.queryContainer.getValue(); + assertEquals("{\"name\": \"foo\"}", query.getQueryObject().toJson()); + assertEquals("{\"name\": -1}", query.getSortObject().toJson()); + assertEquals("collection", collectionContainer.getValue()); + } + + @Test + public void testVarargs() throws Exception { + MongoItemReader reader = getBasicBuilder() + .parameterValues("foo") + .jsonQuery("{ name : ?0 }") + .collection("collection") + .build(); + + ArgumentCaptor collectionContainer = ArgumentCaptor.forClass(String.class); + + when(this.template.find(this.queryContainer.capture(), eq(String.class), collectionContainer.capture())) + .thenReturn(new ArrayList<>()); + + assertNull("reader should not return result", reader.read()); + + Query query = this.queryContainer.getValue(); + assertEquals("{\"name\": \"foo\"}", query.getQueryObject().toJson()); + assertEquals("{\"name\": -1}", query.getSortObject().toJson()); + assertEquals("collection", collectionContainer.getValue()); + } + + @Test + public void testNullTemplate() { + validateExceptionMessage(new MongoItemReaderBuilder().targetType(String.class) + .jsonQuery("{ }") + .sorts(this.sortOptions) + .name("mongoReaderTest") + .pageSize(50), "template is required."); + } + + @Test + public void testNullTargetType() { + validateExceptionMessage(new MongoItemReaderBuilder().template(this.template) + .jsonQuery("{ }") + .sorts(this.sortOptions) + .name("mongoReaderTest") + .pageSize(50), "targetType is required."); + } + + @Test + public void testNullQuery() { + validateExceptionMessage(new MongoItemReaderBuilder().template(this.template) + .targetType(String.class) + .sorts(this.sortOptions) + .name("mongoReaderTest") + .pageSize(50), "A query is required"); + } + + @Test + public void testNullSorts() { + validateExceptionMessage(new MongoItemReaderBuilder().template(this.template) + .targetType(String.class) + .jsonQuery("{ }") + .name("mongoReaderTest") + .pageSize(50), "sorts map is required."); + } + + @Test + public void testNullName() { + validateExceptionMessage(new MongoItemReaderBuilder().template(this.template) + .targetType(String.class) + .jsonQuery("{ }") + .sorts(this.sortOptions) + .pageSize(50), "A name is required when saveState is set to true"); + } + + private void validateExceptionMessage(MongoItemReaderBuilder builder, String message) { + try { + builder.build(); + fail("Exception should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", message, + iae.getMessage()); + } + catch (IllegalStateException ise) { + assertEquals("IllegalStateException message did not match the expected result.", message, + ise.getMessage()); + } + } + + private MongoItemReaderBuilder getBasicBuilder() { + return new MongoItemReaderBuilder().template(this.template) + .targetType(String.class) + .jsonQuery("{ }") + .sorts(this.sortOptions) + .name("mongoReaderTest") + .pageSize(50); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/MongoItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/MongoItemWriterBuilderTests.java new file mode 100644 index 0000000000..7ec9fe84ed --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/MongoItemWriterBuilderTests.java @@ -0,0 +1,100 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import java.util.Arrays; +import java.util.List; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.data.MongoItemWriter; +import org.springframework.data.mongodb.core.MongoOperations; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; + +/** + * @author Glenn Renfro + */ +public class MongoItemWriterBuilderTests { + @Mock + private MongoOperations template; + + private List items; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + this.items = Arrays.asList("foo", "bar"); + } + + @Test + public void testBasicWrite() throws Exception { + MongoItemWriter writer = new MongoItemWriterBuilder().template(this.template).build(); + writer.write(this.items); + + verify(this.template).save(this.items.get(0)); + verify(this.template).save(this.items.get(1)); + verify(this.template, never()).remove(this.items.get(0)); + verify(this.template, never()).remove(this.items.get(1)); + } + + @Test + public void testDelete() throws Exception { + MongoItemWriter writer = new MongoItemWriterBuilder().template(this.template) + .delete(true) + .build(); + + writer.write(this.items); + + verify(this.template).remove(this.items.get(0)); + verify(this.template).remove(this.items.get(1)); + verify(this.template, never()).save(this.items.get(0)); + verify(this.template, never()).save(this.items.get(1)); + } + + @Test + public void testWriteToCollection() throws Exception { + MongoItemWriter writer = new MongoItemWriterBuilder().collection("collection") + .template(this.template) + .build(); + + writer.write(this.items); + + verify(this.template).save(this.items.get(0), "collection"); + verify(this.template).save(this.items.get(1), "collection"); + verify(this.template, never()).remove(this.items.get(0), "collection"); + verify(this.template, never()).remove(this.items.get(1), "collection"); + } + + @Test + public void testNullTemplate() { + try { + new MongoItemWriterBuilder<>().build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", "template is required.", + iae.getMessage()); + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/Neo4jItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/Neo4jItemReaderBuilderTests.java new file mode 100644 index 0000000000..6e5c95975e --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/Neo4jItemReaderBuilderTests.java @@ -0,0 +1,291 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import java.util.Arrays; +import java.util.HashMap; +import java.util.Map; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; +import org.neo4j.ogm.session.Session; +import org.neo4j.ogm.session.SessionFactory; + +import org.springframework.batch.item.data.Neo4jItemReader; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.when; + +/** + * @author Glenn Renfro + */ +public class Neo4jItemReaderBuilderTests { + + @Mock + private Iterable result; + + @Mock + private SessionFactory sessionFactory; + + @Mock + private Session session; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + } + + @Test + public void testFullyQualifiedItemReader() throws Exception { + Neo4jItemReader itemReader = new Neo4jItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .targetType(String.class) + .startStatement("n=node(*)") + .orderByStatement("n.age") + .pageSize(50).name("bar") + .matchStatement("n -- m") + .whereStatement("has(n.name)") + .returnStatement("m").build(); + + when(this.sessionFactory.openSession()).thenReturn(this.session); + when(this.session.query(String.class, + "START n=node(*) MATCH n -- m WHERE has(n.name) RETURN m ORDER BY n.age SKIP 0 LIMIT 50", null)) + .thenReturn(result); + when(result.iterator()).thenReturn(Arrays.asList("foo", "bar", "baz").iterator()); + + assertEquals("The expected value was not returned by reader.", "foo", itemReader.read()); + assertEquals("The expected value was not returned by reader.", "bar", itemReader.read()); + assertEquals("The expected value was not returned by reader.", "baz", itemReader.read()); + } + + @Test + public void testCurrentSize() throws Exception { + Neo4jItemReader itemReader = new Neo4jItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .targetType(String.class) + .startStatement("n=node(*)") + .orderByStatement("n.age") + .pageSize(50).name("bar") + .returnStatement("m") + .currentItemCount(0) + .maxItemCount(1) + .build(); + + when(this.sessionFactory.openSession()).thenReturn(this.session); + when(this.session.query(String.class, "START n=node(*) RETURN m ORDER BY n.age SKIP 0 LIMIT 50", null)) + .thenReturn(result); + when(result.iterator()).thenReturn(Arrays.asList("foo", "bar", "baz").iterator()); + + assertEquals("The expected value was not returned by reader.", "foo", itemReader.read()); + assertNull("The expected value was not should be null.", itemReader.read()); + } + + @Test + public void testResultsWithMatchAndWhereWithParametersWithSession() throws Exception { + Map params = new HashMap<>(); + params.put("foo", "bar"); + Neo4jItemReader itemReader = new Neo4jItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .targetType(String.class) + .startStatement("n=node(*)") + .returnStatement("*") + .orderByStatement("n.age") + .pageSize(50) + .name("foo") + .parameterValues(params) + .matchStatement("n -- m") + .whereStatement("has(n.name)") + .returnStatement("m") + .build(); + + when(this.sessionFactory.openSession()).thenReturn(this.session); + when(this.session.query(String.class, + "START n=node(*) MATCH n -- m WHERE has(n.name) RETURN m ORDER BY n.age SKIP 0 LIMIT 50", params)) + .thenReturn(result); + when(result.iterator()).thenReturn(Arrays.asList("foo", "bar", "baz").iterator()); + + assertEquals("The expected value was not returned by reader.", "foo", itemReader.read()); + } + + @Test + public void testNoSessionFactory() { + try { + new Neo4jItemReaderBuilder() + .targetType(String.class) + .startStatement("n=node(*)") + .returnStatement("*") + .orderByStatement("n.age") + .pageSize(50) + .name("bar").build(); + + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "sessionFactory is required.", iae.getMessage()); + } + } + + @Test + public void testZeroPageSize() { + validateExceptionMessage(new Neo4jItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .targetType(String.class) + .startStatement("n=node(*)") + .returnStatement("*") + .orderByStatement("n.age") + .pageSize(0) + .name("foo") + .matchStatement("n -- m") + .whereStatement("has(n.name)") + .returnStatement("m"), + "pageSize must be greater than zero"); + } + + @Test + public void testZeroMaxItemCount() { + validateExceptionMessage(new Neo4jItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .targetType(String.class) + .startStatement("n=node(*)") + .returnStatement("*") + .orderByStatement("n.age") + .pageSize(5) + .maxItemCount(0) + .name("foo") + .matchStatement("n -- m") + .whereStatement("has(n.name)") + .returnStatement("m"), + "maxItemCount must be greater than zero"); + } + + @Test + public void testCurrentItemCountGreaterThanMaxItemCount() { + validateExceptionMessage(new Neo4jItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .targetType(String.class) + .startStatement("n=node(*)") + .returnStatement("*") + .orderByStatement("n.age") + .pageSize(5) + .maxItemCount(5) + .currentItemCount(6) + .name("foo") + .matchStatement("n -- m") + .whereStatement("has(n.name)") + .returnStatement("m"), + "maxItemCount must be greater than currentItemCount"); + } + + @Test + public void testNullName() { + validateExceptionMessage( + new Neo4jItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .targetType(String.class) + .startStatement("n=node(*)") + .returnStatement("*") + .orderByStatement("n.age") + .pageSize(50), + "A name is required when saveState is set to true"); + + // tests that name is not required if saveState is set to false. + new Neo4jItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .targetType(String.class) + .startStatement("n=node(*)") + .returnStatement("*") + .orderByStatement("n.age") + .saveState(false) + .pageSize(50) + .build(); + } + + @Test + public void testNullTargetType() { + validateExceptionMessage( + new Neo4jItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .startStatement("n=node(*)") + .returnStatement("*") + .orderByStatement("n.age") + .pageSize(50) + .name("bar") + .matchStatement("n -- m") + .whereStatement("has(n.name)") + .returnStatement("m"), + "targetType is required."); + } + + @Test + public void testNullStartStatement() { + validateExceptionMessage( + new Neo4jItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .targetType(String.class) + .returnStatement("*") + .orderByStatement("n.age") + .pageSize(50).name("bar") + .matchStatement("n -- m") + .whereStatement("has(n.name)") + .returnStatement("m"), + "startStatement is required."); + } + + @Test + public void testNullReturnStatement() { + validateExceptionMessage(new Neo4jItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .targetType(String.class) + .startStatement("n=node(*)") + .orderByStatement("n.age") + .pageSize(50).name("bar") + .matchStatement("n -- m") + .whereStatement("has(n.name)"), "returnStatement is required."); + } + + @Test + public void testNullOrderByStatement() { + validateExceptionMessage( + new Neo4jItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .targetType(String.class) + .startStatement("n=node(*)") + .returnStatement("*") + .pageSize(50) + .name("bar") + .matchStatement("n -- m") + .whereStatement("has(n.name)") + .returnStatement("m"), + "orderByStatement is required."); + } + + private void validateExceptionMessage(Neo4jItemReaderBuilder builder, String message) { + try { + builder.build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", message, + iae.getMessage()); + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/Neo4jItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/Neo4jItemWriterBuilderTests.java new file mode 100644 index 0000000000..377adcdda2 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/Neo4jItemWriterBuilderTests.java @@ -0,0 +1,94 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import java.util.ArrayList; +import java.util.List; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; +import org.neo4j.ogm.session.Session; +import org.neo4j.ogm.session.SessionFactory; + +import org.springframework.batch.item.data.Neo4jItemWriter; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +/** + * @author Glenn Renfro + */ +public class Neo4jItemWriterBuilderTests { + + @Mock + private SessionFactory sessionFactory; + @Mock + private Session session; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + } + + @Test + public void testBasicWriter() throws Exception{ + Neo4jItemWriter writer = new Neo4jItemWriterBuilder().sessionFactory(this.sessionFactory).build(); + List items = new ArrayList<>(); + items.add("foo"); + items.add("bar"); + + when(this.sessionFactory.openSession()).thenReturn(this.session); + writer.write(items); + + verify(this.session).save("foo"); + verify(this.session).save("bar"); + verify(this.session, never()).delete("foo"); + verify(this.session, never()).delete("bar"); + } + + @Test + public void testBasicDelete() throws Exception{ + Neo4jItemWriter writer = new Neo4jItemWriterBuilder().delete(true).sessionFactory(this.sessionFactory).build(); + List items = new ArrayList<>(); + items.add("foo"); + items.add("bar"); + + when(this.sessionFactory.openSession()).thenReturn(this.session); + writer.write(items); + + verify(this.session).delete("foo"); + verify(this.session).delete("bar"); + verify(this.session, never()).save("foo"); + verify(this.session, never()).save("bar"); + } + + @Test + public void testNoSessionFactory() { + try { + new Neo4jItemWriterBuilder().build(); + fail("SessionFactory was not set but exception was not thrown."); + } catch (IllegalArgumentException iae) { + assertEquals("sessionFactory is required.", iae.getMessage()); + } + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/RepositoryItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/RepositoryItemReaderBuilderTests.java new file mode 100644 index 0000000000..a7daf8cf32 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/RepositoryItemReaderBuilderTests.java @@ -0,0 +1,296 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.ArgumentCaptor; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.data.RepositoryItemReader; +import org.springframework.data.domain.Page; +import org.springframework.data.domain.PageRequest; +import org.springframework.data.domain.Sort; +import org.springframework.data.repository.PagingAndSortingRepository; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.when; + +/** + * @author Glenn Renfro + * @author Drummond Dawson + */ +public class RepositoryItemReaderBuilderTests { + + private static final String ARG1 = "foo"; + private static final String ARG2 = "bar"; + private static final String ARG3 = "baz"; + + private static final String TEST_CONTENT = "FOOBAR"; + + @Mock + private TestRepository repository; + + @Mock + private Page page; + + private Map sorts; + + private ArgumentCaptor pageRequestContainer; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + this.sorts = new HashMap<>(); + this.sorts.put("id", Sort.Direction.ASC); + this.pageRequestContainer = ArgumentCaptor.forClass(PageRequest.class); + + List testResult = new ArrayList<>(); + testResult.add(TEST_CONTENT); + when(page.getContent()).thenReturn(testResult); + when(page.getSize()).thenReturn(5); + when(this.repository.foo(this.pageRequestContainer.capture())).thenReturn(this.page); + } + + @Test + public void testBasicRead() throws Exception { + RepositoryItemReader reader = new RepositoryItemReaderBuilder<>().repository(this.repository) + .sorts(this.sorts) + .maxItemCount(5) + .methodName("foo") + .name("bar") + .build(); + String result = (String) reader.read(); + assertEquals("Result returned from reader was not expected value.", TEST_CONTENT, result); + assertEquals("page size was not expected value.", 10, this.pageRequestContainer.getValue().getPageSize()); + } + + @Test + public void testRepositoryMethodReference() throws Exception { + RepositoryItemReaderBuilder.RepositoryMethodReference repositoryMethodReference = + new RepositoryItemReaderBuilder.RepositoryMethodReference<>(this.repository); + repositoryMethodReference.methodIs().foo(null); + RepositoryItemReader reader = new RepositoryItemReaderBuilder<>() + .repository(repositoryMethodReference) + .sorts(this.sorts) + .maxItemCount(5) + .name("bar").build(); + String result = (String) reader.read(); + assertEquals("Result returned from reader was not expected value.", TEST_CONTENT, result); + assertEquals("page size was not expected value.", 10, this.pageRequestContainer.getValue().getPageSize()); + } + + @Test + public void testRepositoryMethodReferenceWithArgs() throws Exception { + RepositoryItemReaderBuilder.RepositoryMethodReference repositoryMethodReference = + new RepositoryItemReaderBuilder.RepositoryMethodReference<>(this.repository); + repositoryMethodReference.methodIs().foo(ARG1, ARG2, ARG3, null); + RepositoryItemReader reader = new RepositoryItemReaderBuilder<>() + .repository(repositoryMethodReference) + .sorts(this.sorts) + .maxItemCount(5) + .name("bar").build(); + ArgumentCaptor arg1Captor = ArgumentCaptor.forClass(String.class); + ArgumentCaptor arg2Captor = ArgumentCaptor.forClass(String.class); + ArgumentCaptor arg3Captor = ArgumentCaptor.forClass(String.class); + when(this.repository.foo(arg1Captor.capture(), arg2Captor.capture(), arg3Captor.capture(), + this.pageRequestContainer.capture())).thenReturn(this.page); + + String result = (String) reader.read(); + assertEquals("Result returned from reader was not expected value.", TEST_CONTENT, result); + verifyMultiArgRead(arg1Captor, arg2Captor, arg3Captor, result); + } + + @Test + public void testCurrentItemCount() throws Exception { + RepositoryItemReader reader = new RepositoryItemReaderBuilder<>().repository(this.repository) + .sorts(this.sorts) + .currentItemCount(6) + .maxItemCount(5) + .methodName("foo") + .name("bar") + .build(); + assertNull("Result returned from reader was not null.", reader.read()); + } + + @Test + public void testPageSize() throws Exception { + RepositoryItemReader reader = new RepositoryItemReaderBuilder<>().repository(this.repository) + .sorts(this.sorts) + .maxItemCount(5) + .methodName("foo") + .name("bar") + .pageSize(2) + .build(); + reader.read(); + assertEquals("page size was not expected value.", 2, this.pageRequestContainer.getValue().getPageSize()); + } + + @Test + public void testNoMethodName() throws Exception { + try { + new RepositoryItemReaderBuilder<>().repository(this.repository) + .sorts(this.sorts) + .maxItemCount(10) + .build(); + + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "methodName is required.", iae.getMessage()); + } + try { + new RepositoryItemReaderBuilder<>().repository(this.repository) + .sorts(this.sorts) + .methodName("") + .maxItemCount(5) + .build(); + + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "methodName is required.", iae.getMessage()); + } + } + + @Test + public void testSaveState() throws Exception { + try { + new RepositoryItemReaderBuilder<>().repository(repository) + .methodName("foo") + .sorts(sorts) + .maxItemCount(5) + .build(); + + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalStateException ise) { + assertEquals("IllegalStateException name was not set when saveState was true.", + "A name is required when saveState is set to true.", ise.getMessage()); + } + // No IllegalStateException for a name that is not set, should not be thrown since + // saveState was false. + new RepositoryItemReaderBuilder<>().repository(repository) + .saveState(false) + .methodName("foo") + .sorts(sorts) + .maxItemCount(5) + .build(); + } + + @Test + public void testNullSort() throws Exception { + try { + new RepositoryItemReaderBuilder<>().repository(repository) + .methodName("foo") + .maxItemCount(5) + .build(); + + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException sorts did not match the expected result.", "sorts map is required.", + iae.getMessage()); + } + } + + @Test + public void testNoRepository() throws Exception { + try { + new RepositoryItemReaderBuilder<>().sorts(this.sorts) + .maxItemCount(10) + .methodName("foo") + .build(); + + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "repository is required.", iae.getMessage()); + } + } + + @Test + public void testArguments() throws Exception { + List args = new ArrayList<>(3); + args.add(ARG1); + args.add(ARG2); + args.add(ARG3); + ArgumentCaptor arg1Captor = ArgumentCaptor.forClass(String.class); + ArgumentCaptor arg2Captor = ArgumentCaptor.forClass(String.class); + ArgumentCaptor arg3Captor = ArgumentCaptor.forClass(String.class); + when(this.repository.foo(arg1Captor.capture(), arg2Captor.capture(), arg3Captor.capture(), + this.pageRequestContainer.capture())).thenReturn(this.page); + + RepositoryItemReader reader = new RepositoryItemReaderBuilder<>().repository(this.repository) + .sorts(this.sorts) + .maxItemCount(5) + .methodName("foo") + .name("bar") + .arguments(args) + .build(); + + String result = (String) reader.read(); + verifyMultiArgRead(arg1Captor, arg2Captor, arg3Captor, result); + } + + @Test + public void testVarargArguments() throws Exception { + ArgumentCaptor arg1Captor = ArgumentCaptor.forClass(String.class); + ArgumentCaptor arg2Captor = ArgumentCaptor.forClass(String.class); + ArgumentCaptor arg3Captor = ArgumentCaptor.forClass(String.class); + when(this.repository.foo(arg1Captor.capture(), arg2Captor.capture(), arg3Captor.capture(), + this.pageRequestContainer.capture())).thenReturn(this.page); + + RepositoryItemReader reader = new RepositoryItemReaderBuilder<>().repository(this.repository) + .sorts(this.sorts) + .maxItemCount(5) + .methodName("foo") + .name("bar") + .arguments(ARG1, ARG2, ARG3) + .build(); + + String result = (String) reader.read(); + verifyMultiArgRead(arg1Captor, arg2Captor, arg3Captor, result); + } + + public interface TestRepository extends PagingAndSortingRepository { + + Object foo(PageRequest request); + + Object foo(String arg1, String arg2, String arg3, PageRequest request); + } + + private void verifyMultiArgRead(ArgumentCaptor arg1Captor, ArgumentCaptor arg2Captor, ArgumentCaptor arg3Captor, String result) { + assertEquals("Result returned from reader was not expected value.", TEST_CONTENT, result); + assertEquals("ARG1 for calling method did not match expected result", ARG1, arg1Captor.getValue()); + assertEquals("ARG2 for calling method did not match expected result", ARG2, arg2Captor.getValue()); + assertEquals("ARG3 for calling method did not match expected result", ARG3, arg3Captor.getValue()); + assertEquals("Result Total Pages did not match expected result", 10, + this.pageRequestContainer.getValue().getPageSize()); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/RepositoryItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/RepositoryItemWriterBuilderTests.java new file mode 100644 index 0000000000..705f08afff --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/data/builder/RepositoryItemWriterBuilderTests.java @@ -0,0 +1,125 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.data.builder; + +import java.io.Serializable; +import java.util.Collections; +import java.util.List; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.data.RepositoryItemWriter; +import org.springframework.data.repository.CrudRepository; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.verify; + +/** + * @author Glenn Renfro + */ +public class RepositoryItemWriterBuilderTests { + @Mock + private TestRepository repository; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + } + + @Test + public void testNullRepository() throws Exception { + try { + new RepositoryItemWriterBuilder().methodName("save").build(); + + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "repository is required.", iae.getMessage()); + } + } + + @Test + public void testEmptyMethodName() throws Exception { + try { + new RepositoryItemWriterBuilder().repository(this.repository).build(); + + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "methodName is required.", iae.getMessage()); + } + } + + @Test + public void testWriteItems() throws Exception { + RepositoryItemWriter writer = new RepositoryItemWriterBuilder() + .methodName("save") + .repository(this.repository) + .build(); + + List items = Collections.singletonList("foo"); + + writer.write(items); + + verify(this.repository).save("foo"); + } + + @Test + public void testWriteItemsTestRepository() throws Exception { + RepositoryItemWriter writer = new RepositoryItemWriterBuilder() + .methodName("foo") + .repository(this.repository) + .build(); + + List items = Collections.singletonList("foo"); + + writer.write(items); + + verify(this.repository).foo("foo"); + } + + @Test + public void testWriteItemsTestRepositoryMethodIs() throws Exception { + RepositoryItemWriterBuilder.RepositoryMethodReference + repositoryMethodReference = new RepositoryItemWriterBuilder.RepositoryMethodReference<>( + this.repository); + repositoryMethodReference.methodIs().foo(null); + + RepositoryItemWriter writer = new RepositoryItemWriterBuilder() + .methodName("foo") + .repository(repositoryMethodReference) + .build(); + + List items = Collections.singletonList("foo"); + + writer.write(items); + + verify(this.repository).foo("foo"); + } + + public interface TestRepository extends CrudRepository { + + Object foo(String arg1); + + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractDataSourceItemReaderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractDataSourceItemReaderIntegrationTests.java index 94bbab6eb9..a0da997541 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractDataSourceItemReaderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractDataSourceItemReaderIntegrationTests.java @@ -1,22 +1,39 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.fail; - import javax.sql.DataSource; import org.junit.Before; import org.junit.Test; + import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ItemStream; import org.springframework.batch.item.sample.Foo; import org.springframework.beans.factory.InitializingBean; import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.annotation.DirtiesContext; import org.springframework.test.context.transaction.AfterTransaction; import org.springframework.transaction.annotation.Transactional; -import org.springframework.util.Assert; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; /** * Common scenarios for testing {@link ItemReader} implementations which read @@ -59,7 +76,9 @@ public void onTearDownAfterTransaction() throws Exception { /* * Regular scenario - read all rows and eventually return null. */ - @Transactional @Test + @Test + @Transactional + @DirtiesContext public void testNormalProcessing() throws Exception { getAsInitializingBean(reader).afterPropertiesSet(); getAsItemStream(reader).open(executionContext); @@ -87,7 +106,9 @@ public void testNormalProcessing() throws Exception { * source and restore from restart data - the new input source should * continue where the old one finished. */ - @Transactional @Test + @Test + @Transactional + @DirtiesContext public void testRestart() throws Exception { getAsItemStream(reader).open(executionContext); @@ -116,7 +137,9 @@ public void testRestart() throws Exception { * source and restore from restart data - the new input source should * continue where the old one finished. */ - @Transactional @Test + @Test + @Transactional + @DirtiesContext public void testRestartOnSecondPage() throws Exception { getAsItemStream(reader).open(executionContext); @@ -148,7 +171,9 @@ public void testRestartOnSecondPage() throws Exception { /* * Reading from an input source and then trying to restore causes an error. */ - @Transactional @Test + @Test + @Transactional + @DirtiesContext public void testInvalidRestore() throws Exception { getAsItemStream(reader).open(executionContext); @@ -182,7 +207,9 @@ public void testInvalidRestore() throws Exception { /* * Empty restart data should be handled gracefully. */ - @Transactional @Test + @Test + @Transactional + @DirtiesContext public void testRestoreFromEmptyData() throws Exception { getAsItemStream(reader).open(executionContext); @@ -194,7 +221,9 @@ public void testRestoreFromEmptyData() throws Exception { * Rollback scenario with restart - input source rollbacks to last * commit point. */ - @Transactional @Test + @Test + @Transactional + @DirtiesContext public void testRollbackAndRestart() throws Exception { getAsItemStream(reader).open(executionContext); @@ -204,10 +233,10 @@ public void testRollbackAndRestart() throws Exception { getAsItemStream(reader).update(executionContext); Foo foo2 = reader.read(); - Assert.state(!foo2.equals(foo1)); + assertTrue(!foo2.equals(foo1)); Foo foo3 = reader.read(); - Assert.state(!foo2.equals(foo3)); + assertTrue(!foo2.equals(foo3)); getAsItemStream(reader).close(); @@ -224,7 +253,9 @@ public void testRollbackAndRestart() throws Exception { * Rollback scenario with restart - input source rollbacks to last * commit point. */ - @Transactional @Test + @Test + @Transactional + @DirtiesContext public void testRollbackOnFirstChunkAndRestart() throws Exception { getAsItemStream(reader).open(executionContext); @@ -232,10 +263,10 @@ public void testRollbackOnFirstChunkAndRestart() throws Exception { Foo foo1 = reader.read(); Foo foo2 = reader.read(); - Assert.state(!foo2.equals(foo1)); + assertTrue(!foo2.equals(foo1)); Foo foo3 = reader.read(); - Assert.state(!foo2.equals(foo3)); + assertTrue(!foo2.equals(foo3)); getAsItemStream(reader).close(); @@ -247,8 +278,10 @@ public void testRollbackOnFirstChunkAndRestart() throws Exception { assertEquals(foo1, reader.read()); assertEquals(foo2, reader.read()); } - - @Transactional @Test + + @Test + @Transactional + @DirtiesContext public void testMultipleRestarts() throws Exception { getAsItemStream(reader).open(executionContext); @@ -259,10 +292,10 @@ public void testMultipleRestarts() throws Exception { getAsItemStream(reader).update(executionContext); Foo foo2 = reader.read(); - Assert.state(!foo2.equals(foo1)); + assertTrue(!foo2.equals(foo1)); Foo foo3 = reader.read(); - Assert.state(!foo2.equals(foo3)); + assertTrue(!foo2.equals(foo3)); getAsItemStream(reader).close(); @@ -289,12 +322,12 @@ public void testMultipleRestarts() throws Exception { assertEquals(5, foo5.getValue()); } - //set transaction to false and make sure the tests work @Test + @DirtiesContext public void testTransacted() throws Exception { if (reader instanceof JpaPagingItemReader) { - ((JpaPagingItemReader)reader).setTransacted(false); + ((JpaPagingItemReader)reader).setTransacted(false); this.testNormalProcessing(); }//end if } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractDatabaseItemStreamItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractDatabaseItemStreamItemReaderTests.java index 95fa9e37a3..74d6b3bade 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractDatabaseItemStreamItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractDatabaseItemStreamItemReaderTests.java @@ -1,18 +1,34 @@ +/* + * Copyright 2009-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; -import static org.junit.Assert.assertEquals; - import javax.sql.DataSource; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + import org.springframework.batch.item.AbstractItemStreamItemReaderTests; import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ItemStream; import org.springframework.batch.item.sample.Foo; import org.springframework.context.support.ClassPathXmlApplicationContext; -import org.junit.Before; -import org.junit.After; -import org.junit.Test; + +import static org.junit.Assert.assertEquals; public abstract class AbstractDatabaseItemStreamItemReaderTests extends AbstractItemStreamItemReaderTests { @@ -34,7 +50,6 @@ public void tearDown() throws Exception { /** * Sub-classes can override this and create their own context. - * @throws Exception */ protected void initializeContext() throws Exception { ctx = new ClassPathXmlApplicationContext("org/springframework/batch/item/database/data-source-context.xml"); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractGenericDataSourceItemReaderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractGenericDataSourceItemReaderIntegrationTests.java index 0be966f3f6..cd33037781 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractGenericDataSourceItemReaderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractGenericDataSourceItemReaderIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractHibernateCursorItemReaderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractHibernateCursorItemReaderIntegrationTests.java index e3f6147af6..ee9d444396 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractHibernateCursorItemReaderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractHibernateCursorItemReaderIntegrationTests.java @@ -1,12 +1,27 @@ +/* + * Copyright 2010-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import org.hibernate.SessionFactory; import org.hibernate.StatelessSession; + import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.sample.Foo; import org.springframework.core.io.ClassPathResource; -import org.springframework.core.io.Resource; -import org.springframework.orm.hibernate4.LocalSessionFactoryBean; +import org.springframework.orm.hibernate5.LocalSessionFactoryBean; /** * Tests for {@link HibernateCursorItemReader} using {@link StatelessSession}. @@ -22,13 +37,13 @@ protected ItemReader createItemReader() throws Exception { LocalSessionFactoryBean factoryBean = new LocalSessionFactoryBean(); factoryBean.setDataSource(dataSource); - factoryBean.setMappingLocations(new Resource[] { new ClassPathResource("Foo.hbm.xml", getClass()) }); + factoryBean.setMappingLocations(new ClassPathResource("Foo.hbm.xml", getClass())); customizeSessionFactory(factoryBean); factoryBean.afterPropertiesSet(); SessionFactory sessionFactory = factoryBean.getObject(); - HibernateCursorItemReader hibernateReader = new HibernateCursorItemReader(); + HibernateCursorItemReader hibernateReader = new HibernateCursorItemReader<>(); setQuery(hibernateReader); hibernateReader.setSessionFactory(sessionFactory); hibernateReader.setUseStatelessSession(isUseStatelessSession()); @@ -42,7 +57,7 @@ protected ItemReader createItemReader() throws Exception { protected void customizeSessionFactory(LocalSessionFactoryBean factoryBean) { } - protected void setQuery(HibernateCursorItemReader reader) throws Exception { + protected void setQuery(HibernateCursorItemReader reader) throws Exception { reader.setQueryString("from Foo"); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractJdbcItemReaderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractJdbcItemReaderIntegrationTests.java index 3e9847f742..6244e6d437 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractJdbcItemReaderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractJdbcItemReaderIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractJdbcPagingItemReaderParameterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractJdbcPagingItemReaderParameterTests.java new file mode 100644 index 0000000000..97277ad917 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractJdbcPagingItemReaderParameterTests.java @@ -0,0 +1,34 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database; + +import java.util.Collections; + +import org.junit.Test; + +/** + * @author Jimmy Praet + */ +public abstract class AbstractJdbcPagingItemReaderParameterTests extends AbstractPagingItemReaderParameterTests { + + @Override + @Test + public void testReadAfterJumpSecondPage() throws Exception { + executionContext.put(getName() + ".start.after", Collections.singletonMap("ID", 4)); + super.testReadAfterJumpSecondPage(); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractPagingItemReaderParameterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractPagingItemReaderParameterTests.java index d934daf91e..62b1f80aa4 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractPagingItemReaderParameterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/AbstractPagingItemReaderParameterTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import javax.sql.DataSource; @@ -38,6 +53,9 @@ public void testRead() throws Exception { ((ItemStream)tested).open(executionContext); + Foo foo2 = tested.read(); + Assert.assertEquals(2, foo2.getValue()); + Foo foo3 = tested.read(); Assert.assertEquals(3, foo3.getValue()); @@ -52,11 +70,14 @@ public void testRead() throws Exception { } @Test - public void testReadAfterJump() throws Exception { + public void testReadAfterJumpFirstPage() throws Exception { - executionContext.putInt(tested.getClass().getSimpleName()+".read.count", 2); + executionContext.putInt(getName()+".read.count", 2); ((ItemStream)tested).open(executionContext); + Foo foo4 = tested.read(); + Assert.assertEquals(4, foo4.getValue()); + Foo foo5 = tested.read(); Assert.assertEquals(5, foo5.getValue()); @@ -64,5 +85,22 @@ public void testReadAfterJump() throws Exception { Assert.assertNull(o); } + @Test + public void testReadAfterJumpSecondPage() throws Exception { + + executionContext.putInt(getName()+".read.count", 3); + ((ItemStream)tested).open(executionContext); + + Foo foo5 = tested.read(); + Assert.assertEquals(5, foo5.getValue()); + + Object o = tested.read(); + Assert.assertNull(o); + } + + protected String getName() { + return tested.getClass().getSimpleName(); + } + protected abstract AbstractPagingItemReader getItemReader() throws Exception; } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/CompositeKeyFooDao.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/CompositeKeyFooDao.java index cf119f63a2..6f7e88b920 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/CompositeKeyFooDao.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/CompositeKeyFooDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -44,9 +44,9 @@ public Foo getFoo(Object key) { Map keys = (Map)key; Object[] args = keys.values().toArray(); - RowMapper fooMapper = new RowMapper(){ + RowMapper fooMapper = new RowMapper(){ @Override - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { + public Foo mapRow(ResultSet rs, int rowNum) throws SQLException { Foo foo = new Foo(); foo.setId(rs.getInt(1)); foo.setName(rs.getString(2)); @@ -55,7 +55,7 @@ public Object mapRow(ResultSet rs, int rowNum) throws SQLException { } }; - return (Foo)getJdbcTemplate().query("SELECT ID, NAME, VALUE from T_FOOS where ID = ? and VALUE = ?", + return getJdbcTemplate().query("SELECT ID, NAME, VALUE from T_FOOS where ID = ? and VALUE = ?", args, fooMapper).get(0); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/ExtendedConnectionDataSourceProxyTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/ExtendedConnectionDataSourceProxyTests.java index 0094d0d032..164697d16c 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/ExtendedConnectionDataSourceProxyTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/ExtendedConnectionDataSourceProxyTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import static org.mockito.Mockito.mock; @@ -113,8 +128,7 @@ public void testOperationWithDirectCloseCall() throws SQLException { } @Test - @SuppressWarnings({"unchecked", "rawtypes"}) - public void testSupressOfCloseWithJdbcTemplate() throws Exception { + public void testSuppressOfCloseWithJdbcTemplate() throws Exception { Connection con = mock(Connection.class); DataSource ds = mock(DataSource.class); @@ -181,21 +195,21 @@ public void testSupressOfCloseWithJdbcTemplate() throws Exception { Connection connection = DataSourceUtils.getConnection(csds); csds.startCloseSuppression(connection); - tt.execute(new TransactionCallback() { + tt.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { template.queryForList("select baz from bar"); template.queryForList("select foo from bar"); return null; } }); - tt.execute(new TransactionCallback() { + tt.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { template.queryForList("select ham from foo"); - tt2.execute(new TransactionCallback() { + tt2.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { template.queryForList("select 1 from eggs"); return null; } @@ -204,9 +218,9 @@ public Object doInTransaction(TransactionStatus status) { return null; } }); - tt.execute(new TransactionCallback() { + tt.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { template.queryForList("select spam from ham"); return null; } @@ -335,6 +349,7 @@ public T unwrap(Class iface) throws SQLException { /** * Added due to JDK 7. */ + @SuppressWarnings("unused") public Logger getParentLogger() throws SQLFeatureNotSupportedException { throw new SQLFeatureNotSupportedException(); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/FooDao.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/FooDao.java index 00305952f8..c6d8d890d5 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/FooDao.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/FooDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/FooRowMapper.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/FooRowMapper.java index 5030556613..bcb64f6a8e 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/FooRowMapper.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/FooRowMapper.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import java.sql.ResultSet; @@ -7,10 +22,10 @@ import org.springframework.jdbc.core.RowMapper; -public class FooRowMapper implements RowMapper { +public class FooRowMapper implements RowMapper { @Override - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { + public Foo mapRow(ResultSet rs, int rowNum) throws SQLException { Foo foo = new Foo(); foo.setId(rs.getInt(1)); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderCommonTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderCommonTests.java index 33d05e8cf7..11c39c4623 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderCommonTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderCommonTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import org.hibernate.SessionFactory; @@ -6,7 +21,7 @@ import org.springframework.batch.item.sample.Foo; import org.springframework.core.io.ClassPathResource; import org.springframework.core.io.Resource; -import org.springframework.orm.hibernate4.LocalSessionFactoryBean; +import org.springframework.orm.hibernate5.LocalSessionFactoryBean; public class HibernateCursorItemReaderCommonTests extends AbstractDatabaseItemStreamItemReaderTests { @@ -17,7 +32,7 @@ protected ItemReader getItemReader() throws Exception { String hsqlQuery = "from Foo"; - HibernateCursorItemReader reader = new HibernateCursorItemReader(); + HibernateCursorItemReader reader = new HibernateCursorItemReader<>(); reader.setQueryString(hsqlQuery); reader.setSessionFactory(sessionFactory); reader.setUseStatelessSession(true); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderIntegrationTests.java index 9857b3d7fa..6ad8078a2c 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import static org.junit.Assert.fail; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderNamedQueryIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderNamedQueryIntegrationTests.java index 18f466c4a1..84b4048e84 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderNamedQueryIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderNamedQueryIntegrationTests.java @@ -1,12 +1,29 @@ +/* + * Copyright 2009-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; +import org.springframework.batch.item.sample.Foo; + /** * Tests {@link HibernateCursorItemReader} configured with named query. */ public class HibernateCursorItemReaderNamedQueryIntegrationTests extends AbstractHibernateCursorItemReaderIntegrationTests { @Override - protected void setQuery(HibernateCursorItemReader reader) { + protected void setQuery(HibernateCursorItemReader reader) { reader.setQueryName("allFoos"); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderNativeQueryIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderNativeQueryIntegrationTests.java index a1b450a493..3b06ea4711 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderNativeQueryIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderNativeQueryIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import org.springframework.batch.item.database.orm.HibernateNativeQueryProvider; @@ -10,13 +25,13 @@ public class HibernateCursorItemReaderNativeQueryIntegrationTests extends AbstractHibernateCursorItemReaderIntegrationTests { @Override - protected void setQuery(HibernateCursorItemReader hibernateReader) throws Exception { + protected void setQuery(HibernateCursorItemReader hibernateReader) throws Exception { String nativeQuery = "select * from T_FOOS"; //creating a native query provider as it would be created in configuration HibernateNativeQueryProvider queryProvider = - new HibernateNativeQueryProvider(); + new HibernateNativeQueryProvider<>(); queryProvider.setSqlQuery(nativeQuery); queryProvider.setEntityClass(Foo.class); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderParametersIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderParametersIntegrationTests.java index ae8c6284b9..2fb46bec1a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderParametersIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderParametersIntegrationTests.java @@ -1,9 +1,26 @@ +/* + * Copyright 2010-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import java.util.Collections; import org.hibernate.StatelessSession; +import org.springframework.batch.item.sample.Foo; + /** * Tests for {@link HibernateCursorItemReader} using {@link StatelessSession}. * @@ -14,9 +31,9 @@ public class HibernateCursorItemReaderParametersIntegrationTests extends AbstractHibernateCursorItemReaderIntegrationTests { @Override - protected void setQuery(HibernateCursorItemReader reader) { + protected void setQuery(HibernateCursorItemReader reader) { reader.setQueryString("from Foo where name like :name"); - reader.setParameterValues(Collections.singletonMap("name", (Object) "bar%")); + reader.setParameterValues(Collections.singletonMap("name", "bar%")); } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderStatefulIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderStatefulIntegrationTests.java index f0caa6df0d..5eb08e6f0a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderStatefulIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderStatefulIntegrationTests.java @@ -1,15 +1,31 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - -import org.hibernate.Query; import org.hibernate.Session; import org.hibernate.SessionFactory; +import org.hibernate.query.Query; import org.junit.Test; + import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.sample.Foo; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + /** * Tests for {@link HibernateCursorItemReader} using standard hibernate {@link Session}. * @@ -25,12 +41,13 @@ protected boolean isUseStatelessSession() { //Ensure close is called on the stateful session correctly. @Test + @SuppressWarnings("unchecked") public void testStatefulClose(){ SessionFactory sessionFactory = mock(SessionFactory.class); Session session = mock(Session.class); - Query scrollableResults = mock(Query.class); - HibernateCursorItemReader itemReader = new HibernateCursorItemReader(); + Query scrollableResults = mock(Query.class); + HibernateCursorItemReader itemReader = new HibernateCursorItemReader<>(); itemReader.setSessionFactory(sessionFactory); itemReader.setQueryString("testQuery"); itemReader.setUseStatelessSession(false); @@ -38,7 +55,6 @@ public void testStatefulClose(){ when(sessionFactory.openSession()).thenReturn(session); when(session.createQuery("testQuery")).thenReturn(scrollableResults); when(scrollableResults.setFetchSize(0)).thenReturn(scrollableResults); - when(session.close()).thenReturn(null); itemReader.open(new ExecutionContext()); itemReader.close(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderStatefulNamedQueryIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderStatefulNamedQueryIntegrationTests.java index cafc8c3477..b15ec16d85 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderStatefulNamedQueryIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorItemReaderStatefulNamedQueryIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; /** diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorProjectionItemReaderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorProjectionItemReaderIntegrationTests.java index 6f975f7ebb..5975b4a571 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorProjectionItemReaderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateCursorProjectionItemReaderIntegrationTests.java @@ -1,23 +1,39 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.fail; - import javax.sql.DataSource; import org.hibernate.SessionFactory; import org.hibernate.StatelessSession; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.item.ExecutionContext; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.core.io.ClassPathResource; import org.springframework.core.io.Resource; -import org.springframework.orm.hibernate4.LocalSessionFactoryBean; +import org.springframework.orm.hibernate5.LocalSessionFactoryBean; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.fail; + /** * Tests for {@link HibernateCursorItemReader} using {@link StatelessSession}. * @@ -52,7 +68,7 @@ private void initializeItemReader(HibernateCursorItemReader reader, @Test public void testMultipleItemsInProjection() throws Exception { - HibernateCursorItemReader reader = new HibernateCursorItemReader(); + HibernateCursorItemReader reader = new HibernateCursorItemReader<>(); initializeItemReader(reader, "select f.value, f.name from Foo f"); Object[] foo1 = reader.read(); assertEquals(1, foo1[0]); @@ -60,7 +76,7 @@ public void testMultipleItemsInProjection() throws Exception { @Test public void testSingleItemInProjection() throws Exception { - HibernateCursorItemReader reader = new HibernateCursorItemReader(); + HibernateCursorItemReader reader = new HibernateCursorItemReader<>(); initializeItemReader(reader, "select f.value from Foo f"); Object foo1 = reader.read(); assertEquals(1, foo1); @@ -68,7 +84,7 @@ public void testSingleItemInProjection() throws Exception { @Test public void testSingleItemInProjectionWithArrayType() throws Exception { - HibernateCursorItemReader reader = new HibernateCursorItemReader(); + HibernateCursorItemReader reader = new HibernateCursorItemReader<>(); initializeItemReader(reader, "select f.value from Foo f"); try { Object[] foo1 = reader.read(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateItemReaderHelperTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateItemReaderHelperTests.java index 94ce695ae1..6c73f1ceef 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateItemReaderHelperTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateItemReaderHelperTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,7 +36,7 @@ */ public class HibernateItemReaderHelperTests { - private HibernateItemReaderHelper helper = new HibernateItemReaderHelper(); + private HibernateItemReaderHelper helper = new HibernateItemReaderHelper<>(); private SessionFactory sessionFactory = mock(SessionFactory.class); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateItemWriterTests.java index 2725ac435e..b9b97a88ce 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernateItemWriterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,12 +15,6 @@ */ package org.springframework.batch.item.database; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - import java.util.Arrays; import java.util.Collections; import java.util.List; @@ -29,7 +23,12 @@ import org.hibernate.SessionFactory; import org.junit.Before; import org.junit.Test; -import org.springframework.orm.hibernate3.HibernateOperations; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; /** * @author Dave Syer @@ -39,8 +38,6 @@ */ public class HibernateItemWriterTests { - HibernateOperations ht; - HibernateItemWriter writer; SessionFactory factory; @@ -48,10 +45,11 @@ public class HibernateItemWriterTests { @Before public void setUp() throws Exception { - writer = new HibernateItemWriter(); - ht = mock(HibernateOperations.class,"ht"); + writer = new HibernateItemWriter<>(); factory = mock(SessionFactory.class); currentSession = mock(Session.class); + + when(this.factory.getCurrentSession()).thenReturn(this.currentSession); } /** @@ -62,14 +60,14 @@ public void setUp() throws Exception { */ @Test public void testAfterPropertiesSet() throws Exception { - writer = new HibernateItemWriter(); + writer = new HibernateItemWriter<>(); try { writer.afterPropertiesSet(); fail("Expected IllegalArgumentException"); } catch (IllegalStateException e) { // expected - assertTrue("Wrong message for exception: " + e.getMessage(), e.getMessage().indexOf("HibernateOperations") >= 0); + assertTrue("Wrong message for exception: " + e.getMessage(), e.getMessage().indexOf("SessionFactory") >= 0); } } @@ -81,29 +79,31 @@ public void testAfterPropertiesSet() throws Exception { */ @Test public void testAfterPropertiesSetWithDelegate() throws Exception { - writer.setHibernateTemplate(ht); + writer.setSessionFactory(this.factory); writer.afterPropertiesSet(); } + @SuppressWarnings("deprecation") @Test public void testWriteAndFlushSunnyDayHibernate3() throws Exception { - writer.setHibernateTemplate(ht); - when(ht.contains("foo")).thenReturn(true); - when(ht.contains("bar")).thenReturn(false); - ht.saveOrUpdate("bar"); - ht.flush(); - ht.clear(); + this.writer.setSessionFactory(this.factory); + when(this.currentSession.contains("foo")).thenReturn(true); + when(this.currentSession.contains("bar")).thenReturn(false); + this.currentSession.saveOrUpdate("bar"); + this.currentSession.flush(); + this.currentSession.clear(); List items = Arrays.asList(new String[] { "foo", "bar" }); writer.write(items); } + @SuppressWarnings("deprecation") @Test public void testWriteAndFlushWithFailureHibernate3() throws Exception { - writer.setHibernateTemplate(ht); + this.writer.setSessionFactory(this.factory); final RuntimeException ex = new RuntimeException("ERROR"); - when(ht.contains("foo")).thenThrow(ex); + when(this.currentSession.contains("foo")).thenThrow(ex); try { writer.write(Collections.singletonList("foo")); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernatePagingItemReaderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernatePagingItemReaderIntegrationTests.java index c3b8156889..5d4f627ce4 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernatePagingItemReaderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/HibernatePagingItemReaderIntegrationTests.java @@ -1,12 +1,27 @@ +/* + * Copyright 2010-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import org.hibernate.SessionFactory; import org.hibernate.StatelessSession; + import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.sample.Foo; import org.springframework.core.io.ClassPathResource; -import org.springframework.core.io.Resource; -import org.springframework.orm.hibernate4.LocalSessionFactoryBean; +import org.springframework.orm.hibernate5.LocalSessionFactoryBean; /** * Tests for {@link HibernateCursorItemReader} using {@link StatelessSession}. @@ -22,13 +37,13 @@ protected ItemReader createItemReader() throws Exception { LocalSessionFactoryBean factoryBean = new LocalSessionFactoryBean(); factoryBean.setDataSource(dataSource); - factoryBean.setMappingLocations(new Resource[] { new ClassPathResource("Foo.hbm.xml", getClass()) }); + factoryBean.setMappingLocations(new ClassPathResource("Foo.hbm.xml", getClass())); customizeSessionFactory(factoryBean); factoryBean.afterPropertiesSet(); SessionFactory sessionFactory = factoryBean.getObject(); - HibernatePagingItemReader hibernateReader = new HibernatePagingItemReader(); + HibernatePagingItemReader hibernateReader = new HibernatePagingItemReader<>(); setQuery(hibernateReader); hibernateReader.setPageSize(2); hibernateReader.setSessionFactory(sessionFactory); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/IbatisBatchItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/IbatisBatchItemWriterTests.java deleted file mode 100644 index 82bc1fdec2..0000000000 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/IbatisBatchItemWriterTests.java +++ /dev/null @@ -1,196 +0,0 @@ -/* - * Copyright 2006-2008 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.database; - -import static org.junit.Assert.*; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - -import java.util.Collections; -import java.util.List; - -import javax.sql.DataSource; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.dao.EmptyResultDataAccessException; -import org.springframework.orm.ibatis.SqlMapClientTemplate; - -import com.ibatis.sqlmap.client.SqlMapClient; -import com.ibatis.sqlmap.client.SqlMapSession; -import com.ibatis.sqlmap.engine.execution.BatchResult; - -/** - * @author Thomas Risberg - * @author Will Schipp - */ -public class IbatisBatchItemWriterTests { - - private IbatisBatchItemWriter writer = new IbatisBatchItemWriter(); - - private DataSource ds; - - private SqlMapClientTemplate smct; - - private SqlMapClient smc; - - private String statementId = "updateFoo"; - - @SuppressWarnings("unused") - private class Foo { - private Long id; - private String bar; - - public Foo(String bar) { - this.id = 1L; - this.bar = bar; - } - - public Long getId() { - return id; - } - - public void setId(Long id) { - this.id = id; - } - - public String getBar() { - return bar; - } - - public void setBar(String bar) { - this.bar = bar; - } - - @Override - public boolean equals(Object obj) { - if (!(obj instanceof Foo)) { - return false; - } - Foo compare = (Foo) obj; - if (this.bar.equals(compare.getBar())) { - return true; - } - return false; - } - - @Override - public int hashCode() { - return bar.hashCode(); - } - - } - - @Before - public void setUp() throws Exception { - smc = mock(SqlMapClient.class); - ds = mock(DataSource.class); - smct = new SqlMapClientTemplate(ds, smc); - writer.setStatementId(statementId); - writer.setSqlMapClientTemplate(smct); - writer.afterPropertiesSet(); - } - - /** - * Test method for - * {@link org.springframework.batch.item.database.JdbcBatchItemWriter#afterPropertiesSet()} - * . - * @throws Exception - */ - @Test - public void testAfterPropertiesSet() throws Exception { - writer = new IbatisBatchItemWriter(); - try { - writer.afterPropertiesSet(); - fail("Expected IllegalArgumentException"); - } - catch (IllegalArgumentException e) { - // expected - String message = e.getMessage(); - assertTrue("Message does not contain 'SqlMapClient'.", message.indexOf("SqlMapClient") >= 0); - } - writer.setSqlMapClientTemplate(smct); - try { - writer.afterPropertiesSet(); - fail("Expected IllegalArgumentException"); - } - catch (IllegalArgumentException e) { - // expected - String message = e.getMessage(); - assertTrue("Message does not contain 'statementId'.", message.indexOf("statementId") >= 0); - } - writer.setStatementId("statementId"); - writer.afterPropertiesSet(); - } - - @Test - public void testWriteAndFlush() throws Exception { - SqlMapSession sms = mock(SqlMapSession.class); - when(smc.openSession()).thenReturn(sms); - sms.close(); - when(sms.getCurrentConnection()).thenReturn(null); - sms.setUserConnection(null); - sms.startBatch(); - when(sms.update("updateFoo", new Foo("bar"))).thenReturn(-2); - List results = Collections.singletonList(new BatchResult("updateFoo", "update foo")); - results.get(0).setUpdateCounts(new int[] {1}); - when(sms.executeBatchDetailed()).thenReturn(results); - writer.write(Collections.singletonList(new Foo("bar"))); - } - - @Test - public void testWriteAndFlushWithEmptyUpdate() throws Exception { - SqlMapSession sms = mock(SqlMapSession.class); - when(smc.openSession()).thenReturn(sms); - sms.close(); - when(sms.getCurrentConnection()).thenReturn(null); - sms.setUserConnection(null); - sms.startBatch(); - when(sms.update("updateFoo", new Foo("bar"))).thenReturn(1); - List results = Collections.singletonList(new BatchResult("updateFoo", "update foo")); - results.get(0).setUpdateCounts(new int[] {0}); - when(sms.executeBatchDetailed()).thenReturn(results); - try { - writer.write(Collections.singletonList(new Foo("bar"))); - fail("Expected EmptyResultDataAccessException"); - } - catch (EmptyResultDataAccessException e) { - // expected - String message = e.getMessage(); - assertTrue("Wrong message: " + message, message.indexOf("did not update") >= 0); - } - } - - @Test - public void testWriteAndFlushWithFailure() throws Exception { - final RuntimeException ex = new RuntimeException("ERROR"); - SqlMapSession sms = mock(SqlMapSession.class); - when(smc.openSession()).thenReturn(sms); - sms.close(); - when(sms.getCurrentConnection()).thenReturn(null); - sms.setUserConnection(null); - sms.startBatch(); - when(sms.update("updateFoo", new Foo("bar"))).thenThrow(ex); - try { - writer.write(Collections.singletonList(new Foo("bar"))); - fail("Expected RuntimeException"); - } - catch (RuntimeException e) { - assertEquals("ERROR", e.getMessage()); - } - } - -} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/IbatisPagingItemReaderAsyncTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/IbatisPagingItemReaderAsyncTests.java deleted file mode 100644 index 79f9cbc0c0..0000000000 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/IbatisPagingItemReaderAsyncTests.java +++ /dev/null @@ -1,158 +0,0 @@ -package org.springframework.batch.item.database; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; - -import java.util.ArrayList; -import java.util.HashSet; -import java.util.List; -import java.util.Set; -import java.util.concurrent.Callable; -import java.util.concurrent.CompletionService; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.ExecutorCompletionService; -import java.util.concurrent.Executors; - -import javax.sql.DataSource; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.junit.After; -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.springframework.batch.item.sample.Foo; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.core.io.ClassPathResource; -import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.orm.ibatis.SqlMapClientFactoryBean; -import org.springframework.test.context.ContextConfiguration; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; - -import com.ibatis.sqlmap.client.SqlMapClient; -import org.springframework.test.jdbc.JdbcTestUtils; - -@RunWith(SpringJUnit4ClassRunner.class) -@ContextConfiguration(locations = "JdbcPagingItemReaderCommonTests-context.xml") -public class IbatisPagingItemReaderAsyncTests { - - /** - * The number of items to read - */ - private static final int ITEM_COUNT = 10; - - /** - * The number of threads to create - */ - private static final int THREAD_COUNT = 3; - - private static Log logger = LogFactory.getLog(IbatisPagingItemReaderAsyncTests.class); - - @Autowired - private DataSource dataSource; - - private int maxId; - - @Before - public void init() { - JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); - maxId = jdbcTemplate.queryForInt("SELECT MAX(ID) from T_FOOS"); - for (int i = maxId + 1; i <= ITEM_COUNT; i++) { - jdbcTemplate.update("INSERT into T_FOOS (ID,NAME,VALUE) values (?, ?, ?)", i, "foo" + i, i); - } - assertEquals(ITEM_COUNT, JdbcTestUtils.countRowsInTable(jdbcTemplate, "T_FOOS")); - } - - @After - public void destroy() { - JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); - jdbcTemplate.update("DELETE from T_FOOS where ID>?", maxId); - } - - @Test - public void testAsyncReader() throws Throwable { - List throwables = new ArrayList(); - int max = 10; - for (int i = 0; i < max; i++) { - try { - doTest(); - } - catch (Throwable e) { - throwables.add(e); - } - } - if (!throwables.isEmpty()) { - throw new IllegalStateException(String.format("Failed %d out of %d", throwables.size(), max), throwables - .get(0)); - } - } - - /** - * @throws Exception - * @throws InterruptedException - * @throws ExecutionException - */ - private void doTest() throws Exception, InterruptedException, ExecutionException { - final IbatisPagingItemReader reader = getItemReader(); - CompletionService> completionService = new ExecutorCompletionService>(Executors - .newFixedThreadPool(THREAD_COUNT)); - for (int i = 0; i < THREAD_COUNT; i++) { - completionService.submit(new Callable>() { - @Override - public List call() throws Exception { - List list = new ArrayList(); - Foo next = null; - do { - next = reader.read(); - Thread.sleep(10L); // try to make it fairer - logger.debug("Reading item: " + next); - if (next != null) { - list.add(next); - } - } while (next != null); - return list; - } - }); - } - int count = 0; - Set results = new HashSet(); - for (int i = 0; i < THREAD_COUNT; i++) { - List items = completionService.take().get(); - count += items.size(); - logger.debug("Finished items count: " + items.size()); - logger.debug("Finished items: " + items); - assertNotNull(items); - results.addAll(items); - } - assertEquals(ITEM_COUNT, count); - assertEquals(ITEM_COUNT, results.size()); - reader.close(); - } - - private IbatisPagingItemReader getItemReader() throws Exception { - SqlMapClientFactoryBean factory = new SqlMapClientFactoryBean(); - factory.setConfigLocation(new ClassPathResource("ibatis-config.xml", getClass())); - factory.setDataSource(dataSource); - factory.afterPropertiesSet(); - SqlMapClient sqlMapClient = createSqlMapClient(); - - IbatisPagingItemReader reader = new IbatisPagingItemReader(); - reader.setQueryId("getPagedFoos"); - reader.setPageSize(2); - reader.setSqlMapClient(sqlMapClient); - reader.setSaveState(true); - - reader.afterPropertiesSet(); - - return reader; - } - - private SqlMapClient createSqlMapClient() throws Exception { - SqlMapClientFactoryBean factory = new SqlMapClientFactoryBean(); - factory.setConfigLocation(new ClassPathResource("ibatis-config.xml", getClass())); - factory.setDataSource(dataSource); - factory.afterPropertiesSet(); - return (SqlMapClient) factory.getObject(); - } - -} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/IbatisPagingItemReaderCommonTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/IbatisPagingItemReaderCommonTests.java deleted file mode 100644 index 8bda98f181..0000000000 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/IbatisPagingItemReaderCommonTests.java +++ /dev/null @@ -1,54 +0,0 @@ -package org.springframework.batch.item.database; - -import org.junit.runners.JUnit4; -import org.junit.runner.RunWith; -import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.item.ItemReader; -import org.springframework.batch.item.sample.Foo; -import org.springframework.core.io.ClassPathResource; -import org.springframework.orm.ibatis.SqlMapClientFactoryBean; - -import com.ibatis.sqlmap.client.SqlMapClient; - -@RunWith(JUnit4.class) -public class IbatisPagingItemReaderCommonTests extends AbstractDatabaseItemStreamItemReaderTests { - - @Override - protected ItemReader getItemReader() throws Exception { - SqlMapClientFactoryBean factory = new SqlMapClientFactoryBean(); - factory.setConfigLocation(new ClassPathResource("ibatis-config.xml", getClass())); - factory.setDataSource(getDataSource()); - factory.afterPropertiesSet(); - SqlMapClient sqlMapClient = createSqlMapClient(); - - IbatisPagingItemReader reader = new IbatisPagingItemReader(); - reader.setQueryId("getPagedFoos"); - reader.setPageSize(2); - reader.setSqlMapClient(sqlMapClient); - reader.setSaveState(true); - - reader.afterPropertiesSet(); - - return reader; - } - - private SqlMapClient createSqlMapClient() throws Exception { - SqlMapClientFactoryBean factory = new SqlMapClientFactoryBean(); - factory.setConfigLocation(new ClassPathResource("ibatis-config.xml", getClass())); - factory.setDataSource(getDataSource()); - factory.afterPropertiesSet(); - return (SqlMapClient) factory.getObject(); - } - - @Override - protected void pointToEmptyInput(ItemReader tested) throws Exception { - IbatisPagingItemReader reader = (IbatisPagingItemReader) tested; - reader.close(); - - reader.setQueryId("getNoFoos"); - reader.afterPropertiesSet(); - - reader.open(new ExecutionContext()); - } - -} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/IbatisPagingItemReaderParameterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/IbatisPagingItemReaderParameterTests.java deleted file mode 100644 index 2657601027..0000000000 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/IbatisPagingItemReaderParameterTests.java +++ /dev/null @@ -1,45 +0,0 @@ -package org.springframework.batch.item.database; - -import java.util.Collections; - -import org.junit.runner.RunWith; -import org.springframework.batch.item.sample.Foo; -import org.springframework.core.io.ClassPathResource; -import org.springframework.orm.ibatis.SqlMapClientFactoryBean; -import org.springframework.test.context.ContextConfiguration; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; - -import com.ibatis.sqlmap.client.SqlMapClient; - -@RunWith(SpringJUnit4ClassRunner.class) -@ContextConfiguration(locations = "/org/springframework/batch/item/database/data-source-context.xml") -public class IbatisPagingItemReaderParameterTests extends AbstractPagingItemReaderParameterTests { - - @Override - protected AbstractPagingItemReader getItemReader() throws Exception { - SqlMapClientFactoryBean factory = new SqlMapClientFactoryBean(); - factory.setConfigLocation(new ClassPathResource("ibatis-config.xml", getClass())); - factory.setDataSource(dataSource); - factory.afterPropertiesSet(); - SqlMapClient sqlMapClient = createSqlMapClient(); - - IbatisPagingItemReader reader = new IbatisPagingItemReader(); - reader.setQueryId("getPagedFoos3AndUp"); - reader.setParameterValues(Collections.singletonMap("limit", 3)); - reader.setSqlMapClient(sqlMapClient); - reader.setSaveState(true); - - reader.afterPropertiesSet(); - - return reader; - } - - private SqlMapClient createSqlMapClient() throws Exception { - SqlMapClientFactoryBean factory = new SqlMapClientFactoryBean(); - factory.setConfigLocation(new ClassPathResource("ibatis-config.xml", getClass())); - factory.setDataSource(dataSource); - factory.afterPropertiesSet(); - return (SqlMapClient) factory.getObject(); - } - -} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcBatchItemWriterClassicTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcBatchItemWriterClassicTests.java index bbb47d23af..e4a58ab58a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcBatchItemWriterClassicTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcBatchItemWriterClassicTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -41,11 +41,11 @@ */ public class JdbcBatchItemWriterClassicTests { - private JdbcBatchItemWriter writer = new JdbcBatchItemWriter(); + private JdbcBatchItemWriter writer = new JdbcBatchItemWriter<>(); private JdbcTemplate jdbcTemplate; - protected List list = new ArrayList(); + protected List list = new ArrayList<>(); private PreparedStatement ps; @@ -54,7 +54,7 @@ public void setUp() throws Exception { ps = mock(PreparedStatement.class); jdbcTemplate = new JdbcTemplate() { @Override - public Object execute(String sql, PreparedStatementCallback action) throws DataAccessException { + public T execute(String sql, PreparedStatementCallback action) throws DataAccessException { list.add(sql); try { return action.doInPreparedStatement(ps); @@ -83,7 +83,7 @@ public void setValues(String item, PreparedStatement ps) throws SQLException { */ @Test public void testAfterPropertiesSet() throws Exception { - writer = new JdbcBatchItemWriter(); + writer = new JdbcBatchItemWriter<>(); try { writer.afterPropertiesSet(); fail("Expected IllegalArgumentException"); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcBatchItemWriterNamedParameterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcBatchItemWriterNamedParameterTests.java index 7163a631b9..7e14337c3e 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcBatchItemWriterNamedParameterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcBatchItemWriterNamedParameterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,14 +15,6 @@ */ package org.springframework.batch.item.database; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; -import static org.mockito.Matchers.argThat; -import static org.mockito.Matchers.eq; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - import java.util.Collections; import java.util.HashMap; import java.util.Map; @@ -32,20 +24,30 @@ import org.junit.Before; import org.junit.Test; import org.mockito.ArgumentCaptor; + import org.springframework.dao.EmptyResultDataAccessException; import org.springframework.jdbc.core.namedparam.BeanPropertySqlParameterSource; +import org.springframework.jdbc.core.namedparam.MapSqlParameterSource; import org.springframework.jdbc.core.namedparam.NamedParameterJdbcOperations; import org.springframework.jdbc.core.namedparam.SqlParameterSource; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; +import static org.mockito.hamcrest.MockitoHamcrest.argThat; + /** * @author Thomas Risberg * @author Will Schipp * @author Michael Minella */ -@SuppressWarnings({"rawtypes", "serial", "unchecked"}) public class JdbcBatchItemWriterNamedParameterTests { - private JdbcBatchItemWriter writer = new JdbcBatchItemWriter(); + private JdbcBatchItemWriter writer = new JdbcBatchItemWriter<>(); private NamedParameterJdbcOperations namedParameterJdbcOperations; @@ -85,7 +87,7 @@ public void setUp() throws Exception { writer.setSql(sql); writer.setJdbcTemplate(namedParameterJdbcOperations); writer.setItemSqlParameterSourceProvider( - new BeanPropertyItemSqlParameterSourceProvider()); + new BeanPropertyItemSqlParameterSourceProvider<>()); writer.afterPropertiesSet(); } @@ -93,11 +95,10 @@ public void setUp() throws Exception { * Test method for * {@link org.springframework.batch.item.database.JdbcBatchItemWriter#afterPropertiesSet()} * . - * @throws Exception */ @Test public void testAfterPropertiesSet() throws Exception { - writer = new JdbcBatchItemWriter(); + writer = new JdbcBatchItemWriter<>(); try { writer.afterPropertiesSet(); fail("Expected IllegalArgumentException"); @@ -105,7 +106,7 @@ public void testAfterPropertiesSet() throws Exception { catch (IllegalArgumentException e) { // expected String message = e.getMessage(); - assertTrue("Message does not contain 'NamedParameterJdbcTemplate'.", message.indexOf("NamedParameterJdbcTemplate") >= 0); + assertTrue("Message does not contain 'NamedParameterJdbcTemplate'.", message.contains("NamedParameterJdbcTemplate")); } writer.setJdbcTemplate(namedParameterJdbcOperations); try { @@ -115,7 +116,7 @@ public void testAfterPropertiesSet() throws Exception { catch (IllegalArgumentException e) { // expected String message = e.getMessage().toLowerCase(); - assertTrue("Message does not contain 'sql'.", message.indexOf("sql") >= 0); + assertTrue("Message does not contain 'sql'.", message.contains("sql")); } writer.setSql("select * from foo where id = :id"); @@ -124,25 +125,58 @@ public void testAfterPropertiesSet() throws Exception { @Test public void testWriteAndFlush() throws Exception { - writer.setItemSqlParameterSourceProvider(null); + when(namedParameterJdbcOperations.batchUpdate(eq(sql), + eqSqlParameterSourceArray(new SqlParameterSource[] {new BeanPropertySqlParameterSource(new Foo("bar"))}))) + .thenReturn(new int[] {1}); + writer.write(Collections.singletonList(new Foo("bar"))); + } + + @SuppressWarnings({ "rawtypes", "serial", "unchecked" }) + @Test + public void testWriteAndFlushMap() throws Exception { + JdbcBatchItemWriter> mapWriter = new JdbcBatchItemWriter<>(); + + mapWriter.setSql(sql); + mapWriter.setJdbcTemplate(namedParameterJdbcOperations); + mapWriter.afterPropertiesSet(); + ArgumentCaptor captor = ArgumentCaptor.forClass(Map[].class); when(namedParameterJdbcOperations.batchUpdate(eq(sql), captor.capture())) .thenReturn(new int[] {1}); - writer.write(Collections.singletonList(new HashMap() {{put("foo", "bar");}})); + mapWriter.write(Collections.singletonList(new HashMap() {{put("foo", "bar");}})); assertEquals(1, captor.getValue().length); Map results = captor.getValue()[0]; assertEquals("bar", results.get("foo")); } + @SuppressWarnings({ "rawtypes", "serial", "unchecked" }) @Test - public void testWriteAndFlushMap() throws Exception { - when(namedParameterJdbcOperations.batchUpdate(eq(sql), - eqSqlParameterSourceArray(new SqlParameterSource[] {new BeanPropertySqlParameterSource(new Foo("bar"))}))) + public void testWriteAndFlushMapWithItemSqlParameterSourceProvider() throws Exception { + JdbcBatchItemWriter> mapWriter = new JdbcBatchItemWriter<>(); + + mapWriter.setSql(sql); + mapWriter.setJdbcTemplate(namedParameterJdbcOperations); + mapWriter.setItemSqlParameterSourceProvider(new ItemSqlParameterSourceProvider>() { + @Override + public SqlParameterSource createSqlParameterSource(Map item) { + return new MapSqlParameterSource(item); + } + }); + mapWriter.afterPropertiesSet(); + + ArgumentCaptor captor = ArgumentCaptor.forClass(SqlParameterSource[].class); + + when(namedParameterJdbcOperations.batchUpdate(any(String.class), + captor.capture())) .thenReturn(new int[] {1}); - writer.write(Collections.singletonList(new Foo("bar"))); + mapWriter.write(Collections.singletonList(new HashMap() {{put("foo", "bar");}})); + + assertEquals(1, captor.getValue().length); + SqlParameterSource results = captor.getValue()[0]; + assertEquals("bar", results.getValue("foo")); } @Test @@ -157,7 +191,7 @@ public void testWriteAndFlushWithEmptyUpdate() throws Exception { catch (EmptyResultDataAccessException e) { // expected String message = e.getMessage(); - assertTrue("Wrong message: " + message, message.indexOf("did not update") >= 0); + assertTrue("Wrong message: " + message, message.contains("did not update")); } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcCursorItemReaderCommonTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcCursorItemReaderCommonTests.java index beff99fb90..beef9ab704 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcCursorItemReaderCommonTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcCursorItemReaderCommonTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import org.junit.Test; @@ -14,7 +29,7 @@ public class JdbcCursorItemReaderCommonTests extends AbstractDatabaseItemStreamI @Override protected ItemReader getItemReader() throws Exception { - JdbcCursorItemReader result = new JdbcCursorItemReader(); + JdbcCursorItemReader result = new JdbcCursorItemReader<>(); result.setDataSource(getDataSource()); result.setSql("select ID, NAME, VALUE from T_FOOS"); result.setIgnoreWarnings(true); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcCursorItemReaderConfigTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcCursorItemReaderConfigTests.java index 014f924144..b20e823514 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcCursorItemReaderConfigTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcCursorItemReaderConfigTests.java @@ -1,17 +1,30 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - import java.sql.Connection; import java.sql.PreparedStatement; import java.sql.ResultSet; - import javax.sql.DataSource; import org.junit.Test; -import org.junit.runners.JUnit4; import org.junit.runner.RunWith; +import org.junit.runners.JUnit4; +import org.mockito.ArgumentCaptor; + import org.springframework.batch.item.ExecutionContext; import org.springframework.jdbc.datasource.DataSourceTransactionManager; import org.springframework.transaction.PlatformTransactionManager; @@ -19,6 +32,13 @@ import org.springframework.transaction.support.TransactionCallback; import org.springframework.transaction.support.TransactionTemplate; +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.reset; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + @RunWith(JUnit4.class) public class JdbcCursorItemReaderConfigTests { @@ -38,15 +58,15 @@ public void testUsesCurrentTransaction() throws Exception { con.commit(); PlatformTransactionManager tm = new DataSourceTransactionManager(ds); TransactionTemplate tt = new TransactionTemplate(tm); - final JdbcCursorItemReader reader = new JdbcCursorItemReader(); + final JdbcCursorItemReader reader = new JdbcCursorItemReader<>(); reader.setDataSource(new ExtendedConnectionDataSourceProxy(ds)); reader.setUseSharedExtendedConnection(true); reader.setSql("select foo from bar"); final ExecutionContext ec = new ExecutionContext(); tt.execute( - new TransactionCallback() { + new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { reader.open(ec); reader.close(); return null; @@ -70,14 +90,14 @@ public void testUsesItsOwnTransaction() throws Exception { con.commit(); PlatformTransactionManager tm = new DataSourceTransactionManager(ds); TransactionTemplate tt = new TransactionTemplate(tm); - final JdbcCursorItemReader reader = new JdbcCursorItemReader(); + final JdbcCursorItemReader reader = new JdbcCursorItemReader<>(); reader.setDataSource(ds); reader.setSql("select foo from bar"); final ExecutionContext ec = new ExecutionContext(); tt.execute( - new TransactionCallback() { + new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { reader.open(ec); reader.close(); return null; @@ -85,4 +105,39 @@ public Object doInTransaction(TransactionStatus status) { }); } + @Test + public void testOverrideConnectionAutoCommit() throws Exception { + boolean initialAutoCommit= false; + boolean neededAutoCommit = true; + + DataSource ds = mock(DataSource.class); + Connection con = mock(Connection.class); + when(con.getAutoCommit()).thenReturn(initialAutoCommit); + PreparedStatement ps = mock(PreparedStatement.class); + when(con.prepareStatement("select foo from bar", ResultSet.TYPE_FORWARD_ONLY, + ResultSet.CONCUR_READ_ONLY)).thenReturn(ps); + when(ds.getConnection()).thenReturn(con); + + final JdbcCursorItemReader reader = new JdbcCursorItemReader<>(); + reader.setDataSource(ds); + reader.setSql("select foo from bar"); + reader.setConnectionAutoCommit(neededAutoCommit); + + // Check "open" outside of a transaction (see AbstractStep#execute()) + final ExecutionContext ec = new ExecutionContext(); + reader.open(ec); + + ArgumentCaptor autoCommitCaptor = ArgumentCaptor.forClass(Boolean.class); + verify(con, times(1)).setAutoCommit(autoCommitCaptor.capture()); + assertEquals(neededAutoCommit, autoCommitCaptor.getValue()); + + reset(con); + reader.close(); + + // Check restored autocommit value + autoCommitCaptor = ArgumentCaptor.forClass(Boolean.class); + verify(con, times(1)).setAutoCommit(autoCommitCaptor.capture()); + assertEquals(initialAutoCommit, autoCommitCaptor.getValue()); + } + } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcCursorItemReaderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcCursorItemReaderIntegrationTests.java index 7db814b489..15dba3a626 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcCursorItemReaderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcCursorItemReaderIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import org.junit.runner.RunWith; @@ -15,7 +30,7 @@ public class JdbcCursorItemReaderIntegrationTests extends AbstractGenericDataSou @Override protected ItemReader createItemReader() throws Exception { - JdbcCursorItemReader result = new JdbcCursorItemReader(); + JdbcCursorItemReader result = new JdbcCursorItemReader<>(); result.setDataSource(dataSource); result.setSql("select ID, NAME, VALUE from T_FOOS"); result.setIgnoreWarnings(true); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderAsyncTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderAsyncTests.java index c1b9ad4a7d..9113c20dcc 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderAsyncTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderAsyncTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import static org.junit.Assert.assertEquals; @@ -25,15 +40,16 @@ import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.database.support.HsqlPagingQueryProvider; import org.springframework.batch.item.sample.Foo; -import org.springframework.test.jdbc.JdbcTestUtils; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.test.jdbc.JdbcTestUtils; @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "JdbcPagingItemReaderCommonTests-context.xml") @@ -64,7 +80,7 @@ public class JdbcPagingItemReaderAsyncTests { @Before public void init() { JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); - maxId = jdbcTemplate.queryForInt("SELECT MAX(ID) from T_FOOS"); + maxId = jdbcTemplate.queryForObject("SELECT MAX(ID) from T_FOOS", Integer.class); for (int i = maxId + 1; i <= ITEM_COUNT; i++) { jdbcTemplate.update("INSERT into T_FOOS (ID,NAME,VALUE) values (?, ?, ?)", i, "foo" + i, i); } @@ -79,7 +95,7 @@ public void destroy() { @Test public void testAsyncReader() throws Throwable { - List throwables = new ArrayList(); + List throwables = new ArrayList<>(); int max = 10; for (int i = 0; i < max; i++) { try { @@ -102,13 +118,13 @@ public void testAsyncReader() throws Throwable { */ private void doTest() throws Exception, InterruptedException, ExecutionException { final ItemReader reader = getItemReader(); - CompletionService> completionService = new ExecutorCompletionService>(Executors + CompletionService> completionService = new ExecutorCompletionService<>(Executors .newFixedThreadPool(THREAD_COUNT)); for (int i = 0; i < THREAD_COUNT; i++) { completionService.submit(new Callable>() { @Override public List call() throws Exception { - List list = new ArrayList(); + List list = new ArrayList<>(); Foo next = null; do { next = reader.read(); @@ -123,7 +139,7 @@ public List call() throws Exception { }); } int count = 0; - Set results = new HashSet(); + Set results = new HashSet<>(); for (int i = 0; i < THREAD_COUNT; i++) { List items = completionService.take().get(); count += items.size(); @@ -138,16 +154,16 @@ public List call() throws Exception { protected ItemReader getItemReader() throws Exception { - JdbcPagingItemReader reader = new JdbcPagingItemReader(); + JdbcPagingItemReader reader = new JdbcPagingItemReader<>(); reader.setDataSource(dataSource); HsqlPagingQueryProvider queryProvider = new HsqlPagingQueryProvider(); queryProvider.setSelectClause("select ID, NAME, VALUE"); queryProvider.setFromClause("from T_FOOS"); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("ID", Order.ASCENDING); queryProvider.setSortKeys(sortKeys); reader.setQueryProvider(queryProvider); - reader.setRowMapper(new ParameterizedRowMapper() { + reader.setRowMapper(new RowMapper() { @Override public Foo mapRow(ResultSet rs, int i) throws SQLException { Foo foo = new Foo(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderClassicParameterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderClassicParameterTests.java index 8f6bcbcd61..ab88f64157 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderClassicParameterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderClassicParameterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,12 +21,15 @@ import java.util.LinkedHashMap; import java.util.Map; +import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.item.database.support.HsqlPagingQueryProvider; import org.springframework.batch.item.sample.Foo; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.test.util.ReflectionTestUtils; /** * @author Dave Syer @@ -36,24 +39,34 @@ */ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "/org/springframework/batch/item/database/JdbcPagingItemReaderParameterTests-context.xml") -public class JdbcPagingItemReaderClassicParameterTests extends AbstractPagingItemReaderParameterTests { +public class JdbcPagingItemReaderClassicParameterTests extends AbstractJdbcPagingItemReaderParameterTests { + // force jumpToItemQuery in JdbcPagingItemReader.doJumpToPage(int) + private static boolean forceJumpToItemQuery = false; + @Override protected AbstractPagingItemReader getItemReader() throws Exception { - - JdbcPagingItemReader reader = new JdbcPagingItemReader(); + JdbcPagingItemReader reader = new JdbcPagingItemReader() { + @Override + protected void doJumpToPage(int itemIndex) { + if (forceJumpToItemQuery) { + ReflectionTestUtils.setField(this, "startAfterValues", null); + } + super.doJumpToPage(itemIndex); + } + }; reader.setDataSource(dataSource); HsqlPagingQueryProvider queryProvider = new HsqlPagingQueryProvider(); queryProvider.setSelectClause("select ID, NAME, VALUE"); queryProvider.setFromClause("from T_FOOS"); queryProvider.setWhereClause("where VALUE >= ?"); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("ID", Order.ASCENDING); queryProvider.setSortKeys(sortKeys); - reader.setParameterValues(Collections.singletonMap("limit", 3)); + reader.setParameterValues(Collections.singletonMap("limit", 2)); reader.setQueryProvider(queryProvider); reader.setRowMapper( - new ParameterizedRowMapper() { + new RowMapper() { @Override public Foo mapRow(ResultSet rs, int i) throws SQLException { Foo foo = new Foo(); @@ -71,5 +84,20 @@ public Foo mapRow(ResultSet rs, int i) throws SQLException { return reader; } + + @Test + public void testReadAfterJumpSecondPageWithJumpToItemQuery() throws Exception { + try { + forceJumpToItemQuery = true; + super.testReadAfterJumpSecondPage(); + } finally { + forceJumpToItemQuery = false; + } + } + + @Override + protected String getName() { + return "JdbcPagingItemReader"; + } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests.java index 8950badab1..90d12caca0 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,13 +23,14 @@ import javax.sql.DataSource; import org.junit.runner.RunWith; + import org.springframework.batch.item.AbstractItemStreamItemReaderTests; import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.database.support.HsqlPagingQueryProvider; import org.springframework.batch.item.sample.Foo; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @@ -48,17 +49,17 @@ public class JdbcPagingItemReaderCommonTests extends AbstractItemStreamItemReade @Override protected ItemReader getItemReader() throws Exception { - JdbcPagingItemReader reader = new JdbcPagingItemReader(); + JdbcPagingItemReader reader = new JdbcPagingItemReader<>(); reader.setDataSource(dataSource); HsqlPagingQueryProvider queryProvider = new HsqlPagingQueryProvider(); queryProvider.setSelectClause("select ID, NAME, VALUE"); queryProvider.setFromClause("from T_FOOS"); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("ID", Order.ASCENDING); queryProvider.setSortKeys(sortKeys); reader.setQueryProvider(queryProvider); reader.setRowMapper( - new ParameterizedRowMapper() { + new RowMapper() { @Override public Foo mapRow(ResultSet rs, int i) throws SQLException { Foo foo = new Foo(); @@ -85,7 +86,7 @@ protected void pointToEmptyInput(ItemReader tested) throws Exception { queryProvider.setSelectClause("select ID, NAME, VALUE"); queryProvider.setFromClause("from T_FOOS"); queryProvider.setWhereClause("where ID = -1"); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("ID", Order.ASCENDING); queryProvider.setSortKeys(sortKeys); reader.setQueryProvider(queryProvider); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderConfigTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderConfigTests.java index 432cac21dc..2caf2b7c95 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderConfigTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderConfigTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import static org.junit.Assert.assertEquals; @@ -18,13 +33,13 @@ public class JdbcPagingItemReaderConfigTests { @Autowired - private JdbcPagingItemReader jdbcPagingItemReder; + private JdbcPagingItemReader jdbcPagingItemReader; @Test public void testConfig() { - assertNotNull(jdbcPagingItemReder); + assertNotNull(jdbcPagingItemReader); NamedParameterJdbcTemplate namedParameterJdbcTemplate = (NamedParameterJdbcTemplate) - ReflectionTestUtils.getField(jdbcPagingItemReder, "namedParameterJdbcTemplate"); + ReflectionTestUtils.getField(jdbcPagingItemReader, "namedParameterJdbcTemplate"); JdbcTemplate jdbcTemplate = (JdbcTemplate) namedParameterJdbcTemplate.getJdbcOperations(); assertEquals(1000, jdbcTemplate.getMaxRows()); assertEquals(100, jdbcTemplate.getFetchSize()); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderIntegrationTests.java index 02e23040d4..9d375a1515 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderIntegrationTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,7 +23,7 @@ import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.database.support.HsqlPagingQueryProvider; import org.springframework.batch.item.sample.Foo; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; /** * Tests for {@link JpaPagingItemReader}. @@ -36,17 +36,17 @@ public class JdbcPagingItemReaderIntegrationTests extends AbstractGenericDataSou @Override protected ItemReader createItemReader() throws Exception { - JdbcPagingItemReader inputSource = new JdbcPagingItemReader(); + JdbcPagingItemReader inputSource = new JdbcPagingItemReader<>(); inputSource.setDataSource(dataSource); HsqlPagingQueryProvider queryProvider = new HsqlPagingQueryProvider(); queryProvider.setSelectClause("select ID, NAME, VALUE"); queryProvider.setFromClause("from T_FOOS"); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("ID", Order.ASCENDING); queryProvider.setSortKeys(sortKeys); inputSource.setQueryProvider(queryProvider); inputSource.setRowMapper( - new ParameterizedRowMapper() { + new RowMapper() { @Override public Foo mapRow(ResultSet rs, int i) throws SQLException { Foo foo = new Foo(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderNamedParameterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderNamedParameterTests.java index a78fa9956a..5d3ac43157 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderNamedParameterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderNamedParameterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,12 +21,15 @@ import java.util.LinkedHashMap; import java.util.Map; +import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.item.database.support.HsqlPagingQueryProvider; import org.springframework.batch.item.sample.Foo; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.test.util.ReflectionTestUtils; /** * @author Dave Syer @@ -35,24 +38,34 @@ */ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "/org/springframework/batch/item/database/JdbcPagingItemReaderParameterTests-context.xml") -public class JdbcPagingItemReaderNamedParameterTests extends AbstractPagingItemReaderParameterTests { +public class JdbcPagingItemReaderNamedParameterTests extends AbstractJdbcPagingItemReaderParameterTests { + + // force jumpToItemQuery in JdbcPagingItemReader.doJumpToPage(int) + private static boolean forceJumpToItemQuery = false; @Override protected AbstractPagingItemReader getItemReader() throws Exception { - - JdbcPagingItemReader reader = new JdbcPagingItemReader(); + JdbcPagingItemReader reader = new JdbcPagingItemReader() { + @Override + protected void doJumpToPage(int itemIndex) { + if (forceJumpToItemQuery) { + ReflectionTestUtils.setField(this, "startAfterValues", null); + } + super.doJumpToPage(itemIndex); + } + }; reader.setDataSource(dataSource); HsqlPagingQueryProvider queryProvider = new HsqlPagingQueryProvider(); queryProvider.setSelectClause("select ID, NAME, VALUE"); queryProvider.setFromClause("from T_FOOS"); queryProvider.setWhereClause("where VALUE >= :limit"); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("ID", Order.ASCENDING); queryProvider.setSortKeys(sortKeys); - reader.setParameterValues(Collections.singletonMap("limit", 3)); + reader.setParameterValues(Collections.singletonMap("limit", 2)); reader.setQueryProvider(queryProvider); reader.setRowMapper( - new ParameterizedRowMapper() { + new RowMapper() { @Override public Foo mapRow(ResultSet rs, int i) throws SQLException { Foo foo = new Foo(); @@ -68,7 +81,22 @@ public Foo mapRow(ResultSet rs, int i) throws SQLException { reader.setSaveState(true); return reader; - + + } + + @Test + public void testReadAfterJumpSecondPageWithJumpToItemQuery() throws Exception { + try { + forceJumpToItemQuery = true; + super.testReadAfterJumpSecondPage(); + } finally { + forceJumpToItemQuery = false; + } } + + @Override + protected String getName() { + return "JdbcPagingItemReader"; + } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderOrderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderOrderIntegrationTests.java index 0839197dd9..c770817c18 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderOrderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcPagingItemReaderOrderIntegrationTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,7 +23,7 @@ import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.database.support.HsqlPagingQueryProvider; import org.springframework.batch.item.sample.Foo; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; /** * Tests for {@link JpaPagingItemReader} with sort key not equal to ID. @@ -36,18 +36,18 @@ public class JdbcPagingItemReaderOrderIntegrationTests extends AbstractGenericDa @Override protected ItemReader createItemReader() throws Exception { - JdbcPagingItemReader inputSource = new JdbcPagingItemReader(); + JdbcPagingItemReader inputSource = new JdbcPagingItemReader<>(); inputSource.setDataSource(dataSource); HsqlPagingQueryProvider queryProvider = new HsqlPagingQueryProvider(); queryProvider.setSelectClause("select ID, NAME, VALUE"); queryProvider.setFromClause("from T_FOOS"); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("VALUE", Order.ASCENDING); sortKeys.put("NAME", Order.DESCENDING); queryProvider.setSortKeys(sortKeys); inputSource.setQueryProvider(queryProvider); inputSource.setRowMapper( - new ParameterizedRowMapper() { + new RowMapper() { @Override public Foo mapRow(ResultSet rs, int i) throws SQLException { Foo foo = new Foo(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcParameterUtilsTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcParameterUtilsTests.java index ee30fc266a..33a318380b 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcParameterUtilsTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JdbcParameterUtilsTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -53,7 +53,7 @@ public void testCountParameterPlaceholders() { assertEquals(2, JdbcParameterUtils.countParameterPlaceholders("The big :parameter, &sameparameter, &sameparameter bad wolf", null)); assertEquals(2, JdbcParameterUtils.countParameterPlaceholders("The big :parameter, :sameparameter, :sameparameter bad wolf", null)); assertEquals(0, JdbcParameterUtils.countParameterPlaceholders("xxx & yyy", null)); - List l = new ArrayList(); + List l = new ArrayList<>(); assertEquals(3, JdbcParameterUtils.countParameterPlaceholders("select :par1, :par2 :par3", l)); assertEquals(3, l.size()); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaItemWriterTests.java index 00c4a788a0..032a1b6f19 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaItemWriterTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ import static org.junit.Assert.assertTrue; import static org.junit.Assert.fail; import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; import java.util.Arrays; @@ -36,7 +37,8 @@ /** * @author Thomas Risberg * @author Will Schipp - * + * @author Chris Cranford + * @author Mahmoud Ben Hassine */ public class JpaItemWriterTests { @@ -49,14 +51,14 @@ public void setUp() throws Exception { if (TransactionSynchronizationManager.isSynchronizationActive()) { TransactionSynchronizationManager.clearSynchronization(); } - writer = new JpaItemWriter(); + writer = new JpaItemWriter<>(); emf = mock(EntityManagerFactory.class,"emf"); writer.setEntityManagerFactory(emf); } @Test public void testAfterPropertiesSet() throws Exception { - writer = new JpaItemWriter(); + writer = new JpaItemWriter<>(); try { writer.afterPropertiesSet(); fail("Expected IllegalArgumentException"); @@ -84,6 +86,18 @@ public void testWriteAndFlushSunnyDay() throws Exception { TransactionSynchronizationManager.unbindResource(emf); } + @Test + public void testPersist() throws Exception { + writer.setUsePersist(true); + EntityManager em = mock(EntityManager.class, "em"); + TransactionSynchronizationManager.bindResource(emf, new EntityManagerHolder(em)); + List items = Arrays.asList("persist1", "persist2"); + writer.write(items); + verify(em).persist(items.get(0)); + verify(em).persist(items.get(1)); + TransactionSynchronizationManager.unbindResource(emf); + } + @Test public void testWriteAndFlushWithFailure() throws Exception { final RuntimeException ex = new RuntimeException("ERROR"); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderAsyncTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderAsyncTests.java index 73af883fae..f1642d121d 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderAsyncTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderAsyncTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import static org.junit.Assert.assertEquals; @@ -57,7 +72,7 @@ public class JpaPagingItemReaderAsyncTests { @Before public void init() { JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); - maxId = jdbcTemplate.queryForInt("SELECT MAX(ID) from T_FOOS"); + maxId = jdbcTemplate.queryForObject("SELECT MAX(ID) from T_FOOS", Integer.class); for (int i = maxId + 1; i <= ITEM_COUNT; i++) { jdbcTemplate.update("INSERT into T_FOOS (ID,NAME,VALUE) values (?, ?, ?)", i, "foo" + i, i); } @@ -72,7 +87,7 @@ public void destroy() { @Test public void testAsyncReader() throws Throwable { - List throwables = new ArrayList(); + List throwables = new ArrayList<>(); int max = 10; for (int i = 0; i < max; i++) { try { @@ -95,13 +110,13 @@ public void testAsyncReader() throws Throwable { */ private void doTest() throws Exception, InterruptedException, ExecutionException { final JpaPagingItemReader reader = getItemReader(); - CompletionService> completionService = new ExecutorCompletionService>(Executors + CompletionService> completionService = new ExecutorCompletionService<>(Executors .newFixedThreadPool(THREAD_COUNT)); for (int i = 0; i < THREAD_COUNT; i++) { completionService.submit(new Callable>() { @Override public List call() throws Exception { - List list = new ArrayList(); + List list = new ArrayList<>(); Foo next = null; do { next = reader.read(); @@ -116,7 +131,7 @@ public List call() throws Exception { }); } int count = 0; - Set results = new HashSet(); + Set results = new HashSet<>(); for (int i = 0; i < THREAD_COUNT; i++) { List items = completionService.take().get(); count += items.size(); @@ -134,7 +149,7 @@ private JpaPagingItemReader getItemReader() throws Exception { String jpqlQuery = "select f from Foo f"; - JpaPagingItemReader reader = new JpaPagingItemReader(); + JpaPagingItemReader reader = new JpaPagingItemReader<>(); reader.setQueryString(jpqlQuery); reader.setEntityManagerFactory(entityManagerFactory); reader.setPageSize(3); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests.java index 547aaaa5ca..9a7acc1db6 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import javax.persistence.EntityManagerFactory; @@ -23,7 +38,7 @@ protected ItemReader getItemReader() throws Exception { String jpqlQuery = "select f from Foo f"; - JpaPagingItemReader reader = new JpaPagingItemReader(); + JpaPagingItemReader reader = new JpaPagingItemReader<>(); reader.setQueryString(jpqlQuery); reader.setEntityManagerFactory(entityManagerFactory); reader.setPageSize(3); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderIntegrationTests.java index eb7ade6261..a3d13f64e5 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import java.util.Collections; @@ -28,9 +43,9 @@ protected ItemReader createItemReader() throws Exception { String jpqlQuery = "select f from Foo f where name like :name"; - JpaPagingItemReader inputSource = new JpaPagingItemReader(); + JpaPagingItemReader inputSource = new JpaPagingItemReader<>(); inputSource.setQueryString(jpqlQuery); - inputSource.setParameterValues(Collections.singletonMap("name", (Object)"bar%")); + inputSource.setParameterValues(Collections.singletonMap("name", "bar%")); inputSource.setEntityManagerFactory(entityManagerFactory); inputSource.setPageSize(3); inputSource.afterPropertiesSet(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderNativeQueryIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderNativeQueryIntegrationTests.java index cf01a2b960..eab7a7b140 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderNativeQueryIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderNativeQueryIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import java.util.Collections; @@ -27,15 +42,15 @@ protected AbstractPagingItemReader getItemReader() throws Exception { String sqlQuery = "select * from T_FOOS where value >= :limit"; - JpaPagingItemReader reader = new JpaPagingItemReader(); + JpaPagingItemReader reader = new JpaPagingItemReader<>(); //creating a native query provider as it would be created in configuration - JpaNativeQueryProvider queryProvider= new JpaNativeQueryProvider(); + JpaNativeQueryProvider queryProvider= new JpaNativeQueryProvider<>(); queryProvider.setSqlQuery(sqlQuery); queryProvider.setEntityClass(Foo.class); queryProvider.afterPropertiesSet(); - reader.setParameterValues(Collections.singletonMap("limit", 3)); + reader.setParameterValues(Collections.singletonMap("limit", 2)); reader.setEntityManagerFactory(entityManagerFactory); reader.setPageSize(3); reader.setQueryProvider(queryProvider); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderParameterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderParameterTests.java index ab407cbaab..8c474249e5 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderParameterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/JpaPagingItemReaderParameterTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import java.util.Collections; @@ -22,9 +37,9 @@ protected AbstractPagingItemReader getItemReader() throws Exception { String jpqlQuery = "select f from Foo f where f.value >= :limit"; - JpaPagingItemReader reader = new JpaPagingItemReader(); + JpaPagingItemReader reader = new JpaPagingItemReader<>(); reader.setQueryString(jpqlQuery); - reader.setParameterValues(Collections.singletonMap("limit", 3)); + reader.setParameterValues(Collections.singletonMap("limit", 2)); reader.setEntityManagerFactory(entityManagerFactory); reader.setPageSize(3); reader.afterPropertiesSet(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/SingleKeyFooDao.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/SingleKeyFooDao.java index 0004a8a67b..557da3a4d8 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/SingleKeyFooDao.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/SingleKeyFooDao.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import java.sql.ResultSet; @@ -12,9 +27,9 @@ public class SingleKeyFooDao extends JdbcDaoSupport implements FooDao { @Override public Foo getFoo(Object key){ - RowMapper fooMapper = new RowMapper(){ + RowMapper fooMapper = new RowMapper(){ @Override - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { + public Foo mapRow(ResultSet rs, int rowNum) throws SQLException { Foo foo = new Foo(); foo.setId(rs.getInt(1)); foo.setName(rs.getString(2)); @@ -23,7 +38,7 @@ public Object mapRow(ResultSet rs, int rowNum) throws SQLException { } }; - return (Foo)getJdbcTemplate().query("SELECT ID, NAME, VALUE from T_FOOS where ID = ?", + return getJdbcTemplate().query("SELECT ID, NAME, VALUE from T_FOOS where ID = ?", new Object[] {key}, fooMapper).get(0); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/StoredProcedureItemReaderCommonTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/StoredProcedureItemReaderCommonTests.java index 0c1c226ad0..fe1fa7ba46 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/StoredProcedureItemReaderCommonTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/StoredProcedureItemReaderCommonTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import java.sql.PreparedStatement; @@ -20,7 +35,7 @@ public class StoredProcedureItemReaderCommonTests extends AbstractDatabaseItemSt @Override protected ItemReader getItemReader() throws Exception { - StoredProcedureItemReader result = new StoredProcedureItemReader(); + StoredProcedureItemReader result = new StoredProcedureItemReader<>(); result.setDataSource(getDataSource()); result.setProcedureName("read_foos"); result.setRowMapper(new FooRowMapper()); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/StoredProcedureItemReaderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/StoredProcedureItemReaderIntegrationTests.java index 880c0d3bb1..939740c805 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/StoredProcedureItemReaderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/StoredProcedureItemReaderIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import org.junit.runner.RunWith; @@ -13,7 +28,7 @@ public class StoredProcedureItemReaderIntegrationTests @Override protected ItemReader createItemReader() throws Exception { - StoredProcedureItemReader reader = new StoredProcedureItemReader(); + StoredProcedureItemReader reader = new StoredProcedureItemReader<>(); reader.setDataSource(dataSource); reader.setProcedureName("read_foos"); reader.setRowMapper(new FooRowMapper()); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/StoredprocedureItemReaderConfigTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/StoredprocedureItemReaderConfigTests.java index 210f35f550..69af1ceb36 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/StoredprocedureItemReaderConfigTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/StoredprocedureItemReaderConfigTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.database; import static org.mockito.Mockito.mock; @@ -48,15 +63,15 @@ public void testUsesCurrentTransaction() throws Exception { con.commit(); PlatformTransactionManager tm = new DataSourceTransactionManager(ds); TransactionTemplate tt = new TransactionTemplate(tm); - final StoredProcedureItemReader reader = new StoredProcedureItemReader(); + final StoredProcedureItemReader reader = new StoredProcedureItemReader<>(); reader.setDataSource(new ExtendedConnectionDataSourceProxy(ds)); reader.setUseSharedExtendedConnection(true); reader.setProcedureName("foo_bar"); final ExecutionContext ec = new ExecutionContext(); tt.execute( - new TransactionCallback() { + new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { reader.open(ec); reader.close(); return null; @@ -84,14 +99,14 @@ public void testUsesItsOwnTransaction() throws Exception { con.commit(); PlatformTransactionManager tm = new DataSourceTransactionManager(ds); TransactionTemplate tt = new TransactionTemplate(tm); - final StoredProcedureItemReader reader = new StoredProcedureItemReader(); + final StoredProcedureItemReader reader = new StoredProcedureItemReader<>(); reader.setDataSource(ds); reader.setProcedureName("foo_bar"); final ExecutionContext ec = new ExecutionContext(); tt.execute( - new TransactionCallback() { + new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { reader.open(ec); reader.close(); return null; @@ -119,7 +134,7 @@ public void testHandlesRefCursorPosition() throws Exception { con.commit(); PlatformTransactionManager tm = new DataSourceTransactionManager(ds); TransactionTemplate tt = new TransactionTemplate(tm); - final StoredProcedureItemReader reader = new StoredProcedureItemReader(); + final StoredProcedureItemReader reader = new StoredProcedureItemReader<>(); reader.setDataSource(ds); reader.setProcedureName("foo_bar"); reader.setParameters(new SqlParameter[] { @@ -135,9 +150,9 @@ public void setValues(PreparedStatement ps) reader.setRefCursorPosition(3); final ExecutionContext ec = new ExecutionContext(); tt.execute( - new TransactionCallback() { + new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { reader.open(ec); reader.close(); return null; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/HibernateCursorItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/HibernateCursorItemReaderBuilderTests.java new file mode 100644 index 0000000000..3d95b8dc5c --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/HibernateCursorItemReaderBuilderTests.java @@ -0,0 +1,263 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.util.HashMap; +import java.util.Map; +import javax.sql.DataSource; + +import org.hibernate.SessionFactory; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.database.HibernateCursorItemReader; +import org.springframework.batch.item.database.orm.HibernateNativeQueryProvider; +import org.springframework.batch.item.sample.Foo; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.core.io.ClassPathResource; +import org.springframework.core.io.Resource; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory; +import org.springframework.jdbc.datasource.init.DataSourceInitializer; +import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator; +import org.springframework.orm.hibernate5.LocalSessionFactoryBean; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; + +/** + * @author Michael Minella + */ +public class HibernateCursorItemReaderBuilderTests { + + private SessionFactory sessionFactory; + + private ConfigurableApplicationContext context; + + @Before + public void setUp() { + this.context = new AnnotationConfigApplicationContext(TestDataSourceConfiguration.class); + this.sessionFactory = (SessionFactory) context.getBean("sessionFactory"); + } + + @After + public void tearDown() { + if(this.context != null) { + this.context.close(); + } + } + + @Test + public void testConfiguration() throws Exception { + HibernateCursorItemReader reader = new HibernateCursorItemReaderBuilder() + .name("fooReader") + .sessionFactory(this.sessionFactory) + .fetchSize(2) + .currentItemCount(2) + .maxItemCount(4) + .queryName("allFoos") + .useStatelessSession(true) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + + reader.open(executionContext); + Foo item1 = reader.read(); + Foo item2 = reader.read(); + assertNull(reader.read()); + reader.update(executionContext); + reader.close(); + + assertEquals(3, item1.getId()); + assertEquals("bar3", item1.getName()); + assertEquals(3, item1.getValue()); + assertEquals(4, item2.getId()); + assertEquals("bar4", item2.getName()); + assertEquals(4, item2.getValue()); + + assertEquals(2, executionContext.size()); + } + + @Test + public void testConfigurationNoSaveState() throws Exception { + Map parameters = new HashMap<>(); + parameters.put("value", 2); + + HibernateCursorItemReader reader = new HibernateCursorItemReaderBuilder() + .name("fooReader") + .sessionFactory(this.sessionFactory) + .queryString("from Foo foo where foo.id > :value") + .parameterValues(parameters) + .saveState(false) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + + reader.open(executionContext); + + int i = 0; + while(reader.read() != null) { + i++; + } + + reader.update(executionContext); + reader.close(); + + assertEquals(3, i); + assertEquals(0, executionContext.size()); + } + + @Test + public void testConfigurationQueryProvider() throws Exception { + + HibernateNativeQueryProvider provider = new HibernateNativeQueryProvider<>(); + provider.setEntityClass(Foo.class); + provider.setSqlQuery("select * from T_FOOS"); + provider.afterPropertiesSet(); + + HibernateCursorItemReader reader = new HibernateCursorItemReaderBuilder() + .name("fooReader") + .sessionFactory(this.sessionFactory) + .queryProvider(provider) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + + reader.open(executionContext); + + int i = 0; + while(reader.read() != null) { + i++; + } + + reader.update(executionContext); + reader.close(); + + assertEquals(5, i); + } + + @Test + public void testConfigurationNativeQuery() throws Exception { + HibernateCursorItemReader reader = new HibernateCursorItemReaderBuilder() + .name("fooReader") + .sessionFactory(this.sessionFactory) + .nativeQuery("select * from T_FOOS") + .entityClass(Foo.class) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + + reader.open(executionContext); + + int i = 0; + while(reader.read() != null) { + i++; + } + + reader.update(executionContext); + reader.close(); + + assertEquals(5, i); + } + + @Test + public void testValidation() { + try { + new HibernateCursorItemReaderBuilder().fetchSize(-2).build(); + fail("fetch size must be >= 0"); + } + catch (IllegalStateException ise) { + assertEquals("fetchSize must not be negative", ise.getMessage()); + } + + try { + new HibernateCursorItemReaderBuilder().build(); + fail("sessionFactory is required"); + } + catch (IllegalStateException ise) { + assertEquals("A SessionFactory must be provided", ise.getMessage()); + } + + try { + new HibernateCursorItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .saveState(true) + .build(); + fail("name is required when saveState is true"); + } + catch (IllegalStateException ise) { + assertEquals("A name is required when saveState is set to true.", ise.getMessage()); + } + + try { + new HibernateCursorItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .saveState(false) + .build(); + fail("A HibernateQueryProvider, queryName, queryString, " + + "or both the nativeQuery and entityClass must be configured"); + } + catch (IllegalStateException ise) { + assertEquals("A HibernateQueryProvider, queryName, queryString, " + + "or both the nativeQuery and entityClass must be configured", ise.getMessage()); + } + + } + + @Configuration + public static class TestDataSourceConfiguration { + + @Bean + public DataSource dataSource() { + return new EmbeddedDatabaseFactory().getDatabase(); + } + + @Bean + public DataSourceInitializer initializer(DataSource dataSource) { + DataSourceInitializer dataSourceInitializer = new DataSourceInitializer(); + dataSourceInitializer.setDataSource(dataSource); + + Resource create = new ClassPathResource("org/springframework/batch/item/database/init-foo-schema-hsqldb.sql"); + dataSourceInitializer.setDatabasePopulator(new ResourceDatabasePopulator(create)); + + return dataSourceInitializer; + } + + @Bean + public SessionFactory sessionFactory() throws Exception { + LocalSessionFactoryBean factoryBean = new LocalSessionFactoryBean(); + factoryBean.setDataSource(dataSource()); + factoryBean.setMappingLocations(new ClassPathResource("/org/springframework/batch/item/database/Foo.hbm.xml", getClass())); + factoryBean.afterPropertiesSet(); + + return factoryBean.getObject(); + + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/HibernateItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/HibernateItemWriterBuilderTests.java new file mode 100644 index 0000000000..ff76a491c0 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/HibernateItemWriterBuilderTests.java @@ -0,0 +1,114 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.util.ArrayList; +import java.util.List; + +import org.hibernate.Session; +import org.hibernate.SessionFactory; +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.database.HibernateItemWriter; +import org.springframework.batch.item.sample.Foo; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +/** + * @author Michael Minella + */ +public class HibernateItemWriterBuilderTests { + + @Mock + private SessionFactory sessionFactory; + + @Mock + private Session session; + + @Before + public void setUp() { + MockitoAnnotations.initMocks(this); + when(this.sessionFactory.getCurrentSession()).thenReturn(this.session); + } + + @Test + public void testConfiguration() { + HibernateItemWriter itemWriter = new HibernateItemWriterBuilder() + .sessionFactory(this.sessionFactory) + .build(); + + itemWriter.afterPropertiesSet(); + + List foos = getFoos(); + + itemWriter.write(foos); + + verify(this.session).saveOrUpdate(foos.get(0)); + verify(this.session).saveOrUpdate(foos.get(1)); + verify(this.session).saveOrUpdate(foos.get(2)); + } + + @Test + public void testConfigurationClearSession() { + HibernateItemWriter itemWriter = new HibernateItemWriterBuilder() + .sessionFactory(this.sessionFactory) + .clearSession(false) + .build(); + + itemWriter.afterPropertiesSet(); + + List foos = getFoos(); + + itemWriter.write(foos); + + verify(this.session).saveOrUpdate(foos.get(0)); + verify(this.session).saveOrUpdate(foos.get(1)); + verify(this.session).saveOrUpdate(foos.get(2)); + verify(this.session, never()).clear(); + } + + @Test + public void testValidation() { + try { + new HibernateItemWriterBuilder() + .build(); + fail("sessionFactory is required"); + } + catch (IllegalStateException ise) { + assertEquals("Incorrect message", "SessionFactory must be provided", ise.getMessage()); + } + } + + private List getFoos() { + List foos = new ArrayList<>(3); + + for(int i = 1; i < 4; i++) { + Foo foo = new Foo(); + foo.setName("foo" + i); + foo.setValue(i); + foos.add(foo); + } + + return foos; + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/HibernatePagingItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/HibernatePagingItemReaderBuilderTests.java new file mode 100644 index 0000000000..56cc4b497e --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/HibernatePagingItemReaderBuilderTests.java @@ -0,0 +1,247 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.util.HashMap; +import java.util.Map; +import javax.sql.DataSource; + +import org.hibernate.SessionFactory; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.database.HibernateItemReaderHelper; +import org.springframework.batch.item.database.HibernatePagingItemReader; +import org.springframework.batch.item.database.orm.HibernateNativeQueryProvider; +import org.springframework.batch.item.sample.Foo; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.core.io.ClassPathResource; +import org.springframework.core.io.Resource; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory; +import org.springframework.jdbc.datasource.init.DataSourceInitializer; +import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator; +import org.springframework.orm.hibernate5.LocalSessionFactoryBean; +import org.springframework.test.util.ReflectionTestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; + +/** + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +public class HibernatePagingItemReaderBuilderTests { + + private SessionFactory sessionFactory; + + private ConfigurableApplicationContext context; + + @Before + public void setUp() { + this.context = new AnnotationConfigApplicationContext(HibernatePagingItemReaderBuilderTests.TestDataSourceConfiguration.class); + this.sessionFactory = (SessionFactory) context.getBean("sessionFactory"); + } + + @After + public void tearDown() { + if(this.context != null) { + this.context.close(); + } + } + + @Test + @SuppressWarnings("unchecked") + public void testConfiguration() throws Exception { + HibernatePagingItemReader reader = new HibernatePagingItemReaderBuilder() + .name("fooReader") + .sessionFactory(this.sessionFactory) + .fetchSize(2) + .currentItemCount(2) + .maxItemCount(4) + .pageSize(5) + .queryName("allFoos") + .useStatelessSession(false) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + + reader.open(executionContext); + Foo item1 = reader.read(); + Foo item2 = reader.read(); + assertNull(reader.read()); + reader.update(executionContext); + reader.close(); + + assertEquals(3, item1.getId()); + assertEquals("bar3", item1.getName()); + assertEquals(3, item1.getValue()); + assertEquals(4, item2.getId()); + assertEquals("bar4", item2.getName()); + assertEquals(4, item2.getValue()); + + assertEquals(2, executionContext.size()); + assertEquals(5, ReflectionTestUtils.getField(reader, "pageSize")); + + HibernateItemReaderHelper helper = (HibernateItemReaderHelper) ReflectionTestUtils.getField(reader, "helper"); + assertEquals(false, ReflectionTestUtils.getField(helper, "useStatelessSession")); + } + + @Test + public void testConfigurationNoSaveState() throws Exception { + Map parameters = new HashMap<>(); + parameters.put("value", 2); + + HibernatePagingItemReader reader = new HibernatePagingItemReaderBuilder() + .name("fooReader") + .sessionFactory(this.sessionFactory) + .queryString("from Foo foo where foo.id > :value") + .parameterValues(parameters) + .saveState(false) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + + reader.open(executionContext); + + int i = 0; + while(reader.read() != null) { + i++; + } + + reader.update(executionContext); + reader.close(); + + assertEquals(3, i); + assertEquals(0, executionContext.size()); + } + + @Test + public void testConfigurationQueryProvider() throws Exception { + + HibernateNativeQueryProvider provider = new HibernateNativeQueryProvider<>(); + provider.setEntityClass(Foo.class); + provider.setSqlQuery("select * from T_FOOS"); + provider.afterPropertiesSet(); + + HibernatePagingItemReader reader = new HibernatePagingItemReaderBuilder() + .name("fooReader") + .sessionFactory(this.sessionFactory) + .queryProvider(provider) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + + reader.open(executionContext); + + int i = 0; + while(reader.read() != null) { + i++; + } + + reader.update(executionContext); + reader.close(); + + assertEquals(5, i); + } + + @Test + public void testValidation() { + try { + new HibernatePagingItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .fetchSize(-2) + .build(); + fail("fetch size must be >= 0"); + } + catch (IllegalStateException ise) { + assertEquals("fetchSize must not be negative", ise.getMessage()); + } + + try { + new HibernatePagingItemReaderBuilder().build(); + fail("A SessionFactory must be provided"); + } + catch (IllegalArgumentException ise) { + assertEquals("A SessionFactory must be provided", ise.getMessage()); + } + + try { + new HibernatePagingItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .saveState(true) + .build(); + fail("name is required when saveState is set to true"); + } + catch (IllegalArgumentException ise) { + assertEquals("A name is required when saveState is set to true", ise.getMessage()); + } + + try { + new HibernatePagingItemReaderBuilder() + .sessionFactory(this.sessionFactory) + .saveState(false) + .build(); + fail("queryString or queryName must be set"); + } + catch (IllegalStateException ise) { + assertEquals("queryString or queryName must be set", ise.getMessage()); + } + + } + + @Configuration + public static class TestDataSourceConfiguration { + + @Bean + public DataSource dataSource() { + return new EmbeddedDatabaseFactory().getDatabase(); + } + + @Bean + public DataSourceInitializer initializer(DataSource dataSource) { + DataSourceInitializer dataSourceInitializer = new DataSourceInitializer(); + dataSourceInitializer.setDataSource(dataSource); + + Resource create = new ClassPathResource("org/springframework/batch/item/database/init-foo-schema-hsqldb.sql"); + dataSourceInitializer.setDatabasePopulator(new ResourceDatabasePopulator(create)); + + return dataSourceInitializer; + } + + @Bean + public SessionFactory sessionFactory() throws Exception { + LocalSessionFactoryBean factoryBean = new LocalSessionFactoryBean(); + factoryBean.setDataSource(dataSource()); + factoryBean.setMappingLocations(new ClassPathResource("/org/springframework/batch/item/database/Foo.hbm.xml", getClass())); + factoryBean.afterPropertiesSet(); + + return factoryBean.getObject(); + + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JdbcBatchItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JdbcBatchItemWriterBuilderTests.java new file mode 100644 index 0000000000..ea07d509ba --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JdbcBatchItemWriterBuilderTests.java @@ -0,0 +1,330 @@ +/* + * Copyright 2016 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import javax.sql.DataSource; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.database.JdbcBatchItemWriter; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.Resource; +import org.springframework.dao.EmptyResultDataAccessException; +import org.springframework.jdbc.core.JdbcOperations; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.jdbc.core.namedparam.MapSqlParameterSource; +import org.springframework.jdbc.core.namedparam.NamedParameterJdbcOperations; +import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory; +import org.springframework.jdbc.datasource.init.DataSourceInitializer; +import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator; +import org.springframework.test.util.ReflectionTestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * @author Michael Minella + */ +public class JdbcBatchItemWriterBuilderTests { + + private DataSource dataSource; + + private ConfigurableApplicationContext context; + + @Before + public void setUp() { + this.context = new AnnotationConfigApplicationContext(TestDataSourceConfiguration.class); + this.dataSource = (DataSource) context.getBean("dataSource"); + } + + @After + public void tearDown() { + if(this.context != null) { + this.context.close(); + } + } + + @Test + public void testBasicMap() throws Exception { + JdbcBatchItemWriter> writer = new JdbcBatchItemWriterBuilder>() + .columnMapped() + .dataSource(this.dataSource) + .sql("INSERT INTO FOO (first, second, third) VALUES (:first, :second, :third)") + .build(); + + writer.afterPropertiesSet(); + + List> items = buildMapItems(); + writer.write(items); + + verifyWrite(); + } + + @Test + public void testCustomJdbcTemplate() throws Exception { + NamedParameterJdbcOperations template = new NamedParameterJdbcTemplate(this.dataSource); + + JdbcBatchItemWriter> writer = new JdbcBatchItemWriterBuilder>() + .columnMapped() + .namedParametersJdbcTemplate(template) + .sql("INSERT INTO FOO (first, second, third) VALUES (:first, :second, :third)") + .build(); + + writer.afterPropertiesSet(); + + List> items = buildMapItems(); + writer.write(items); + + verifyWrite(); + + Object usedTemplate = ReflectionTestUtils.getField(writer, "namedParameterJdbcTemplate"); + assertTrue(template == usedTemplate); + } + + @Test + public void testBasicPojo() throws Exception { + JdbcBatchItemWriter writer = new JdbcBatchItemWriterBuilder() + .beanMapped() + .dataSource(this.dataSource) + .sql("INSERT INTO FOO (first, second, third) VALUES (:first, :second, :third)") + .build(); + + writer.afterPropertiesSet(); + + List items = new ArrayList<>(3); + + items.add(new Foo(1, "two", "three")); + items.add(new Foo(4, "five", "six")); + items.add(new Foo(7, "eight", "nine")); + + writer.write(items); + + verifyWrite(); + } + + @Test(expected = EmptyResultDataAccessException.class) + public void testAssertUpdates() throws Exception { + JdbcBatchItemWriter writer = new JdbcBatchItemWriterBuilder() + .beanMapped() + .dataSource(this.dataSource) + .sql("UPDATE FOO SET second = :second, third = :third WHERE first = :first") + .assertUpdates(true) + .build(); + + writer.afterPropertiesSet(); + + List items = new ArrayList<>(1); + + items.add(new Foo(1, "two", "three")); + + writer.write(items); + } + + @Test + public void testCustomPreparedStatementSetter() throws Exception { + JdbcBatchItemWriter> writer = new JdbcBatchItemWriterBuilder>() + .itemPreparedStatementSetter((item, ps) -> { + ps.setInt(0, (int) item.get("first")); + ps.setString(1, (String) item.get("second")); + ps.setString(2, (String) item.get("third")); + }) + .dataSource(this.dataSource) + .sql("INSERT INTO FOO (first, second, third) VALUES (:first, :second, :third)") + .build(); + + writer.afterPropertiesSet(); + + List> items = buildMapItems(); + writer.write(items); + + verifyWrite(); + } + + @Test + public void testCustomPSqlParameterSourceProvider() throws Exception { + JdbcBatchItemWriter> writer = new JdbcBatchItemWriterBuilder>() + .itemSqlParameterSourceProvider(MapSqlParameterSource::new) + .dataSource(this.dataSource) + .sql("INSERT INTO FOO (first, second, third) VALUES (:first, :second, :third)") + .build(); + + writer.afterPropertiesSet(); + + List> items = buildMapItems(); + writer.write(items); + + verifyWrite(); + } + + @Test + public void testBuildAssertions() { + try { + new JdbcBatchItemWriterBuilder>() + .itemSqlParameterSourceProvider(MapSqlParameterSource::new) + .build(); + } + catch (IllegalStateException ise) { + assertEquals("Either a DataSource or a NamedParameterJdbcTemplate is required", + ise.getMessage()); + } + catch (Exception e) { + fail("Incorrect exception was thrown when missing DataSource and JdbcTemplate: " + + e.getMessage()); + } + + try { + new JdbcBatchItemWriterBuilder>() + .itemSqlParameterSourceProvider(MapSqlParameterSource::new) + .dataSource(this.dataSource) + .build(); + } + catch (IllegalArgumentException ise) { + assertEquals("A SQL statement is required", ise.getMessage()); + } + catch (Exception e) { + fail("Incorrect exception was thrown when testing missing SQL: " + + e); + } + + try { + new JdbcBatchItemWriterBuilder>() + .dataSource(this.dataSource) + .sql("INSERT INTO FOO VALUES (?, ?, ?)") + .columnMapped() + .beanMapped() + .build(); + } + catch (IllegalStateException ise) { + assertEquals("Either an item can be mapped via db column or via bean spec, can't be both", + ise.getMessage()); + } + catch (Exception e) { + fail("Incorrect exception was thrown both mapping types are used" + + e.getMessage()); + } + } + + private void verifyWrite() { + verifyRow(1, "two", "three"); + verifyRow(4, "five", "six"); + verifyRow(7, "eight", "nine"); + } + + private List> buildMapItems() { + List> items = new ArrayList<>(3); + + Map item = new HashMap<>(3); + item.put("first", 1); + item.put("second", "two"); + item.put("third", "three"); + items.add(item); + + item = new HashMap<>(3); + item.put("first", 4); + item.put("second", "five"); + item.put("third", "six"); + items.add(item); + + item = new HashMap<>(3); + item.put("first", 7); + item.put("second", "eight"); + item.put("third", "nine"); + items.add(item); + return items; + } + + private void verifyRow(int i, String i1, String nine) { + JdbcOperations template = new JdbcTemplate(this.dataSource); + + assertEquals(1, (int) template.queryForObject( + "select count(*) from foo where first = ? and second = ? and third = ?", + new Object[] {i, i1, nine}, Integer.class)); + } + + public static class Foo { + private int first; + private String second; + private String third; + + public Foo(int first, String second, String third) { + this.first = first; + this.second = second; + this.third = third; + } + + public int getFirst() { + return first; + } + + public void setFirst(int first) { + this.first = first; + } + + public String getSecond() { + return second; + } + + public void setSecond(String second) { + this.second = second; + } + + public String getThird() { + return third; + } + + public void setThird(String third) { + this.third = third; + } + } + + @Configuration + public static class TestDataSourceConfiguration { + + private static final String CREATE_SQL = "CREATE TABLE FOO (\n" + + "\tID BIGINT IDENTITY NOT NULL PRIMARY KEY ,\n" + + "\tFIRST BIGINT ,\n" + + "\tSECOND VARCHAR(5) NOT NULL,\n" + + "\tTHIRD VARCHAR(5) NOT NULL) ;"; + + @Bean + public DataSource dataSource() { + return new EmbeddedDatabaseFactory().getDatabase(); + } + + @Bean + public DataSourceInitializer initializer(DataSource dataSource) { + DataSourceInitializer dataSourceInitializer = new DataSourceInitializer(); + dataSourceInitializer.setDataSource(dataSource); + + Resource create = new ByteArrayResource(CREATE_SQL.getBytes()); + dataSourceInitializer.setDatabasePopulator(new ResourceDatabasePopulator(create)); + + return dataSourceInitializer; + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JdbcCursorItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JdbcCursorItemReaderBuilderTests.java new file mode 100644 index 0000000000..24a0452db5 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JdbcCursorItemReaderBuilderTests.java @@ -0,0 +1,435 @@ +/* + * Copyright 2016-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.sql.PreparedStatement; +import java.sql.SQLException; +import java.sql.Types; +import java.util.Arrays; +import javax.sql.DataSource; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.database.JdbcCursorItemReader; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.Resource; +import org.springframework.jdbc.core.PreparedStatementSetter; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory; +import org.springframework.jdbc.datasource.init.DataSourceInitializer; +import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator; +import org.springframework.test.util.ReflectionTestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * @author Michael Minella + * @author Drummond Dawson + */ +public class JdbcCursorItemReaderBuilderTests { + + private DataSource dataSource; + + private ConfigurableApplicationContext context; + + @Before + public void setUp() { + this.context = new AnnotationConfigApplicationContext(TestDataSourceConfiguration.class); + this.dataSource = (DataSource) context.getBean("dataSource"); + } + + @After + public void tearDown() { + if(this.context != null) { + this.context.close(); + } + } + + @Test + public void testSimpleScenario() throws Exception { + JdbcCursorItemReader reader = new JdbcCursorItemReaderBuilder() + .dataSource(this.dataSource) + .name("fooReader") + .sql("SELECT * FROM FOO ORDER BY FIRST") + .rowMapper((rs, rowNum) -> { + Foo foo = new Foo(); + + foo.setFirst(rs.getInt("FIRST")); + foo.setSecond(rs.getString("SECOND")); + foo.setThird(rs.getString("THIRD")); + + return foo; + }) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + + validateFoo(reader.read(), 1, "2", "3"); + validateFoo(reader.read(), 4, "5", "6"); + validateFoo(reader.read(), 7, "8", "9"); + + assertNull(reader.read()); + } + + @Test + public void testMaxRows() throws Exception { + JdbcCursorItemReader reader = new JdbcCursorItemReaderBuilder() + .dataSource(this.dataSource) + .name("fooReader") + .sql("SELECT * FROM FOO ORDER BY FIRST") + .maxRows(2) + .saveState(false) + .rowMapper((rs, rowNum) -> { + Foo foo = new Foo(); + + foo.setFirst(rs.getInt("FIRST")); + foo.setSecond(rs.getString("SECOND")); + foo.setThird(rs.getString("THIRD")); + + return foo; + }) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + + validateFoo(reader.read(), 1, "2", "3"); + validateFoo(reader.read(), 4, "5", "6"); + assertNull(reader.read()); + + reader.close(); + assertEquals(0, executionContext.size()); + } + + @Test + public void testQueryArgumentsList() throws Exception { + JdbcCursorItemReader reader = new JdbcCursorItemReaderBuilder() + .dataSource(this.dataSource) + .name("fooReader") + .sql("SELECT * FROM FOO WHERE FIRST > ? ORDER BY FIRST") + .queryArguments(Arrays.asList(3)) + .rowMapper((rs, rowNum) -> { + Foo foo = new Foo(); + + foo.setFirst(rs.getInt("FIRST")); + foo.setSecond(rs.getString("SECOND")); + foo.setThird(rs.getString("THIRD")); + + return foo; + }) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + + validateFoo(reader.read(), 4, "5", "6"); + validateFoo(reader.read(), 7, "8", "9"); + + assertNull(reader.read()); + } + + @Test + public void testQueryArgumentsArray() throws Exception { + JdbcCursorItemReader reader = new JdbcCursorItemReaderBuilder() + .dataSource(this.dataSource) + .name("fooReader") + .sql("SELECT * FROM FOO WHERE FIRST > ? ORDER BY FIRST") + .queryArguments(3) + .rowMapper((rs, rowNum) -> { + Foo foo = new Foo(); + + foo.setFirst(rs.getInt("FIRST")); + foo.setSecond(rs.getString("SECOND")); + foo.setThird(rs.getString("THIRD")); + + return foo; + }) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + + validateFoo(reader.read(), 4, "5", "6"); + validateFoo(reader.read(), 7, "8", "9"); + + assertNull(reader.read()); + } + + @Test + public void testQueryArgumentsTypedArray() throws Exception { + JdbcCursorItemReader reader = new JdbcCursorItemReaderBuilder() + .dataSource(this.dataSource) + .name("fooReader") + .sql("SELECT * FROM FOO WHERE FIRST > ? ORDER BY FIRST") + .queryArguments(new Integer[] {3}, new int[] {Types.BIGINT}) + .rowMapper((rs, rowNum) -> { + Foo foo = new Foo(); + + foo.setFirst(rs.getInt("FIRST")); + foo.setSecond(rs.getString("SECOND")); + foo.setThird(rs.getString("THIRD")); + + return foo; + }) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + + validateFoo(reader.read(), 4, "5", "6"); + validateFoo(reader.read(), 7, "8", "9"); + + assertNull(reader.read()); + } + + @Test + public void testPreparedStatementSetter() throws Exception { + JdbcCursorItemReader reader = new JdbcCursorItemReaderBuilder() + .dataSource(this.dataSource) + .name("fooReader") + .sql("SELECT * FROM FOO WHERE FIRST > ? ORDER BY FIRST") + .preparedStatementSetter(new PreparedStatementSetter() { + @Override + public void setValues(PreparedStatement ps) throws SQLException { + ps.setInt(1, 3); + } + }) + .rowMapper((rs, rowNum) -> { + Foo foo = new Foo(); + + foo.setFirst(rs.getInt("FIRST")); + foo.setSecond(rs.getString("SECOND")); + foo.setThird(rs.getString("THIRD")); + + return foo; + }) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + + validateFoo(reader.read(), 4, "5", "6"); + validateFoo(reader.read(), 7, "8", "9"); + + assertNull(reader.read()); + } + + @Test + public void testMaxItemCount() throws Exception { + JdbcCursorItemReader reader = new JdbcCursorItemReaderBuilder() + .dataSource(this.dataSource) + .name("fooReader") + .sql("SELECT * FROM FOO ORDER BY FIRST") + .maxItemCount(2) + .rowMapper((rs, rowNum) -> { + Foo foo = new Foo(); + + foo.setFirst(rs.getInt("FIRST")); + foo.setSecond(rs.getString("SECOND")); + foo.setThird(rs.getString("THIRD")); + + return foo; + }) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + + validateFoo(reader.read(), 1, "2", "3"); + validateFoo(reader.read(), 4, "5", "6"); + + assertNull(reader.read()); + } + + @Test + public void testCurrentItemCount() throws Exception { + JdbcCursorItemReader reader = new JdbcCursorItemReaderBuilder() + .dataSource(this.dataSource) + .name("fooReader") + .sql("SELECT * FROM FOO ORDER BY FIRST") + .currentItemCount(1) + .rowMapper((rs, rowNum) -> { + Foo foo = new Foo(); + + foo.setFirst(rs.getInt("FIRST")); + foo.setSecond(rs.getString("SECOND")); + foo.setThird(rs.getString("THIRD")); + + return foo; + }) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + + validateFoo(reader.read(), 4, "5", "6"); + validateFoo(reader.read(), 7, "8", "9"); + + assertNull(reader.read()); + } + + @Test + public void testOtherProperties() { + JdbcCursorItemReader reader = new JdbcCursorItemReaderBuilder() + .dataSource(this.dataSource) + .name("fooReader") + .sql("SELECT * FROM FOO ORDER BY FIRST") + .fetchSize(1) + .queryTimeout(2) + .ignoreWarnings(true) + .driverSupportsAbsolute(true) + .useSharedExtendedConnection(true) + .beanRowMapper(Foo.class) + .build(); + + assertEquals(1, ReflectionTestUtils.getField(reader, "fetchSize")); + assertEquals(2, ReflectionTestUtils.getField(reader, "queryTimeout")); + assertTrue((boolean) ReflectionTestUtils.getField(reader, "ignoreWarnings")); + assertTrue((boolean) ReflectionTestUtils.getField(reader, "driverSupportsAbsolute")); + } + + @Test + public void testValidation() { + try { + new JdbcCursorItemReaderBuilder().saveState(true).build(); + } + catch (IllegalArgumentException iae) { + assertEquals("A name is required when saveState is set to true", iae.getMessage()); + } + catch (Exception e) { + fail(); + } + + try { + new JdbcCursorItemReaderBuilder() + .saveState(false) + .build(); + } + catch (IllegalArgumentException iae) { + assertEquals("A query is required", iae.getMessage()); + } + catch (Exception e) { + fail(); + } + + try { + new JdbcCursorItemReaderBuilder() + .saveState(false) + .sql("select 1") + .build(); + } + catch (IllegalArgumentException iae) { + assertEquals("A datasource is required", iae.getMessage()); + } + catch (Exception e) { + fail(); + } + + try { + new JdbcCursorItemReaderBuilder() + .saveState(false) + .sql("select 1") + .dataSource(this.dataSource) + .build(); + } + catch (IllegalArgumentException iae) { + assertEquals("A rowmapper is required", iae.getMessage()); + } + catch (Exception e) { + fail(); + } + } + + private void validateFoo(Foo item, int first, String second, String third) { + assertEquals(first, item.getFirst()); + assertEquals(second, item.getSecond()); + assertEquals(third, item.getThird()); + } + + public static class Foo { + private int first; + private String second; + private String third; + + public int getFirst() { + return first; + } + + public void setFirst(int first) { + this.first = first; + } + + public String getSecond() { + return second; + } + + public void setSecond(String second) { + this.second = second; + } + + public String getThird() { + return third; + } + + public void setThird(String third) { + this.third = third; + } + } + + @Configuration + public static class TestDataSourceConfiguration { + + private static final String CREATE_SQL = "CREATE TABLE FOO (\n" + + "\tID BIGINT IDENTITY NOT NULL PRIMARY KEY ,\n" + + "\tFIRST BIGINT ,\n" + + "\tSECOND VARCHAR(5) NOT NULL,\n" + + "\tTHIRD VARCHAR(5) NOT NULL) ;"; + + private static final String INSERT_SQL = + "INSERT INTO FOO (FIRST, SECOND, THIRD) VALUES (1, '2', '3');" + + "INSERT INTO FOO (FIRST, SECOND, THIRD) VALUES (4, '5', '6');" + + "INSERT INTO FOO (FIRST, SECOND, THIRD) VALUES (7, '8', '9');"; + + @Bean + public DataSource dataSource() { + return new EmbeddedDatabaseFactory().getDatabase(); + } + + @Bean + public DataSourceInitializer initializer(DataSource dataSource) { + DataSourceInitializer dataSourceInitializer = new DataSourceInitializer(); + dataSourceInitializer.setDataSource(dataSource); + + Resource create = new ByteArrayResource(CREATE_SQL.getBytes()); + Resource insert = new ByteArrayResource(INSERT_SQL.getBytes()); + dataSourceInitializer.setDatabasePopulator(new ResourceDatabasePopulator(create, insert)); + + return dataSourceInitializer; + } + + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JdbcPagingItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JdbcPagingItemReaderBuilderTests.java new file mode 100644 index 0000000000..81a3fbbb50 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JdbcPagingItemReaderBuilderTests.java @@ -0,0 +1,418 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.util.HashMap; +import java.util.Map; +import javax.sql.DataSource; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.database.JdbcPagingItemReader; +import org.springframework.batch.item.database.Order; +import org.springframework.batch.item.database.support.AbstractSqlPagingQueryProvider; +import org.springframework.batch.item.database.support.HsqlPagingQueryProvider; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.Resource; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory; +import org.springframework.jdbc.datasource.init.DataSourceInitializer; +import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator; +import org.springframework.test.util.ReflectionTestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * @author Michael Minella + */ +public class JdbcPagingItemReaderBuilderTests { + + private DataSource dataSource; + + private ConfigurableApplicationContext context; + + @Before + public void setUp() { + this.context = new AnnotationConfigApplicationContext(TestDataSourceConfiguration.class); + this.dataSource = (DataSource) context.getBean("dataSource"); + } + + @After + public void tearDown() { + if(this.context != null) { + this.context.close(); + } + } + + @Test + public void testBasicConfigurationQueryProvider() throws Exception { + Map sortKeys = new HashMap<>(1); + sortKeys.put("ID", Order.DESCENDING); + + AbstractSqlPagingQueryProvider provider = new HsqlPagingQueryProvider(); + provider.setSelectClause("SELECT ID, FIRST, SECOND, THIRD"); + provider.setFromClause("FOO"); + provider.setSortKeys(sortKeys); + + JdbcPagingItemReader reader = new JdbcPagingItemReaderBuilder() + .name("fooReader") + .currentItemCount(1) + .dataSource(this.dataSource) + .queryProvider(provider) + .fetchSize(2) + .maxItemCount(2) + .rowMapper((rs, rowNum) -> new Foo(rs.getInt(1), + rs.getInt(2), + rs.getString(3), + rs.getString(4))) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + Foo item1 = reader.read(); + assertNull(reader.read()); + reader.update(executionContext); + reader.close(); + + assertEquals(3, item1.getId()); + assertEquals(10, item1.getFirst()); + assertEquals("11", item1.getSecond()); + assertEquals("12", item1.getThird()); + assertTrue((int) ReflectionTestUtils.getField(reader, "fetchSize") == 2); + + assertEquals(2, executionContext.size()); + } + + @Test + public void testBasicConfiguration() throws Exception { + Map sortKeys = new HashMap<>(1); + sortKeys.put("ID", Order.DESCENDING); + + JdbcPagingItemReader reader = new JdbcPagingItemReaderBuilder() + .name("fooReader") + .currentItemCount(1) + .dataSource(this.dataSource) + .maxItemCount(2) + .selectClause("SELECT ID, FIRST, SECOND, THIRD") + .fromClause("FOO") + .sortKeys(sortKeys) + .rowMapper((rs, rowNum) -> new Foo(rs.getInt(1), + rs.getInt(2), + rs.getString(3), + rs.getString(4))) + .build(); + + reader.afterPropertiesSet(); + + reader.open(new ExecutionContext()); + Foo item1 = reader.read(); + assertNull(reader.read()); + + assertEquals(3, item1.getId()); + assertEquals(10, item1.getFirst()); + assertEquals("11", item1.getSecond()); + assertEquals("12", item1.getThird()); + } + + @Test + public void testPageSize() throws Exception { + Map sortKeys = new HashMap<>(1); + sortKeys.put("ID", Order.DESCENDING); + + JdbcPagingItemReader reader = new JdbcPagingItemReaderBuilder() + .name("fooReader") + .dataSource(this.dataSource) + .pageSize(1) + .maxItemCount(2) + .selectClause("SELECT ID, FIRST, SECOND, THIRD") + .fromClause("FOO") + .sortKeys(sortKeys) + .rowMapper((rs, rowNum) -> new Foo(rs.getInt(1), + rs.getInt(2), + rs.getString(3), + rs.getString(4))) + .build(); + + reader.afterPropertiesSet(); + + reader.open(new ExecutionContext()); + Foo item1 = reader.read(); + Foo item2 = reader.read(); + assertNull(reader.read()); + + assertEquals(4, item1.getId()); + assertEquals(13, item1.getFirst()); + assertEquals("14", item1.getSecond()); + assertEquals("15", item1.getThird()); + + assertEquals(3, item2.getId()); + assertEquals(10, item2.getFirst()); + assertEquals("11", item2.getSecond()); + assertEquals("12", item2.getThird()); + } + + @Test + public void testSaveState() throws Exception { + Map sortKeys = new HashMap<>(1); + sortKeys.put("ID", Order.DESCENDING); + + JdbcPagingItemReader reader = new JdbcPagingItemReaderBuilder() + .dataSource(this.dataSource) + .pageSize(1) + .maxItemCount(2) + .selectClause("SELECT ID, FIRST, SECOND, THIRD") + .fromClause("FOO") + .sortKeys(sortKeys) + .saveState(false) + .rowMapper((rs, rowNum) -> new Foo(rs.getInt(1), + rs.getInt(2), + rs.getString(3), + rs.getString(4))) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + Foo item1 = reader.read(); + Foo item2 = reader.read(); + assertNull(reader.read()); + reader.update(executionContext); + reader.close(); + + assertEquals(4, item1.getId()); + assertEquals(13, item1.getFirst()); + assertEquals("14", item1.getSecond()); + assertEquals("15", item1.getThird()); + + assertEquals(3, item2.getId()); + assertEquals(10, item2.getFirst()); + assertEquals("11", item2.getSecond()); + assertEquals("12", item2.getThird()); + + assertEquals(0, executionContext.size()); + } + + @Test + public void testParameters() throws Exception { + Map sortKeys = new HashMap<>(1); + sortKeys.put("ID", Order.DESCENDING); + + Map parameterValues = new HashMap<>(); + parameterValues.put("min", 1); + parameterValues.put("max", 10); + + JdbcPagingItemReader reader = new JdbcPagingItemReaderBuilder() + .name("fooReader") + .dataSource(this.dataSource) + .pageSize(1) + .maxItemCount(1) + .selectClause("SELECT ID, FIRST, SECOND, THIRD") + .fromClause("FOO") + .whereClause("FIRST > :min AND FIRST < :max") + .sortKeys(sortKeys) + .parameterValues(parameterValues) + .rowMapper((rs, rowNum) -> new Foo(rs.getInt(1), + rs.getInt(2), + rs.getString(3), + rs.getString(4))) + .build(); + + reader.afterPropertiesSet(); + + reader.open(new ExecutionContext()); + Foo item1 = reader.read(); + assertNull(reader.read()); + + assertEquals(2, item1.getId()); + assertEquals(7, item1.getFirst()); + assertEquals("8", item1.getSecond()); + assertEquals("9", item1.getThird()); + } + + @Test + public void testValidation() { + + try { + new JdbcPagingItemReaderBuilder().build(); + fail(); + } + catch (IllegalArgumentException iae) { + assertEquals("dataSource is required", iae.getMessage()); + } + + try { + new JdbcPagingItemReaderBuilder() + .pageSize(-2) + .build(); + fail(); + } + catch (IllegalArgumentException iae) { + assertEquals("pageSize must be greater than zero", iae.getMessage()); + } + + try { + new JdbcPagingItemReaderBuilder() + .pageSize(2) + .build(); + fail(); + } + catch (IllegalArgumentException ise) { + assertEquals("dataSource is required", ise.getMessage()); + } + + try { + new JdbcPagingItemReaderBuilder() + .pageSize(2) + .dataSource(this.dataSource) + .build(); + fail(); + } + catch (IllegalArgumentException ise) { + assertEquals("A name is required when saveState is set to true", ise.getMessage()); + } + + try { + new JdbcPagingItemReaderBuilder() + .saveState(false) + .pageSize(2) + .dataSource(this.dataSource) + .build(); + fail(); + } + catch (IllegalArgumentException ise) { + assertEquals("selectClause is required when not providing a PagingQueryProvider", ise.getMessage()); + } + + try { + new JdbcPagingItemReaderBuilder() + .name("fooReader") + .pageSize(2) + .dataSource(this.dataSource) + .selectClause("SELECT *") + .build(); + fail(); + } + catch (IllegalArgumentException ise) { + assertEquals("fromClause is required when not providing a PagingQueryProvider", ise.getMessage()); + } + + try { + new JdbcPagingItemReaderBuilder() + .saveState(false) + .pageSize(2) + .dataSource(this.dataSource) + .selectClause("SELECT *") + .fromClause("FOO") + .build(); + fail(); + } + catch (IllegalArgumentException ise) { + assertEquals("sortKeys are required when not providing a PagingQueryProvider", ise.getMessage()); + } + } + + public static class Foo { + private int id; + private int first; + private String second; + private String third; + + public Foo(int id, int first, String second, String third) { + this.id = id; + this.first = first; + this.second = second; + this.third = third; + } + + public int getId() { + return id; + } + + public void setId(int id) { + this.id = id; + } + + public int getFirst() { + return first; + } + + public void setFirst(int first) { + this.first = first; + } + + public String getSecond() { + return second; + } + + public void setSecond(String second) { + this.second = second; + } + + public String getThird() { + return third; + } + + public void setThird(String third) { + this.third = third; + } + } + + @Configuration + public static class TestDataSourceConfiguration { + + private static final String CREATE_SQL = "CREATE TABLE FOO (\n" + + "\tID BIGINT IDENTITY NOT NULL PRIMARY KEY ,\n" + + "\tFIRST BIGINT ,\n" + + "\tSECOND VARCHAR(5) NOT NULL,\n" + + "\tTHIRD VARCHAR(5) NOT NULL) ;"; + + private static final String INSERT_SQL = + "INSERT INTO FOO (FIRST, SECOND, THIRD) VALUES (1, '2', '3');" + + "INSERT INTO FOO (FIRST, SECOND, THIRD) VALUES (4, '5', '6');" + + "INSERT INTO FOO (FIRST, SECOND, THIRD) VALUES (7, '8', '9');" + + "INSERT INTO FOO (FIRST, SECOND, THIRD) VALUES (10, '11', '12');" + + "INSERT INTO FOO (FIRST, SECOND, THIRD) VALUES (13, '14', '15');"; + + @Bean + public DataSource dataSource() { + return new EmbeddedDatabaseFactory().getDatabase(); + } + + @Bean + public DataSourceInitializer initializer(DataSource dataSource) { + DataSourceInitializer dataSourceInitializer = new DataSourceInitializer(); + dataSourceInitializer.setDataSource(dataSource); + + Resource create = new ByteArrayResource(CREATE_SQL.getBytes()); + Resource insert = new ByteArrayResource(INSERT_SQL.getBytes()); + dataSourceInitializer.setDatabasePopulator(new ResourceDatabasePopulator(create, insert)); + + return dataSourceInitializer; + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JpaItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JpaItemWriterBuilderTests.java new file mode 100644 index 0000000000..94e5c0a513 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JpaItemWriterBuilderTests.java @@ -0,0 +1,87 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.util.Arrays; +import java.util.List; +import javax.persistence.EntityManager; +import javax.persistence.EntityManagerFactory; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.database.JpaItemWriter; +import org.springframework.orm.jpa.EntityManagerHolder; +import org.springframework.transaction.support.TransactionSynchronizationManager; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.verify; + +/** + * @author Mahmoud Ben Hassine + */ +public class JpaItemWriterBuilderTests { + + @Mock + private EntityManagerFactory entityManagerFactory; + + @Mock + private EntityManager entityManager; + + @Before + public void setUp() { + MockitoAnnotations.initMocks(this); + TransactionSynchronizationManager.bindResource(this.entityManagerFactory, + new EntityManagerHolder(this.entityManager)); + } + + @After + public void tearDown() { + TransactionSynchronizationManager.unbindResource(this.entityManagerFactory); + } + + @Test + public void testConfiguration() throws Exception { + JpaItemWriter itemWriter = new JpaItemWriterBuilder() + .entityManagerFactory(this.entityManagerFactory) + .build(); + + itemWriter.afterPropertiesSet(); + + List items = Arrays.asList("foo", "bar"); + + itemWriter.write(items); + + verify(this.entityManager).merge(items.get(0)); + verify(this.entityManager).merge(items.get(1)); + } + + @Test + public void testValidation() { + try { + new JpaItemWriterBuilder() + .build(); + fail("Should fail if no EntityManagerFactory is provided"); + } + catch (IllegalStateException ise) { + assertEquals("Incorrect message", "EntityManagerFactory must be provided", ise.getMessage()); + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JpaPagingItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JpaPagingItemReaderBuilderTests.java new file mode 100644 index 0000000000..3e9399e0fa --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/JpaPagingItemReaderBuilderTests.java @@ -0,0 +1,243 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import java.util.HashMap; +import java.util.Map; +import javax.persistence.EntityManagerFactory; +import javax.sql.DataSource; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.database.JpaPagingItemReader; +import org.springframework.batch.item.database.orm.JpaNativeQueryProvider; +import org.springframework.batch.item.sample.Foo; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.core.io.ClassPathResource; +import org.springframework.core.io.Resource; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory; +import org.springframework.jdbc.datasource.init.DataSourceInitializer; +import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator; +import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean; +import org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter; +import org.springframework.test.util.ReflectionTestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; + +/** + * @author Michael Minella + */ +public class JpaPagingItemReaderBuilderTests { + + private EntityManagerFactory entityManagerFactory; + + private ConfigurableApplicationContext context; + + @Before + public void setUp() { + this.context = new AnnotationConfigApplicationContext(JpaPagingItemReaderBuilderTests.TestDataSourceConfiguration.class); + this.entityManagerFactory = (EntityManagerFactory) context.getBean("entityManagerFactory"); + } + + @After + public void tearDown() { + if(this.context != null) { + this.context.close(); + } + } + + @Test + public void testConfiguration() throws Exception { + JpaPagingItemReader reader = new JpaPagingItemReaderBuilder() + .name("fooReader") + .entityManagerFactory(this.entityManagerFactory) + .currentItemCount(2) + .maxItemCount(4) + .pageSize(5) + .transacted(false) + .queryString("select f from Foo f ") + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + + reader.open(executionContext); + Foo item1 = reader.read(); + Foo item2 = reader.read(); + assertNull(reader.read()); + reader.update(executionContext); + reader.close(); + + assertEquals(3, item1.getId()); + assertEquals("bar3", item1.getName()); + assertEquals(3, item1.getValue()); + assertEquals(4, item2.getId()); + assertEquals("bar4", item2.getName()); + assertEquals(4, item2.getValue()); + + assertEquals(2, executionContext.size()); + assertEquals(5, ReflectionTestUtils.getField(reader, "pageSize")); + assertFalse((Boolean) ReflectionTestUtils.getField(reader, "transacted")); + } + + @Test + public void testConfigurationNoSaveState() throws Exception { + Map parameters = new HashMap<>(); + parameters.put("value", 2); + + JpaPagingItemReader reader = new JpaPagingItemReaderBuilder() + .name("fooReader") + .entityManagerFactory(this.entityManagerFactory) + .queryString("select f from Foo f where f.id > :value") + .parameterValues(parameters) + .saveState(false) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + + reader.open(executionContext); + + int i = 0; + while(reader.read() != null) { + i++; + } + + reader.update(executionContext); + reader.close(); + + assertEquals(3, i); + assertEquals(0, executionContext.size()); + } + + @Test + public void testConfigurationQueryProvider() throws Exception { + + JpaNativeQueryProvider provider = new JpaNativeQueryProvider<>(); + provider.setEntityClass(Foo.class); + provider.setSqlQuery("select * from T_FOOS"); + provider.afterPropertiesSet(); + + JpaPagingItemReader reader = new JpaPagingItemReaderBuilder() + .name("fooReader") + .entityManagerFactory(this.entityManagerFactory) + .queryProvider(provider) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + + reader.open(executionContext); + + int i = 0; + while(reader.read() != null) { + i++; + } + + reader.update(executionContext); + reader.close(); + + assertEquals(5, i); + } + + @Test + public void testValidation() { + try { + new JpaPagingItemReaderBuilder() + .entityManagerFactory(this.entityManagerFactory) + .pageSize(-2) + .build(); + fail("pageSize must be >= 0"); + } + catch (IllegalArgumentException iae) { + assertEquals("pageSize must be greater than zero", iae.getMessage()); + } + + try { + new JpaPagingItemReaderBuilder().build(); + fail("An EntityManagerFactory is required"); + } + catch (IllegalArgumentException iae) { + assertEquals("An EntityManagerFactory is required", iae.getMessage()); + } + + try { + new JpaPagingItemReaderBuilder() + .entityManagerFactory(this.entityManagerFactory) + .saveState(true) + .build(); + fail("A name is required when saveState is set to true"); + } + catch (IllegalArgumentException iae) { + assertEquals("A name is required when saveState is set to true", iae.getMessage()); + } + + try { + new JpaPagingItemReaderBuilder() + .entityManagerFactory(this.entityManagerFactory) + .saveState(false) + .build(); + fail("Query string is required when queryProvider is null"); + } + catch (IllegalArgumentException iae) { + assertEquals("Query string is required when queryProvider is null", iae.getMessage()); + } + } + + @Configuration + public static class TestDataSourceConfiguration { + + @Bean + public DataSource dataSource() { + return new EmbeddedDatabaseFactory().getDatabase(); + } + + @Bean + public DataSourceInitializer initializer(DataSource dataSource) { + DataSourceInitializer dataSourceInitializer = new DataSourceInitializer(); + dataSourceInitializer.setDataSource(dataSource); + + Resource create = new ClassPathResource("org/springframework/batch/item/database/init-foo-schema-hsqldb.sql"); + dataSourceInitializer.setDatabasePopulator(new ResourceDatabasePopulator(create)); + + return dataSourceInitializer; + } + + @Bean + public LocalContainerEntityManagerFactoryBean entityManagerFactory() throws Exception { + LocalContainerEntityManagerFactoryBean entityManagerFactoryBean = + new LocalContainerEntityManagerFactoryBean(); + + entityManagerFactoryBean.setDataSource(dataSource()); + entityManagerFactoryBean.setPersistenceUnitName("bar"); + entityManagerFactoryBean.setJpaVendorAdapter(new HibernateJpaVendorAdapter()); + + return entityManagerFactoryBean; + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/StoredProcedureItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/StoredProcedureItemReaderBuilderTests.java new file mode 100644 index 0000000000..4637a02075 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/builder/StoredProcedureItemReaderBuilderTests.java @@ -0,0 +1,249 @@ +/* + * Copyright 2017-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.builder; + +import javax.sql.DataSource; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import test.jdbc.datasource.DataSourceInitializer; +import test.jdbc.datasource.DerbyDataSourceFactoryBean; +import test.jdbc.datasource.DerbyShutdownBean; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.database.FooRowMapper; +import org.springframework.batch.item.database.StoredProcedureItemReader; +import org.springframework.batch.item.sample.Foo; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.core.io.ClassPathResource; +import org.springframework.jdbc.core.ArgumentPreparedStatementSetter; +import org.springframework.jdbc.core.SqlParameter; +import org.springframework.jdbc.datasource.DataSourceTransactionManager; +import org.springframework.test.util.ReflectionTestUtils; +import org.springframework.transaction.PlatformTransactionManager; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +public class StoredProcedureItemReaderBuilderTests { + + private DataSource dataSource; + + private ConfigurableApplicationContext context; + + @Before + public void setUp() { + this.context = new AnnotationConfigApplicationContext(TestDataSourceConfiguration.class); + this.dataSource = (DataSource) this.context.getBean("dataSource"); + } + + @After + public void tearDown() { + this.context.close(); + } + + @Test + public void testSunnyScenario() throws Exception { + StoredProcedureItemReader reader = new StoredProcedureItemReaderBuilder() + .name("foo_reader") + .dataSource(this.dataSource) + .procedureName("read_foos") + .rowMapper(new FooRowMapper()) + .verifyCursorPosition(false) + .build(); + + reader.open(new ExecutionContext()); + + Foo item1 = reader.read(); + assertEquals(1, item1.getId()); + assertEquals("bar1", item1.getName()); + assertEquals(1, item1.getValue()); + + reader.close(); + } + + @Test + public void testConfiguration() { + ArgumentPreparedStatementSetter preparedStatementSetter = new ArgumentPreparedStatementSetter(null); + + SqlParameter[] parameters = new SqlParameter[0]; + + StoredProcedureItemReader reader = new StoredProcedureItemReaderBuilder() + .name("foo_reader") + .dataSource(this.dataSource) + .procedureName("read_foos") + .rowMapper(new FooRowMapper()) + .verifyCursorPosition(false) + .refCursorPosition(3) + .useSharedExtendedConnection(true) + .preparedStatementSetter(preparedStatementSetter) + .parameters(parameters) + .function() + .fetchSize(5) + .driverSupportsAbsolute(true) + .currentItemCount(6) + .ignoreWarnings(false) + .maxItemCount(7) + .queryTimeout(8) + .maxRows(9) + .build(); + + assertEquals(3, ReflectionTestUtils.getField(reader, "refCursorPosition")); + assertEquals(preparedStatementSetter, ReflectionTestUtils.getField(reader, "preparedStatementSetter")); + assertEquals(parameters, ReflectionTestUtils.getField(reader, "parameters")); + assertEquals(5, ReflectionTestUtils.getField(reader, "fetchSize")); + assertEquals(6, ReflectionTestUtils.getField(reader, "currentItemCount")); + assertEquals(7, ReflectionTestUtils.getField(reader, "maxItemCount")); + assertEquals(8, ReflectionTestUtils.getField(reader, "queryTimeout")); + assertEquals(9, ReflectionTestUtils.getField(reader, "maxRows")); + assertTrue((Boolean) ReflectionTestUtils.getField(reader, "useSharedExtendedConnection")); + assertTrue((Boolean) ReflectionTestUtils.getField(reader, "function")); + assertTrue((Boolean) ReflectionTestUtils.getField(reader, "driverSupportsAbsolute")); + assertFalse((Boolean) ReflectionTestUtils.getField(reader, "ignoreWarnings")); + } + + @Test + public void testNoSaveState() throws Exception { + StoredProcedureItemReader reader = new StoredProcedureItemReaderBuilder() + .dataSource(this.dataSource) + .procedureName("read_foos") + .rowMapper(new FooRowMapper()) + .verifyCursorPosition(false) + .saveState(false) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + + reader.read(); + reader.read(); + + reader.update(executionContext); + + assertEquals(0, executionContext.size()); + + reader.close(); + } + + @Test + public void testValidation() { + try { + new StoredProcedureItemReaderBuilder() + .build(); + + fail("Exception was not thrown for missing the name"); + } + catch (IllegalArgumentException iae) { + assertEquals("A name is required when saveSate is set to true", + iae.getMessage()); + } + + try { + new StoredProcedureItemReaderBuilder() + .saveState(false) + .build(); + + fail("Exception was not thrown for missing the stored procedure name"); + } + catch (IllegalArgumentException iae) { + assertEquals("The name of the stored procedure must be provided", + iae.getMessage()); + } + + try { + new StoredProcedureItemReaderBuilder() + .saveState(false) + .procedureName("read_foos") + .build(); + + fail("Exception was not thrown for missing the DataSource"); + } + catch (IllegalArgumentException iae) { + assertEquals("A datasource is required", + iae.getMessage()); + } + + try { + new StoredProcedureItemReaderBuilder() + .saveState(false) + .procedureName("read_foos") + .dataSource(this.dataSource) + .build(); + + fail("Exception was not thrown for missing the RowMapper"); + } + catch (IllegalArgumentException iae) { + assertEquals("A rowmapper is required", + iae.getMessage()); + } + } + + @Configuration + public static class TestDataSourceConfiguration { + + @Bean + public DerbyDataSourceFactoryBean dataSource() { + DerbyDataSourceFactoryBean derbyDataSourceFactoryBean = new DerbyDataSourceFactoryBean(); + + derbyDataSourceFactoryBean.setDataDirectory("build/derby-home"); + + return derbyDataSourceFactoryBean; + } + + @Bean + public DerbyShutdownBean dbShutdown(DataSource dataSource) { + DerbyShutdownBean shutdownBean = new DerbyShutdownBean(); + + shutdownBean.setDataSource(dataSource); + + return shutdownBean; + } + + @Bean + public PlatformTransactionManager transactionManager(DataSource dataSource) { + DataSourceTransactionManager transactionManager = new DataSourceTransactionManager(); + + transactionManager.setDataSource(dataSource); + + return transactionManager; + } + + @Bean + public DataSourceInitializer initializer(DataSource dataSource) { + DataSourceInitializer initializer = new DataSourceInitializer(); + + initializer.setDataSource(dataSource); + initializer.setInitScripts(new ClassPathResource[]{ + new ClassPathResource("org/springframework/batch/item/database/init-foo-schema-derby.sql") + }); + initializer.setDestroyScripts(new ClassPathResource[]{ + new ClassPathResource("org/springframework/batch/item/database/drop-foo-schema-derby.sql") + }); + + return initializer; + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/AbstractSqlPagingQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/AbstractSqlPagingQueryProviderTests.java index a82e5354b8..ae8887586f 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/AbstractSqlPagingQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/AbstractSqlPagingQueryProviderTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -44,7 +44,7 @@ public void setUp() { pagingQueryProvider.setFromClause("foo"); pagingQueryProvider.setWhereClause("bar = 1"); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("id", Order.ASCENDING); pagingQueryProvider.setSortKeys(sortKeys); pageSize = 100; @@ -66,7 +66,7 @@ public void testQueryContainsSortKeyDesc(){ @Test public void testGenerateFirstPageQueryWithMultipleSortKeys() { - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("name", Order.ASCENDING); sortKeys.put("id", Order.DESCENDING); pagingQueryProvider.setSortKeys(sortKeys); @@ -76,7 +76,7 @@ public void testGenerateFirstPageQueryWithMultipleSortKeys() { @Test public void testGenerateRemainingPagesQueryWithMultipleSortKeys() { - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("name", Order.ASCENDING); sortKeys.put("id", Order.DESCENDING); pagingQueryProvider.setSortKeys(sortKeys); @@ -86,7 +86,7 @@ public void testGenerateRemainingPagesQueryWithMultipleSortKeys() { @Test public void testGenerateJumpToItemQueryWithMultipleSortKeys() { - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("name", Order.ASCENDING); sortKeys.put("id", Order.DESCENDING); pagingQueryProvider.setSortKeys(sortKeys); @@ -96,7 +96,7 @@ public void testGenerateJumpToItemQueryWithMultipleSortKeys() { @Test public void testGenerateJumpToItemQueryForFirstPageWithMultipleSortKeys() { - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("name", Order.ASCENDING); sortKeys.put("id", Order.DESCENDING); pagingQueryProvider.setSortKeys(sortKeys); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/ColumnMapExecutionContextRowMapperTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/ColumnMapExecutionContextRowMapperTests.java index 2f8681437b..d501b23b23 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/ColumnMapExecutionContextRowMapperTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/ColumnMapExecutionContextRowMapperTests.java @@ -1,11 +1,21 @@ -/** - * +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ package org.springframework.batch.item.database.support; import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - import java.sql.PreparedStatement; import java.util.HashMap; import java.util.LinkedHashMap; @@ -32,14 +42,14 @@ protected void setUp() throws Exception { ps = mock(PreparedStatement.class); mapper = new ColumnMapItemPreparedStatementSetter(); - key = new LinkedHashMap(2); + key = new LinkedHashMap<>(2); key.put("1", Integer.valueOf(1)); key.put("2", Integer.valueOf(2)); } public void testCreateExecutionContextFromEmptyKeys() throws Exception { - mapper.setValues(new HashMap(), ps); + mapper.setValues(new HashMap<>(), ps); } public void testCreateSetter() throws Exception { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/Db2PagingQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/Db2PagingQueryProviderTests.java index 1486a5a359..01018c1e52 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/Db2PagingQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/Db2PagingQueryProviderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2012 the original author or authors. + * Copyright 2012-2015 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,7 +40,7 @@ public void testGenerateFirstPageQuery() { @Test @Override public void testGenerateRemainingPagesQuery() { - String sql = "SELECT id, name, age FROM foo WHERE bar = 1 AND ((id > ?)) ORDER BY id ASC FETCH FIRST 100 ROWS ONLY"; + String sql = "SELECT id, name, age FROM foo WHERE (bar = 1) AND ((id > ?)) ORDER BY id ASC FETCH FIRST 100 ROWS ONLY"; String s = pagingQueryProvider.generateRemainingPagesQuery(pageSize); assertEquals(sql, s); } @@ -104,7 +104,7 @@ public String getFirstPageSqlWithMultipleSortKeys() { @Override public String getRemainingSqlWithMultipleSortKeys() { - return "SELECT id, name, age FROM foo WHERE bar = 1 AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC FETCH FIRST 100 ROWS ONLY"; + return "SELECT id, name, age FROM foo WHERE (bar = 1) AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC FETCH FIRST 100 ROWS ONLY"; } @Override diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/DefaultDataFieldMaxValueIncrementerFactoryTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/DefaultDataFieldMaxValueIncrementerFactoryTests.java index 8ded8be59b..d69ba7db18 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/DefaultDataFieldMaxValueIncrementerFactoryTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/DefaultDataFieldMaxValueIncrementerFactoryTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,20 +21,20 @@ import static org.mockito.Mockito.mock; -import org.springframework.jdbc.support.incrementer.DB2SequenceMaxValueIncrementer; +import org.springframework.jdbc.support.incrementer.Db2LuwMaxValueIncrementer; +import org.springframework.jdbc.support.incrementer.Db2MainframeMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.DerbyMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.HsqlMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.MySQLMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.OracleSequenceMaxValueIncrementer; -import org.springframework.jdbc.support.incrementer.PostgreSQLSequenceMaxValueIncrementer; +import org.springframework.jdbc.support.incrementer.PostgresSequenceMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.SqlServerMaxValueIncrementer; import org.springframework.jdbc.support.incrementer.SybaseMaxValueIncrementer; -import org.springframework.jdbc.support.incrementer.DB2MainframeSequenceMaxValueIncrementer; /** * @author Lucas Ward * @author Will Schipp - * + * @author Drummond Dawson */ public class DefaultDataFieldMaxValueIncrementerFactoryTests extends TestCase { @@ -61,6 +61,7 @@ public void testSupportedDatabaseType(){ assertTrue(factory.isSupportedIncrementerType("hsql")); assertTrue(factory.isSupportedIncrementerType("sqlserver")); assertTrue(factory.isSupportedIncrementerType("sybase")); + assertTrue(factory.isSupportedIncrementerType("sqlite")); } public void testUnsupportedDatabaseType(){ @@ -88,11 +89,11 @@ public void testNullIncrementerName(){ } public void testDb2(){ - assertTrue(factory.getIncrementer("db2", "NAME") instanceof DB2SequenceMaxValueIncrementer); + assertTrue(factory.getIncrementer("db2", "NAME") instanceof Db2LuwMaxValueIncrementer); } public void testDb2zos(){ - assertTrue(factory.getIncrementer("db2zos", "NAME") instanceof DB2MainframeSequenceMaxValueIncrementer); + assertTrue(factory.getIncrementer("db2zos", "NAME") instanceof Db2MainframeMaxValueIncrementer); } public void testMysql(){ @@ -113,7 +114,7 @@ public void testHsql(){ } public void testPostgres(){ - assertTrue(factory.getIncrementer("postgres", "NAME") instanceof PostgreSQLSequenceMaxValueIncrementer); + assertTrue(factory.getIncrementer("postgres", "NAME") instanceof PostgresSequenceMaxValueIncrementer); } public void testMsSqlServer(){ @@ -124,5 +125,8 @@ public void testSybase(){ assertTrue(factory.getIncrementer("sybase", "NAME") instanceof SybaseMaxValueIncrementer); } + public void testSqlite(){ + assertTrue(factory.getIncrementer("sqlite", "NAME") instanceof SqliteMaxValueIncrementer); + } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/DerbyPagingQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/DerbyPagingQueryProviderTests.java index 4536f80134..3583cd5d3b 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/DerbyPagingQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/DerbyPagingQueryProviderTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/H2PagingQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/H2PagingQueryProviderTests.java index 49f344e5aa..8e6604040b 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/H2PagingQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/H2PagingQueryProviderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2015 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,7 +40,7 @@ public void testGenerateFirstPageQuery() { @Test @Override public void testGenerateRemainingPagesQuery() { - String sql = "SELECT TOP 100 id, name, age FROM foo WHERE bar = 1 AND ((id > ?)) ORDER BY id ASC"; + String sql = "SELECT TOP 100 id, name, age FROM foo WHERE (bar = 1) AND ((id > ?)) ORDER BY id ASC"; String s = pagingQueryProvider.generateRemainingPagesQuery(pageSize); assertEquals(sql, s); } @@ -72,7 +72,7 @@ public void testGenerateFirstPageQueryWithGroupBy() { @Test public void testGenerateRemainingPagesQueryWithGroupBy() { pagingQueryProvider.setGroupClause("dep"); - String sql = "SELECT TOP 100 id, name, age FROM foo WHERE bar = 1 AND ((id > ?)) GROUP BY dep ORDER BY id ASC"; + String sql = "SELECT TOP 100 id, name, age FROM foo WHERE (bar = 1) AND ((id > ?)) GROUP BY dep ORDER BY id ASC"; String s = pagingQueryProvider.generateRemainingPagesQuery(pageSize); assertEquals(sql, s); } @@ -102,7 +102,7 @@ public String getFirstPageSqlWithMultipleSortKeys() { @Override public String getRemainingSqlWithMultipleSortKeys() { - return "SELECT TOP 100 id, name, age FROM foo WHERE bar = 1 AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC"; + return "SELECT TOP 100 id, name, age FROM foo WHERE (bar = 1) AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC"; } @Override diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/HibernateNativeQueryProviderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/HibernateNativeQueryProviderIntegrationTests.java index b9b7ec9d45..03f455072d 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/HibernateNativeQueryProviderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/HibernateNativeQueryProviderIntegrationTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,7 +23,7 @@ import javax.sql.DataSource; -import org.hibernate.Query; +import org.hibernate.query.Query; import org.hibernate.SessionFactory; import org.junit.Before; import org.junit.Test; @@ -33,7 +33,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.core.io.ClassPathResource; import org.springframework.core.io.Resource; -import org.springframework.orm.hibernate4.LocalSessionFactoryBean; +import org.springframework.orm.hibernate5.LocalSessionFactoryBean; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.transaction.annotation.Transactional; @@ -58,7 +58,7 @@ public void setDataSource(DataSource dataSource) { } public HibernateNativeQueryProviderIntegrationTests() { - hibernateQueryProvider = new HibernateNativeQueryProvider(); + hibernateQueryProvider = new HibernateNativeQueryProvider<>(); hibernateQueryProvider.setEntityClass(Foo.class); } @@ -84,9 +84,9 @@ public void shouldRetrieveAndMapAllFoos() throws Exception { hibernateQueryProvider.afterPropertiesSet(); hibernateQueryProvider.setSession(sessionFactory.openSession()); - Query query = hibernateQueryProvider.createQuery(); + Query query = hibernateQueryProvider.createQuery(); - List expectedFoos = new ArrayList(); + List expectedFoos = new ArrayList<>(); expectedFoos.add(new Foo(1, "bar1", 1)); expectedFoos.add(new Foo(2, "bar2", 2)); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/HibernateNativeQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/HibernateNativeQueryProviderTests.java index e566781778..d2ff37ce15 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/HibernateNativeQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/HibernateNativeQueryProviderTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,15 +16,16 @@ package org.springframework.batch.item.database.support; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - -import org.hibernate.SQLQuery; import org.hibernate.Session; import org.hibernate.StatelessSession; +import org.hibernate.query.NativeQuery; import org.junit.Test; + import org.springframework.batch.item.database.orm.HibernateNativeQueryProvider; -import org.springframework.util.Assert; + +import static org.junit.Assert.assertNotNull; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; /** * @author Anatoly Polinsky @@ -36,39 +37,41 @@ public class HibernateNativeQueryProviderTests { protected HibernateNativeQueryProvider hibernateQueryProvider; public HibernateNativeQueryProviderTests() { - hibernateQueryProvider = new HibernateNativeQueryProvider(); + hibernateQueryProvider = new HibernateNativeQueryProvider<>(); hibernateQueryProvider.setEntityClass(Foo.class); } @Test + @SuppressWarnings("unchecked") public void testCreateQueryWithStatelessSession() { String sqlQuery = "select * from T_FOOS"; hibernateQueryProvider.setSqlQuery(sqlQuery); StatelessSession session = mock(StatelessSession.class); - SQLQuery query = mock(SQLQuery.class); + NativeQuery query = mock(NativeQuery.class); - when(session.createSQLQuery(sqlQuery)).thenReturn(query); + when(session.createNativeQuery(sqlQuery)).thenReturn(query); when(query.addEntity(Foo.class)).thenReturn(query); hibernateQueryProvider.setStatelessSession(session); - Assert.notNull(hibernateQueryProvider.createQuery()); + assertNotNull(hibernateQueryProvider.createQuery()); } @Test + @SuppressWarnings("unchecked") public void shouldCreateQueryWithStatefulSession() { String sqlQuery = "select * from T_FOOS"; hibernateQueryProvider.setSqlQuery(sqlQuery); Session session = mock(Session.class); - SQLQuery query = mock(SQLQuery.class); + NativeQuery query = mock(NativeQuery.class); - when(session.createSQLQuery(sqlQuery)).thenReturn(query); + when(session.createNativeQuery(sqlQuery)).thenReturn(query); when(query.addEntity(Foo.class)).thenReturn(query); hibernateQueryProvider.setSession(session); - Assert.notNull(hibernateQueryProvider.createQuery()); + assertNotNull(hibernateQueryProvider.createQuery()); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/HsqlPagingQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/HsqlPagingQueryProviderTests.java index f95d51320f..33be0b46dd 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/HsqlPagingQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/HsqlPagingQueryProviderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2015 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -39,7 +39,7 @@ public void testGenerateFirstPageQuery() { @Test @Override public void testGenerateRemainingPagesQuery() { - String sql = "SELECT TOP 100 id, name, age FROM foo WHERE bar = 1 AND ((id > ?)) ORDER BY id ASC"; + String sql = "SELECT TOP 100 id, name, age FROM foo WHERE (bar = 1) AND ((id > ?)) ORDER BY id ASC"; String s = pagingQueryProvider.generateRemainingPagesQuery(pageSize); assertEquals(sql, s); } @@ -101,7 +101,7 @@ public String getFirstPageSqlWithMultipleSortKeys() { @Override public String getRemainingSqlWithMultipleSortKeys() { - return "SELECT TOP 100 id, name, age FROM foo WHERE bar = 1 AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC"; + return "SELECT TOP 100 id, name, age FROM foo WHERE (bar = 1) AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC"; } @Override diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/JpaNativeQueryProviderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/JpaNativeQueryProviderIntegrationTests.java index aec09a0f6f..1346325f10 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/JpaNativeQueryProviderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/JpaNativeQueryProviderIntegrationTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -47,7 +47,7 @@ public class JpaNativeQueryProviderIntegrationTests { private JpaNativeQueryProvider jpaQueryProvider; public JpaNativeQueryProviderIntegrationTests() { - jpaQueryProvider = new JpaNativeQueryProvider(); + jpaQueryProvider = new JpaNativeQueryProvider<>(); jpaQueryProvider.setEntityClass(Foo.class); } @@ -62,7 +62,7 @@ public void shouldRetrieveAndMapAllFoos() throws Exception { Query query = jpaQueryProvider.createQuery(); - List expectedFoos = new ArrayList(); + List expectedFoos = new ArrayList<>(); expectedFoos.add(new Foo(1, "bar1", 1)); expectedFoos.add(new Foo(2, "bar2", 2)); @@ -89,7 +89,7 @@ public void shouldExecuteParameterizedQuery() throws Exception { Query query = jpaQueryProvider.createQuery(); query.setParameter("limit", 3); - List expectedFoos = new ArrayList(); + List expectedFoos = new ArrayList<>(); expectedFoos.add(new Foo(3, "bar3", 3)); expectedFoos.add(new Foo(4, "bar4", 4)); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/JpaNativeQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/JpaNativeQueryProviderTests.java index 0390266d9b..fe3bfd3883 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/JpaNativeQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/JpaNativeQueryProviderTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,17 +16,18 @@ package org.springframework.batch.item.database.support; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - import javax.persistence.EntityManager; import javax.persistence.Query; import org.junit.Test; + import org.springframework.batch.item.database.orm.JpaNativeQueryProvider; import org.springframework.batch.item.sample.Foo; import org.springframework.util.Assert; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + /** * @author Anatoly Polinsky * @author Dave Syer @@ -37,7 +38,7 @@ public class JpaNativeQueryProviderTests { private JpaNativeQueryProvider jpaQueryProvider; public JpaNativeQueryProviderTests() { - jpaQueryProvider = new JpaNativeQueryProvider(); + jpaQueryProvider = new JpaNativeQueryProvider<>(); jpaQueryProvider.setEntityClass(Foo.class); } @@ -53,6 +54,6 @@ public void testCreateQuery() { when(entityManager.createNativeQuery(sqlQuery, Foo.class)).thenReturn(query); jpaQueryProvider.setEntityManager(entityManager); - Assert.notNull(jpaQueryProvider.createQuery()); + Assert.notNull(jpaQueryProvider.createQuery(), "Query was null"); } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/MySqlPagingQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/MySqlPagingQueryProviderTests.java index d48d5009ab..4593b94d9c 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/MySqlPagingQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/MySqlPagingQueryProviderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2015 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,10 +15,15 @@ */ package org.springframework.batch.item.database.support; -import static org.junit.Assert.assertEquals; +import java.util.HashMap; +import java.util.Map; import org.junit.Test; +import org.springframework.batch.item.database.Order; + +import static org.junit.Assert.assertEquals; + /** * @author Thomas Risberg * @author Michael Minella @@ -39,7 +44,7 @@ public void testGenerateFirstPageQuery() { @Test @Override public void testGenerateRemainingPagesQuery() { - String sql = "SELECT id, name, age FROM foo WHERE bar = 1 AND ((id > ?)) ORDER BY id ASC LIMIT 100"; + String sql = "SELECT id, name, age FROM foo WHERE (bar = 1) AND ((id > ?)) ORDER BY id ASC LIMIT 100"; String s = pagingQueryProvider.generateRemainingPagesQuery(pageSize); assertEquals(sql, s); } @@ -94,6 +99,25 @@ public void testGenerateJumpToItemQueryForFirstPageWithGroupBy() { assertEquals(sql, s); } + @Test + public void testFirstPageSqlWithAliases() { + Map sorts = new HashMap<>(); + sorts.put("owner.id", Order.ASCENDING); + + this.pagingQueryProvider = new MySqlPagingQueryProvider(); + this.pagingQueryProvider.setSelectClause("SELECT owner.id as ownerid, first_name, last_name, dog_name "); + this.pagingQueryProvider.setFromClause("FROM dog_owner owner INNER JOIN dog ON owner.id = dog.id "); + this.pagingQueryProvider.setSortKeys(sorts); + + String firstPage = this.pagingQueryProvider.generateFirstPageQuery(5); + String jumpToItemQuery = this.pagingQueryProvider.generateJumpToItemQuery(7, 5); + String remainingPagesQuery = this.pagingQueryProvider.generateRemainingPagesQuery(5); + + assertEquals("SELECT owner.id as ownerid, first_name, last_name, dog_name FROM dog_owner owner INNER JOIN dog ON owner.id = dog.id ORDER BY owner.id ASC LIMIT 5", firstPage); + assertEquals("SELECT owner.id FROM dog_owner owner INNER JOIN dog ON owner.id = dog.id ORDER BY owner.id ASC LIMIT 4, 1", jumpToItemQuery); + assertEquals("SELECT owner.id as ownerid, first_name, last_name, dog_name FROM dog_owner owner INNER JOIN dog ON owner.id = dog.id WHERE ((owner.id > ?)) ORDER BY owner.id ASC LIMIT 5", remainingPagesQuery); + } + @Override public String getFirstPageSqlWithMultipleSortKeys() { return "SELECT id, name, age FROM foo WHERE bar = 1 ORDER BY name ASC, id DESC LIMIT 100"; @@ -101,7 +125,7 @@ public String getFirstPageSqlWithMultipleSortKeys() { @Override public String getRemainingSqlWithMultipleSortKeys() { - return "SELECT id, name, age FROM foo WHERE bar = 1 AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC LIMIT 100"; + return "SELECT id, name, age FROM foo WHERE (bar = 1) AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC LIMIT 100"; } @Override diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/OraclePagingQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/OraclePagingQueryProviderTests.java index 4ee1a012e9..1eb9260e9a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/OraclePagingQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/OraclePagingQueryProviderTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/PostgresPagingQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/PostgresPagingQueryProviderTests.java index 3eaea7a1cd..d43ea4c8ed 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/PostgresPagingQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/PostgresPagingQueryProviderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2012 the original author or authors. + * Copyright 2012-2015 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -39,7 +39,7 @@ public void testGenerateFirstPageQuery() { @Test @Override public void testGenerateRemainingPagesQuery() { - String sql = "SELECT id, name, age FROM foo WHERE bar = 1 AND ((id > ?)) ORDER BY id ASC LIMIT 100"; + String sql = "SELECT id, name, age FROM foo WHERE (bar = 1) AND ((id > ?)) ORDER BY id ASC LIMIT 100"; String s = pagingQueryProvider.generateRemainingPagesQuery(pageSize); assertEquals(sql, s); } @@ -101,7 +101,7 @@ public String getFirstPageSqlWithMultipleSortKeys() { @Override public String getRemainingSqlWithMultipleSortKeys() { - return "SELECT id, name, age FROM foo WHERE bar = 1 AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC LIMIT 100"; + return "SELECT id, name, age FROM foo WHERE (bar = 1) AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC LIMIT 100"; } @Override diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlPagingQueryProviderFactoryBeanTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlPagingQueryProviderFactoryBeanTests.java index bd24154e1c..f10d67435a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlPagingQueryProviderFactoryBeanTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlPagingQueryProviderFactoryBeanTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,9 +19,6 @@ import static org.junit.Assert.assertNotNull; import static org.junit.Assert.assertTrue; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - import java.util.LinkedHashMap; import java.util.Map; @@ -46,7 +43,7 @@ public SqlPagingQueryProviderFactoryBeanTests() throws Exception { factory.setSelectClause("id, name, age"); factory.setFromClause("foo"); factory.setWhereClause("bar = 1"); - Map sortKeys = new LinkedHashMap(); + Map sortKeys = new LinkedHashMap<>(); sortKeys.put("id", Order.ASCENDING); factory.setSortKeys(sortKeys); DataSource dataSource = DatabaseTypeTestUtils.getMockDataSource(DatabaseType.HSQL.getProductName(), "100.0.0"); @@ -55,7 +52,7 @@ public SqlPagingQueryProviderFactoryBeanTests() throws Exception { @Test public void testFactory() throws Exception { - PagingQueryProvider provider = (PagingQueryProvider) factory.getObject(); + PagingQueryProvider provider = factory.getObject(); assertNotNull(provider); } @@ -72,28 +69,28 @@ public void testSingleton() throws Exception { @Test(expected=IllegalArgumentException.class) public void testNoDataSource() throws Exception { factory.setDataSource(null); - PagingQueryProvider provider = (PagingQueryProvider) factory.getObject(); + PagingQueryProvider provider = factory.getObject(); assertNotNull(provider); } @Test(expected=IllegalArgumentException.class) public void testNoSortKey() throws Exception { factory.setSortKeys(null); - PagingQueryProvider provider = (PagingQueryProvider) factory.getObject(); + PagingQueryProvider provider = factory.getObject(); assertNotNull(provider); } @Test public void testWhereClause() throws Exception { factory.setWhereClause("x=y"); - PagingQueryProvider provider = (PagingQueryProvider) factory.getObject(); + PagingQueryProvider provider = factory.getObject(); String query = provider.generateFirstPageQuery(100); assertTrue("Wrong query: "+query, query.contains("x=y")); } @Test public void testAscending() throws Exception { - PagingQueryProvider provider = (PagingQueryProvider) factory.getObject(); + PagingQueryProvider provider = factory.getObject(); String query = provider.generateFirstPageQuery(100); assertTrue("Wrong query: "+query, query.contains("ASC")); } @@ -101,14 +98,14 @@ public void testAscending() throws Exception { @Test(expected=IllegalArgumentException.class) public void testWrongDatabaseType() throws Exception { factory.setDatabaseType("NoSuchDb"); - PagingQueryProvider provider = (PagingQueryProvider) factory.getObject(); + PagingQueryProvider provider = factory.getObject(); assertNotNull(provider); } @Test(expected=IllegalArgumentException.class) public void testMissingMetaData() throws Exception { factory.setDataSource(DatabaseTypeTestUtils.getMockDataSource(new MetaDataAccessException("foo"))); - PagingQueryProvider provider = (PagingQueryProvider) factory.getObject(); + PagingQueryProvider provider = factory.getObject(); assertNotNull(provider); } @@ -116,7 +113,7 @@ public void testMissingMetaData() throws Exception { public void testAllDatabaseTypes() throws Exception { for (DatabaseType type : DatabaseType.values()) { factory.setDatabaseType(type.name()); - PagingQueryProvider provider = (PagingQueryProvider) factory.getObject(); + PagingQueryProvider provider = factory.getObject(); assertNotNull(provider); } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlPagingQueryUtilsTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlPagingQueryUtilsTests.java index 0acb9db464..e8b6d90849 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlPagingQueryUtilsTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlPagingQueryUtilsTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2015 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -39,7 +39,7 @@ public class SqlPagingQueryUtilsTests { @Before public void setUp() { - sortKeys = new LinkedHashMap(); + sortKeys = new LinkedHashMap<>(); sortKeys.put("ID", Order.ASCENDING); } @@ -53,7 +53,7 @@ public void testGenerateLimitSqlQuery() { qp.setWhereClause("BAZ IS NOT NULL"); assertEquals("SELECT FOO FROM BAR WHERE BAZ IS NOT NULL ORDER BY ID ASC LIMIT 100", SqlPagingQueryUtils .generateLimitSqlQuery(qp, false, "LIMIT 100")); - assertEquals("SELECT FOO FROM BAR WHERE BAZ IS NOT NULL AND ((ID > ?)) ORDER BY ID ASC LIMIT 100", + assertEquals("SELECT FOO FROM BAR WHERE (BAZ IS NOT NULL) AND ((ID > ?)) ORDER BY ID ASC LIMIT 100", SqlPagingQueryUtils.generateLimitSqlQuery(qp, true, "LIMIT 100")); } @@ -67,7 +67,7 @@ public void testGenerateTopSqlQuery() { qp.setWhereClause("BAZ IS NOT NULL"); assertEquals("SELECT TOP 100 FOO FROM BAR WHERE BAZ IS NOT NULL ORDER BY ID ASC", SqlPagingQueryUtils .generateTopSqlQuery(qp, false, "TOP 100")); - assertEquals("SELECT TOP 100 FOO FROM BAR WHERE BAZ IS NOT NULL AND ((ID > ?)) ORDER BY ID ASC", + assertEquals("SELECT TOP 100 FOO FROM BAR WHERE (BAZ IS NOT NULL) AND ((ID > ?)) ORDER BY ID ASC", SqlPagingQueryUtils.generateTopSqlQuery(qp, true, "TOP 100")); } @@ -108,7 +108,7 @@ public void testGenerateTopSqlQueryDescending() { qp.setWhereClause("BAZ IS NOT NULL"); assertEquals("SELECT TOP 100 FOO FROM BAR WHERE BAZ IS NOT NULL ORDER BY ID DESC", SqlPagingQueryUtils .generateTopSqlQuery(qp, false, "TOP 100")); - assertEquals("SELECT TOP 100 FOO FROM BAR WHERE BAZ IS NOT NULL AND ((ID < ?)) ORDER BY ID DESC", + assertEquals("SELECT TOP 100 FOO FROM BAR WHERE (BAZ IS NOT NULL) AND ((ID < ?)) ORDER BY ID DESC", SqlPagingQueryUtils.generateTopSqlQuery(qp, true, "TOP 100")); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlServerPagingQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlServerPagingQueryProviderTests.java index 10a411c56f..f18dcac8e0 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlServerPagingQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlServerPagingQueryProviderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2012 the original author or authors. + * Copyright 2012-2015 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,7 +40,7 @@ public void testGenerateFirstPageQuery() { @Test @Override public void testGenerateRemainingPagesQuery() { - String sql = "SELECT TOP 100 id, name, age FROM foo WHERE bar = 1 AND ((id > ?)) ORDER BY id ASC"; + String sql = "SELECT TOP 100 id, name, age FROM foo WHERE (bar = 1) AND ((id > ?)) ORDER BY id ASC"; String s = pagingQueryProvider.generateRemainingPagesQuery(pageSize); assertEquals(sql, s); } @@ -104,7 +104,7 @@ public String getFirstPageSqlWithMultipleSortKeys() { @Override public String getRemainingSqlWithMultipleSortKeys() { - return "SELECT TOP 100 id, name, age FROM foo WHERE bar = 1 AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC"; + return "SELECT TOP 100 id, name, age FROM foo WHERE (bar = 1) AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC"; } @Override diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlWindowingPagingQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlWindowingPagingQueryProviderTests.java index 42cf7d74a1..06e504d1e6 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlWindowingPagingQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlWindowingPagingQueryProviderTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,10 +15,14 @@ */ package org.springframework.batch.item.database.support; -import static org.junit.Assert.assertEquals; - import org.junit.Assert; import org.junit.Test; +import org.springframework.batch.item.database.Order; + +import java.util.HashMap; +import java.util.Map; + +import static org.junit.Assert.assertEquals; /** * @author Thomas Risberg @@ -97,6 +101,20 @@ public void testGenerateJumpToItemQueryForFirstPageWithGroupBy() { String s = pagingQueryProvider.generateJumpToItemQuery(45, pageSize); assertEquals(sql, s); } + + @Test + public void testGenerateJumpToItemQueryForTableQualifierReplacement() { + pagingQueryProvider.setFromClause("foo_e E, foo_i I"); + pagingQueryProvider.setWhereClause("E.id=I.id"); + + Map sortKeys = new HashMap<>(); + sortKeys.put("E.id", Order.DESCENDING); + pagingQueryProvider.setSortKeys(sortKeys); + + String sql="SELECT TMP_SUB.id FROM ( SELECT E.id, ROW_NUMBER() OVER ( ORDER BY id DESC) AS ROW_NUMBER FROM foo_e E, foo_i I WHERE E.id=I.id) AS TMP_SUB WHERE TMP_SUB.ROW_NUMBER = 1 ORDER BY TMP_SUB.id DESC"; + String s = pagingQueryProvider.generateJumpToItemQuery(45, pageSize); + assertEquals(sql, s); + } @Override public String getFirstPageSqlWithMultipleSortKeys() { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqliteMaxValueIncrementerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqliteMaxValueIncrementerTests.java new file mode 100644 index 0000000000..6fb46ce26e --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqliteMaxValueIncrementerTests.java @@ -0,0 +1,64 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.support; + +import static org.junit.Assert.assertEquals; + +import java.io.File; + +import org.junit.AfterClass; +import org.junit.BeforeClass; +import org.junit.Test; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.jdbc.datasource.SimpleDriverDataSource; + +/** + * @author Luke Taylor + */ +public class SqliteMaxValueIncrementerTests { + static String dbFile; + static SimpleDriverDataSource dataSource; + static JdbcTemplate template; + + @BeforeClass + public static void setUp() { + dbFile = System.getProperty("java.io.tmpdir") + File.separator + "batch_sqlite_inc.db"; + dataSource = new SimpleDriverDataSource(); + dataSource.setDriverClass(org.sqlite.JDBC.class); + dataSource.setUrl("jdbc:sqlite:" + dbFile); + template = new JdbcTemplate(dataSource); + template.execute("create table max_value (id integer primary key autoincrement)"); + } + + @AfterClass + public static void removeDbFile() { + File db = new File(dbFile); + if (db.exists()) { + db.delete(); + } + dataSource = null; + template = null; + } + + @Test + public void testNextKey() throws Exception { + SqliteMaxValueIncrementer mvi = new SqliteMaxValueIncrementer(dataSource, "max_value", "id"); + assertEquals(1, mvi.getNextKey()); + assertEquals(2, mvi.getNextKey()); + assertEquals(3, mvi.getNextKey()); + assertEquals(1, template.queryForObject("select count(*) from max_value", Integer.class).intValue()); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlitePagingQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlitePagingQueryProviderTests.java new file mode 100644 index 0000000000..7ff6becfb2 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SqlitePagingQueryProviderTests.java @@ -0,0 +1,117 @@ +/* + * Copyright 2014-2015 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.database.support; + +import static org.junit.Assert.assertEquals; + +import org.junit.Test; + +/** + * @author Thomas Risberg + * @author Michael Minella + * @author Luke Taylor + */ +public class SqlitePagingQueryProviderTests extends AbstractSqlPagingQueryProviderTests { + + public SqlitePagingQueryProviderTests() { + pagingQueryProvider = new MySqlPagingQueryProvider(); + } + + @Test + @Override + public void testGenerateFirstPageQuery() { + String sql = "SELECT id, name, age FROM foo WHERE bar = 1 ORDER BY id ASC LIMIT 100"; + String s = pagingQueryProvider.generateFirstPageQuery(pageSize); + assertEquals(sql, s); + } + + @Test @Override + public void testGenerateRemainingPagesQuery() { + String sql = "SELECT id, name, age FROM foo WHERE (bar = 1) AND ((id > ?)) ORDER BY id ASC LIMIT 100"; + String s = pagingQueryProvider.generateRemainingPagesQuery(pageSize); + assertEquals(sql, s); + } + + @Test @Override + public void testGenerateJumpToItemQuery() { + String sql = "SELECT id FROM foo WHERE bar = 1 ORDER BY id ASC LIMIT 99, 1"; + String s = pagingQueryProvider.generateJumpToItemQuery(145, pageSize); + assertEquals(sql, s); + } + + @Test @Override + public void testGenerateJumpToItemQueryForFirstPage() { + String sql = "SELECT id FROM foo WHERE bar = 1 ORDER BY id ASC LIMIT 0, 1"; + String s = pagingQueryProvider.generateJumpToItemQuery(45, pageSize); + assertEquals(sql, s); + } + + @Override + @Test + public void testGenerateFirstPageQueryWithGroupBy() { + pagingQueryProvider.setGroupClause("dep"); + String sql = "SELECT id, name, age FROM foo WHERE bar = 1 GROUP BY dep ORDER BY id ASC LIMIT 100"; + String s = pagingQueryProvider.generateFirstPageQuery(pageSize); + assertEquals(sql, s); + } + + @Override + @Test + public void testGenerateRemainingPagesQueryWithGroupBy() { + pagingQueryProvider.setGroupClause("dep"); + String sql = "SELECT * FROM (SELECT id, name, age FROM foo WHERE bar = 1 GROUP BY dep) AS MAIN_QRY WHERE ((id > ?)) ORDER BY id ASC LIMIT 100"; + String s = pagingQueryProvider.generateRemainingPagesQuery(pageSize); + assertEquals(sql, s); + } + + @Override + @Test + public void testGenerateJumpToItemQueryWithGroupBy() { + pagingQueryProvider.setGroupClause("dep"); + String sql = "SELECT id FROM foo WHERE bar = 1 GROUP BY dep ORDER BY id ASC LIMIT 99, 1"; + String s = pagingQueryProvider.generateJumpToItemQuery(145, pageSize); + assertEquals(sql, s); + } + + @Override + @Test + public void testGenerateJumpToItemQueryForFirstPageWithGroupBy() { + pagingQueryProvider.setGroupClause("dep"); + String sql = "SELECT id FROM foo WHERE bar = 1 GROUP BY dep ORDER BY id ASC LIMIT 0, 1"; + String s = pagingQueryProvider.generateJumpToItemQuery(45, pageSize); + assertEquals(sql, s); + } + + @Override + public String getFirstPageSqlWithMultipleSortKeys() { + return "SELECT id, name, age FROM foo WHERE bar = 1 ORDER BY name ASC, id DESC LIMIT 100"; + } + + @Override + public String getRemainingSqlWithMultipleSortKeys() { + return "SELECT id, name, age FROM foo WHERE (bar = 1) AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC LIMIT 100"; + } + + @Override + public String getJumpToItemQueryWithMultipleSortKeys() { + return "SELECT name, id FROM foo WHERE bar = 1 ORDER BY name ASC, id DESC LIMIT 99, 1"; + } + + @Override + public String getJumpToItemQueryForFirstPageWithMultipleSortKeys() { + return "SELECT name, id FROM foo WHERE bar = 1 ORDER BY name ASC, id DESC LIMIT 0, 1"; + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SybasePagingQueryProviderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SybasePagingQueryProviderTests.java index 83eaa12dd8..3600a2a2e8 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SybasePagingQueryProviderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/database/support/SybasePagingQueryProviderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2012 the original author or authors. + * Copyright 2012-2015 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,7 +40,7 @@ public void testGenerateFirstPageQuery() { @Test @Override public void testGenerateRemainingPagesQuery() { - String sql = "SELECT TOP 100 id, name, age FROM foo WHERE bar = 1 AND ((id > ?)) ORDER BY id ASC"; + String sql = "SELECT TOP 100 id, name, age FROM foo WHERE (bar = 1) AND ((id > ?)) ORDER BY id ASC"; String s = pagingQueryProvider.generateRemainingPagesQuery(pageSize); assertEquals("", sql, s); } @@ -104,7 +104,7 @@ public String getFirstPageSqlWithMultipleSortKeys() { @Override public String getRemainingSqlWithMultipleSortKeys() { - return "SELECT TOP 100 id, name, age FROM foo WHERE bar = 1 AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC"; + return "SELECT TOP 100 id, name, age FROM foo WHERE (bar = 1) AND ((name > ?) OR (name = ? AND id < ?)) ORDER BY name ASC, id DESC"; } @Override diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/AbstractMultiResourceItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/AbstractMultiResourceItemWriterTests.java index b122fc1377..08ca568b3a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/AbstractMultiResourceItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/AbstractMultiResourceItemWriterTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2016 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file; import java.io.BufferedReader; @@ -15,7 +30,7 @@ */ public class AbstractMultiResourceItemWriterTests { - protected MultiResourceItemWriter tested = new MultiResourceItemWriter(); + protected MultiResourceItemWriter tested; protected File file; @@ -24,13 +39,16 @@ public class AbstractMultiResourceItemWriterTests { protected ExecutionContext executionContext = new ExecutionContext(); protected void setUp(ResourceAwareItemWriterItemStream delegate) throws Exception { - file = File.createTempFile(MultiResourceItemWriterFlatFileTests.class.getSimpleName(), null); + tested = new MultiResourceItemWriter<>(); tested.setResource(new FileSystemResource(file)); tested.setDelegate(delegate); tested.setResourceSuffixCreator(suffixCreator); tested.setItemCountLimitPerResource(2); tested.setSaveState(true); - tested.open(executionContext); + } + + public void createFile() throws Exception { + file = File.createTempFile(MultiResourceItemWriterFlatFileTests.class.getSimpleName(), null); } protected String readFile(File f) throws Exception { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/DefaultBufferedReaderFactoryTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/DefaultBufferedReaderFactoryTests.java index c249421475..0ca3073fd6 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/DefaultBufferedReaderFactoryTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/DefaultBufferedReaderFactoryTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,6 +31,7 @@ public class DefaultBufferedReaderFactoryTests { @Test public void testCreate() throws Exception { DefaultBufferedReaderFactory factory = new DefaultBufferedReaderFactory(); + @SuppressWarnings("resource") BufferedReader reader = factory.create(new ByteArrayResource("a\nb\nc".getBytes()), "UTF-8"); assertEquals("a", reader.readLine()); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileItemReaderCommonTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileItemReaderCommonTests.java index 00afeb397d..3499ab8231 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileItemReaderCommonTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileItemReaderCommonTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file; import org.springframework.batch.item.AbstractItemStreamItemReaderTests; @@ -16,7 +31,7 @@ public class FlatFileItemReaderCommonTests extends AbstractItemStreamItemReaderT @Override protected ItemReader getItemReader() throws Exception { - FlatFileItemReader tested = new FlatFileItemReader(); + FlatFileItemReader tested = new FlatFileItemReader<>(); Resource resource = new ByteArrayResource(FOOS.getBytes()); tested.setResource(resource); tested.setLineMapper(new LineMapper() { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileItemReaderTests.java index 7c8bd5b94a..32865878bf 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileItemReaderTests.java @@ -1,560 +1,584 @@ -package org.springframework.batch.item.file; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - -import java.io.IOException; -import java.io.InputStream; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.item.ItemCountAware; -import org.springframework.batch.item.ItemStreamException; -import org.springframework.batch.item.file.mapping.PassThroughLineMapper; -import org.springframework.batch.item.file.separator.RecordSeparatorPolicy; -import org.springframework.core.io.AbstractResource; -import org.springframework.core.io.ByteArrayResource; -import org.springframework.core.io.FileSystemResource; -import org.springframework.core.io.Resource; -import org.springframework.util.ClassUtils; -import org.springframework.util.StringUtils; - -/** - * Tests for {@link FlatFileItemReader}. - */ -public class FlatFileItemReaderTests { - - // common value used for writing to a file - private String TEST_STRING = "FlatFileInputTemplate-TestData"; - - private FlatFileItemReader reader = new FlatFileItemReader(); - - private FlatFileItemReader itemReader = new FlatFileItemReader(); - - private ExecutionContext executionContext = new ExecutionContext(); - - @Before - public void setUp() { - - reader.setResource(getInputResource("testLine1\ntestLine2\ntestLine3\ntestLine4\ntestLine5\ntestLine6")); - reader.setLineMapper(new PassThroughLineMapper()); - - itemReader.setResource(getInputResource("testLine1\ntestLine2\ntestLine3\ntestLine4\ntestLine5\ntestLine6")); - itemReader.setLineMapper(new ItemLineMapper()); - } - - @Test - public void testRestartWithCustomRecordSeparatorPolicy() throws Exception { - - reader.setRecordSeparatorPolicy(new RecordSeparatorPolicy() { - // 1 record = 2 lines - boolean pair = true; - - @Override - public boolean isEndOfRecord(String line) { - pair = !pair; - return pair; - } - - @Override - public String postProcess(String record) { - return record; - } - - @Override - public String preProcess(String record) { - return record; - } - }); - - reader.open(executionContext); - - assertEquals("testLine1testLine2", reader.read()); - assertEquals("testLine3testLine4", reader.read()); - - reader.update(executionContext); - - reader.close(); - - reader.open(executionContext); - - assertEquals("testLine5testLine6", reader.read()); - } - - @Test - public void testCustomRecordSeparatorPolicyEndOfFile() throws Exception { - - reader.setRecordSeparatorPolicy(new RecordSeparatorPolicy() { - // 1 record = 2 lines - boolean pair = true; - - @Override - public boolean isEndOfRecord(String line) { - pair = !pair; - return pair; - } - - @Override - public String postProcess(String record) { - return record; - } - - @Override - public String preProcess(String record) { - return record; - } - }); - - reader.setResource(getInputResource("testLine1\ntestLine2\ntestLine3\n")); - reader.open(executionContext); - - assertEquals("testLine1testLine2", reader.read()); - - try { - reader.read(); - fail("Expected Exception"); - } - catch (FlatFileParseException e) { - // File ends in the middle of a record - assertEquals(3, e.getLineNumber()); - assertEquals("testLine3", e.getInput()); - } - - } - - @Test - public void testCustomRecordSeparatorBlankLine() throws Exception { - - reader.setRecordSeparatorPolicy(new RecordSeparatorPolicy() { - - @Override - public boolean isEndOfRecord(String line) { - return StringUtils.hasText(line); - } - - @Override - public String postProcess(String record) { - return StringUtils.hasText(record) ? record : null; - } - - @Override - public String preProcess(String record) { - return record; - } - }); - - reader.setResource(getInputResource("testLine1\ntestLine2\ntestLine3\n\n")); - reader.open(executionContext); - - assertEquals("testLine1", reader.read()); - assertEquals("testLine2", reader.read()); - assertEquals("testLine3", reader.read()); - assertEquals(null, reader.read()); - - } - - @Test - public void testCustomRecordSeparatorMultilineBlankLineAfterEnd() throws Exception { - - reader.setRecordSeparatorPolicy(new RecordSeparatorPolicy() { - - // 1 record = 2 lines - boolean pair = true; - - @Override - public boolean isEndOfRecord(String line) { - if (StringUtils.hasText(line)) { - pair = !pair; - } - return pair; - } - - @Override - public String postProcess(String record) { - return StringUtils.hasText(record) ? record : null; - } - - @Override - public String preProcess(String record) { - return record; - } - }); - - reader.setResource(getInputResource("testLine1\ntestLine2\n\n")); - reader.open(executionContext); - - assertEquals("testLine1testLine2", reader.read()); - assertEquals(null, reader.read()); - - } - - @Test - public void testRestartWithSkippedLines() throws Exception { - - reader.setLinesToSkip(2); - reader.open(executionContext); - - // read some records - reader.read(); - reader.read(); - // get restart data - reader.update(executionContext); - // read next two records - reader.read(); - reader.read(); - - assertEquals(2, executionContext.getInt(ClassUtils.getShortName(FlatFileItemReader.class) + ".read.count")); - // close input - reader.close(); - - reader.setResource(getInputResource("header\nignoreme\ntestLine1\ntestLine2\ntestLine3\ntestLine4\ntestLine5\ntestLine6")); - - // init for restart - reader.open(executionContext); - - // read remaining records - assertEquals("testLine3", reader.read()); - assertEquals("testLine4", reader.read()); - - reader.update(executionContext); - assertEquals(4, executionContext.getInt(ClassUtils.getShortName(FlatFileItemReader.class) + ".read.count")); - } - - @Test - public void testCurrentItemCount() throws Exception { - - reader.setCurrentItemCount(2); - reader.open(executionContext); - - // read some records - reader.read(); - reader.read(); - // get restart data - reader.update(executionContext); - - assertEquals(4, executionContext.getInt(ClassUtils.getShortName(FlatFileItemReader.class) + ".read.count")); - // close input - reader.close(); - - } - - @Test - public void testMaxItemCount() throws Exception { - - reader.setMaxItemCount(2); - reader.open(executionContext); - - // read some records - reader.read(); - reader.read(); - // get restart data - reader.update(executionContext); - assertNull(reader.read()); - - assertEquals(2, executionContext.getInt(ClassUtils.getShortName(FlatFileItemReader.class) + ".read.count")); - // close input - reader.close(); - - } - - @Test - public void testMaxItemCountFromContext() throws Exception { - - reader.setMaxItemCount(2); - executionContext.putInt(reader.getClass().getSimpleName() + ".read.count.max", Integer.MAX_VALUE); - reader.open(executionContext); - // read some records - reader.read(); - reader.read(); - assertNotNull(reader.read()); - // close input - reader.close(); - - } - - @Test - public void testCurrentItemCountFromContext() throws Exception { - - reader.setCurrentItemCount(2); - executionContext.putInt(reader.getClass().getSimpleName() + ".read.count", 3); - reader.open(executionContext); - // read some records - assertEquals("testLine4", reader.read()); - // close input - reader.close(); - - } - - @Test - public void testMaxAndCurrentItemCount() throws Exception { - - reader.setMaxItemCount(2); - reader.setCurrentItemCount(2); - reader.open(executionContext); - // read some records - assertNull(reader.read()); - // close input - reader.close(); - - } - - @Test - public void testNonExistentResource() throws Exception { - - Resource resource = new NonExistentResource(); - - reader.setResource(resource); - - // afterPropertiesSet should only throw an exception if the Resource is - // null - reader.afterPropertiesSet(); - - reader.setStrict(false); - reader.open(executionContext); - assertNull(reader.read()); - reader.close(); - } - - @Test - public void testOpenBadIOInput() throws Exception { - - reader.setResource(new AbstractResource() { - @Override - public String getDescription() { - return null; - } - - @Override - public InputStream getInputStream() throws IOException { - throw new IOException(); - } - - @Override - public boolean exists() { - return true; - } - }); - - try { - reader.open(executionContext); - fail(); - } - catch (ItemStreamException ex) { - // expected - } - - // read() should then return a null - assertNull(reader.read()); - reader.close(); - - } - - @Test - public void testDirectoryResource() throws Exception { - - FileSystemResource resource = new FileSystemResource("target/data"); - resource.getFile().mkdirs(); - assertTrue(resource.getFile().isDirectory()); - reader.setResource(resource); - reader.afterPropertiesSet(); - - reader.setStrict(false); - reader.open(executionContext); - assertNull(reader.read()); - - } - - @Test - public void testRuntimeFileCreation() throws Exception { - - Resource resource = new NonExistentResource(); - - reader.setResource(resource); - - // afterPropertiesSet should only throw an exception if the Resource is - // null - reader.afterPropertiesSet(); - - // replace the resource to simulate runtime resource creation - reader.setResource(getInputResource(TEST_STRING)); - reader.open(executionContext); - assertEquals(TEST_STRING, reader.read()); - } - - /** - * In strict mode, resource must exist at the time reader is opened. - */ - @Test(expected = ItemStreamException.class) - public void testStrictness() throws Exception { - - Resource resource = new NonExistentResource(); - - reader.setResource(resource); - reader.setStrict(true); - - reader.afterPropertiesSet(); - - reader.open(executionContext); - } - - /** - * Exceptions from {@link LineMapper} are wrapped as {@link FlatFileParseException} containing contextual info about - * the problematic line and its line number. - */ - @Test - public void testMappingExceptionWrapping() throws Exception { - LineMapper exceptionLineMapper = new LineMapper() { - @Override - public String mapLine(String line, int lineNumber) throws Exception { - if (lineNumber == 2) { - throw new Exception("Couldn't map line 2"); - } - return line; - } - }; - reader.setLineMapper(exceptionLineMapper); - reader.afterPropertiesSet(); - - reader.open(executionContext); - assertNotNull(reader.read()); - - try { - reader.read(); - fail(); - } - catch (FlatFileParseException expected) { - assertEquals(2, expected.getLineNumber()); - assertEquals("testLine2", expected.getInput()); - assertEquals("Couldn't map line 2", expected.getCause().getMessage()); - assertEquals("Parsing error at line: 2 in resource=[resource loaded from byte array], input=[testLine2]", expected.getMessage()); - } - } - - @Test - public void testItemCountAware() throws Exception { - itemReader.open(executionContext); - Item item1 = itemReader.read(); - assertEquals("testLine1", item1.getValue()); - assertEquals(1, item1.getItemCount()); - Item item2 = itemReader.read(); - assertEquals("testLine2", item2.getValue()); - assertEquals(2, item2.getItemCount()); - itemReader.update(executionContext); - itemReader.close(); - - itemReader.open(executionContext); - Item item3 = itemReader.read(); - assertEquals("testLine3", item3.getValue()); - assertEquals(3, item3.getItemCount()); - } - - @Test - public void testItemCountAwareMultiLine() throws Exception { - itemReader.setRecordSeparatorPolicy(new RecordSeparatorPolicy() { - - // 1 record = 2 lines - boolean pair = true; - - @Override - public boolean isEndOfRecord(String line) { - if (StringUtils.hasText(line)) { - pair = !pair; - } - return pair; - } - - @Override - public String postProcess(String record) { - return StringUtils.hasText(record) ? record : null; - } - - @Override - public String preProcess(String record) { - return record; - } - }); - - itemReader.open(executionContext); - Item item1 = itemReader.read(); - assertEquals("testLine1testLine2", item1.getValue()); - assertEquals(1, item1.getItemCount()); - Item item2 = itemReader.read(); - assertEquals("testLine3testLine4", item2.getValue()); - assertEquals(2, item2.getItemCount()); - itemReader.update(executionContext); - itemReader.close(); - - itemReader.open(executionContext); - Item item3 = itemReader.read(); - assertEquals("testLine5testLine6", item3.getValue()); - assertEquals(3, item3.getItemCount()); - } - - private Resource getInputResource(String input) { - return new ByteArrayResource(input.getBytes()); - } - - private static class NonExistentResource extends AbstractResource { - - public NonExistentResource() { - } - - @Override - public boolean exists() { - return false; - } - - @Override - public String getDescription() { - return "NonExistentResource"; - } - - @Override - public InputStream getInputStream() throws IOException { - return null; - } - } - - private static class Item implements ItemCountAware { - - private String value; - - private int itemCount; - - public Item(String value) { - this.value = value; - } - - @SuppressWarnings("unused") - public void setValue(String value) { - this.value = value; - } - - public String getValue() { - return value; - } - - @Override - public void setItemCount(int count) { - this.itemCount = count; - } - - public int getItemCount() { - return itemCount; - } - - } - - private static final class ItemLineMapper implements LineMapper { - - @Override - public Item mapLine(String line, int lineNumber) throws Exception { - return new Item(line); - } - - } -} +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.file; + +import static org.hamcrest.Matchers.endsWith; +import static org.hamcrest.Matchers.startsWith; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertThat; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +import java.io.IOException; +import java.io.InputStream; + +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemCountAware; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.file.mapping.PassThroughLineMapper; +import org.springframework.batch.item.file.separator.RecordSeparatorPolicy; +import org.springframework.core.io.AbstractResource; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.FileSystemResource; +import org.springframework.core.io.Resource; +import org.springframework.util.ClassUtils; +import org.springframework.util.StringUtils; + +/** + * Tests for {@link FlatFileItemReader}. + */ +public class FlatFileItemReaderTests { + + // common value used for writing to a file + private String TEST_STRING = "FlatFileInputTemplate-TestData"; + + private FlatFileItemReader reader = new FlatFileItemReader<>(); + + private FlatFileItemReader itemReader = new FlatFileItemReader<>(); + + private ExecutionContext executionContext = new ExecutionContext(); + + private Resource inputResource2 = getInputResource("testLine1\ntestLine2\ntestLine3\ntestLine4\ntestLine5\ntestLine6"); + + private Resource inputResource1 = getInputResource("testLine1\ntestLine2\ntestLine3\ntestLine4\ntestLine5\ntestLine6"); + + @Before + public void setUp() { + + reader.setResource(inputResource1); + reader.setLineMapper(new PassThroughLineMapper()); + + itemReader.setResource(inputResource2); + itemReader.setLineMapper(new ItemLineMapper()); + } + + @Test + public void testRestartWithCustomRecordSeparatorPolicy() throws Exception { + + reader.setRecordSeparatorPolicy(new RecordSeparatorPolicy() { + // 1 record = 2 lines + boolean pair = true; + + @Override + public boolean isEndOfRecord(String line) { + pair = !pair; + return pair; + } + + @Override + public String postProcess(String record) { + return record; + } + + @Override + public String preProcess(String record) { + return record; + } + }); + + reader.open(executionContext); + + assertEquals("testLine1testLine2", reader.read()); + assertEquals("testLine3testLine4", reader.read()); + + reader.update(executionContext); + + reader.close(); + + reader.open(executionContext); + + assertEquals("testLine5testLine6", reader.read()); + } + + @Test + public void testCustomRecordSeparatorPolicyEndOfFile() throws Exception { + + reader.setRecordSeparatorPolicy(new RecordSeparatorPolicy() { + // 1 record = 2 lines + boolean pair = true; + + @Override + public boolean isEndOfRecord(String line) { + pair = !pair; + return pair; + } + + @Override + public String postProcess(String record) { + return record; + } + + @Override + public String preProcess(String record) { + return record; + } + }); + + reader.setResource(getInputResource("testLine1\ntestLine2\ntestLine3\n")); + reader.open(executionContext); + + assertEquals("testLine1testLine2", reader.read()); + + try { + reader.read(); + fail("Expected Exception"); + } + catch (FlatFileParseException e) { + // File ends in the middle of a record + assertEquals(3, e.getLineNumber()); + assertEquals("testLine3", e.getInput()); + } + + } + + @Test + public void testCustomRecordSeparatorBlankLine() throws Exception { + + reader.setRecordSeparatorPolicy(new RecordSeparatorPolicy() { + + @Override + public boolean isEndOfRecord(String line) { + return StringUtils.hasText(line); + } + + @Override + public String postProcess(String record) { + return StringUtils.hasText(record) ? record : null; + } + + @Override + public String preProcess(String record) { + return record; + } + }); + + reader.setResource(getInputResource("testLine1\ntestLine2\ntestLine3\n\n")); + reader.open(executionContext); + + assertEquals("testLine1", reader.read()); + assertEquals("testLine2", reader.read()); + assertEquals("testLine3", reader.read()); + assertEquals(null, reader.read()); + + } + + @Test + public void testCustomRecordSeparatorMultilineBlankLineAfterEnd() throws Exception { + + reader.setRecordSeparatorPolicy(new RecordSeparatorPolicy() { + + // 1 record = 2 lines + boolean pair = true; + + @Override + public boolean isEndOfRecord(String line) { + if (StringUtils.hasText(line)) { + pair = !pair; + } + return pair; + } + + @Override + public String postProcess(String record) { + return StringUtils.hasText(record) ? record : null; + } + + @Override + public String preProcess(String record) { + return record; + } + }); + + reader.setResource(getInputResource("testLine1\ntestLine2\n\n")); + reader.open(executionContext); + + assertEquals("testLine1testLine2", reader.read()); + assertEquals(null, reader.read()); + + } + + @Test + public void testRestartWithSkippedLines() throws Exception { + + reader.setLinesToSkip(2); + reader.open(executionContext); + + // read some records + reader.read(); + reader.read(); + // get restart data + reader.update(executionContext); + // read next two records + reader.read(); + reader.read(); + + assertEquals(2, executionContext.getInt(ClassUtils.getShortName(FlatFileItemReader.class) + ".read.count")); + // close input + reader.close(); + + reader.setResource(getInputResource("header\nignoreme\ntestLine1\ntestLine2\ntestLine3\ntestLine4\ntestLine5\ntestLine6")); + + // init for restart + reader.open(executionContext); + + // read remaining records + assertEquals("testLine3", reader.read()); + assertEquals("testLine4", reader.read()); + + reader.update(executionContext); + assertEquals(4, executionContext.getInt(ClassUtils.getShortName(FlatFileItemReader.class) + ".read.count")); + } + + @Test + public void testCurrentItemCount() throws Exception { + + reader.setCurrentItemCount(2); + reader.open(executionContext); + + // read some records + reader.read(); + reader.read(); + // get restart data + reader.update(executionContext); + + assertEquals(4, executionContext.getInt(ClassUtils.getShortName(FlatFileItemReader.class) + ".read.count")); + // close input + reader.close(); + + } + + @Test + public void testMaxItemCount() throws Exception { + + reader.setMaxItemCount(2); + reader.open(executionContext); + + // read some records + reader.read(); + reader.read(); + // get restart data + reader.update(executionContext); + assertNull(reader.read()); + + assertEquals(2, executionContext.getInt(ClassUtils.getShortName(FlatFileItemReader.class) + ".read.count")); + // close input + reader.close(); + + } + + @Test + public void testMaxItemCountFromContext() throws Exception { + + reader.setMaxItemCount(2); + executionContext.putInt(reader.getClass().getSimpleName() + ".read.count.max", Integer.MAX_VALUE); + reader.open(executionContext); + // read some records + reader.read(); + reader.read(); + assertNotNull(reader.read()); + // close input + reader.close(); + + } + + @Test + public void testCurrentItemCountFromContext() throws Exception { + + reader.setCurrentItemCount(2); + executionContext.putInt(reader.getClass().getSimpleName() + ".read.count", 3); + reader.open(executionContext); + // read some records + assertEquals("testLine4", reader.read()); + // close input + reader.close(); + + } + + @Test + public void testMaxAndCurrentItemCount() throws Exception { + + reader.setMaxItemCount(2); + reader.setCurrentItemCount(2); + reader.open(executionContext); + // read some records + assertNull(reader.read()); + // close input + reader.close(); + + } + + @Test + public void testNonExistentResource() throws Exception { + + Resource resource = new NonExistentResource(); + + reader.setResource(resource); + + // afterPropertiesSet should only throw an exception if the Resource is + // null + reader.afterPropertiesSet(); + + reader.setStrict(false); + reader.open(executionContext); + assertNull(reader.read()); + reader.close(); + } + + @Test + public void testOpenBadIOInput() throws Exception { + + reader.setResource(new AbstractResource() { + @Override + public String getDescription() { + return null; + } + + @Override + public InputStream getInputStream() throws IOException { + throw new IOException(); + } + + @Override + public boolean exists() { + return true; + } + }); + + try { + reader.open(executionContext); + fail(); + } + catch (ItemStreamException ex) { + // expected + } + + // read() should then return a null + assertNull(reader.read()); + reader.close(); + + } + + @Test + public void testDirectoryResource() throws Exception { + + FileSystemResource resource = new FileSystemResource("build/data"); + resource.getFile().mkdirs(); + assertTrue(resource.getFile().isDirectory()); + reader.setResource(resource); + reader.afterPropertiesSet(); + + reader.setStrict(false); + reader.open(executionContext); + assertNull(reader.read()); + + } + + @Test + public void testRuntimeFileCreation() throws Exception { + + Resource resource = new NonExistentResource(); + + reader.setResource(resource); + + // afterPropertiesSet should only throw an exception if the Resource is + // null + reader.afterPropertiesSet(); + + // replace the resource to simulate runtime resource creation + reader.setResource(getInputResource(TEST_STRING)); + reader.open(executionContext); + assertEquals(TEST_STRING, reader.read()); + } + + /** + * In strict mode, resource must exist at the time reader is opened. + */ + @Test(expected = ItemStreamException.class) + public void testStrictness() throws Exception { + + Resource resource = new NonExistentResource(); + + reader.setResource(resource); + reader.setStrict(true); + + reader.afterPropertiesSet(); + + reader.open(executionContext); + } + + /** + * Exceptions from {@link LineMapper} are wrapped as {@link FlatFileParseException} containing contextual info about + * the problematic line and its line number. + */ + @Test + public void testMappingExceptionWrapping() throws Exception { + LineMapper exceptionLineMapper = new LineMapper() { + @Override + public String mapLine(String line, int lineNumber) throws Exception { + if (lineNumber == 2) { + throw new Exception("Couldn't map line 2"); + } + return line; + } + }; + reader.setLineMapper(exceptionLineMapper); + reader.afterPropertiesSet(); + + reader.open(executionContext); + assertNotNull(reader.read()); + + try { + reader.read(); + fail(); + } + catch (FlatFileParseException expected) { + assertEquals(2, expected.getLineNumber()); + assertEquals("testLine2", expected.getInput()); + assertEquals("Couldn't map line 2", expected.getCause().getMessage()); + assertThat(expected.getMessage(), startsWith("Parsing error at line: 2 in resource=[")); + assertThat(expected.getMessage(), endsWith("], input=[testLine2]")); + } + } + + @Test + public void testItemCountAware() throws Exception { + itemReader.open(executionContext); + Item item1 = itemReader.read(); + assertEquals("testLine1", item1.getValue()); + assertEquals(1, item1.getItemCount()); + Item item2 = itemReader.read(); + assertEquals("testLine2", item2.getValue()); + assertEquals(2, item2.getItemCount()); + itemReader.update(executionContext); + itemReader.close(); + + itemReader.open(executionContext); + Item item3 = itemReader.read(); + assertEquals("testLine3", item3.getValue()); + assertEquals(3, item3.getItemCount()); + } + + @Test + public void testItemCountAwareMultiLine() throws Exception { + itemReader.setRecordSeparatorPolicy(new RecordSeparatorPolicy() { + + // 1 record = 2 lines + boolean pair = true; + + @Override + public boolean isEndOfRecord(String line) { + if (StringUtils.hasText(line)) { + pair = !pair; + } + return pair; + } + + @Override + public String postProcess(String record) { + return StringUtils.hasText(record) ? record : null; + } + + @Override + public String preProcess(String record) { + return record; + } + }); + + itemReader.open(executionContext); + Item item1 = itemReader.read(); + assertEquals("testLine1testLine2", item1.getValue()); + assertEquals(1, item1.getItemCount()); + Item item2 = itemReader.read(); + assertEquals("testLine3testLine4", item2.getValue()); + assertEquals(2, item2.getItemCount()); + itemReader.update(executionContext); + itemReader.close(); + + itemReader.open(executionContext); + Item item3 = itemReader.read(); + assertEquals("testLine5testLine6", item3.getValue()); + assertEquals(3, item3.getItemCount()); + } + + private Resource getInputResource(String input) { + return new ByteArrayResource(input.getBytes()); + } + + private static class NonExistentResource extends AbstractResource { + + public NonExistentResource() { + } + + @Override + public boolean exists() { + return false; + } + + @Override + public String getDescription() { + return "NonExistentResource"; + } + + @Override + public InputStream getInputStream() throws IOException { + return null; + } + } + + private static class Item implements ItemCountAware { + + private String value; + + private int itemCount; + + public Item(String value) { + this.value = value; + } + + @SuppressWarnings("unused") + public void setValue(String value) { + this.value = value; + } + + public String getValue() { + return value; + } + + @Override + public void setItemCount(int count) { + this.itemCount = count; + } + + public int getItemCount() { + return itemCount; + } + + } + + private static final class ItemLineMapper implements LineMapper { + + @Override + public Item mapLine(String line, int lineNumber) throws Exception { + return new Item(line); + } + + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileItemWriterTests.java index 4e9c0a7642..1edadb1fc7 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileItemWriterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,13 +16,6 @@ package org.springframework.batch.item.file; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - import java.io.BufferedReader; import java.io.File; import java.io.FileInputStream; @@ -38,6 +31,7 @@ import org.junit.After; import org.junit.Before; import org.junit.Test; + import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemStreamException; import org.springframework.batch.item.UnexpectedInputException; @@ -52,6 +46,13 @@ import org.springframework.transaction.support.TransactionTemplate; import org.springframework.util.ClassUtils; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + /** * Tests of regular usage for {@link FlatFileItemWriter} Exception cases will be in separate TestCase classes with * different setUp and tearDown methods @@ -63,7 +64,7 @@ public class FlatFileItemWriterTests { // object under test - private FlatFileItemWriter writer = new FlatFileItemWriter(); + private FlatFileItemWriter writer = new FlatFileItemWriter<>(); // String to be written into file by the FlatFileInputTemplate private static final String TEST_STRING = "FlatFileOutputTemplateTest-OutputData"; @@ -86,7 +87,7 @@ public void setUp() throws Exception { writer.setResource(new FileSystemResource(outputFile)); writer.setLineSeparator("\n"); - writer.setLineAggregator(new PassThroughLineAggregator()); + writer.setLineAggregator(new PassThroughLineAggregator<>()); writer.afterPropertiesSet(); writer.setSaveState(true); writer.setEncoding("UTF-8"); @@ -176,6 +177,8 @@ public void testWriteWithAppend() throws Exception { @Test public void testWriteWithAppendRestartOnSecondChunk() throws Exception { + // This should be overridden via the writer#setAppendAllowed(true) + writer.setShouldDeleteIfExists(true); writer.setAppendAllowed(true); writer.open(executionContext); writer.write(Collections.singletonList("test1")); @@ -355,9 +358,9 @@ private void writeStringTransactionCheck(final String expectedInTransaction) { PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); writer.open(executionContext); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { writer.write(Collections.singletonList(TEST_STRING)); assertEquals(expectedInTransaction, readLine()); @@ -388,9 +391,9 @@ public void writeFooter(Writer writer) throws IOException { PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { // write some lines writer.write(Arrays.asList(new String[] { "testLine1", "testLine2", "testLine3" })); @@ -411,9 +414,9 @@ public Object doInTransaction(TransactionStatus status) { // init with correct data writer.open(executionContext); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { // write more lines writer.write(Arrays.asList(new String[] { "testLine6", "testLine7", "testLine8" })); @@ -468,9 +471,9 @@ public void writeFooter(Writer writer) throws IOException { PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { // write some lines writer.write(Arrays.asList(new String[] { "téstLine1", "téstLine2", "téstLine3" })); @@ -491,9 +494,9 @@ public Object doInTransaction(TransactionStatus status) { // init with correct data writer.open(executionContext); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { // write more lines writer.write(Arrays.asList(new String[] { "téstLine6", "téstLine7", "téstLine8" })); @@ -522,9 +525,9 @@ public Object doInTransaction(TransactionStatus status) { @Test public void testOpenWithNonWritableFile() throws Exception { - writer = new FlatFileItemWriter(); - writer.setLineAggregator(new PassThroughLineAggregator()); - FileSystemResource file = new FileSystemResource("target/no-such-file.foo"); + writer = new FlatFileItemWriter<>(); + writer.setLineAggregator(new PassThroughLineAggregator<>()); + FileSystemResource file = new FileSystemResource("build/no-such-file.foo"); writer.setResource(file); new File(file.getFile().getParent()).mkdirs(); file.getFile().createNewFile(); @@ -544,7 +547,7 @@ public void testOpenWithNonWritableFile() throws Exception { @Test public void testAfterPropertiesSetChecksMandatory() throws Exception { - writer = new FlatFileItemWriter(); + writer = new FlatFileItemWriter<>(); try { writer.afterPropertiesSet(); fail("Expected IllegalArgumentException"); @@ -556,9 +559,9 @@ public void testAfterPropertiesSetChecksMandatory() throws Exception { @Test public void testDefaultStreamContext() throws Exception { - writer = new FlatFileItemWriter(); + writer = new FlatFileItemWriter<>(); writer.setResource(new FileSystemResource(outputFile)); - writer.setLineAggregator(new PassThroughLineAggregator()); + writer.setLineAggregator(new PassThroughLineAggregator<>()); writer.afterPropertiesSet(); writer.setSaveState(true); writer.open(executionContext); @@ -808,6 +811,7 @@ public String aggregate(String item) { return item; } }); + @SuppressWarnings("serial") List items = new ArrayList() { { add("1"); @@ -834,7 +838,7 @@ public String aggregate(String item) { * If append=true a new output file should still be created on the first run (not restart). */ public void testAppendToNotYetExistingFile() throws Exception { - Resource toBeCreated = new FileSystemResource("target/FlatFileItemWriterTests.out"); + Resource toBeCreated = new FileSystemResource("build/FlatFileItemWriterTests.out"); outputFile = toBeCreated.getFile(); //enable easy content reading and auto-delete the file diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileParseExceptionTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileParseExceptionTests.java index fe21a4f47e..4d934defab 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileParseExceptionTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/FlatFileParseExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderFlatFileTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderFlatFileTests.java index 46769d2b00..70f5124e05 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderFlatFileTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderFlatFileTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file; import java.util.Comparator; @@ -18,8 +33,8 @@ public class MultiResourceItemReaderFlatFileTests extends @Override protected ItemReader getItemReader() throws Exception { - MultiResourceItemReader multiReader = new MultiResourceItemReader(); - FlatFileItemReader fileReader = new FlatFileItemReader(); + MultiResourceItemReader multiReader = new MultiResourceItemReader<>(); + FlatFileItemReader fileReader = new FlatFileItemReader<>(); fileReader.setLineMapper(new LineMapper() { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderIntegrationTests.java index 05f98fb29b..7fa57665d6 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderIntegrationTests.java @@ -1,7 +1,22 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file; import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNull; +import static org.junit.Assert.*; import static org.junit.Assert.assertSame; import static org.junit.Assert.assertTrue; import static org.junit.Assert.fail; @@ -12,13 +27,18 @@ import org.junit.Before; import org.junit.Test; + import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.NonTransientResourceException; +import org.springframework.batch.item.ParseException; +import org.springframework.batch.item.UnexpectedInputException; import org.springframework.batch.item.file.mapping.PassThroughLineMapper; import org.springframework.core.io.AbstractResource; import org.springframework.core.io.ByteArrayResource; import org.springframework.core.io.FileSystemResource; import org.springframework.core.io.Resource; +import org.springframework.lang.Nullable; import org.springframework.test.util.ReflectionTestUtils; /** @@ -26,9 +46,9 @@ */ public class MultiResourceItemReaderIntegrationTests { - private MultiResourceItemReader tested = new MultiResourceItemReader(); + private MultiResourceItemReader tested = new MultiResourceItemReader<>(); - private FlatFileItemReader itemReader = new FlatFileItemReader(); + private FlatFileItemReader itemReader = new FlatFileItemReader<>(); private ExecutionContext ctx = new ExecutionContext(); @@ -229,8 +249,8 @@ public void testResourceOrderingWithCustomComparator() { */ @Override public int compare(Resource o1, Resource o2) { - Resource r1 = (Resource) o1; - Resource r2 = (Resource) o2; + Resource r1 = o1; + Resource r2 = o2; return -r1.getDescription().compareTo(r2.getDescription()); } @@ -274,12 +294,31 @@ public void testNonExistentResources() throws Exception { tested.close(); } + /** + * Test {@link org.springframework.batch.item.ItemStream} lifecycle symmetry + */ + @Test + public void testNonExistentResourcesItemStreamLifecycle() throws Exception { + ItemStreamReaderImpl delegate = new ItemStreamReaderImpl(); + tested.setDelegate(delegate); + tested.setResources(new Resource[] { }); + itemReader.setStrict(false); + tested.open(new ExecutionContext()); + + assertNull(tested.read()); + assertFalse(delegate.openCalled); + assertFalse(delegate.closeCalled); + assertFalse(delegate.updateCalled); + + tested.close(); + } + /** * Directory resource behaves as if it was empty. */ @Test public void testDirectoryResources() throws Exception { - FileSystemResource resource = new FileSystemResource("target/data"); + FileSystemResource resource = new FileSystemResource("build/data"); resource.getFile().mkdirs(); assertTrue(resource.getFile().isDirectory()); tested.setResources(new Resource[] { resource }); @@ -441,4 +480,35 @@ public void testRestartAfterFailureWithoutRead() throws Exception { assertEquals("1", tested.read()); } + private static class ItemStreamReaderImpl implements ResourceAwareItemReaderItemStream { + + private boolean openCalled = false; + private boolean updateCalled = false; + private boolean closeCalled = false; + + @Nullable + @Override + public String read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException { + return null; + } + + @Override + public void open(ExecutionContext executionContext) throws ItemStreamException { + openCalled = true; + } + + @Override + public void update(ExecutionContext executionContext) throws ItemStreamException { + updateCalled = true; + } + + @Override + public void close() throws ItemStreamException { + closeCalled = true; + } + + @Override + public void setResource(Resource resource) { + } + } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderResourceAwareTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderResourceAwareTests.java index 4e85bc437e..2aad05b342 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderResourceAwareTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderResourceAwareTests.java @@ -1,19 +1,26 @@ +/* + * Copyright 2012-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file; import org.junit.Before; import org.junit.Test; import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.item.ItemStreamException; import org.springframework.batch.item.ResourceAware; -import org.springframework.batch.item.file.mapping.PassThroughLineMapper; -import org.springframework.core.io.AbstractResource; import org.springframework.core.io.ByteArrayResource; -import org.springframework.core.io.FileSystemResource; import org.springframework.core.io.Resource; -import org.springframework.test.util.ReflectionTestUtils; - -import java.io.IOException; -import java.io.InputStream; import java.util.Comparator; import static org.junit.Assert.*; @@ -25,9 +32,9 @@ */ public class MultiResourceItemReaderResourceAwareTests { - private MultiResourceItemReader tested = new MultiResourceItemReader(); + private MultiResourceItemReader tested = new MultiResourceItemReader<>(); - private FlatFileItemReader itemReader = new FlatFileItemReader(); + private FlatFileItemReader itemReader = new FlatFileItemReader<>(); private ExecutionContext ctx = new ExecutionContext(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderXmlTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderXmlTests.java index 0aa09ba1ec..11bcd3ac6c 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderXmlTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemReaderXmlTests.java @@ -1,5 +1,22 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file; +import static org.junit.Assert.assertTrue; + import java.io.IOException; import java.util.Comparator; @@ -8,16 +25,14 @@ import javax.xml.stream.events.StartElement; import javax.xml.transform.Source; -import junit.framework.Assert; - -import org.junit.runners.JUnit4; import org.junit.runner.RunWith; +import org.junit.runners.JUnit4; import org.springframework.batch.item.AbstractItemStreamItemReaderTests; import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.sample.Foo; import org.springframework.batch.item.xml.StaxEventItemReader; -import org.springframework.batch.item.xml.StaxUtils; +import org.springframework.batch.item.xml.StaxTestUtils; import org.springframework.core.io.ByteArrayResource; import org.springframework.core.io.Resource; import org.springframework.oxm.Unmarshaller; @@ -28,9 +43,9 @@ public class MultiResourceItemReaderXmlTests extends AbstractItemStreamItemReade @Override protected ItemReader getItemReader() throws Exception { - MultiResourceItemReader multiReader = new MultiResourceItemReader(); + MultiResourceItemReader multiReader = new MultiResourceItemReader<>(); - StaxEventItemReader reader = new StaxEventItemReader(); + StaxEventItemReader reader = new StaxEventItemReader<>(); reader.setFragmentRootElementName("foo"); reader.setUnmarshaller(new Unmarshaller() { @@ -40,8 +55,8 @@ public Object unmarshal(Source source) throws XmlMappingException, IOException { Attribute attr; try { - XMLEventReader eventReader = StaxUtils.getXmlEventReader(source ); - Assert.assertTrue(eventReader.nextEvent().isStartDocument()); + XMLEventReader eventReader = StaxTestUtils.getXmlEventReader(source ); + assertTrue(eventReader.nextEvent().isStartDocument()); StartElement event = eventReader.nextEvent().asStartElement(); attr = (Attribute) event.getAttributes().next(); } @@ -54,8 +69,7 @@ public Object unmarshal(Source source) throws XmlMappingException, IOException { } @Override - @SuppressWarnings("rawtypes") - public boolean supports(Class clazz) { + public boolean supports(Class clazz) { return true; } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemWriterFlatFileTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemWriterFlatFileTests.java index d5c87c00f2..51c6066751 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemWriterFlatFileTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemWriterFlatFileTests.java @@ -1,250 +1,277 @@ -package org.springframework.batch.item.file; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; - -import java.io.File; -import java.io.IOException; -import java.io.Writer; -import java.util.Arrays; -import java.util.List; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.item.file.transform.PassThroughLineAggregator; -import org.springframework.batch.support.transaction.ResourcelessTransactionManager; -import org.springframework.transaction.TransactionStatus; -import org.springframework.transaction.support.TransactionCallback; -import org.springframework.transaction.support.TransactionTemplate; - -/** - * Tests for {@link MultiResourceItemWriter} delegating to - * {@link FlatFileItemWriter}. - */ -public class MultiResourceItemWriterFlatFileTests extends AbstractMultiResourceItemWriterTests { - - /** - * @author dsyer - * - */ - private final class WriterCallback implements TransactionCallback { - private List list; - - public WriterCallback(List list) { - super(); - this.list = list; - } - - @Override - public Object doInTransaction(TransactionStatus status) { - try { - tested.write(list); - } - catch (Exception e) { - throw new IllegalStateException("Unexpected"); - } - return null; - } - } - - private FlatFileItemWriter delegate; - - @Before - public void setUp() throws Exception { - delegate = new FlatFileItemWriter(); - delegate.setLineAggregator(new PassThroughLineAggregator()); - } - - @Test - public void testBasicMultiResourceWriteScenario() throws Exception { - - super.setUp(delegate); - - tested.write(Arrays.asList("1", "2", "3")); - - File part1 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(1)); - assertTrue(part1.exists()); - assertEquals("123", readFile(part1)); - - tested.write(Arrays.asList("4")); - File part2 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(2)); - assertTrue(part2.exists()); - assertEquals("4", readFile(part2)); - - tested.write(Arrays.asList("5")); - assertEquals("45", readFile(part2)); - - tested.write(Arrays.asList("6", "7", "8", "9")); - File part3 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(3)); - assertTrue(part3.exists()); - assertEquals("6789", readFile(part3)); - } - - @Test - public void testUpdateAfterDelegateClose() throws Exception { - - super.setUp(delegate); - - tested.update(executionContext); - assertEquals(0, executionContext.getInt(tested.getExecutionContextKey("resource.item.count"))); - assertEquals(1, executionContext.getInt(tested.getExecutionContextKey("resource.index"))); - tested.write(Arrays.asList("1", "2", "3")); - tested.update(executionContext); - assertEquals(0, executionContext.getInt(tested.getExecutionContextKey("resource.item.count"))); - assertEquals(2, executionContext.getInt(tested.getExecutionContextKey("resource.index"))); - - } - - @Test - public void testMultiResourceWriteScenarioWithFooter() throws Exception { - - delegate.setFooterCallback(new FlatFileFooterCallback() { - @Override - public void writeFooter(Writer writer) throws IOException { - writer.write("f"); - } - }); - super.setUp(delegate); - - tested.write(Arrays.asList("1", "2", "3")); - - File part1 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(1)); - assertTrue(part1.exists()); - - tested.write(Arrays.asList("4")); - File part2 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(2)); - assertTrue(part2.exists()); - - tested.close(); - - assertEquals("123f", readFile(part1)); - assertEquals("4f", readFile(part2)); - - } - - @Test - public void testTransactionalMultiResourceWriteScenarioWithFooter() throws Exception { - - delegate.setFooterCallback(new FlatFileFooterCallback() { - @Override - public void writeFooter(Writer writer) throws IOException { - writer.write("f"); - } - }); - super.setUp(delegate); - - ResourcelessTransactionManager transactionManager = new ResourcelessTransactionManager(); - - new TransactionTemplate(transactionManager).execute(new WriterCallback(Arrays.asList("1", "2", "3"))); - - File part1 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(1)); - assertTrue(part1.exists()); - - new TransactionTemplate(transactionManager).execute(new WriterCallback(Arrays.asList("4"))); - File part2 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(2)); - assertTrue(part2.exists()); - - tested.close(); - - assertEquals("123f", readFile(part1)); - assertEquals("4f", readFile(part2)); - - } - - @Test - public void testRestart() throws Exception { - - super.setUp(delegate); - - tested.write(Arrays.asList("1", "2", "3")); - - File part1 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(1)); - assertTrue(part1.exists()); - assertEquals("123", readFile(part1)); - - tested.write(Arrays.asList("4")); - File part2 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(2)); - assertTrue(part2.exists()); - assertEquals("4", readFile(part2)); - - tested.update(executionContext); - tested.close(); - tested.open(executionContext); - - tested.write(Arrays.asList("5")); - assertEquals("45", readFile(part2)); - - tested.write(Arrays.asList("6", "7", "8", "9")); - File part3 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(3)); - assertTrue(part3.exists()); - assertEquals("6789", readFile(part3)); - } - - @Test - public void testRestartWithFooter() throws Exception { - - delegate.setFooterCallback(new FlatFileFooterCallback() { - @Override - public void writeFooter(Writer writer) throws IOException { - writer.write("f"); - } - }); - super.setUp(delegate); - - tested.write(Arrays.asList("1", "2", "3")); - - File part1 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(1)); - assertTrue(part1.exists()); - assertEquals("123f", readFile(part1)); - - tested.write(Arrays.asList("4")); - File part2 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(2)); - assertTrue(part2.exists()); - assertEquals("4", readFile(part2)); - - tested.update(executionContext); - tested.close(); - tested.open(executionContext); - - tested.write(Arrays.asList("5")); - assertEquals("45f", readFile(part2)); - - tested.write(Arrays.asList("6", "7", "8", "9")); - File part3 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(3)); - assertTrue(part3.exists()); - assertEquals("6789f", readFile(part3)); - } - - @Test - public void testTransactionalRestartWithFooter() throws Exception { - - delegate.setFooterCallback(new FlatFileFooterCallback() { - @Override - public void writeFooter(Writer writer) throws IOException { - writer.write("f"); - } - }); - super.setUp(delegate); - - ResourcelessTransactionManager transactionManager = new ResourcelessTransactionManager(); - - new TransactionTemplate(transactionManager).execute(new WriterCallback(Arrays.asList("1", "2", "3"))); - - File part1 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(1)); - assertTrue(part1.exists()); - assertEquals("123f", readFile(part1)); - - new TransactionTemplate(transactionManager).execute(new WriterCallback(Arrays.asList("4"))); - File part2 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(2)); - assertTrue(part2.exists()); - assertEquals("4", readFile(part2)); - - tested.update(executionContext); - tested.close(); - tested.open(executionContext); - - new TransactionTemplate(transactionManager).execute(new WriterCallback(Arrays.asList("5"))); - assertEquals("45f", readFile(part2)); - } - -} +/* + * Copyright 2008-2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.file; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +import java.io.File; +import java.io.IOException; +import java.io.Writer; +import java.util.Arrays; +import java.util.List; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.item.file.transform.PassThroughLineAggregator; +import org.springframework.batch.support.transaction.ResourcelessTransactionManager; +import org.springframework.transaction.TransactionStatus; +import org.springframework.transaction.support.TransactionCallback; +import org.springframework.transaction.support.TransactionTemplate; + +/** + * Tests for {@link MultiResourceItemWriter} delegating to + * {@link FlatFileItemWriter}. + */ +public class MultiResourceItemWriterFlatFileTests extends AbstractMultiResourceItemWriterTests { + + /** + * @author dsyer + * + */ + private final class WriterCallback implements TransactionCallback { + private List list; + + public WriterCallback(List list) { + super(); + this.list = list; + } + + @Override + public Void doInTransaction(TransactionStatus status) { + try { + tested.write(list); + } + catch (Exception e) { + throw new IllegalStateException("Unexpected"); + } + return null; + } + } + + private FlatFileItemWriter delegate; + + @Before + public void setUp() throws Exception { + super.createFile(); + delegate = new FlatFileItemWriter<>(); + delegate.setLineAggregator(new PassThroughLineAggregator<>()); + } + + @Test + public void testBasicMultiResourceWriteScenario() throws Exception { + + super.setUp(delegate); + tested.open(executionContext); + + tested.write(Arrays.asList("1", "2", "3")); + + File part1 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(1)); + assertTrue(part1.exists()); + assertEquals("123", readFile(part1)); + + tested.write(Arrays.asList("4")); + File part2 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(2)); + assertTrue(part2.exists()); + assertEquals("4", readFile(part2)); + + tested.write(Arrays.asList("5")); + assertEquals("45", readFile(part2)); + + tested.write(Arrays.asList("6", "7", "8", "9")); + File part3 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(3)); + assertTrue(part3.exists()); + assertEquals("6789", readFile(part3)); + } + + @Test + public void testUpdateAfterDelegateClose() throws Exception { + + super.setUp(delegate); + tested.open(executionContext); + + tested.update(executionContext); + assertEquals(0, executionContext.getInt(tested.getExecutionContextKey("resource.item.count"))); + assertEquals(1, executionContext.getInt(tested.getExecutionContextKey("resource.index"))); + tested.write(Arrays.asList("1", "2", "3")); + tested.update(executionContext); + assertEquals(0, executionContext.getInt(tested.getExecutionContextKey("resource.item.count"))); + assertEquals(2, executionContext.getInt(tested.getExecutionContextKey("resource.index"))); + + } + + @Test + public void testMultiResourceWriteScenarioWithFooter() throws Exception { + + delegate.setFooterCallback(new FlatFileFooterCallback() { + @Override + public void writeFooter(Writer writer) throws IOException { + writer.write("f"); + } + }); + super.setUp(delegate); + tested.open(executionContext); + + tested.write(Arrays.asList("1", "2", "3")); + + File part1 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(1)); + assertTrue(part1.exists()); + + tested.write(Arrays.asList("4")); + File part2 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(2)); + assertTrue(part2.exists()); + + tested.close(); + + assertEquals("123f", readFile(part1)); + assertEquals("4f", readFile(part2)); + + } + + @Test + public void testTransactionalMultiResourceWriteScenarioWithFooter() throws Exception { + + delegate.setFooterCallback(new FlatFileFooterCallback() { + @Override + public void writeFooter(Writer writer) throws IOException { + writer.write("f"); + } + }); + super.setUp(delegate); + tested.open(executionContext); + + ResourcelessTransactionManager transactionManager = new ResourcelessTransactionManager(); + + new TransactionTemplate(transactionManager).execute(new WriterCallback(Arrays.asList("1", "2", "3"))); + + File part1 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(1)); + assertTrue(part1.exists()); + + new TransactionTemplate(transactionManager).execute(new WriterCallback(Arrays.asList("4"))); + File part2 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(2)); + assertTrue(part2.exists()); + + tested.close(); + + assertEquals("123f", readFile(part1)); + assertEquals("4f", readFile(part2)); + + } + + @Test + public void testRestart() throws Exception { + + super.setUp(delegate); + tested.open(executionContext); + + tested.write(Arrays.asList("1", "2", "3")); + + File part1 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(1)); + assertTrue(part1.exists()); + assertEquals("123", readFile(part1)); + + tested.write(Arrays.asList("4")); + File part2 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(2)); + assertTrue(part2.exists()); + assertEquals("4", readFile(part2)); + + tested.update(executionContext); + tested.close(); + + tested.open(executionContext); + + tested.write(Arrays.asList("5")); + assertEquals("45", readFile(part2)); + + tested.write(Arrays.asList("6", "7", "8", "9")); + File part3 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(3)); + assertTrue(part3.exists()); + assertEquals("6789", readFile(part3)); + } + + @Test + public void testRestartWithFooter() throws Exception { + + delegate.setFooterCallback(new FlatFileFooterCallback() { + @Override + public void writeFooter(Writer writer) throws IOException { + writer.write("f"); + } + }); + + super.setUp(delegate); + tested.open(executionContext); + + tested.write(Arrays.asList("1", "2", "3")); + + File part1 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(1)); + assertTrue(part1.exists()); + assertEquals("123f", readFile(part1)); + + tested.write(Arrays.asList("4")); + File part2 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(2)); + assertTrue(part2.exists()); + assertEquals("4", readFile(part2)); + + tested.update(executionContext); + tested.close(); + + tested.open(executionContext); + + tested.write(Arrays.asList("5")); + assertEquals("45f", readFile(part2)); + + tested.write(Arrays.asList("6", "7", "8", "9")); + File part3 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(3)); + assertTrue(part3.exists()); + assertEquals("6789f", readFile(part3)); + } + + @Test + public void testTransactionalRestartWithFooter() throws Exception { + + delegate.setFooterCallback(new FlatFileFooterCallback() { + @Override + public void writeFooter(Writer writer) throws IOException { + writer.write("f"); + } + }); + super.setUp(delegate); + tested.open(executionContext); + + ResourcelessTransactionManager transactionManager = new ResourcelessTransactionManager(); + + new TransactionTemplate(transactionManager).execute(new WriterCallback(Arrays.asList("1", "2", "3"))); + + File part1 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(1)); + assertTrue(part1.exists()); + assertEquals("123f", readFile(part1)); + + new TransactionTemplate(transactionManager).execute(new WriterCallback(Arrays.asList("4"))); + File part2 = new File(file.getAbsolutePath() + suffixCreator.getSuffix(2)); + assertTrue(part2.exists()); + assertEquals("4", readFile(part2)); + + tested.update(executionContext); + tested.close(); + + tested.open(executionContext); + + new TransactionTemplate(transactionManager).execute(new WriterCallback(Arrays.asList("5"))); + assertEquals("45f", readFile(part2)); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemWriterXmlTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemWriterXmlTests.java index af2ed8cfc6..9b8730fdae 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemWriterXmlTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/MultiResourceItemWriterXmlTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertTrue; @@ -13,7 +28,7 @@ import org.junit.Before; import org.junit.Test; import org.springframework.batch.item.xml.StaxEventItemWriter; -import org.springframework.batch.item.xml.StaxUtils; +import org.springframework.batch.item.xml.StaxTestUtils; import org.springframework.oxm.Marshaller; import org.springframework.oxm.XmlMappingException; import org.springframework.util.Assert; @@ -32,7 +47,8 @@ public class MultiResourceItemWriterXmlTests extends AbstractMultiResourceItemWr @Before public void setUp() throws Exception { - delegate = new StaxEventItemWriter(); + super.createFile(); + delegate = new StaxEventItemWriter<>(); delegate.setMarshaller(new SimpleMarshaller()); } @@ -47,7 +63,7 @@ public void marshal(Object graph, Result result) throws XmlMappingException, IOE try { XMLEventFactory factory = XMLEventFactory.newInstance(); - XMLEventWriter writer = StaxUtils.getXmlEventWriter(result); + XMLEventWriter writer = StaxTestUtils.getXmlEventWriter(result); writer.add(factory.createStartDocument("UTF-8")); writer.add(factory.createStartElement("prefix", "namespace", graph.toString())); writer.add(factory.createEndElement("prefix", "namespace", graph.toString())); @@ -59,8 +75,7 @@ public void marshal(Object graph, Result result) throws XmlMappingException, IOE } @Override - @SuppressWarnings("rawtypes") - public boolean supports(Class clazz) { + public boolean supports(Class clazz) { return true; } } @@ -74,8 +89,9 @@ protected String readFile(File f) throws Exception { @Test public void multiResourceWritingWithRestart() throws Exception { - - setUp(delegate); + + super.setUp(delegate); + tested.open(executionContext); tested.write(Arrays.asList("1", "2", "3")); @@ -89,6 +105,7 @@ public void multiResourceWritingWithRestart() throws Exception { tested.update(executionContext); tested.close(); + assertEquals(xmlDocStart + "" + xmlDocEnd, readFile(part2)); assertEquals(xmlDocStart + "" + xmlDocEnd, readFile(part1)); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/ResourcesItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/ResourcesItemReaderTests.java index a4b95067a1..b35351f3ac 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/ResourcesItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/ResourcesItemReaderTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file; import static org.junit.Assert.*; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/SimpleBinaryBufferedReaderFactoryTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/SimpleBinaryBufferedReaderFactoryTests.java index e4daecc05b..250fbb86a1 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/SimpleBinaryBufferedReaderFactoryTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/SimpleBinaryBufferedReaderFactoryTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,6 +31,7 @@ public class SimpleBinaryBufferedReaderFactoryTests { @Test public void testCreate() throws Exception { SimpleBinaryBufferedReaderFactory factory = new SimpleBinaryBufferedReaderFactory(); + @SuppressWarnings("resource") BufferedReader reader = factory.create(new ByteArrayResource("a\nb".getBytes()), "UTF-8"); assertEquals("a", reader.readLine()); assertEquals("b", reader.readLine()); @@ -41,6 +42,7 @@ public void testCreate() throws Exception { public void testCreateWithLineEnding() throws Exception { SimpleBinaryBufferedReaderFactory factory = new SimpleBinaryBufferedReaderFactory(); factory.setLineEnding("||"); + @SuppressWarnings("resource") BufferedReader reader = factory.create(new ByteArrayResource("a||b".getBytes()), "UTF-8"); assertEquals("a", reader.readLine()); assertEquals("b", reader.readLine()); @@ -51,6 +53,7 @@ public void testCreateWithLineEnding() throws Exception { public void testMarkResetWithLineEnding() throws Exception { SimpleBinaryBufferedReaderFactory factory = new SimpleBinaryBufferedReaderFactory(); factory.setLineEnding("||"); + @SuppressWarnings("resource") BufferedReader reader = factory.create(new ByteArrayResource("a||b||c".getBytes()), "UTF-8"); assertEquals("a", reader.readLine()); reader.mark(1024); @@ -65,6 +68,7 @@ public void testMarkResetWithLineEnding() throws Exception { public void testCreateWithLineEndingAtEnd() throws Exception { SimpleBinaryBufferedReaderFactory factory = new SimpleBinaryBufferedReaderFactory(); factory.setLineEnding("||"); + @SuppressWarnings("resource") BufferedReader reader = factory.create(new ByteArrayResource("a||".getBytes()), "UTF-8"); assertEquals("a", reader.readLine()); assertEquals(null, reader.readLine()); @@ -74,6 +78,7 @@ public void testCreateWithLineEndingAtEnd() throws Exception { public void testCreateWithFalseLineEnding() throws Exception { SimpleBinaryBufferedReaderFactory factory = new SimpleBinaryBufferedReaderFactory(); factory.setLineEnding("||"); + @SuppressWarnings("resource") BufferedReader reader = factory.create(new ByteArrayResource("a|b||".getBytes()), "UTF-8"); assertEquals("a|b", reader.readLine()); assertEquals(null, reader.readLine()); @@ -83,6 +88,7 @@ public void testCreateWithFalseLineEnding() throws Exception { public void testCreateWithIncompleteLineEnding() throws Exception { SimpleBinaryBufferedReaderFactory factory = new SimpleBinaryBufferedReaderFactory(); factory.setLineEnding("||"); + @SuppressWarnings("resource") BufferedReader reader = factory.create(new ByteArrayResource("a||b|".getBytes()), "UTF-8"); assertEquals("a", reader.readLine()); assertEquals("b|", reader.readLine()); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/SimpleResourceSuffixCreatorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/SimpleResourceSuffixCreatorTests.java index 8818d6f4c0..23bdf5ba6f 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/SimpleResourceSuffixCreatorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/SimpleResourceSuffixCreatorTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file; import static org.junit.Assert.*; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/builder/FlatFileItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/builder/FlatFileItemReaderBuilderTests.java new file mode 100644 index 0000000000..78e085664f --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/builder/FlatFileItemReaderBuilderTests.java @@ -0,0 +1,526 @@ +/* + * Copyright 2016-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.file.builder; + +import java.io.InputStreamReader; +import java.io.LineNumberReader; +import java.util.ArrayList; +import java.util.List; + +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.file.FlatFileItemReader; +import org.springframework.batch.item.file.separator.DefaultRecordSeparatorPolicy; +import org.springframework.batch.item.file.transform.DefaultFieldSet; +import org.springframework.batch.item.file.transform.DelimitedLineTokenizer; +import org.springframework.batch.item.file.transform.FieldSet; +import org.springframework.batch.item.file.transform.FieldSetFactory; +import org.springframework.batch.item.file.transform.Range; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Scope; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.FileSystemResource; +import org.springframework.core.io.Resource; +import org.springframework.test.util.ReflectionTestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; + +/** + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @author Drummond Dawson + */ +public class FlatFileItemReaderBuilderTests { + + @Test + public void testSimpleFixedLength() throws Exception { + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1 2 3")) + .fixedLength() + .columns(new Range(1, 3), new Range(4, 6), new Range(7)) + .names("first", "second", "third") + .targetType(Foo.class) + .build(); + + reader.open(new ExecutionContext()); + Foo item = reader.read(); + assertEquals(1, item.getFirst()); + assertEquals(2, item.getSecond()); + assertEquals("3", item.getThird()); + assertNull(reader.read()); + } + + @Test + public void testSimpleDelimited() throws Exception { + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1,2,3")) + .delimited() + .names("first", "second", "third") + .targetType(Foo.class) + .build(); + + reader.open(new ExecutionContext()); + Foo item = reader.read(); + assertEquals(1, item.getFirst()); + assertEquals(2, item.getSecond()); + assertEquals("3", item.getThird()); + assertNull(reader.read()); + } + + @Test + public void testSimpleDelimitedWithWhitespaceCharacter() throws Exception { + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1 2 3")) + .delimited() + .delimiter(" ") + .names("first", "second", "third") + .targetType(Foo.class) + .build(); + + reader.open(new ExecutionContext()); + Foo item = reader.read(); + assertEquals(1, item.getFirst()); + assertEquals(2, item.getSecond()); + assertEquals("3", item.getThird()); + assertNull(reader.read()); + } + + @Test + public void testSimpleDelimitedWithTabCharacter() throws Exception { + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1\t2\t3")) + .delimited() + .delimiter(DelimitedLineTokenizer.DELIMITER_TAB) + .names("first", "second", "third") + .targetType(Foo.class) + .build(); + + reader.open(new ExecutionContext()); + Foo item = reader.read(); + assertEquals(1, item.getFirst()); + assertEquals(2, item.getSecond()); + assertEquals("3", item.getThird()); + assertNull(reader.read()); + } + + @Test + public void testAdvancedDelimited() throws Exception { + final List skippedLines = new ArrayList<>(); + + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1,2,3\n4,5,$1,2,3$\n@this is a comment\n6,7, 8")) + .delimited() + .quoteCharacter('$') + .names("first", "second", "third") + .targetType(Foo.class) + .linesToSkip(1) + .skippedLinesCallback(skippedLines::add) + .addComment("@") + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + + Foo item = reader.read(); + assertEquals(4, item.getFirst()); + assertEquals(5, item.getSecond()); + assertEquals("1,2,3", item.getThird()); + + item = reader.read(); + assertEquals(6, item.getFirst()); + assertEquals(7, item.getSecond()); + assertEquals("8", item.getThird()); + + reader.update(executionContext); + + assertNull(reader.read()); + + assertEquals("1,2,3", skippedLines.get(0)); + assertEquals(1, skippedLines.size()); + + assertEquals(1, executionContext.size()); + } + + @Test + public void testAdvancedFixedLength() throws Exception { + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1 2%\n 3\n4 5%\n 6\n@this is a comment\n7 8%\n 9\n")) + .fixedLength() + .columns(new Range(1, 2), new Range(3, 5), new Range(6)) + .names("first", "second", "third") + .targetType(Foo.class) + .recordSeparatorPolicy(new DefaultRecordSeparatorPolicy("\"", "%")) + .bufferedReaderFactory((resource, encoding) -> + new LineNumberReader(new InputStreamReader(resource.getInputStream(), encoding))) + .maxItemCount(2) + .saveState(false) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + + reader.open(executionContext); + Foo item = reader.read(); + assertEquals(1, item.getFirst()); + assertEquals(2, item.getSecond()); + assertEquals("3", item.getThird()); + + item = reader.read(); + assertEquals(4, item.getFirst()); + assertEquals(5, item.getSecond()); + assertEquals("6", item.getThird()); + + reader.update(executionContext); + + assertNull(reader.read()); + assertEquals(0, executionContext.size()); + } + + @Test + public void testStrict() throws Exception { + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(new FileSystemResource("this/file/does/not/exist")) + .delimited() + .names("first", "second", "third") + .targetType(Foo.class) + .strict(false) + .build(); + + reader.open(new ExecutionContext()); + + assertNull(reader.read()); + } + + @Test + public void testCustomLineTokenizerFieldSetMapper() throws Exception { + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("|1|&|2|&| 3|\n|4|&|5|&|foo|")) + .lineTokenizer(line -> new DefaultFieldSet(line.split("&"))) + .fieldSetMapper(fieldSet -> { + Foo item = new Foo(); + + item.setFirst(Integer.valueOf(fieldSet.readString(0).replaceAll("\\|", ""))); + item.setSecond(Integer.valueOf(fieldSet.readString(1).replaceAll("\\|", ""))); + item.setThird(fieldSet.readString(2).replaceAll("\\|", "")); + + return item; + }) + .build(); + + reader.open(new ExecutionContext()); + Foo item = reader.read(); + + assertEquals(1, item.getFirst()); + assertEquals(2, item.getSecond()); + assertEquals(" 3", item.getThird()); + + item = reader.read(); + + assertEquals(4, item.getFirst()); + assertEquals(5, item.getSecond()); + assertEquals("foo", item.getThird()); + + assertNull(reader.read()); + } + + @Test + public void testComments() throws Exception { + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1,2,3\n@this is a comment\n+so is this\n4,5,6")) + .comments("@", "+") + .delimited() + .names("first", "second", "third") + .targetType(Foo.class) + .build(); + + reader.open(new ExecutionContext()); + Foo item = reader.read(); + assertEquals(1, item.getFirst()); + assertEquals(2, item.getSecond()); + assertEquals("3", item.getThird()); + item = reader.read(); + assertEquals(4, item.getFirst()); + assertEquals(5, item.getSecond()); + assertEquals("6", item.getThird()); + assertNull(reader.read()); + } + + @Test + public void testPrototypeBean() throws Exception { + BeanFactory factory = new AnnotationConfigApplicationContext(Beans.class); + + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1,2,3")) + .delimited() + .names("first", "second", "third") + .prototypeBeanName("foo") + .beanFactory(factory) + .build(); + + reader.open(new ExecutionContext()); + Foo item = reader.read(); + assertEquals(1, item.getFirst()); + assertEquals(2, item.getSecond()); + assertEquals("3", item.getThird()); + assertNull(reader.read()); + } + + @Test + public void testBeanWrapperFieldSetMapperStrict() throws Exception { + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1,2,3")) + .delimited() + .names("setFirst", "setSecond", "setThird") + .targetType(Foo.class) + .beanMapperStrict(true) + .build(); + + reader.open(new ExecutionContext()); + Foo item = reader.read(); + assertEquals(1, item.getFirst()); + assertEquals(2, item.getSecond()); + assertEquals("3", item.getThird()); + assertNull(reader.read()); + } + + @Test + public void testDelimitedIncludedFields() throws Exception { + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1,2,3")) + .delimited() + .includedFields(0, 2) + .addIncludedField(1) + .names("first", "second", "third") + .targetType(Foo.class) + .build(); + + reader.open(new ExecutionContext()); + Foo item = reader.read(); + assertEquals(1, item.getFirst()); + assertEquals(2, item.getSecond()); + assertEquals("3", item.getThird()); + assertNull(reader.read()); + } + + @Test + public void testDelimitedFieldSetFactory() throws Exception { + String[] names = {"first", "second", "third"}; + + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1,2,3")) + .delimited() + .fieldSetFactory(new FieldSetFactory() { + private FieldSet fieldSet = new DefaultFieldSet(new String[] {"1", "3", "foo"}, names); + + @Override + public FieldSet create(String[] values, String[] names) { + return fieldSet; + } + + @Override + public FieldSet create(String[] values) { + return fieldSet; + } + }) + .names(names) + .targetType(Foo.class) + .build(); + + reader.open(new ExecutionContext()); + Foo item = reader.read(); + assertEquals(1, item.getFirst()); + assertEquals(3, item.getSecond()); + assertEquals("foo", item.getThird()); + assertNull(reader.read()); + } + + @Test + public void testFixedLengthFieldSetFactory() throws Exception { + String[] names = {"first", "second", "third"}; + + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1 2 3")) + .fixedLength() + .fieldSetFactory(new FieldSetFactory() { + private FieldSet fieldSet = new DefaultFieldSet(new String[] {"1", "3", "foo"}, names); + + @Override + public FieldSet create(String[] values, String[] names) { + return fieldSet; + } + + @Override + public FieldSet create(String[] values) { + return fieldSet; + } + }) + .columns(new Range(1, 3), new Range(4, 6), new Range(7)) + .names("first", "second", "third") + .targetType(Foo.class) + .build(); + + reader.open(new ExecutionContext()); + Foo item = reader.read(); + assertEquals(1, item.getFirst()); + assertEquals(3, item.getSecond()); + assertEquals("foo", item.getThird()); + assertNull(reader.read()); + } + + + @Test + public void testName() throws Exception { + try { + new FlatFileItemReaderBuilder() + .resource(getResource("1 2 3")) + .fixedLength() + .columns(new Range(1, 3), new Range(4, 6), new Range(7)) + .names("first", "second", "third") + .targetType(Foo.class) + .build(); + fail("null name should throw exception"); + } + catch (IllegalStateException iae) { + assertEquals("A name is required when saveState is set to true.", iae.getMessage()); + } + try { + new FlatFileItemReaderBuilder() + .resource(getResource("1 2 3")) + .fixedLength() + .columns(new Range(1, 3), new Range(4, 6), new Range(7)) + .names("first", "second", "third") + .targetType(Foo.class) + .name(null) + .build(); + } + catch (IllegalStateException iae) { + assertEquals("A name is required when saveState is set to true.", iae.getMessage()); + } + assertNotNull("builder should return new instance of FlatFileItemReader", new FlatFileItemReaderBuilder() + .resource(getResource("1 2 3")) + .fixedLength() + .columns(new Range(1, 3), new Range(4, 6), new Range(7)) + .names("first", "second", "third") + .targetType(Foo.class) + .saveState(false) + .build()); + + assertNotNull("builder should return new instance of FlatFileItemReader", new FlatFileItemReaderBuilder() + .resource(getResource("1 2 3")) + .fixedLength() + .columns(new Range(1, 3), new Range(4, 6), new Range(7)) + .names("first", "second", "third") + .targetType(Foo.class) + .name("foobar") + .build()); + + } + + @Test + public void testDefaultEncoding() { + String encoding = FlatFileItemReader.DEFAULT_CHARSET; + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1,2,3")) + .delimited() + .names("first", "second", "third") + .targetType(Foo.class) + .build(); + + assertEquals(encoding, ReflectionTestUtils.getField(reader, "encoding")); + } + + @Test + public void testCustomEncoding() { + String encoding = "UTF-8"; + FlatFileItemReader reader = new FlatFileItemReaderBuilder() + .name("fooReader") + .resource(getResource("1 2 3")) + .encoding(encoding) + .fixedLength() + .columns(new Range(1, 3), new Range(4, 6), new Range(7)) + .names("first", "second", "third") + .targetType(Foo.class) + .build(); + + assertEquals(encoding, ReflectionTestUtils.getField(reader, "encoding")); + } + + private Resource getResource(String contents) { + return new ByteArrayResource(contents.getBytes()); + } + + public static class Foo { + private int first; + private int second; + private String third; + + public int getFirst() { + return first; + } + + public void setFirst(int first) { + this.first = first; + } + + public int getSecond() { + return second; + } + + public void setSecond(int second) { + this.second = second; + } + + public String getThird() { + return third; + } + + public void setThird(String third) { + this.third = third; + } + } + + @Configuration + public static class Beans { + + @Bean + @Scope("prototype") + public Foo foo() { + return new Foo(); + } + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/builder/FlatFileItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/builder/FlatFileItemWriterBuilderTests.java new file mode 100644 index 0000000000..20d23c2204 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/builder/FlatFileItemWriterBuilderTests.java @@ -0,0 +1,312 @@ +/* + * Copyright 2016-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.file.builder; + +import java.io.BufferedReader; +import java.io.File; +import java.io.IOException; +import java.io.InputStreamReader; +import java.util.Arrays; + +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.file.FlatFileItemWriter; +import org.springframework.batch.item.file.transform.PassThroughLineAggregator; +import org.springframework.core.io.FileSystemResource; +import org.springframework.core.io.Resource; +import org.springframework.test.util.ReflectionTestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +/** + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @author Drummond Dawson + */ +public class FlatFileItemWriterBuilderTests { + + // reads the output file to check the result + private BufferedReader reader; + + @Test(expected = IllegalArgumentException.class) + public void testMissingLineAggregator() { + new FlatFileItemWriterBuilder() + .build(); + } + + @Test(expected = IllegalStateException.class) + public void testMultipleLineAggregators() throws IOException { + Resource output = new FileSystemResource(File.createTempFile("foo", "txt")); + + new FlatFileItemWriterBuilder() + .name("itemWriter") + .resource(output) + .delimited() + .delimiter(";") + .names("foo", "bar") + .formatted() + .format("%2s%2s") + .names("foo", "bar") + .build(); + } + + @Test + public void test() throws Exception { + + Resource output = new FileSystemResource(File.createTempFile("foo", "txt")); + + FlatFileItemWriter writer = new FlatFileItemWriterBuilder() + .name("foo") + .resource(output) + .lineSeparator("$") + .lineAggregator(new PassThroughLineAggregator<>()) + .encoding("UTF-16LE") + .headerCallback(writer1 -> writer1.append("HEADER")) + .footerCallback(writer12 -> writer12.append("FOOTER")) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + + writer.open(executionContext); + + writer.write(Arrays.asList(new Foo(1, 2, "3"), new Foo(4, 5, "6"))); + + writer.close(); + + assertEquals("HEADER$Foo{first=1, second=2, third='3'}$Foo{first=4, second=5, third='6'}$FOOTER", readLine("UTF-16LE", output)); + } + + @Test + public void testDelimitedOutputWithDefaultDelimiter() throws Exception { + + Resource output = new FileSystemResource(File.createTempFile("foo", "txt")); + + FlatFileItemWriter writer = new FlatFileItemWriterBuilder() + .name("foo") + .resource(output) + .lineSeparator("$") + .delimited() + .names("first", "second", "third") + .encoding("UTF-16LE") + .headerCallback(writer1 -> writer1.append("HEADER")) + .footerCallback(writer12 -> writer12.append("FOOTER")) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + + writer.open(executionContext); + + writer.write(Arrays.asList(new Foo(1, 2, "3"), new Foo(4, 5, "6"))); + + writer.close(); + + assertEquals("HEADER$1,2,3$4,5,6$FOOTER", readLine("UTF-16LE", output)); + } + + @Test + public void testDelimitedOutputWithDefaultFieldExtractor() throws Exception { + + Resource output = new FileSystemResource(File.createTempFile("foo", "txt")); + + FlatFileItemWriter writer = new FlatFileItemWriterBuilder() + .name("foo") + .resource(output) + .lineSeparator("$") + .delimited() + .delimiter(";") + .names("first", "second", "third") + .encoding("UTF-16LE") + .headerCallback(writer1 -> writer1.append("HEADER")) + .footerCallback(writer12 -> writer12.append("FOOTER")) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + + writer.open(executionContext); + + writer.write(Arrays.asList(new Foo(1, 2, "3"), new Foo(4, 5, "6"))); + + writer.close(); + + assertEquals("HEADER$1;2;3$4;5;6$FOOTER", readLine("UTF-16LE", output)); + } + + @Test + public void testDelimitedOutputWithCustomFieldExtractor() throws Exception { + + Resource output = new FileSystemResource(File.createTempFile("foo", "txt")); + + FlatFileItemWriter writer = new FlatFileItemWriterBuilder() + .name("foo") + .resource(output) + .lineSeparator("$") + .delimited() + .delimiter(" ") + .fieldExtractor(item -> new Object[] {item.getFirst(), item.getThird()}) + .encoding("UTF-16LE") + .headerCallback(writer1 -> writer1.append("HEADER")) + .footerCallback(writer12 -> writer12.append("FOOTER")) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + + writer.open(executionContext); + + writer.write(Arrays.asList(new Foo(1, 2, "3"), new Foo(4, 5, "6"))); + + writer.close(); + + assertEquals("HEADER$1 3$4 6$FOOTER", readLine("UTF-16LE", output)); + } + + @Test + public void testFormattedOutputWithDefaultFieldExtractor() throws Exception { + + Resource output = new FileSystemResource(File.createTempFile("foo", "txt")); + + FlatFileItemWriter writer = new FlatFileItemWriterBuilder() + .name("foo") + .resource(output) + .lineSeparator("$") + .formatted() + .format("%2s%2s%2s") + .names("first", "second", "third") + .encoding("UTF-16LE") + .headerCallback(writer1 -> writer1.append("HEADER")) + .footerCallback(writer12 -> writer12.append("FOOTER")) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + + writer.open(executionContext); + + writer.write(Arrays.asList(new Foo(1, 2, "3"), new Foo(4, 5, "6"))); + + writer.close(); + + assertEquals("HEADER$ 1 2 3$ 4 5 6$FOOTER", readLine("UTF-16LE", output)); + } + + @Test + public void testFormattedOutputWithCustomFieldExtractor() throws Exception { + + Resource output = new FileSystemResource(File.createTempFile("foo", "txt")); + + FlatFileItemWriter writer = new FlatFileItemWriterBuilder() + .name("foo") + .resource(output) + .lineSeparator("$") + .formatted() + .format("%3s%3s") + .fieldExtractor(item -> new Object[] {item.getFirst(), item.getThird()}) + .encoding("UTF-16LE") + .headerCallback(writer1 -> writer1.append("HEADER")) + .footerCallback(writer12 -> writer12.append("FOOTER")) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + + writer.open(executionContext); + + writer.write(Arrays.asList(new Foo(1, 2, "3"), new Foo(4, 5, "6"))); + + writer.close(); + + assertEquals("HEADER$ 1 3$ 4 6$FOOTER", readLine("UTF-16LE", output)); + } + + @Test + public void testFlags() throws Exception { + + Resource output = new FileSystemResource(File.createTempFile("foo", "txt")); + + FlatFileItemWriter writer = new FlatFileItemWriterBuilder() + .name("foo") + .resource(output) + .shouldDeleteIfEmpty(true) + .shouldDeleteIfExists(false) + .saveState(false) + .forceSync(true) + .append(true) + .transactional(false) + .lineAggregator(new PassThroughLineAggregator<>()) + .build(); + + assertFalse((Boolean) ReflectionTestUtils.getField(writer, "saveState")); + assertTrue((Boolean) ReflectionTestUtils.getField(writer, "append")); + assertFalse((Boolean) ReflectionTestUtils.getField(writer, "transactional")); + assertTrue((Boolean) ReflectionTestUtils.getField(writer, "shouldDeleteIfEmpty")); + assertFalse((Boolean) ReflectionTestUtils.getField(writer, "shouldDeleteIfExists")); + assertTrue((Boolean) ReflectionTestUtils.getField(writer, "forceSync")); + } + + private String readLine(String encoding, Resource outputFile ) throws IOException { + + if (reader == null) { + reader = new BufferedReader(new InputStreamReader(outputFile.getInputStream(), encoding)); + } + + return reader.readLine(); + } + + public static class Foo { + private int first; + private int second; + private String third; + + public Foo(int first, int second, String third) { + this.first = first; + this.second = second; + this.third = third; + } + + public int getFirst() { + return first; + } + + public void setFirst(int first) { + this.first = first; + } + + public int getSecond() { + return second; + } + + public void setSecond(int second) { + this.second = second; + } + + public String getThird() { + return third; + } + + public void setThird(String third) { + this.third = third; + } + + @Override + public String toString() { + return "Foo{" + + "first=" + first + + ", second=" + second + + ", third='" + third + '\'' + + '}'; + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/builder/MultiResourceItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/builder/MultiResourceItemReaderBuilderTests.java new file mode 100644 index 0000000000..95905bbf8b --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/builder/MultiResourceItemReaderBuilderTests.java @@ -0,0 +1,98 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.file.builder; + +import java.util.Comparator; + +import org.junit.Test; + +import org.springframework.batch.item.AbstractItemStreamItemReaderTests; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.file.FlatFileItemReader; +import org.springframework.batch.item.file.MultiResourceItemReader; +import org.springframework.batch.item.sample.Foo; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.Resource; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.mock; + +/** + * @author Glenn Renfro + */ +public class MultiResourceItemReaderBuilderTests extends AbstractItemStreamItemReaderTests { + + @Override + protected ItemReader getItemReader() throws Exception { + + FlatFileItemReader fileReader = new FlatFileItemReader<>(); + + fileReader.setLineMapper((line, lineNumber) -> { + Foo foo = new Foo(); + foo.setValue(Integer.valueOf(line)); + return foo; + }); + fileReader.setSaveState(true); + + Resource r1 = new ByteArrayResource("1\n2\n".getBytes()); + Resource r2 = new ByteArrayResource("".getBytes()); + Resource r3 = new ByteArrayResource("3\n".getBytes()); + Resource r4 = new ByteArrayResource("4\n5\n".getBytes()); + + Comparator comparator = (arg0, arg1) -> { + return 0; // preserve original ordering + }; + return new MultiResourceItemReaderBuilder().delegate(fileReader) + .resources(new Resource[] { r1, r2, r3, r4 }).saveState(true).comparator(comparator).name("FOO") + .build(); + } + + @Test + public void testNullDelegate() { + try { + new MultiResourceItemReaderBuilder().resources(new Resource[]{}).build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException ise) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "delegate is required.", ise.getMessage()); + } + } + + @Test + @SuppressWarnings("unchecked") + public void testNullResources() { + try { + new MultiResourceItemReaderBuilder().delegate(mock(FlatFileItemReader.class)).build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException ise) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "resources array is required.", ise.getMessage()); + } + } + + @Override + protected void pointToEmptyInput(ItemReader tested) throws Exception { + MultiResourceItemReader multiReader = (MultiResourceItemReader) tested; + multiReader.close(); + multiReader.setResources(new Resource[] { new ByteArrayResource("".getBytes()) }); + multiReader.open(new ExecutionContext()); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/builder/MultiResourceItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/builder/MultiResourceItemWriterBuilderTests.java new file mode 100644 index 0000000000..5dcb8358c7 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/builder/MultiResourceItemWriterBuilderTests.java @@ -0,0 +1,266 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.file.builder; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; +import java.util.Arrays; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.file.FlatFileItemWriter; +import org.springframework.batch.item.file.MultiResourceItemWriter; +import org.springframework.batch.item.file.MultiResourceItemWriterFlatFileTests; +import org.springframework.batch.item.file.ResourceSuffixCreator; +import org.springframework.batch.item.file.SimpleResourceSuffixCreator; +import org.springframework.batch.item.file.transform.PassThroughLineAggregator; +import org.springframework.core.io.FileSystemResource; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * @author Glenn Renfro + */ +public class MultiResourceItemWriterBuilderTests { + + private MultiResourceItemWriter writer; + + private File file; + + private ResourceSuffixCreator suffixCreator = new ResourceSuffixCreator() { + @Override + public String getSuffix(int index) { + return "A" + index; + } + }; + + private ExecutionContext executionContext = new ExecutionContext(); + + private FlatFileItemWriter delegate; + + @Before + public void setUp() throws Exception { + this.delegate = new FlatFileItemWriter<>(); + this.delegate.setLineAggregator(new PassThroughLineAggregator<>()); + this.file = File.createTempFile(MultiResourceItemWriterFlatFileTests.class.getSimpleName(), null); + this.writer = null; + } + + @After + public void tearDown() { + if (this.writer != null) { + this.writer.close(); + } + } + + @Test + public void testBasicMultiResourceWriteScenario() throws Exception { + + this.writer = new MultiResourceItemWriterBuilder().delegate(this.delegate) + .resource(new FileSystemResource(this.file)).resourceSuffixCreator(this.suffixCreator) + .itemCountLimitPerResource(2).saveState(true).name("foo").build(); + + this.writer.open(this.executionContext); + + this.writer.write(Arrays.asList("1", "2", "3")); + + File part1 = new File(this.file.getAbsolutePath() + this.suffixCreator.getSuffix(1)); + assertTrue(part1.exists()); + assertEquals("123", readFile(part1)); + + this.writer.write(Arrays.asList("4")); + File part2 = new File(this.file.getAbsolutePath() + this.suffixCreator.getSuffix(2)); + assertTrue(part2.exists()); + assertEquals("4", readFile(part2)); + + this.writer.write(Arrays.asList("5")); + assertEquals("45", readFile(part2)); + + this.writer.write(Arrays.asList("6", "7", "8", "9")); + File part3 = new File(this.file.getAbsolutePath() + this.suffixCreator.getSuffix(3)); + assertTrue(part3.exists()); + assertEquals("6789", readFile(part3)); + } + + @Test + public void testBasicDefaultSuffixCreator() throws Exception { + + SimpleResourceSuffixCreator simpleResourceSuffixCreator = new SimpleResourceSuffixCreator(); + this.writer = new MultiResourceItemWriterBuilder().delegate(this.delegate) + .resource(new FileSystemResource(this.file)).itemCountLimitPerResource(2).saveState(true).name("foo") + .build(); + + this.writer.open(this.executionContext); + + this.writer.write(Arrays.asList("1", "2", "3")); + + File part1 = new File(this.file.getAbsolutePath() + simpleResourceSuffixCreator.getSuffix(1)); + assertTrue(part1.exists()); + assertEquals("123", readFile(part1)); + + this.writer.write(Arrays.asList("4")); + File part2 = new File(this.file.getAbsolutePath() + simpleResourceSuffixCreator.getSuffix(2)); + assertTrue(part2.exists()); + assertEquals("4", readFile(part2)); + } + + @Test + public void testUpdateAfterDelegateClose() throws Exception { + + this.writer = new MultiResourceItemWriterBuilder().delegate(this.delegate) + .resource(new FileSystemResource(this.file)).resourceSuffixCreator(this.suffixCreator) + .itemCountLimitPerResource(2).saveState(true).name("foo").build(); + + this.writer.update(this.executionContext); + assertEquals(0, this.executionContext.getInt(this.writer.getExecutionContextKey("resource.item.count"))); + assertEquals(1, this.executionContext.getInt(this.writer.getExecutionContextKey("resource.index"))); + this.writer.write(Arrays.asList("1", "2", "3")); + this.writer.update(this.executionContext); + assertEquals(0, this.executionContext.getInt(this.writer.getExecutionContextKey("resource.item.count"))); + assertEquals(2, this.executionContext.getInt(this.writer.getExecutionContextKey("resource.index"))); + } + + @Test + public void testRestart() throws Exception { + + this.writer = new MultiResourceItemWriterBuilder().delegate(this.delegate) + .resource(new FileSystemResource(this.file)).resourceSuffixCreator(this.suffixCreator) + .itemCountLimitPerResource(2).saveState(true).name("foo").build(); + + this.writer.write(Arrays.asList("1", "2", "3")); + + File part1 = new File(this.file.getAbsolutePath() + this.suffixCreator.getSuffix(1)); + assertTrue(part1.exists()); + assertEquals("123", readFile(part1)); + + this.writer.write(Arrays.asList("4")); + File part2 = new File(this.file.getAbsolutePath() + this.suffixCreator.getSuffix(2)); + assertTrue(part2.exists()); + assertEquals("4", readFile(part2)); + + this.writer.update(this.executionContext); + this.writer.close(); + this.writer.open(this.executionContext); + + this.writer.write(Arrays.asList("5")); + assertEquals("45", readFile(part2)); + + this.writer.write(Arrays.asList("6", "7", "8", "9")); + File part3 = new File(this.file.getAbsolutePath() + this.suffixCreator.getSuffix(3)); + assertTrue(part3.exists()); + assertEquals("6789", readFile(part3)); + } + + @Test + public void testRestartNoSaveState() throws Exception { + + this.writer = new MultiResourceItemWriterBuilder().delegate(this.delegate) + .resource(new FileSystemResource(this.file)).resourceSuffixCreator(this.suffixCreator) + .itemCountLimitPerResource(2).saveState(false).name("foo").build(); + + this.writer.write(Arrays.asList("1", "2", "3")); + + File part1 = new File(this.file.getAbsolutePath() + this.suffixCreator.getSuffix(1)); + assertTrue(part1.exists()); + assertEquals("123", readFile(part1)); + + this.writer.write(Arrays.asList("4")); + File part2 = new File(this.file.getAbsolutePath() + this.suffixCreator.getSuffix(2)); + assertTrue(part2.exists()); + assertEquals("4", readFile(part2)); + + this.writer.update(this.executionContext); + this.writer.close(); + this.writer.open(this.executionContext); + + this.writer.write(Arrays.asList("5")); + assertEquals("4", readFile(part2)); + + this.writer.write(Arrays.asList("6", "7", "8", "9")); + File part3 = new File(this.file.getAbsolutePath() + this.suffixCreator.getSuffix(1)); + assertTrue(part3.exists()); + assertEquals("56789", readFile(part3)); + } + + @Test + public void testSaveStateNoName() { + try { + new MultiResourceItemWriterBuilder().delegate(this.delegate) + .resource(new FileSystemResource(this.file)).resourceSuffixCreator(this.suffixCreator) + .itemCountLimitPerResource(2).saveState(true).build(); + + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException ise) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "A name is required when saveState is true.", ise.getMessage()); + } + } + + @Test + public void testNoResource() throws Exception { + try { + new MultiResourceItemWriterBuilder().delegate(this.delegate) + .resourceSuffixCreator(this.suffixCreator).itemCountLimitPerResource(2).build(); + + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException ise) { + assertEquals("IllegalArgumentException message did not match the expected result.", "resource is required.", + ise.getMessage()); + } + } + + @Test + public void testNoDelegateNoName() { + try { + new MultiResourceItemWriterBuilder().resource(new FileSystemResource(this.file)) + .resourceSuffixCreator(this.suffixCreator).itemCountLimitPerResource(2).saveState(false).build(); + + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException ise) { + assertEquals("IllegalArgumentException message did not match the expected result.", "delegate is required.", + ise.getMessage()); + } + } + + private String readFile(File f) throws Exception { + BufferedReader reader = new BufferedReader(new FileReader(f)); + StringBuilder result = new StringBuilder(); + try { + while (true) { + String line = reader.readLine(); + if (line == null) { + break; + } + result.append(line); + } + } + finally { + reader.close(); + } + return result.toString(); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapperConcurrentTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapperConcurrentTests.java index d8b10d434d..f83d7d64fe 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapperConcurrentTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapperConcurrentTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -33,7 +33,7 @@ public class BeanWrapperFieldSetMapperConcurrentTests { @Test public void testConcurrentUsage() throws Exception { - final BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + final BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setStrict(true); mapper.setTargetType(GreenBean.class); // mapper.setDistanceLimit(0); @@ -42,7 +42,7 @@ public void testConcurrentUsage() throws Exception { lineTokenizer.setNames(names); ExecutorService executorService = Executors.newFixedThreadPool(5); - Collection> results = new ArrayList>(); + Collection> results = new ArrayList<>(); for (int i = 0; i < 10; i++) { Future result = executorService.submit(new Callable() { @Override diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapperFuzzyMatchingTest.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapperFuzzyMatchingTest.java index 4dbf70c764..8733f12130 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapperFuzzyMatchingTest.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapperFuzzyMatchingTest.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,7 +26,7 @@ public class BeanWrapperFieldSetMapperFuzzyMatchingTest { @Test(expected = NotWritablePropertyException.class) public void testFuzzyMatchingWithKeyCandidateCollision() throws BindException { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setStrict(true); mapper.setTargetType(GreenBean.class); DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer(); @@ -38,7 +38,7 @@ public void testFuzzyMatchingWithKeyCandidateCollision() throws BindException { @Test public void testFuzzyMatchingWithLowerLimit() throws BindException { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setDistanceLimit(0); mapper.setStrict(false); mapper.setTargetType(GreenBean.class); @@ -51,7 +51,7 @@ public void testFuzzyMatchingWithLowerLimit() throws BindException { @Test public void testFuzzyMatchingWithPropertyCollision() throws BindException { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setStrict(true); mapper.setTargetType(BlueBean.class); DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapperTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapperTests.java index f0e94bb1a9..867225007b 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapperTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/BeanWrapperFieldSetMapperTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,10 +16,6 @@ package org.springframework.batch.item.file.mapping; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - import java.beans.PropertyEditor; import java.math.BigDecimal; import java.text.NumberFormat; @@ -31,8 +27,12 @@ import java.util.Locale; import java.util.Map; import java.util.Properties; +import java.util.TimeZone; +import org.junit.After; +import org.junit.Before; import org.junit.Test; + import org.springframework.batch.item.file.transform.DefaultFieldSet; import org.springframework.batch.item.file.transform.FieldSet; import org.springframework.beans.BeanWrapperImpl; @@ -43,39 +43,72 @@ import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; import org.springframework.context.support.StaticApplicationContext; +import org.springframework.core.convert.ConversionService; +import org.springframework.core.convert.TypeDescriptor; +import org.springframework.core.convert.support.DefaultConversionService; +import org.springframework.lang.Nullable; import org.springframework.test.util.ReflectionTestUtils; import org.springframework.validation.BindException; import org.springframework.validation.DataBinder; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + public class BeanWrapperFieldSetMapperTests { + private static final TimeZone UTC_TIME_ZONE = TimeZone.getTimeZone("UTC"); + + private TimeZone defaultTimeZone = TimeZone.getDefault(); + + @Before + public void setUp() { + TimeZone.setDefault(UTC_TIME_ZONE); + } + + @After + public void tearDown() { + TimeZone.setDefault(defaultTimeZone); + } + @Test public void testNameAndTypeSpecified() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + boolean errorCaught = false; + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setTargetType(TestObject.class); mapper.setPrototypeBeanName("foo"); try { mapper.afterPropertiesSet(); } - catch (IllegalStateException e) { - // expected + catch (IllegalStateException ise) { + errorCaught = true; + assertEquals("Both name and type cannot be specified together.", ise.getMessage()); + } + if (!errorCaught) { + fail(); } } @Test public void testNameNorTypeSpecified() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + boolean errorCaught = false; + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); try { mapper.afterPropertiesSet(); } - catch (IllegalStateException e) { - // expected + catch (IllegalStateException ise) { + errorCaught = true; + assertEquals("Either name or type must be provided.", ise.getMessage()); + } + if (!errorCaught) { + fail(); } - } + +} @Test public void testVanillaBeanCreatedFromType() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setTargetType(TestObject.class); mapper.afterPropertiesSet(); @@ -89,7 +122,7 @@ public void testVanillaBeanCreatedFromType() throws Exception { @Test public void testNullPropertyAutoCreated() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setTargetType(TestNestedA.class); mapper.afterPropertiesSet(); @@ -101,7 +134,8 @@ public void testNullPropertyAutoCreated() throws Exception { @Test public void testMapperWithSingleton() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + @SuppressWarnings("resource") StaticApplicationContext context = new StaticApplicationContext(); mapper.setBeanFactory(context); context.getBeanFactory().registerSingleton("bean", new TestObject()); @@ -117,7 +151,8 @@ public void testMapperWithSingleton() throws Exception { @Test public void testPropertyNameMatching() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + @SuppressWarnings("resource") StaticApplicationContext context = new StaticApplicationContext(); mapper.setBeanFactory(context); mapper.setDistanceLimit(2); @@ -133,7 +168,7 @@ public void testPropertyNameMatching() throws Exception { } @Test - @SuppressWarnings("unchecked") + @SuppressWarnings({"unchecked", "resource"}) public void testMapperWithPrototype() throws Exception { ApplicationContext context = new ClassPathXmlApplicationContext("bean-wrapper.xml", getClass()); @@ -156,7 +191,8 @@ public void testMapperWithNestedBeanPaths() throws Exception { testNestedA.setTestObjectB(testNestedB); testNestedB.setTestObjectC(new TestNestedC()); - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + @SuppressWarnings("resource") StaticApplicationContext context = new StaticApplicationContext(); mapper.setBeanFactory(context); context.getBeanFactory().registerSingleton("bean", testNestedA); @@ -178,7 +214,8 @@ public void testMapperWithNestedBeanPaths() throws Exception { public void testMapperWithSimilarNamePropertyMatches() throws Exception { TestNestedA testNestedA = new TestNestedA(); - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + @SuppressWarnings("resource") StaticApplicationContext context = new StaticApplicationContext(); mapper.setBeanFactory(context); mapper.setDistanceLimit(2); @@ -188,7 +225,7 @@ public void testMapperWithSimilarNamePropertyMatches() throws Exception { FieldSet fieldSet = new DefaultFieldSet(new String[] { "This is some dummy string", "1" }, new String[] { "VALUE_A", "VALUE_B" }); - TestNestedA result = (TestNestedA) mapper.mapFieldSet(fieldSet); + TestNestedA result = mapper.mapFieldSet(fieldSet); assertEquals("This is some dummy string", result.getValueA()); assertEquals(1, result.getValueB()); @@ -198,7 +235,8 @@ public void testMapperWithSimilarNamePropertyMatches() throws Exception { public void testMapperWithNotVerySimilarNamePropertyMatches() throws Exception { TestNestedC testNestedC = new TestNestedC(); - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + @SuppressWarnings("resource") StaticApplicationContext context = new StaticApplicationContext(); mapper.setBeanFactory(context); context.getBeanFactory().registerSingleton("bean", testNestedC); @@ -220,7 +258,8 @@ public void testMapperWithNestedBeanPathsAndPropertyMatches() throws Exception { testNestedA.setTestObjectB(testNestedB); testNestedB.setTestObjectC(new TestNestedC()); - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + @SuppressWarnings("resource") StaticApplicationContext context = new StaticApplicationContext(); mapper.setBeanFactory(context); context.getBeanFactory().registerSingleton("bean", testNestedA); @@ -242,7 +281,8 @@ public void testMapperWithNestedBeanPathsAndPropertyMisMatches() throws Exceptio TestNestedB testNestedB = new TestNestedB(); testNestedA.setTestObjectB(testNestedB); - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + @SuppressWarnings("resource") StaticApplicationContext context = new StaticApplicationContext(); mapper.setBeanFactory(context); context.getBeanFactory().registerSingleton("bean", testNestedA); @@ -265,7 +305,8 @@ public void testMapperWithNestedBeanPathsAndPropertyPrefixMisMatches() throws Ex TestNestedB testNestedB = new TestNestedB(); testNestedA.setTestObjectB(testNestedB); - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + @SuppressWarnings("resource") StaticApplicationContext context = new StaticApplicationContext(); mapper.setBeanFactory(context); context.getBeanFactory().registerSingleton("bean", testNestedA); @@ -299,13 +340,14 @@ public void testPlainBeanWrapper() throws Exception { public void testNestedList() throws Exception { TestNestedList nestedList = new TestNestedList(); - List nestedC = new ArrayList(); + List nestedC = new ArrayList<>(); nestedC.add(new TestNestedC()); nestedC.add(new TestNestedC()); nestedC.add(new TestNestedC()); nestedList.setNestedC(nestedC); - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + @SuppressWarnings("resource") StaticApplicationContext context = new StaticApplicationContext(); mapper.setBeanFactory(context); context.getBeanFactory().registerSingleton("bean", nestedList); @@ -333,6 +375,7 @@ protected void initBinder(DataBinder binder) { ReflectionTestUtils.setField(binder, "autoGrowNestedPaths", true); } }; + @SuppressWarnings("resource") StaticApplicationContext context = new StaticApplicationContext(); mapper.setBeanFactory(context); context.getBeanFactory().registerSingleton("bean", nestedList); @@ -352,11 +395,11 @@ protected void initBinder(DataBinder binder) { @Test public void testPaddedLongWithNoEditor() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setTargetType(TestObject.class); FieldSet fieldSet = new DefaultFieldSet(new String[] { "00009" }, new String[] { "varLong" }); - TestObject bean = (TestObject) mapper.mapFieldSet(fieldSet); + TestObject bean = mapper.mapFieldSet(fieldSet); // since Spring 2.5.5 this is OK (before that BATCH-261) assertEquals(9, bean.getVarLong()); } @@ -364,14 +407,14 @@ public void testPaddedLongWithNoEditor() throws Exception { @Test public void testPaddedLongWithEditor() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setTargetType(TestObject.class); FieldSet fieldSet = new DefaultFieldSet(new String[] { "00009" }, new String[] { "varLong" }); mapper.setCustomEditors(Collections.singletonMap(Long.TYPE, new CustomNumberEditor(Long.class, NumberFormat .getNumberInstance(), true))); - TestObject bean = (TestObject) mapper.mapFieldSet(fieldSet); + TestObject bean = mapper.mapFieldSet(fieldSet); assertEquals(9, bean.getVarLong()); } @@ -379,14 +422,14 @@ public void testPaddedLongWithEditor() throws Exception { @Test public void testPaddedLongWithDefaultAndCustomEditor() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setTargetType(TestObject.class); FieldSet fieldSet = new DefaultFieldSet(new String[] { "00009", "78" }, new String[] { "varLong", "varInt" }); mapper.setCustomEditors(Collections.singletonMap(Long.TYPE, new CustomNumberEditor(Long.class, NumberFormat .getNumberInstance(), true))); - TestObject bean = (TestObject) mapper.mapFieldSet(fieldSet); + TestObject bean = mapper.mapFieldSet(fieldSet); assertEquals(9, bean.getVarLong()); assertEquals(78, bean.getVarInt()); @@ -395,27 +438,99 @@ public void testPaddedLongWithDefaultAndCustomEditor() throws Exception { @Test public void testNumberFormatWithDefaultAndCustomEditor() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setTargetType(TestObject.class); FieldSet fieldSet = new DefaultFieldSet(new String[] { "9.876,1", "7,890.1" }, new String[] { "varDouble", "varFloat" }); - Map, PropertyEditor> editors = new HashMap, PropertyEditor>(); + Map, PropertyEditor> editors = new HashMap<>(); editors.put(Double.TYPE, new CustomNumberEditor(Double.class, NumberFormat.getInstance(Locale.GERMAN), true)); editors.put(Float.TYPE, new CustomNumberEditor(Float.class, NumberFormat.getInstance(Locale.UK), true)); mapper.setCustomEditors(editors); - TestObject bean = (TestObject) mapper.mapFieldSet(fieldSet); + TestObject bean = mapper.mapFieldSet(fieldSet); assertEquals(9876.1, bean.getVarDouble(), 0.01); assertEquals(7890.1, bean.getVarFloat(), 0.01); } + + @Test + public void testConversionWithTestConverter() throws Exception { + + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + mapper.setTargetType(TestObject.class); + + FieldSet fieldSet = new DefaultFieldSet(new String[] { "SHOULD BE CONVERTED" }, new String[] { "varString" }); + + mapper.setConversionService(new TestConversion()); + mapper.afterPropertiesSet(); + TestObject bean = mapper.mapFieldSet(fieldSet); + + assertEquals("Expecting the conversion to have returned \"CONVERTED\"", bean.getVarString(), "CONVERTED"); + } + + @Test + public void testDefaultConversion() throws Exception { + + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + mapper.setTargetType(TestObject.class); + + final String sampleString = "myString"; + Date date = new Date(); + BigDecimal bigDecimal = new BigDecimal(12345L); + String dateString = date.toString(); + + + FieldSet fieldSet = new DefaultFieldSet(new String[] { "12", "12345", "true", "Z", "123", "12345", "12345", "12", dateString, "12345", sampleString}, + new String[] { "varInt", "varLong", "varBoolean", "varChar","varByte","varFloat", "varDouble", "varShort", "varDate", "varBigDecimal", "varString" }); + + mapper.setConversionService(new DefaultConversionService()); + mapper.afterPropertiesSet(); + + TestObject bean = mapper.mapFieldSet(fieldSet); + + assertEquals("Expected 12 for varInt", bean.getVarInt(), 12); + assertEquals("Expected 12345 for varLong", bean.getVarLong(), 12345L); + assertEquals("Expected true for varBoolean", bean.isVarBoolean(), true); + assertEquals("Expected Z for varChar", bean.getVarChar(), 'Z'); + assertEquals("Expected A for varByte", bean.getVarByte(), 123); + assertEquals("Expected 12345 for varFloat", bean.getVarFloat(), 12345F, 1F); + assertEquals("Expected 12345 for varDouble", bean.getVarDouble(), 12345D, 1D); + assertEquals("Expected 12 for varShort", bean.getVarShort(), 12); + assertEquals("Expected currentDate for varDate", bean.getVarDate().toString(), dateString); + assertEquals("Expected 12345 for varBigDecimal", bean.getVarBigDecimal(), bigDecimal); + assertEquals("Expected " + sampleString + " for varString", bean.getVarString(), sampleString); + + } + + @Test + public void testConversionAndCustomEditor() throws Exception { + + boolean errorCaught = false; + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); + mapper.setTargetType(TestObject.class); + + mapper.setConversionService(new TestConversion()); + mapper.setCustomEditors(Collections.singletonMap(Long.TYPE, new CustomNumberEditor(Long.class, NumberFormat + .getNumberInstance(), true))); + try { + mapper.afterPropertiesSet(); + } + catch (IllegalStateException ise) { + errorCaught = true; + assertEquals("Both customEditor and conversionService cannot be specified together.", ise.getMessage()); + } + if (!errorCaught) { + fail(); + } + } + @Test public void testBinderWithErrors() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setTargetType(TestObject.class); FieldSet fieldSet = new DefaultFieldSet(new String[] { "foo", "7890.1" }, new String[] { "varDouble", @@ -472,7 +587,7 @@ public void registerCustomEditors(PropertyEditorRegistry registry) { @Test public void testStrict() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setStrict(true); mapper.setTargetType(TestObject.class); mapper.afterPropertiesSet(); @@ -490,7 +605,7 @@ public void testStrict() throws Exception { @Test public void testNotStrict() throws Exception { - BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper(); + BeanWrapperFieldSetMapper mapper = new BeanWrapperFieldSetMapper<>(); mapper.setStrict(false); mapper.setTargetType(TestObject.class); mapper.afterPropertiesSet(); @@ -505,7 +620,7 @@ public void testNotStrict() throws Exception { private static class TestNestedList { - List nestedC = new ArrayList(); + List nestedC = new ArrayList<>(); public List getNestedC() { return nestedC; @@ -722,4 +837,30 @@ public void setVarInt(int varInt) { this.varInt = varInt; } } + + public static class TestConversion implements ConversionService{ + + @Override + public boolean canConvert(@Nullable Class sourceType, Class targetType) { + return true; + } + + @Override + public boolean canConvert(@Nullable TypeDescriptor sourceType, TypeDescriptor targetType) { + return true; + } + + @Nullable + @Override + @SuppressWarnings("unchecked") + public T convert(@Nullable Object source, Class targetType) { + return (T)"CONVERTED"; + } + + @Nullable + @Override + public Object convert(@Nullable Object source, @Nullable TypeDescriptor sourceType, TypeDescriptor targetType) { + return "CONVERTED"; + } + } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/DefaultLineMapperTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/DefaultLineMapperTests.java index 4cb6e92998..12fdb2c590 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/DefaultLineMapperTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/DefaultLineMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file.mapping; import static org.mockito.Mockito.mock; @@ -15,7 +30,7 @@ */ public class DefaultLineMapperTests { - private DefaultLineMapper tested = new DefaultLineMapper(); + private DefaultLineMapper tested = new DefaultLineMapper<>(); @Test(expected=IllegalArgumentException.class) public void testMandatoryTokenizer() throws Exception { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/JsonLineMapperTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/JsonLineMapperTests.java index 2c4ac6e691..5d68be2c8b 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/JsonLineMapperTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/JsonLineMapperTests.java @@ -1,12 +1,27 @@ +/* + * Copyright 2009-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file.mapping; -import static org.junit.Assert.assertEquals; - import java.util.Map; -import org.codehaus.jackson.JsonParseException; +import com.fasterxml.jackson.core.JsonParseException; import org.junit.Test; +import static org.junit.Assert.assertEquals; + public class JsonLineMapperTests { private JsonLineMapper mapper = new JsonLineMapper(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PassThroughFieldSetMapperTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PassThroughFieldSetMapperTests.java index d4a62ad2b3..3b6b80c351 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PassThroughFieldSetMapperTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PassThroughFieldSetMapperTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PassThroughLineMapperTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PassThroughLineMapperTests.java index 5e88bb64a2..7c45f53756 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PassThroughLineMapperTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PassThroughLineMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file.mapping; import static org.junit.Assert.*; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapperTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapperTests.java index 7aa2e5ee3a..1cffddc2f1 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapperTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapperTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,7 +16,7 @@ package org.springframework.batch.item.file.mapping; -import static junit.framework.Assert.assertEquals; +import static org.junit.Assert.assertEquals; import java.util.Collections; import java.util.HashMap; @@ -28,6 +28,7 @@ import org.springframework.batch.item.file.transform.FieldSet; import org.springframework.batch.item.file.transform.LineTokenizer; import org.springframework.batch.item.file.transform.Name; +import org.springframework.lang.Nullable; /** * @author Dan Garrette @@ -36,7 +37,7 @@ */ public class PatternMatchingCompositeLineMapperTests { - private PatternMatchingCompositeLineMapper mapper = new PatternMatchingCompositeLineMapper(); + private PatternMatchingCompositeLineMapper mapper = new PatternMatchingCompositeLineMapper<>(); @Test(expected = IllegalArgumentException.class) public void testNoMappers() throws Exception { @@ -48,22 +49,22 @@ public void testNoMappers() throws Exception { @Test public void testKeyFound() throws Exception { - Map tokenizers = new HashMap(); + Map tokenizers = new HashMap<>(); tokenizers.put("foo*", new LineTokenizer() { @Override - public FieldSet tokenize(String line) { + public FieldSet tokenize(@Nullable String line) { return new DefaultFieldSet(new String[] { "a", "b" }); } }); tokenizers.put("bar*", new LineTokenizer() { @Override - public FieldSet tokenize(String line) { + public FieldSet tokenize(@Nullable String line) { return new DefaultFieldSet(new String[] { "c", "d" }); } }); mapper.setTokenizers(tokenizers); - Map> fieldSetMappers = new HashMap>(); + Map> fieldSetMappers = new HashMap<>(); fieldSetMappers.put("foo*", new FieldSetMapper() { @Override public Name mapFieldSet(FieldSet fs) { @@ -84,22 +85,22 @@ public Name mapFieldSet(FieldSet fs) { @Test(expected = IllegalStateException.class) public void testMapperKeyNotFound() throws Exception { - Map tokenizers = new HashMap(); + Map tokenizers = new HashMap<>(); tokenizers.put("foo*", new LineTokenizer() { @Override - public FieldSet tokenize(String line) { + public FieldSet tokenize(@Nullable String line) { return new DefaultFieldSet(new String[] { "a", "b" }); } }); tokenizers.put("bar*", new LineTokenizer() { @Override - public FieldSet tokenize(String line) { + public FieldSet tokenize(@Nullable String line) { return new DefaultFieldSet(new String[] { "c", "d" }); } }); mapper.setTokenizers(tokenizers); - Map> fieldSetMappers = new HashMap>(); + Map> fieldSetMappers = new HashMap<>(); fieldSetMappers.put("foo*", new FieldSetMapper() { @Override public Name mapFieldSet(FieldSet fs) { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PropertyMatchesTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PropertyMatchesTests.java index b2a8a4488f..c50955883f 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PropertyMatchesTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/mapping/PropertyMatchesTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/DefaultRecordSeparatorPolicyTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/DefaultRecordSeparatorPolicyTests.java index fee7d672cf..99584bd474 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/DefaultRecordSeparatorPolicyTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/DefaultRecordSeparatorPolicyTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/JsonRecordSeparatorPolicyTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/JsonRecordSeparatorPolicyTests.java index 9ec34e591f..290d189a88 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/JsonRecordSeparatorPolicyTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/JsonRecordSeparatorPolicyTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file.separator; import static org.junit.Assert.*; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/SimpleRecordSeparatorPolicyTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/SimpleRecordSeparatorPolicyTests.java index 7f824f1174..36ce460fa8 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/SimpleRecordSeparatorPolicyTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/SimpleRecordSeparatorPolicyTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/SuffixRecordSeparatorPolicyTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/SuffixRecordSeparatorPolicyTests.java index dd0c03115c..6c4f716a3c 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/SuffixRecordSeparatorPolicyTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/separator/SuffixRecordSeparatorPolicyTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/BeanWrapperFieldExtractorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/BeanWrapperFieldExtractorTests.java index 0d99cb4704..834db77855 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/BeanWrapperFieldExtractorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/BeanWrapperFieldExtractorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,9 +16,9 @@ package org.springframework.batch.item.file.transform; -import static junit.framework.Assert.assertEquals; -import static junit.framework.Assert.assertTrue; -import static junit.framework.Assert.fail; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; import org.junit.Test; import org.springframework.beans.NotReadablePropertyException; @@ -29,7 +29,7 @@ */ public class BeanWrapperFieldExtractorTests { - private BeanWrapperFieldExtractor extractor = new BeanWrapperFieldExtractor(); + private BeanWrapperFieldExtractor extractor = new BeanWrapperFieldExtractor<>(); @Test public void testExtract() throws Exception { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/CommonLineTokenizerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/CommonLineTokenizerTests.java index 5528a0c564..2f383783ee 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/CommonLineTokenizerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/CommonLineTokenizerTests.java @@ -1,5 +1,21 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file.transform; +import java.util.ArrayList; import java.util.List; import junit.framework.TestCase; @@ -25,13 +41,13 @@ protected List doTokenize(String line) { assertFalse(tokenizer.hasNames()); - tokenizer.setNames(null); + tokenizer.setNames((String) null); assertFalse(tokenizer.hasNames()); - tokenizer.setNames(new String[0]); + tokenizer.setNames(new ArrayList().toArray(new String[0])); assertFalse(tokenizer.hasNames()); - tokenizer.setNames(new String[]{"name1", "name2"}); + tokenizer.setNames("name1", "name2"); assertTrue(tokenizer.hasNames()); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DefaultFieldSetFactoryTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DefaultFieldSetFactoryTests.java index c959adbed1..dd0e5c7b9b 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DefaultFieldSetFactoryTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DefaultFieldSetFactoryTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DefaultFieldSetTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DefaultFieldSetTests.java index 2ed6ca9ecd..13110fce7a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DefaultFieldSetTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DefaultFieldSetTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -430,6 +430,37 @@ public void testReadDateWithPatternAndDefault() throws Exception { assertEquals(date, fieldSet.readDate("BlankInput", "dd-MM-yyyy", date)); } + @Test + public void testReadDateInvalidWithDefault() throws Exception { + Date defaultDate = new Date(); + try { + fieldSet.readDate(1, defaultDate); + fail("Should throw IllegalArgumentException"); + } catch (IllegalArgumentException e) { + assertTrue(e.getMessage().indexOf("yyyy-MM-dd") > 0); + } + try { + fieldSet.readDate("String", defaultDate); + fail("Should throw IllegalArgumentException"); + } catch (IllegalArgumentException e) { + assertTrue(e.getMessage().indexOf("yyyy-MM-dd") > 0); + assertTrue(e.getMessage().indexOf("name: [String]") > 0); + } + try { + fieldSet.readDate(1, "dd-MM-yyyy", defaultDate); + fail("Should throw IllegalArgumentException"); + } catch (IllegalArgumentException e) { + assertTrue(e.getMessage().indexOf("dd-MM-yyyy") > 0); + } + try { + fieldSet.readDate("String", "dd-MM-yyyy", defaultDate); + fail("Should throw IllegalArgumentException"); + } catch (IllegalArgumentException e) { + assertTrue(e.getMessage().indexOf("dd-MM-yyyy") > 0); + assertTrue(e.getMessage().indexOf("name: [String]") > 0); + } + } + @Test public void testStrictReadDateWithPattern() throws Exception { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DelimitedLineAggregatorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DelimitedLineAggregatorTests.java index 47b975196a..94ca082ee3 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DelimitedLineAggregatorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DelimitedLineAggregatorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -37,7 +37,7 @@ public Object[] extract(String[] item) { @Before public void setup() { - aggregator = new DelimitedLineAggregator(); + aggregator = new DelimitedLineAggregator<>(); aggregator.setFieldExtractor(defaultFieldExtractor); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DelimitedLineTokenizerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DelimitedLineTokenizerTests.java index eb687f0afc..588b93a204 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DelimitedLineTokenizerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/DelimitedLineTokenizerTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,12 +16,12 @@ package org.springframework.batch.item.file.transform; +import org.junit.Test; + import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertTrue; import static org.junit.Assert.fail; -import org.junit.Test; - public class DelimitedLineTokenizerTests { @@ -46,6 +46,18 @@ public void testTokenizeRegularUse() { assertTrue(TOKEN_MATCHES, tokens.readString(1).equals("")); } + @Test + public void testBlankString() { + FieldSet tokens = tokenizer.tokenize(" "); + assertTrue(TOKEN_MATCHES, tokens.readString(0).equals("")); + } + + @Test + public void testEmptyString() { + FieldSet tokens = tokenizer.tokenize("\"\""); + assertTrue(TOKEN_MATCHES, tokens.readString(0).equals("")); + } + @Test public void testInvalidConstructorArgument() { try { @@ -81,6 +93,7 @@ public void testTooFewNames() { catch (IncorrectTokenCountException e) { assertEquals(2, e.getExpectedCount()); assertEquals(3, e.getActualCount()); + assertEquals("a,b,c", e.getInput()); } } @@ -104,6 +117,7 @@ public void testTooManyNames() { catch(IncorrectTokenCountException e){ assertEquals(4, e.getExpectedCount()); assertEquals(3, e.getActualCount()); + assertEquals("a,b,c", e.getInput()); } } @@ -129,6 +143,19 @@ public void testDelimitedLineTokenizerChar() { assertEquals(3, line.getFieldCount()); } + @Test(expected=IllegalArgumentException.class) + public void testDelimitedLineTokenizerNullDelimiter() { + AbstractLineTokenizer tokenizer = new DelimitedLineTokenizer(null); + tokenizer.tokenize("a b c"); + } + + @Test(expected=IllegalArgumentException.class) + public void testDelimitedLineTokenizerEmptyString() throws Exception { + DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer(""); + tokenizer.afterPropertiesSet(); + tokenizer.tokenize("a b c"); + } + @Test public void testDelimitedLineTokenizerString() { AbstractLineTokenizer tokenizer = new DelimitedLineTokenizer(" b "); @@ -279,6 +306,7 @@ public void testEmptyLineWithNames(){ catch(IncorrectTokenCountException ex){ assertEquals(2, ex.getExpectedCount()); assertEquals(0, ex.getActualCount()); + assertEquals("", ex.getInput()); } } @@ -362,4 +390,12 @@ public void testTokenizeWithIncludedFieldsAndTooManyNames() { assertEquals("c", line.readString("bar")); } -} \ No newline at end of file + @Test + public void testTokenizeOverMultipleLines() { + tokenizer = new DelimitedLineTokenizer(";"); + FieldSet line = tokenizer.tokenize("value1;\"value2\nvalue2cont\";value3;value4"); + assertEquals(4, line.getFieldCount()); + assertEquals("value2\nvalue2cont", line.readString(1)); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/FixedLengthTokenizerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/FixedLengthTokenizerTests.java index a1f4669f21..9d08c104c5 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/FixedLengthTokenizerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/FixedLengthTokenizerTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -41,6 +41,7 @@ public void testTokenizeEmptyString() { catch (IncorrectLineLengthException ex) { assertEquals(15, ex.getExpectedLength()); assertEquals(0, ex.getActualLength()); + assertEquals("", ex.getInput()); } } @@ -60,6 +61,7 @@ public void testTokenizeSmallerStringThanRanges() { catch (IncorrectLineLengthException ex) { assertEquals(15, ex.getExpectedLength()); assertEquals(5, ex.getActualLength()); + assertEquals("12345", ex.getInput()); } } @@ -97,6 +99,7 @@ public void testTokenizeNullString() { fail("Expected IncorrectLineLengthException"); } catch (IncorrectLineLengthException ex) { + assertEquals("", ex.getInput()); } } @@ -135,6 +138,7 @@ public void testLongerLines() throws Exception { catch (IncorrectLineLengthException ex) { assertEquals(30, ex.getExpectedLength()); assertEquals(35, ex.getActualLength()); + assertEquals(line, ex.getInput()); } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/FormatterLineAggregatorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/FormatterLineAggregatorTests.java index b7bc5dc7d7..4b4c368fd7 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/FormatterLineAggregatorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/FormatterLineAggregatorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -41,7 +41,7 @@ public Object[] extract(String[] item) { @Before public void setup() { - aggregator = new FormatterLineAggregator(); + aggregator = new FormatterLineAggregator<>(); aggregator.setFieldExtractor(defaultFieldExtractor); } @@ -132,7 +132,7 @@ public Object[] extract(String[] item) { for (int i = 0; i < strings.length; i++) { strings[i] = item[i]; if (item[i].length() < widths[i]) { - StringBuffer buffer = new StringBuffer(strings[i]); + StringBuilder buffer = new StringBuilder(strings[i]); for (int j = 0; j < (widths[i] - item[i].length() + 1) / 2; j++) { buffer.append(" "); } @@ -167,7 +167,7 @@ public Object[] extract(String[] item) { for (int i = 0; i < strings.length; i++) { strings[i] = item[i]; if (item[i].length() < widths[i]) { - StringBuffer buffer = new StringBuffer(strings[i]); + StringBuilder buffer = new StringBuilder(strings[i]); for (int j = 0; j < widths[i] - item[i].length(); j++) { buffer.append("."); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/Name.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/Name.java index 4a37322d3d..493ff48b9d 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/Name.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/Name.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file.transform; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/PassThroughFieldExtractorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/PassThroughFieldExtractorTests.java index 43bc160628..6c2b58898e 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/PassThroughFieldExtractorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/PassThroughFieldExtractorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,7 +15,7 @@ */ package org.springframework.batch.item.file.transform; -import static junit.framework.Assert.assertTrue; +import static org.junit.Assert.assertTrue; import java.util.Arrays; import java.util.LinkedHashMap; @@ -32,36 +32,36 @@ public class PassThroughFieldExtractorTests { @Test public void testExtractString() { - PassThroughFieldExtractor extractor = new PassThroughFieldExtractor(); + PassThroughFieldExtractor extractor = new PassThroughFieldExtractor<>(); Object[] result = extractor.extract("abc"); assertTrue(Arrays.equals(new Object[] { "abc" }, result)); } @Test public void testExtractArray() { - PassThroughFieldExtractor extractor = new PassThroughFieldExtractor(); + PassThroughFieldExtractor extractor = new PassThroughFieldExtractor<>(); Object[] result = extractor.extract(new String[] { "a", "b", null, "d" }); assertTrue(Arrays.equals(new Object[] { "a", "b", null, "d" }, result)); } @Test public void testExtractFieldSet() { - PassThroughFieldExtractor
        extractor = new PassThroughFieldExtractor
        (); + PassThroughFieldExtractor
        extractor = new PassThroughFieldExtractor<>(); Object[] result = extractor.extract(new DefaultFieldSet(new String[] { "a", "b", "", "d" })); assertTrue(Arrays.equals(new Object[] { "a", "b", "", "d" }, result)); } @Test public void testExtractCollection() { - PassThroughFieldExtractor> extractor = new PassThroughFieldExtractor>(); + PassThroughFieldExtractor> extractor = new PassThroughFieldExtractor<>(); Object[] result = extractor.extract(Arrays.asList("a", "b", null, "d")); assertTrue(Arrays.equals(new Object[] { "a", "b", null, "d" }, result)); } @Test public void testExtractMap() { - PassThroughFieldExtractor> extractor = new PassThroughFieldExtractor>(); - Map map = new LinkedHashMap(); + PassThroughFieldExtractor> extractor = new PassThroughFieldExtractor<>(); + Map map = new LinkedHashMap<>(); map.put("A", "a"); map.put("B", "b"); map.put("C", null); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/PassThroughLineAggregatorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/PassThroughLineAggregatorTests.java index ed0a289211..44b9361c88 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/PassThroughLineAggregatorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/PassThroughLineAggregatorTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file.transform; import junit.framework.TestCase; @@ -7,7 +22,7 @@ public class PassThroughLineAggregatorTests extends TestCase { - private LineAggregator mapper = new PassThroughLineAggregator(); + private LineAggregator mapper = new PassThroughLineAggregator<>(); public void testUnmapItemAsFieldSet() throws Exception { Object item = new Object(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/PatternMatchingCompositeLineTokenizerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/PatternMatchingCompositeLineTokenizerTests.java index 7f9e717b45..76cde5cf8a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/PatternMatchingCompositeLineTokenizerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/PatternMatchingCompositeLineTokenizerTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,7 +16,7 @@ package org.springframework.batch.item.file.transform; -import static junit.framework.Assert.assertEquals; +import static org.junit.Assert.assertEquals; import java.util.Collections; import java.util.HashMap; @@ -24,6 +24,7 @@ import java.util.Map; import org.junit.Test; +import org.springframework.lang.Nullable; /** * @author Ben Hale @@ -42,11 +43,11 @@ public void testNoTokenizers() throws Exception { @Test public void testEmptyKeyMatchesAnyLine() throws Exception { - Map map = new HashMap(); + Map map = new HashMap<>(); map.put("*", new DelimitedLineTokenizer()); map.put("foo", new LineTokenizer() { @Override - public FieldSet tokenize(String line) { + public FieldSet tokenize(@Nullable String line) { return null; } }); @@ -59,10 +60,10 @@ public FieldSet tokenize(String line) { @Test public void testEmptyKeyDoesNotMatchWhenAlternativeAvailable() throws Exception { - Map map = new LinkedHashMap(); + Map map = new LinkedHashMap<>(); map.put("*", new LineTokenizer() { @Override - public FieldSet tokenize(String line) { + public FieldSet tokenize(@Nullable String line) { return null; } }); @@ -84,7 +85,7 @@ public void testNoMatch() throws Exception { public void testMatchWithPrefix() throws Exception { tokenizer.setTokenizers(Collections.singletonMap("foo*", (LineTokenizer) new LineTokenizer() { @Override - public FieldSet tokenize(String line) { + public FieldSet tokenize(@Nullable String line) { return new DefaultFieldSet(new String[] { line }); } })); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/RangeArrayPropertyEditorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/RangeArrayPropertyEditorTests.java index cd7f88097a..017acb68a6 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/RangeArrayPropertyEditorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/RangeArrayPropertyEditorTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.file.transform; import junit.framework.TestCase; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/RecursiveCollectionItemTransformerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/RecursiveCollectionItemTransformerTests.java index dd8a4a088d..d31822ece8 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/RecursiveCollectionItemTransformerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/RecursiveCollectionItemTransformerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,7 +30,7 @@ public class RecursiveCollectionItemTransformerTests extends TestCase { private static final String LINE_SEPARATOR = System.getProperty("line.separator"); - private RecursiveCollectionLineAggregator aggregator = new RecursiveCollectionLineAggregator(); + private RecursiveCollectionLineAggregator aggregator = new RecursiveCollectionLineAggregator<>(); public void testSetDelegateAndPassInString() throws Exception { aggregator.setDelegate(new LineAggregator() { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/RegexLineTokenizerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/RegexLineTokenizerTests.java index a2e72ae7fb..f2b518e94f 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/RegexLineTokenizerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/file/transform/RegexLineTokenizerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/function/FunctionItemProcessorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/function/FunctionItemProcessorTests.java new file mode 100644 index 0000000000..01ca7e95a7 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/function/FunctionItemProcessorTests.java @@ -0,0 +1,57 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.function; + +import java.util.function.Function; + +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.ItemProcessor; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; + +/** + * @author Michael Minella + */ +public class FunctionItemProcessorTests { + + private Function function; + + @Before + public void setUp() { + this.function = o -> o.toString(); + } + + @Test + public void testConstructorValidation() { + try { + new FunctionItemProcessor<>(null); + fail("null should not be accepted as a constructor arg"); + } + catch (IllegalArgumentException iae) {} + } + + @Test + public void testFunctionItemProcessor() throws Exception { + ItemProcessor itemProcessor = + new FunctionItemProcessor<>(this.function); + + assertEquals("1", itemProcessor.process(1L)); + assertEquals("foo", itemProcessor.process("foo")); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsItemReaderTests.java index ae0531bf91..3f3ac0e5ee 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsItemReaderTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -33,7 +33,7 @@ public class JmsItemReaderTests { - JmsItemReader itemReader = new JmsItemReader(); + JmsItemReader itemReader = new JmsItemReader<>(); @Test public void testNoItemTypeSunnyDay() { @@ -61,7 +61,7 @@ public void testSetItemSubclassTypeSunnyDay() { Date date = new java.sql.Date(0L); when(jmsTemplate.receiveAndConvert()).thenReturn(date); - JmsItemReader itemReader = new JmsItemReader(); + JmsItemReader itemReader = new JmsItemReader<>(); itemReader.setJmsTemplate(jmsTemplate); itemReader.setItemType(Date.class); assertEquals(date, itemReader.read()); @@ -73,7 +73,7 @@ public void testSetItemTypeMismatch() { JmsOperations jmsTemplate = mock(JmsOperations.class); when(jmsTemplate.receiveAndConvert()).thenReturn("foo"); - JmsItemReader itemReader = new JmsItemReader(); + JmsItemReader itemReader = new JmsItemReader<>(); itemReader.setJmsTemplate(jmsTemplate); itemReader.setItemType(Date.class); try { @@ -92,7 +92,7 @@ public void testNextMessageSunnyDay() { Message message = mock(Message.class); when(jmsTemplate.receive()).thenReturn(message); - JmsItemReader itemReader = new JmsItemReader(); + JmsItemReader itemReader = new JmsItemReader<>(); itemReader.setJmsTemplate(jmsTemplate); itemReader.setItemType(Message.class); assertEquals(message, itemReader.read()); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsItemWriterTests.java index 2f462f12ce..f21dd65c9d 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsItemWriterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,7 +26,7 @@ public class JmsItemWriterTests { - JmsItemWriter itemWriter = new JmsItemWriter(); + JmsItemWriter itemWriter = new JmsItemWriter<>(); @Test public void testNoItemTypeSunnyDay() throws Exception { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsMethodArgumentsKeyGeneratorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsMethodArgumentsKeyGeneratorTests.java index 2335ad5947..aa9a13b445 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsMethodArgumentsKeyGeneratorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsMethodArgumentsKeyGeneratorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -38,7 +38,7 @@ public void testGetKeyFromMessage() throws Exception { Message message = mock(Message.class); when(message.getJMSMessageID()).thenReturn("foo"); - JmsItemReader itemReader = new JmsItemReader(); + JmsItemReader itemReader = new JmsItemReader<>(); itemReader.setItemType(Message.class); assertEquals("foo", methodArgumentsKeyGenerator.getKey(new Object[]{message})); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsMethodInvocationRecovererTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsMethodInvocationRecovererTests.java index ed438e971a..26fbfdeabc 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsMethodInvocationRecovererTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsMethodInvocationRecovererTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,7 +27,7 @@ */ public class JmsMethodInvocationRecovererTests { - private JmsMethodInvocationRecoverer itemReader = new JmsMethodInvocationRecoverer(); + private JmsMethodInvocationRecoverer itemReader = new JmsMethodInvocationRecoverer<>(); @Test public void testRecoverWithNoDestination() throws Exception { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsNewMethodArgumentsIdentifierTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsNewMethodArgumentsIdentifierTests.java index 486623a2cc..466e15ec92 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsNewMethodArgumentsIdentifierTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/JmsNewMethodArgumentsIdentifierTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,7 +31,7 @@ */ public class JmsNewMethodArgumentsIdentifierTests { - private JmsNewMethodArgumentsIdentifier newMethodArgumentsIdentifier = new JmsNewMethodArgumentsIdentifier(); + private JmsNewMethodArgumentsIdentifier newMethodArgumentsIdentifier = new JmsNewMethodArgumentsIdentifier<>(); @Test public void testIsNewForMessage() throws Exception { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/builder/JmsItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/builder/JmsItemReaderBuilderTests.java new file mode 100644 index 0000000000..d024784285 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/builder/JmsItemReaderBuilderTests.java @@ -0,0 +1,104 @@ +/* + * Copyright 2017-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.jms.builder; + +import java.util.Date; + +import javax.jms.Message; + +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.jms.JmsItemReader; +import org.springframework.jms.core.JmsOperations; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +/** + * @author Glenn Renfro + * @author Mahmoud Ben Hassine + */ +public class JmsItemReaderBuilderTests { + + private JmsOperations defaultJmsTemplate; + + @Before + public void setupJmsTemplate() { + this.defaultJmsTemplate = mock(JmsOperations.class); + when(this.defaultJmsTemplate.receiveAndConvert()).thenReturn("foo"); + } + + @Test + public void testBasicRead() { + JmsItemReader itemReader = new JmsItemReaderBuilder().jmsTemplate(this.defaultJmsTemplate) + .build(); + assertEquals("foo", itemReader.read()); + } + + @Test + public void testSetItemSubclassType() { + JmsOperations jmsTemplate = mock(JmsOperations.class); + + Date date = new java.sql.Date(0L); + when(jmsTemplate.receiveAndConvert()).thenReturn(date); + + JmsItemReader itemReader = new JmsItemReaderBuilder().jmsTemplate(jmsTemplate).itemType(Date.class) + .build(); + assertEquals(date, itemReader.read()); + } + + @Test + public void testSetItemTypeMismatch() { + JmsItemReader itemReader = new JmsItemReaderBuilder().jmsTemplate(this.defaultJmsTemplate) + .itemType(Date.class).build(); + try { + itemReader.read(); + fail("Expected IllegalStateException"); + } + catch (IllegalStateException e) { + // expected + assertTrue(e.getMessage().indexOf("wrong type") >= 0); + } + } + + @Test + public void testMessageType() { + JmsOperations jmsTemplate = mock(JmsOperations.class); + Message message = mock(Message.class); + when(jmsTemplate.receive()).thenReturn(message); + + JmsItemReader itemReader = new JmsItemReaderBuilder().jmsTemplate(jmsTemplate) + .itemType(Message.class).build(); + assertEquals(message, itemReader.read()); + } + + @Test + public void testNullJmsTemplate() { + try { + new JmsItemReaderBuilder().itemType(String.class).build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException ise) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "jmsTemplate is required.", ise.getMessage()); + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/builder/JmsItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/builder/JmsItemWriterBuilderTests.java new file mode 100644 index 0000000000..42c891faa5 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/jms/builder/JmsItemWriterBuilderTests.java @@ -0,0 +1,61 @@ +/* + * Copyright 2017-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.jms.builder; + +import java.util.Arrays; + +import org.junit.Test; +import org.mockito.ArgumentCaptor; + +import org.springframework.batch.item.jms.JmsItemWriter; +import org.springframework.jms.core.JmsOperations; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; + +/** + * @author Glenn Renfro + * @author Mahmoud Ben Hassine + */ +public class JmsItemWriterBuilderTests { + + @Test + public void testNoItem() throws Exception { + JmsOperations jmsTemplate = mock(JmsOperations.class); + JmsItemWriter itemWriter = new JmsItemWriterBuilder().jmsTemplate(jmsTemplate).build(); + ArgumentCaptor argCaptor = ArgumentCaptor.forClass(String.class); + itemWriter.write(Arrays.asList("foo", "bar")); + verify(jmsTemplate, times(2)).convertAndSend(argCaptor.capture()); + assertEquals("Expected foo", "foo", argCaptor.getAllValues().get(0)); + assertEquals("Expected bar", "bar", argCaptor.getAllValues().get(1)); + } + + @Test + public void testNullJmsTemplate() { + try { + new JmsItemWriterBuilder().build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException ise) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "jmsTemplate is required.", ise.getMessage()); + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/GsonJsonItemReaderCommonTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/GsonJsonItemReaderCommonTests.java new file mode 100644 index 0000000000..c4b6153bcf --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/GsonJsonItemReaderCommonTests.java @@ -0,0 +1,31 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import org.springframework.batch.item.sample.Foo; + +/** + * @author Mahmoud Ben Hassine + */ +public class GsonJsonItemReaderCommonTests extends JsonItemReaderCommonTests { + + @Override + protected JsonObjectReader getJsonObjectReader() { + return new GsonJsonObjectReader<>(Foo.class); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/GsonJsonObjectMarshallerTest.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/GsonJsonObjectMarshallerTest.java new file mode 100644 index 0000000000..bd138ba161 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/GsonJsonObjectMarshallerTest.java @@ -0,0 +1,65 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import org.junit.Assert; +import org.junit.Test; + +/** + * @author Mahmoud Ben Hassine + */ +public class GsonJsonObjectMarshallerTest { + + @Test + public void testJsonMarshalling() { + // given + GsonJsonObjectMarshaller jsonObjectMarshaller = new GsonJsonObjectMarshaller<>(); + + // when + String foo = jsonObjectMarshaller.marshal(new Foo(1, "foo")); + + // then + Assert.assertEquals("{\"id\":1,\"name\":\"foo\"}", foo); + } + + public static class Foo { + private int id; + private String name; + + public Foo(int id, String name) { + this.id = id; + this.name = name; + } + + public int getId() { + return id; + } + + public void setId(int id) { + this.id = id; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JacksonJsonItemReaderCommonTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JacksonJsonItemReaderCommonTests.java new file mode 100644 index 0000000000..100d0f566f --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JacksonJsonItemReaderCommonTests.java @@ -0,0 +1,31 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import org.springframework.batch.item.sample.Foo; + +/** + * @author Mahmoud Ben Hassine + */ +public class JacksonJsonItemReaderCommonTests extends JsonItemReaderCommonTests { + + @Override + protected JsonObjectReader getJsonObjectReader() { + return new JacksonJsonObjectReader<>(Foo.class); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JacksonJsonObjectMarshallerTest.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JacksonJsonObjectMarshallerTest.java new file mode 100644 index 0000000000..1c82a47aab --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JacksonJsonObjectMarshallerTest.java @@ -0,0 +1,65 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import org.junit.Assert; +import org.junit.Test; + +/** + * @author Mahmoud Ben Hassine + */ +public class JacksonJsonObjectMarshallerTest { + + @Test + public void testJsonMarshalling() { + // given + JacksonJsonObjectMarshaller jsonObjectMarshaller = new JacksonJsonObjectMarshaller<>(); + + // when + String foo = jsonObjectMarshaller.marshal(new Foo(1, "foo")); + + // then + Assert.assertEquals("{\"id\":1,\"name\":\"foo\"}", foo); + } + + public static class Foo { + private int id; + private String name; + + public Foo(int id, String name) { + this.id = id; + this.name = name; + } + + public int getId() { + return id; + } + + public void setId(int id) { + this.id = id; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JsonFileItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JsonFileItemWriterTests.java new file mode 100644 index 0000000000..c281d55f2c --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JsonFileItemWriterTests.java @@ -0,0 +1,74 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import java.io.File; +import java.nio.file.Files; +import java.util.Arrays; + +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mock; +import org.mockito.Mockito; +import org.mockito.junit.MockitoJUnitRunner; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.core.io.FileSystemResource; +import org.springframework.core.io.Resource; + +/** + * @author Mahmoud Ben Hassine + */ +@RunWith(MockitoJUnitRunner.class) +public class JsonFileItemWriterTests { + + private Resource resource; + @Mock + private JsonObjectMarshaller jsonObjectMarshaller; + + @Before + public void setUp() throws Exception { + File file = Files.createTempFile("test", "json").toFile(); + this.resource = new FileSystemResource(file); + } + + @Test(expected = IllegalArgumentException.class) + public void resourceMustNotBeNull() { + new JsonFileItemWriter<>(null, this.jsonObjectMarshaller); + } + + @Test(expected = IllegalArgumentException.class) + public void jsonObjectMarshallerMustNotBeNull() { + new JsonFileItemWriter<>(this.resource, null); + } + + @Test + public void itemsShouldBeMarshalledToJsonWithTheJsonObjectMarshaller() throws Exception { + // given + JsonFileItemWriter writer = new JsonFileItemWriter<>(this.resource, this.jsonObjectMarshaller); + + // when + writer.open(new ExecutionContext()); + writer.write(Arrays.asList("foo", "bar")); + writer.close(); + + // then + Mockito.verify(this.jsonObjectMarshaller).marshal("foo"); + Mockito.verify(this.jsonObjectMarshaller).marshal("bar"); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JsonItemReaderCommonTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JsonItemReaderCommonTests.java new file mode 100644 index 0000000000..fad5d1e947 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JsonItemReaderCommonTests.java @@ -0,0 +1,57 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import org.springframework.batch.item.AbstractItemStreamItemReaderTests; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.sample.Foo; +import org.springframework.core.io.ByteArrayResource; + +/** + * @author Mahmoud Ben Hassine + */ +public abstract class JsonItemReaderCommonTests extends AbstractItemStreamItemReaderTests { + + private static final String FOOS = "[" + + " {\"value\":1}," + + " {\"value\":2}," + + " {\"value\":3}," + + " {\"value\":4}," + + " {\"value\":5}" + + "]"; + + protected abstract JsonObjectReader getJsonObjectReader(); + + @Override + protected ItemReader getItemReader() { + ByteArrayResource resource = new ByteArrayResource(FOOS.getBytes()); + JsonObjectReader jsonObjectReader = getJsonObjectReader(); + JsonItemReader itemReader = new JsonItemReader<>(resource, jsonObjectReader); + itemReader.setName("fooJsonItemReader"); + return itemReader; + } + + @Override + protected void pointToEmptyInput(ItemReader tested) { + JsonItemReader reader = (JsonItemReader) tested; + reader.setResource(new ByteArrayResource("[]".getBytes())); + + reader.open(new ExecutionContext()); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JsonItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JsonItemReaderTests.java new file mode 100644 index 0000000000..5e1b5ab259 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/JsonItemReaderTests.java @@ -0,0 +1,159 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json; + +import java.io.InputStream; + +import org.hamcrest.Matchers; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; +import org.junit.runner.RunWith; +import org.mockito.Mock; +import org.mockito.Mockito; +import org.mockito.junit.MockitoJUnitRunner; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.core.io.AbstractResource; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.Resource; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; + +/** + * @author Mahmoud Ben Hassine + */ +@RunWith(MockitoJUnitRunner.class) +public class JsonItemReaderTests { + + @Rule + public ExpectedException expectedException = ExpectedException.none(); + + @Mock + private JsonObjectReader jsonObjectReader; + + private JsonItemReader itemReader; + + @Test + public void testValidation() { + try { + new JsonItemReader<>(null, this.jsonObjectReader); + fail("A resource is required."); + } catch (IllegalArgumentException iae) { + assertEquals("The resource must not be null.", iae.getMessage()); + } + + try { + new JsonItemReader<>(new ByteArrayResource("[{}]".getBytes()), null); + fail("A json object reader is required."); + } catch (IllegalArgumentException iae) { + assertEquals("The json object reader must not be null.", iae.getMessage()); + } + } + + @Test + public void testNonExistentResource() { + // given + this.expectedException.expect(ItemStreamException.class); + this.expectedException.expectMessage("Failed to initialize the reader"); + this.expectedException.expectCause(Matchers.instanceOf(IllegalStateException.class)); + this.itemReader = new JsonItemReader<>(new NonExistentResource(), this.jsonObjectReader); + + // when + this.itemReader.open(new ExecutionContext()); + + // then + // expected exception + } + + @Test + public void testNonReadableResource() { + // given + this.expectedException.expect(ItemStreamException.class); + this.expectedException.expectMessage("Failed to initialize the reader"); + this.expectedException.expectCause(Matchers.instanceOf(IllegalStateException.class)); + this.itemReader = new JsonItemReader<>(new NonReadableResource(), this.jsonObjectReader); + + // when + this.itemReader.open(new ExecutionContext()); + + // then + // expected exception + } + + @Test + public void testReadItem() throws Exception { + // given + Resource resource = new ByteArrayResource("[]".getBytes()); + itemReader = new JsonItemReader<>(resource, this.jsonObjectReader); + + // when + itemReader.read(); + + // then + Mockito.verify(this.jsonObjectReader).read(); + } + + private static class NonExistentResource extends AbstractResource { + + NonExistentResource() { + } + + @Override + public boolean exists() { + return false; + } + + @Override + public String getDescription() { + return "NonExistentResource"; + } + + @Override + public InputStream getInputStream() { + return null; + } + } + + private static class NonReadableResource extends AbstractResource { + + NonReadableResource() { + } + + @Override + public boolean isReadable() { + return false; + } + + @Override + public boolean exists() { + return true; + } + + @Override + public String getDescription() { + return "NonReadableResource"; + } + + @Override + public InputStream getInputStream() { + return null; + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/builder/JsonFileItemWriterBuilderTest.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/builder/JsonFileItemWriterBuilderTest.java new file mode 100644 index 0000000000..7e57a25b65 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/builder/JsonFileItemWriterBuilderTest.java @@ -0,0 +1,116 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json.builder; + +import java.io.File; +import java.nio.file.Files; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mockito; + +import org.springframework.batch.item.file.FlatFileFooterCallback; +import org.springframework.batch.item.file.FlatFileHeaderCallback; +import org.springframework.batch.item.json.JsonFileItemWriter; +import org.springframework.batch.item.json.JsonObjectMarshaller; +import org.springframework.core.io.FileSystemResource; +import org.springframework.core.io.Resource; +import org.springframework.test.util.ReflectionTestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +/** + * @author Mahmoud Ben Hassine + */ +public class JsonFileItemWriterBuilderTest { + + private Resource resource; + private JsonObjectMarshaller jsonObjectMarshaller; + + @Before + public void setUp() throws Exception { + File file = Files.createTempFile("test", "json").toFile(); + this.resource = new FileSystemResource(file); + this.jsonObjectMarshaller = object -> object; + } + + @Test(expected = IllegalArgumentException.class) + public void testMissingResource() { + new JsonFileItemWriterBuilder() + .jsonObjectMarshaller(this.jsonObjectMarshaller) + .build(); + } + + @Test(expected = IllegalArgumentException.class) + public void testMissingJsonObjectMarshaller() { + new JsonFileItemWriterBuilder() + .resource(this.resource) + .build(); + } + + @Test(expected = IllegalArgumentException.class) + public void testMandatoryNameWhenSaveStateIsSet() { + new JsonFileItemWriterBuilder() + .resource(this.resource) + .jsonObjectMarshaller(this.jsonObjectMarshaller) + .build(); + } + + @Test + public void testJsonFileItemWriterCreation() { + // given + boolean append = true; + boolean forceSync = true; + boolean transactional = true; + boolean shouldDeleteIfEmpty = true; + boolean shouldDeleteIfExists = true; + String encoding = "UTF-8"; + String lineSeparator = "#"; + FlatFileHeaderCallback headerCallback = Mockito.mock(FlatFileHeaderCallback.class); + FlatFileFooterCallback footerCallback = Mockito.mock(FlatFileFooterCallback.class); + + // when + JsonFileItemWriter writer = new JsonFileItemWriterBuilder() + .name("jsonFileItemWriter") + .resource(this.resource) + .jsonObjectMarshaller(this.jsonObjectMarshaller) + .append(append) + .encoding(encoding) + .forceSync(forceSync) + .headerCallback(headerCallback) + .footerCallback(footerCallback) + .lineSeparator(lineSeparator) + .shouldDeleteIfEmpty(shouldDeleteIfEmpty) + .shouldDeleteIfExists(shouldDeleteIfExists) + .transactional(transactional) + .build(); + + // then + assertTrue((Boolean) ReflectionTestUtils.getField(writer, "saveState")); + assertTrue((Boolean) ReflectionTestUtils.getField(writer, "append")); + assertTrue((Boolean) ReflectionTestUtils.getField(writer, "transactional")); + assertTrue((Boolean) ReflectionTestUtils.getField(writer, "shouldDeleteIfEmpty")); + assertTrue((Boolean) ReflectionTestUtils.getField(writer, "shouldDeleteIfExists")); + assertTrue((Boolean) ReflectionTestUtils.getField(writer, "forceSync")); + assertEquals(encoding, ReflectionTestUtils.getField(writer, "encoding")); + assertEquals(lineSeparator, ReflectionTestUtils.getField(writer, "lineSeparator")); + assertEquals(headerCallback, ReflectionTestUtils.getField(writer, "headerCallback")); + assertEquals(footerCallback, ReflectionTestUtils.getField(writer, "footerCallback")); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/builder/JsonItemReaderBuilderTest.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/builder/JsonItemReaderBuilderTest.java new file mode 100644 index 0000000000..f5506e7279 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/json/builder/JsonItemReaderBuilderTest.java @@ -0,0 +1,105 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.json.builder; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.json.JsonItemReader; +import org.springframework.batch.item.json.JsonObjectReader; +import org.springframework.core.io.Resource; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.springframework.test.util.ReflectionTestUtils.getField; + +/** + * @author Mahmoud Ben Hassine + */ +public class JsonItemReaderBuilderTest { + + @Mock + private Resource resource; + @Mock + private JsonObjectReader jsonObjectReader; + + @Before + public void setUp() { + MockitoAnnotations.initMocks(this); + } + + @Test + public void testValidation() { + try { + new JsonItemReaderBuilder() + .build(); + fail("A json object reader is required."); + } + catch (IllegalArgumentException iae) { + assertEquals("A json object reader is required.", + iae.getMessage()); + } + + try { + new JsonItemReaderBuilder() + .jsonObjectReader(this.jsonObjectReader) + .build(); + fail("A resource is required."); + } + catch (IllegalArgumentException iae) { + assertEquals("A resource is required.", + iae.getMessage()); + } + + try { + new JsonItemReaderBuilder() + .jsonObjectReader(this.jsonObjectReader) + .resource(this.resource) + .build(); + fail("A name is required when saveState is set to true."); + } + catch (IllegalStateException iae) { + assertEquals("A name is required when saveState is set to true.", + iae.getMessage()); + } + } + + @Test + public void testConfiguration() { + JsonItemReader itemReader = new JsonItemReaderBuilder() + .jsonObjectReader(this.jsonObjectReader) + .resource(this.resource) + .saveState(true) + .strict(true) + .name("jsonItemReader") + .maxItemCount(100) + .currentItemCount(50) + .build(); + + Assert.assertEquals(this.jsonObjectReader, getField(itemReader, "jsonObjectReader")); + Assert.assertEquals(this.resource, getField(itemReader, "resource")); + Assert.assertEquals(100, getField(itemReader, "maxItemCount")); + Assert.assertEquals(50, getField(itemReader, "currentItemCount")); + Assert.assertTrue((Boolean) getField(itemReader, "saveState")); + Assert.assertTrue((Boolean) getField(itemReader, "strict")); + Object executionContext = getField(itemReader, "executionContextUserSupport"); + Assert.assertEquals("jsonItemReader", getField(executionContext, "name")); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/KafkaItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/KafkaItemReaderTests.java new file mode 100644 index 0000000000..75e2563530 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/KafkaItemReaderTests.java @@ -0,0 +1,319 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.kafka; + +import java.time.Duration; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Properties; +import java.util.concurrent.ExecutionException; + +import org.apache.kafka.clients.admin.NewTopic; +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.common.TopicPartition; +import org.apache.kafka.common.serialization.StringDeserializer; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.ClassRule; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.kafka.core.DefaultKafkaProducerFactory; +import org.springframework.kafka.core.KafkaTemplate; +import org.springframework.kafka.core.ProducerFactory; +import org.springframework.kafka.test.rule.EmbeddedKafkaRule; +import org.springframework.kafka.test.utils.KafkaTestUtils; +import org.springframework.util.concurrent.ListenableFuture; + +import static org.hamcrest.Matchers.containsInAnyOrder; +import static org.hamcrest.Matchers.is; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertThat; +import static org.junit.Assert.fail; + +/** + * @author Mathieu Ouellet + * @author Mahmoud Ben Hassine + */ +public class KafkaItemReaderTests { + + @ClassRule + public static EmbeddedKafkaRule embeddedKafka = new EmbeddedKafkaRule(1); + + private KafkaItemReader reader; + private KafkaTemplate template; + private Properties consumerProperties; + + @BeforeClass + public static void setUpTopics() { + embeddedKafka.getEmbeddedKafka().addTopics( + new NewTopic("topic1", 1, (short) 1), + new NewTopic("topic2", 2, (short) 1), + new NewTopic("topic3", 1, (short) 1), + new NewTopic("topic4", 2, (short) 1) + ); + } + + @Before + public void setUp() { + Map producerProperties = KafkaTestUtils.producerProps(embeddedKafka.getEmbeddedKafka()); + ProducerFactory producerFactory = new DefaultKafkaProducerFactory<>(producerProperties); + this.template = new KafkaTemplate<>(producerFactory); + + this.consumerProperties = new Properties(); + this.consumerProperties.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, + embeddedKafka.getEmbeddedKafka().getBrokersAsString()); + this.consumerProperties.setProperty(ConsumerConfig.GROUP_ID_CONFIG, "1"); + this.consumerProperties.setProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, + StringDeserializer.class.getName()); + this.consumerProperties.setProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, + StringDeserializer.class.getName()); + } + + @Test + public void testValidation() { + try { + new KafkaItemReader<>(null, "topic", 0); + fail("Expected exception was not thrown"); + } + catch (IllegalArgumentException exception) { + assertEquals("Consumer properties must not be null", exception.getMessage()); + } + + try { + new KafkaItemReader<>(new Properties(), "topic", 0); + fail("Expected exception was not thrown"); + } + catch (IllegalArgumentException exception) { + assertEquals("bootstrap.servers property must be provided", exception.getMessage()); + } + + Properties consumerProperties = new Properties(); + consumerProperties.put("bootstrap.servers", embeddedKafka.getEmbeddedKafka()); + try { + new KafkaItemReader<>(consumerProperties, "topic", 0); + fail("Expected exception was not thrown"); + } + catch (IllegalArgumentException exception) { + assertEquals("group.id property must be provided", exception.getMessage()); + } + + consumerProperties.put("group.id", "1"); + try { + new KafkaItemReader<>(consumerProperties, "topic", 0); + fail("Expected exception was not thrown"); + } + catch (IllegalArgumentException exception) { + assertEquals("key.deserializer property must be provided", exception.getMessage()); + } + + consumerProperties.put("key.deserializer", StringDeserializer.class.getName()); + try { + new KafkaItemReader<>(consumerProperties, "topic", 0); + fail("Expected exception was not thrown"); + } + catch (IllegalArgumentException exception) { + assertEquals("value.deserializer property must be provided", exception.getMessage()); + } + + consumerProperties.put("value.deserializer", StringDeserializer.class.getName()); + try { + new KafkaItemReader<>(consumerProperties, "", 0); + fail("Expected exception was not thrown"); + } + catch (IllegalArgumentException exception) { + assertEquals("Topic name must not be null or empty", exception.getMessage()); + } + + try { + this.reader = new KafkaItemReader<>(consumerProperties, "topic"); + fail("Expected exception was not thrown"); + } + catch (Exception exception) { + assertEquals("At least one partition must be provided", exception.getMessage()); + } + + try { + this.reader = new KafkaItemReader<>(consumerProperties, "topic", 0); + } + catch (Exception exception) { + fail("Must not throw an exception when configuration is valid"); + } + + try { + this.reader.setPollTimeout(null); + fail("Expected exception was not thrown"); + } + catch (IllegalArgumentException exception) { + assertEquals("pollTimeout must not be null", exception.getMessage()); + } + + try { + this.reader.setPollTimeout(Duration.ZERO); + fail("Expected exception was not thrown"); + } + catch (IllegalArgumentException exception) { + assertEquals("pollTimeout must not be zero", exception.getMessage()); + } + + try { + this.reader.setPollTimeout(Duration.ofSeconds(-1)); + fail("Expected exception was not thrown"); + } + catch (IllegalArgumentException exception) { + assertEquals("pollTimeout must not be negative", exception.getMessage()); + } + } + + @Test + public void testReadFromSinglePartition() { + this.template.setDefaultTopic("topic1"); + this.template.sendDefault("val0"); + this.template.sendDefault("val1"); + this.template.sendDefault("val2"); + this.template.sendDefault("val3"); + + this.reader = new KafkaItemReader<>(this.consumerProperties, "topic1", 0); + this.reader.setPollTimeout(Duration.ofSeconds(1)); + this.reader.open(new ExecutionContext()); + + String item = this.reader.read(); + assertThat(item, is("val0")); + + item = this.reader.read(); + assertThat(item, is("val1")); + + item = this.reader.read(); + assertThat(item, is("val2")); + + item = this.reader.read(); + assertThat(item, is("val3")); + + item = this.reader.read(); + assertNull(item); + + this.reader.close(); + } + + @Test + public void testReadFromMultiplePartitions() { + this.template.setDefaultTopic("topic2"); + this.template.sendDefault("val0"); + this.template.sendDefault("val1"); + this.template.sendDefault("val2"); + this.template.sendDefault("val3"); + + this.reader = new KafkaItemReader<>(this.consumerProperties, "topic2", 0, 1); + this.reader.setPollTimeout(Duration.ofSeconds(1)); + this.reader.open(new ExecutionContext()); + + List items = new ArrayList<>(); + items.add(this.reader.read()); + items.add(this.reader.read()); + items.add(this.reader.read()); + items.add(this.reader.read()); + assertThat(items, containsInAnyOrder("val0", "val1", "val2", "val3")); + String item = this.reader.read(); + assertNull(item); + + this.reader.close(); + } + + @Test + public void testReadFromSinglePartitionAfterRestart() { + this.template.setDefaultTopic("topic3"); + this.template.sendDefault("val0"); + this.template.sendDefault("val1"); + this.template.sendDefault("val2"); + this.template.sendDefault("val3"); + this.template.sendDefault("val4"); + + ExecutionContext executionContext = new ExecutionContext(); + Map offsets = new HashMap<>(); + offsets.put(new TopicPartition("topic3", 0), 1L); + executionContext.put("topic.partition.offsets", offsets); + + // topic3-0: val0, val1, val2, val3, val4 + // ^ + // | + // last committed offset = 1 (should restart from offset = 2) + + this.reader = new KafkaItemReader<>(this.consumerProperties, "topic3", 0); + this.reader.setPollTimeout(Duration.ofSeconds(1)); + this.reader.open(executionContext); + + List items = new ArrayList<>(); + items.add(this.reader.read()); + items.add(this.reader.read()); + items.add(this.reader.read()); + assertThat(items, containsInAnyOrder("val2", "val3", "val4")); + String item = this.reader.read(); + assertNull(item); + + this.reader.close(); + } + + @Test + public void testReadFromMultiplePartitionsAfterRestart() throws ExecutionException, InterruptedException { + List futures = new ArrayList<>(); + futures.add(this.template.send("topic4", 0, null, "val0")); + futures.add(this.template.send("topic4", 0, null, "val2")); + futures.add(this.template.send("topic4", 0, null, "val4")); + futures.add(this.template.send("topic4", 0, null, "val6")); + futures.add(this.template.send("topic4", 1, null, "val1")); + futures.add(this.template.send("topic4", 1, null, "val3")); + futures.add(this.template.send("topic4", 1, null, "val5")); + futures.add(this.template.send("topic4", 1, null, "val7")); + + for (ListenableFuture future : futures) { + future.get(); + } + + ExecutionContext executionContext = new ExecutionContext(); + Map offsets = new HashMap<>(); + offsets.put(new TopicPartition("topic4", 0), 1L); + offsets.put(new TopicPartition("topic4", 1), 2L); + executionContext.put("topic.partition.offsets", offsets); + + // topic4-0: val0, val2, val4, val6 + // ^ + // | + // last committed offset = 1 (should restart from offset = 2) + // topic4-1: val1, val3, val5, val7 + // ^ + // | + // last committed offset = 2 (should restart from offset = 3) + + this.reader = new KafkaItemReader<>(this.consumerProperties, "topic4", 0, 1); + this.reader.setPollTimeout(Duration.ofSeconds(1)); + this.reader.open(executionContext); + + List items = new ArrayList<>(); + items.add(this.reader.read()); + items.add(this.reader.read()); + items.add(this.reader.read()); + assertThat(items, containsInAnyOrder("val4", "val6", "val7")); + String item = this.reader.read(); + assertNull(item); + + this.reader.close(); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/KafkaItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/KafkaItemWriterTests.java new file mode 100644 index 0000000000..5b97c12102 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/KafkaItemWriterTests.java @@ -0,0 +1,113 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.kafka; + +import java.util.Arrays; +import java.util.List; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.core.convert.converter.Converter; +import org.springframework.kafka.core.KafkaTemplate; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +public class KafkaItemWriterTests { + + @Mock + private KafkaTemplate kafkaTemplate; + + private KafkaItemKeyMapper itemKeyMapper; + + private KafkaItemWriter writer; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + when(this.kafkaTemplate.getDefaultTopic()).thenReturn("defaultTopic"); + this.itemKeyMapper = new KafkaItemKeyMapper(); + this.writer = new KafkaItemWriter<>(); + this.writer.setKafkaTemplate(this.kafkaTemplate); + this.writer.setItemKeyMapper(this.itemKeyMapper); + this.writer.setDelete(false); + this.writer.afterPropertiesSet(); + } + + @Test + public void testAfterPropertiesSet() throws Exception { + this.writer = new KafkaItemWriter<>(); + + try { + this.writer.afterPropertiesSet(); + fail("Expected exception was not thrown"); + } + catch (IllegalArgumentException exception) { + assertEquals("itemKeyMapper requires a Converter type.", exception.getMessage()); + } + + this.writer.setItemKeyMapper(this.itemKeyMapper); + try { + this.writer.afterPropertiesSet(); + fail("Expected exception was not thrown"); + } + catch (IllegalArgumentException exception) { + assertEquals("KafkaTemplate must not be null.", exception.getMessage()); + } + + this.writer.setKafkaTemplate(this.kafkaTemplate); + try { + this.writer.afterPropertiesSet(); + } + catch (Exception e) { + fail("Must not throw an exception when correctly configured"); + } + } + + @Test + public void testBasicWrite() throws Exception { + List items = Arrays.asList("val1", "val2"); + + this.writer.write(items); + + verify(this.kafkaTemplate).sendDefault(items.get(0), items.get(0)); + verify(this.kafkaTemplate).sendDefault(items.get(1), items.get(1)); + } + + @Test + public void testBasicDelete() throws Exception { + List items = Arrays.asList("val1", "val2"); + this.writer.setDelete(true); + + this.writer.write(items); + + verify(this.kafkaTemplate).sendDefault(items.get(0), null); + verify(this.kafkaTemplate).sendDefault(items.get(1), null); + } + + static class KafkaItemKeyMapper implements Converter { + + @Override + public String convert(String source) { + return source; + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/builder/KafkaItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/builder/KafkaItemReaderBuilderTests.java new file mode 100644 index 0000000000..4eed038507 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/builder/KafkaItemReaderBuilderTests.java @@ -0,0 +1,238 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.kafka.builder; + +import java.time.Duration; +import java.util.Arrays; +import java.util.List; +import java.util.Properties; + +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.common.TopicPartition; +import org.apache.kafka.common.serialization.StringDeserializer; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; + +import org.springframework.batch.item.kafka.KafkaItemReader; +import org.springframework.test.util.ReflectionTestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.fail; + +/** + * @author Mathieu Ouellet + * @author Mahmoud Ben Hassine + */ +public class KafkaItemReaderBuilderTests { + + @Rule + public ExpectedException thrown = ExpectedException.none(); + + private Properties consumerProperties; + + @Before + public void setUp() throws Exception { + this.consumerProperties = new Properties(); + this.consumerProperties.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"); + this.consumerProperties.setProperty(ConsumerConfig.GROUP_ID_CONFIG, "1"); + this.consumerProperties.setProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, + StringDeserializer.class.getName()); + this.consumerProperties.setProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, + StringDeserializer.class.getName()); + } + + @Test + public void testNullConsumerProperties() { + this.thrown.expect(IllegalArgumentException.class); + this.thrown.expectMessage("Consumer properties must not be null"); + + new KafkaItemReaderBuilder<>() + .name("kafkaItemReader") + .consumerProperties(null) + .build(); + } + + @Test + public void testConsumerPropertiesValidation() { + try { + new KafkaItemReaderBuilder<>() + .name("kafkaItemReader") + .consumerProperties(new Properties()) + .build(); + fail("Expected exception was not thrown"); + } catch (IllegalArgumentException exception) { + assertEquals("bootstrap.servers property must be provided", exception.getMessage()); + } + + Properties consumerProperties = new Properties(); + consumerProperties.put("bootstrap.servers", "foo"); + try { + new KafkaItemReaderBuilder<>() + .name("kafkaItemReader") + .consumerProperties(consumerProperties) + .build(); + fail("Expected exception was not thrown"); + } catch (IllegalArgumentException exception) { + assertEquals("group.id property must be provided", exception.getMessage()); + } + + consumerProperties.put("group.id", "1"); + try { + new KafkaItemReaderBuilder<>() + .name("kafkaItemReader") + .consumerProperties(consumerProperties) + .build(); + fail("Expected exception was not thrown"); + } catch (IllegalArgumentException exception) { + assertEquals("key.deserializer property must be provided", exception.getMessage()); + } + + consumerProperties.put("key.deserializer", StringDeserializer.class.getName()); + try { + new KafkaItemReaderBuilder<>() + .name("kafkaItemReader") + .consumerProperties(consumerProperties) + .build(); + fail("Expected exception was not thrown"); + } catch (IllegalArgumentException exception) { + assertEquals("value.deserializer property must be provided", exception.getMessage()); + } + + consumerProperties.put("value.deserializer", StringDeserializer.class.getName()); + try { + new KafkaItemReaderBuilder<>() + .name("kafkaItemReader") + .consumerProperties(consumerProperties) + .topic("test") + .partitions(0, 1) + .build(); + } catch (Exception exception) { + fail("Must not throw an exception when configuration is valid"); + } + } + + @Test + public void testNullTopicName() { + this.thrown.expect(IllegalArgumentException.class); + this.thrown.expectMessage("Topic name must not be null or empty"); + + new KafkaItemReaderBuilder<>() + .name("kafkaItemReader") + .consumerProperties(this.consumerProperties) + .topic(null) + .build(); + } + + @Test + public void testEmptyTopicName() { + this.thrown.expect(IllegalArgumentException.class); + this.thrown.expectMessage("Topic name must not be null or empty"); + + new KafkaItemReaderBuilder<>() + .name("kafkaItemReader") + .consumerProperties(this.consumerProperties) + .topic("") + .build(); + } + + @Test + public void testNullPollTimeout() { + this.thrown.expect(IllegalArgumentException.class); + this.thrown.expectMessage("pollTimeout must not be null"); + + new KafkaItemReaderBuilder<>() + .name("kafkaItemReader") + .consumerProperties(this.consumerProperties) + .topic("test") + .pollTimeout(null) + .build(); + } + + @Test + public void testNegativePollTimeout() { + this.thrown.expect(IllegalArgumentException.class); + this.thrown.expectMessage("pollTimeout must not be negative"); + + new KafkaItemReaderBuilder<>() + .name("kafkaItemReader") + .consumerProperties(this.consumerProperties) + .topic("test") + .pollTimeout(Duration.ofSeconds(-1)) + .build(); + } + + @Test + public void testZeroPollTimeout() { + this.thrown.expect(IllegalArgumentException.class); + this.thrown.expectMessage("pollTimeout must not be zero"); + + new KafkaItemReaderBuilder<>() + .name("kafkaItemReader") + .consumerProperties(this.consumerProperties) + .topic("test") + .pollTimeout(Duration.ZERO) + .build(); + } + + @Test + public void testEmptyPartitions() { + this.thrown.expect(IllegalArgumentException.class); + this.thrown.expectMessage("At least one partition must be provided"); + + new KafkaItemReaderBuilder<>() + .name("kafkaItemReader") + .consumerProperties(this.consumerProperties) + .topic("test") + .pollTimeout(Duration.ofSeconds(10)) + .build(); + } + + @Test + @SuppressWarnings("unchecked") + public void testKafkaItemReaderCreation() { + // given + boolean saveState = false; + Duration pollTimeout = Duration.ofSeconds(100); + String topic = "test"; + List partitions = Arrays.asList(0, 1); + + // when + KafkaItemReader reader = new KafkaItemReaderBuilder() + .name("kafkaItemReader") + .consumerProperties(this.consumerProperties) + .topic(topic) + .partitions(partitions) + .pollTimeout(pollTimeout) + .saveState(saveState) + .build(); + + // then + assertNotNull(reader); + assertFalse((Boolean) ReflectionTestUtils.getField(reader, "saveState")); + assertEquals(pollTimeout, ReflectionTestUtils.getField(reader, "pollTimeout")); + List topicPartitions = (List) ReflectionTestUtils.getField(reader, "topicPartitions"); + assertEquals(2, topicPartitions.size()); + assertEquals(topic, topicPartitions.get(0).topic()); + assertEquals(partitions.get(0).intValue(), topicPartitions.get(0).partition()); + assertEquals(topic, topicPartitions.get(1).topic()); + assertEquals(partitions.get(1).intValue(), topicPartitions.get(1).partition()); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/builder/KafkaItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/builder/KafkaItemWriterBuilderTests.java new file mode 100644 index 0000000000..1ebe70ee96 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/builder/KafkaItemWriterBuilderTests.java @@ -0,0 +1,92 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.kafka.builder; + +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.kafka.KafkaItemWriter; +import org.springframework.core.convert.converter.Converter; +import org.springframework.kafka.core.KafkaTemplate; +import org.springframework.test.util.ReflectionTestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +/** + * @author Mathieu Ouellet + */ +public class KafkaItemWriterBuilderTests { + + @Rule + public ExpectedException thrown = ExpectedException.none(); + + @Mock + private KafkaTemplate kafkaTemplate; + + private KafkaItemKeyMapper itemKeyMapper; + + @Before + public void setUp() { + MockitoAnnotations.initMocks(this); + this.itemKeyMapper = new KafkaItemKeyMapper(); + } + + @Test + public void testNullKafkaTemplate() { + this.thrown.expect(IllegalArgumentException.class); + this.thrown.expectMessage("kafkaTemplate is required."); + + new KafkaItemWriterBuilder().itemKeyMapper(this.itemKeyMapper).build(); + } + + @Test + public void testNullItemKeyMapper() { + this.thrown.expect(IllegalArgumentException.class); + this.thrown.expectMessage("itemKeyMapper is required."); + + new KafkaItemWriterBuilder().kafkaTemplate(this.kafkaTemplate).build(); + } + + @Test + public void testKafkaItemWriterBuild() { + // given + boolean delete = true; + + // when + KafkaItemWriter writer = new KafkaItemWriterBuilder() + .kafkaTemplate(this.kafkaTemplate).itemKeyMapper(this.itemKeyMapper).delete(delete).build(); + + // then + assertTrue((Boolean) ReflectionTestUtils.getField(writer, "delete")); + assertEquals(this.itemKeyMapper, ReflectionTestUtils.getField(writer, "itemKeyMapper")); + assertEquals(this.kafkaTemplate, ReflectionTestUtils.getField(writer, "kafkaTemplate")); + } + + static class KafkaItemKeyMapper implements Converter { + + @Override + public String convert(String source) { + return source; + } + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/DefaultMailErrorHandlerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/DefaultMailErrorHandlerTests.java index 03f2183164..48235bb09d 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/DefaultMailErrorHandlerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/DefaultMailErrorHandlerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/SimpleMailMessageItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/SimpleMailMessageItemWriterTests.java index bf1a90b277..63ffc0c730 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/SimpleMailMessageItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/SimpleMailMessageItemWriterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,6 +17,7 @@ import static org.junit.Assert.assertEquals; import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; import static org.mockito.AdditionalMatchers.aryEq; @@ -28,11 +29,13 @@ import org.junit.Before; import org.junit.Test; + import org.springframework.mail.MailException; import org.springframework.mail.MailMessage; import org.springframework.mail.MailSendException; import org.springframework.mail.MailSender; import org.springframework.mail.SimpleMailMessage; +import org.springframework.util.ReflectionUtils; /** * @author Dave Syer @@ -59,11 +62,15 @@ public void testSend() throws Exception { SimpleMailMessage bar = new SimpleMailMessage(); SimpleMailMessage[] items = new SimpleMailMessage[] { foo, bar }; - mailSender.send(aryEq(items)); - writer.write(Arrays.asList(items)); - + // Spring 4.1 changed the send method to be vargs instead of an array + if(ReflectionUtils.findMethod(SimpleMailMessage.class, "send", SimpleMailMessage[].class) != null) { + verify(mailSender).send(aryEq(items)); + } + else { + verify(mailSender).send(items); + } } @Test(expected = MailSendException.class) @@ -73,17 +80,23 @@ public void testDefaultErrorHandler() throws Exception { SimpleMailMessage bar = new SimpleMailMessage(); SimpleMailMessage[] items = new SimpleMailMessage[] { foo, bar }; - mailSender.send(aryEq(items)); + // Spring 4.1 changed the send method to be vargs instead of an array + if(ReflectionUtils.findMethod(SimpleMailMessage.class, "send", SimpleMailMessage[].class) != null) { + mailSender.send(aryEq(items)); + } + else { + mailSender.send(items); + } + when(mailSender).thenThrow(new MailSendException(Collections.singletonMap((Object)foo, (Exception)new MessagingException("FOO")))); writer.write(Arrays.asList(items)); - } @Test public void testCustomErrorHandler() throws Exception { - final AtomicReference content = new AtomicReference(); + final AtomicReference content = new AtomicReference<>(); writer.setMailErrorHandler(new MailErrorHandler() { @Override public void handle(MailMessage message, Exception exception) throws MailException { @@ -95,14 +108,19 @@ public void handle(MailMessage message, Exception exception) throws MailExceptio SimpleMailMessage bar = new SimpleMailMessage(); SimpleMailMessage[] items = new SimpleMailMessage[] { foo, bar }; - mailSender.send(aryEq(items)); + // Spring 4.1 changed the send method to be vargs instead of an array + if(ReflectionUtils.findMethod(SimpleMailMessage.class, "send", SimpleMailMessage[].class) != null) { + mailSender.send(aryEq(items)); + } + else { + mailSender.send(items); + } + when(mailSender).thenThrow(new MailSendException(Collections.singletonMap((Object)foo, (Exception)new MessagingException("FOO")))); writer.write(Arrays.asList(items)); assertEquals("FOO", content.get()); - - } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/builder/SimpleMailMessageItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/builder/SimpleMailMessageItemWriterBuilderTests.java new file mode 100644 index 0000000000..a942cab206 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/builder/SimpleMailMessageItemWriterBuilderTests.java @@ -0,0 +1,112 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.mail.builder; + +import java.util.Arrays; +import java.util.Collections; +import java.util.concurrent.atomic.AtomicReference; + +import javax.mail.MessagingException; + +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.mail.MailErrorHandler; +import org.springframework.batch.item.mail.SimpleMailMessageItemWriter; +import org.springframework.mail.MailException; +import org.springframework.mail.MailMessage; +import org.springframework.mail.MailSendException; +import org.springframework.mail.MailSender; +import org.springframework.mail.SimpleMailMessage; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +/** + * @author Glenn Renfro + */ +public class SimpleMailMessageItemWriterBuilderTests { + + private MailSender mailSender; + + private SimpleMailMessage foo; + + private SimpleMailMessage bar; + + private SimpleMailMessage[] items; + + @Before + public void setup() { + mailSender = mock(MailSender.class); + this.foo = new SimpleMailMessage(); + this.bar = new SimpleMailMessage(); + this.items = new SimpleMailMessage[] { this.foo, this.bar }; + } + + @Test + public void testSend() throws Exception { + SimpleMailMessageItemWriter writer = new SimpleMailMessageItemWriterBuilder().mailSender(this.mailSender) + .build(); + + writer.write(Arrays.asList(this.items)); + verify(this.mailSender).send(this.foo, this.bar); + } + + @Test + public void testMailSenderNotSet() throws Exception { + try { + new SimpleMailMessageItemWriterBuilder().build(); + fail("A mailSender is required"); + } + catch (IllegalArgumentException iae) { + assertEquals("A mailSender is required", iae.getMessage()); + } + } + + @Test(expected = MailSendException.class) + public void testErrorHandler() throws Exception { + SimpleMailMessageItemWriter writer = new SimpleMailMessageItemWriterBuilder().mailSender(this.mailSender) + .build(); + + this.mailSender.send(this.foo, this.bar); + when(this.mailSender) + .thenThrow(new MailSendException(Collections.singletonMap(this.foo, new MessagingException("FOO")))); + writer.write(Arrays.asList(this.items)); + } + + @Test + public void testCustomErrorHandler() throws Exception { + final AtomicReference content = new AtomicReference<>(); + SimpleMailMessageItemWriter writer = new SimpleMailMessageItemWriterBuilder() + .mailErrorHandler(new MailErrorHandler() { + @Override + public void handle(MailMessage message, Exception exception) throws MailException { + content.set(exception.getMessage()); + } + }).mailSender(this.mailSender).build(); + + this.mailSender.send(this.foo, this.bar); + when(this.mailSender) + .thenThrow(new MailSendException(Collections.singletonMap(this.foo, new MessagingException("FOO")))); + writer.write(Arrays.asList(this.items)); + assertEquals("FOO", content.get()); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/javamail/MimeMessageItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/javamail/MimeMessageItemWriterTests.java index 60abd34998..3a4459d68f 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/javamail/MimeMessageItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/mail/javamail/MimeMessageItemWriterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,7 +35,9 @@ import org.springframework.mail.MailException; import org.springframework.mail.MailMessage; import org.springframework.mail.MailSendException; +import org.springframework.mail.MailSender; import org.springframework.mail.javamail.JavaMailSender; +import org.springframework.util.ReflectionUtils; /** * @author Dave Syer @@ -78,18 +80,23 @@ public void testDefaultErrorHandler() throws Exception { MimeMessage bar = new MimeMessage(session); MimeMessage[] items = new MimeMessage[] { foo, bar }; - mailSender.send(aryEq(items)); + // Spring 4.1 changed the send method to be vargs instead of an array + if(ReflectionUtils.findMethod(MailSender.class, "send", MimeMessage[].class) != null) { + mailSender.send(aryEq(items)); + } + else { + mailSender.send(items); + } + when(mailSender).thenThrow(new MailSendException(Collections.singletonMap((Object)foo, (Exception)new MessagingException("FOO")))); writer.write(Arrays.asList(items)); - - } @Test public void testCustomErrorHandler() throws Exception { - final AtomicReference content = new AtomicReference(); + final AtomicReference content = new AtomicReference<>(); writer.setMailErrorHandler(new MailErrorHandler() { @Override public void handle(MailMessage message, Exception exception) throws MailException { @@ -101,7 +108,15 @@ public void handle(MailMessage message, Exception exception) throws MailExceptio MimeMessage bar = new MimeMessage(session); MimeMessage[] items = new MimeMessage[] { foo, bar }; - mailSender.send(aryEq(items)); + + // Spring 4.1 changed the send method to be vargs instead of an array + if(ReflectionUtils.findMethod(MailSender.class, "send", MimeMessage[].class) != null) { + mailSender.send(aryEq(items)); + } + else { + mailSender.send(items); + } + when(mailSender).thenThrow(new MailSendException(Collections.singletonMap((Object)foo, (Exception)new MessagingException("FOO")))); writer.write(Arrays.asList(items)); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Customer.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Customer.java index bb0cc5945d..41d3ef22cb 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Customer.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Customer.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Foo.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Foo.java index 9cc25fe583..8328080c48 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Foo.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Foo.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.sample; import javax.persistence.Entity; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/FooService.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/FooService.java index bf8c195285..673c723f21 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/FooService.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/FooService.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.sample; import java.util.ArrayList; @@ -16,9 +31,9 @@ public class FooService { public static final int GENERATION_LIMIT = 10; private int counter = 0; - private List generatedFoos = new ArrayList(GENERATION_LIMIT); - private List processedFoos = new ArrayList(GENERATION_LIMIT); - private List processedFooNameValuePairs = new ArrayList(GENERATION_LIMIT); + private List generatedFoos = new ArrayList<>(GENERATION_LIMIT); + private List processedFoos = new ArrayList<>(GENERATION_LIMIT); + private List processedFooNameValuePairs = new ArrayList<>(GENERATION_LIMIT); public Foo generateFoo() { if (counter++ >= GENERATION_LIMIT) return null; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/LineItem.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/LineItem.java index 3c9d9d1dcf..4ab0e62507 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/LineItem.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/LineItem.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Order.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Order.java index 7f0d4a0678..54df935258 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Order.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Order.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Shipper.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Shipper.java index 3d2fd8866d..c10794242a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Shipper.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/sample/Shipper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ClassifierCompositeItemProcessorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ClassifierCompositeItemProcessorTests.java new file mode 100644 index 0000000000..e47e0eb766 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ClassifierCompositeItemProcessorTests.java @@ -0,0 +1,108 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.support; + +import static org.junit.Assert.assertEquals; + +import java.util.HashMap; +import java.util.Map; + +import org.junit.Test; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.classify.PatternMatchingClassifier; +import org.springframework.classify.SubclassClassifier; +import org.springframework.lang.Nullable; + +/** + * @author Jimmy Praet + */ +public class ClassifierCompositeItemProcessorTests { + + @Test + public void testBasicClassifierCompositeItemProcessor() throws Exception { + ClassifierCompositeItemProcessor processor = new ClassifierCompositeItemProcessor<>(); + + ItemProcessor fooProcessor = new ItemProcessor() { + @Nullable + @Override + public String process(String item) throws Exception { + return "foo: " + item; + } + }; + ItemProcessor defaultProcessor = new ItemProcessor() { + @Nullable + @Override + public String process(String item) throws Exception { + return item; + } + }; + + Map> routingConfiguration = + new HashMap<>(); + routingConfiguration.put("foo", fooProcessor); + routingConfiguration.put("*", defaultProcessor); + processor.setClassifier(new PatternMatchingClassifier<>(routingConfiguration)); + + assertEquals("bar", processor.process("bar")); + assertEquals("foo: foo", processor.process("foo")); + assertEquals("baz", processor.process("baz")); + } + + /** + * Test the ClassifierCompositeItemProcessor with delegates that have more specific generic types for input as well as output. + */ + @Test + public void testGenericsClassifierCompositeItemProcessor() throws Exception { + ClassifierCompositeItemProcessor processor = new ClassifierCompositeItemProcessor<>(); + + ItemProcessor intProcessor = new ItemProcessor() { + @Nullable + @Override + public String process(Integer item) throws Exception { + return "int: " + item; + } + }; + ItemProcessor longProcessor = new ItemProcessor() { + @Nullable + @Override + public StringBuffer process(Long item) throws Exception { + return new StringBuffer("long: " + item); + } + }; + ItemProcessor defaultProcessor = new ItemProcessor() { + @Nullable + @Override + public StringBuilder process(Number item) throws Exception { + return new StringBuilder("number: " + item); + } + }; + + SubclassClassifier> classifier = + new SubclassClassifier<>(); + Map, ItemProcessor> typeMap = + new HashMap<>(); + typeMap.put(Integer.class, intProcessor); + typeMap.put(Long.class, longProcessor); + typeMap.put(Number.class, defaultProcessor); + classifier.setTypeMap(typeMap); + processor.setClassifier(classifier); + + assertEquals("int: 1", processor.process(Integer.valueOf(1)).toString()); + assertEquals("long: 2", processor.process(Long.valueOf(2)).toString()); + assertEquals("number: 3", processor.process(Byte.valueOf((byte) 3)).toString()); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ClassifierCompositeItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ClassifierCompositeItemWriterTests.java index de1cf63955..ffd3c6f482 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ClassifierCompositeItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ClassifierCompositeItemWriterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,8 +15,6 @@ */ package org.springframework.batch.item.support; -import static org.junit.Assert.assertEquals; - import java.util.ArrayList; import java.util.Arrays; import java.util.HashMap; @@ -24,22 +22,27 @@ import java.util.Map; import org.junit.Test; -import org.springframework.classify.PatternMatchingClassifier; + import org.springframework.batch.item.ItemWriter; +import org.springframework.classify.PatternMatchingClassifier; + +import static junit.framework.TestCase.fail; +import static org.junit.Assert.assertEquals; /** * @author Dave Syer + * @author Glenn Renfro * */ public class ClassifierCompositeItemWriterTests { - private ClassifierCompositeItemWriter writer = new ClassifierCompositeItemWriter(); - private List defaults = new ArrayList(); - private List foos = new ArrayList(); + private ClassifierCompositeItemWriter writer = new ClassifierCompositeItemWriter<>(); + private List defaults = new ArrayList<>(); + private List foos = new ArrayList<>(); @Test public void testWrite() throws Exception { - Map> map = new HashMap>(); + Map> map = new HashMap<>(); ItemWriter fooWriter = new ItemWriter() { @Override public void write(List items) throws Exception { @@ -54,10 +57,22 @@ public void write(List items) throws Exception { }; map.put("foo", fooWriter ); map.put("*", defaultWriter); - writer.setClassifier(new PatternMatchingClassifier>(map)); - writer.write(Arrays.asList("foo", "foo", "bar")); + writer.setClassifier(new PatternMatchingClassifier<>(map)); + writer.write(Arrays.asList("foo", "foo", "one", "two", "three")); assertEquals("[foo, foo]", foos.toString()); - assertEquals("[bar]", defaults.toString()); + assertEquals("[one, two, three]", defaults.toString()); } + @Test + public void testSetNullClassifier() throws Exception { + try { + ClassifierCompositeItemWriter writer = new ClassifierCompositeItemWriter<>(); + writer.setClassifier(null); + fail("A classifier is required."); + } + catch (IllegalArgumentException iae) { + assertEquals("Message returned from exception did not match expected result.", "A classifier is required.", + iae.getMessage()); + } + } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/CompositeItemProcessorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/CompositeItemProcessorTests.java index 7dcfa60101..0dc6dcd5f1 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/CompositeItemProcessorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/CompositeItemProcessorTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.support; import static org.junit.Assert.assertEquals; @@ -21,7 +36,7 @@ */ public class CompositeItemProcessorTests { - private CompositeItemProcessor composite = new CompositeItemProcessor(); + private CompositeItemProcessor composite = new CompositeItemProcessor<>(); private ItemProcessor processor1; private ItemProcessor processor2; @@ -46,12 +61,12 @@ public void setUp() throws Exception { @Test public void testTransform() throws Exception { Object item = new Object(); - Object itemAfterFirstTransfromation = new Object(); + Object itemAfterFirstTransformation = new Object(); Object itemAfterSecondTransformation = new Object(); - when(processor1.process(item)).thenReturn(itemAfterFirstTransfromation); + when(processor1.process(item)).thenReturn(itemAfterFirstTransformation); - when(processor2.process(itemAfterFirstTransfromation)).thenReturn(itemAfterSecondTransformation); + when(processor2.process(itemAfterFirstTransformation)).thenReturn(itemAfterSecondTransformation); assertSame(itemAfterSecondTransformation, composite.process(item)); @@ -63,7 +78,7 @@ public void testTransform() throws Exception { @Test @SuppressWarnings({"unchecked", "serial"}) public void testItemProcessorGenerics() throws Exception { - CompositeItemProcessor composite = new CompositeItemProcessor(); + CompositeItemProcessor composite = new CompositeItemProcessor<>(); final ItemProcessor processor1 = mock(ItemProcessor.class); final ItemProcessor processor2 = mock(ItemProcessor.class); composite.setDelegates(new ArrayList>() {{ diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/CompositeItemStreamTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/CompositeItemStreamTests.java index 53ad92105c..523db2b31a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/CompositeItemStreamTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/CompositeItemStreamTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,7 +22,6 @@ import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemStream; -import org.springframework.batch.item.ItemStreamException; import org.springframework.batch.item.ItemStreamSupport; import org.springframework.batch.item.support.CompositeItemStream; @@ -34,7 +33,7 @@ public class CompositeItemStreamTests extends TestCase { private CompositeItemStream manager = new CompositeItemStream(); - private List list = new ArrayList(); + private List list = new ArrayList<>(); public void testRegisterAndOpen() { ItemStreamSupport stream = new ItemStreamSupport() { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/CompositeItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/CompositeItemWriterTests.java index dc418e01a3..e9e8926c20 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/CompositeItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/CompositeItemWriterTests.java @@ -1,8 +1,21 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.support; import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - import java.util.ArrayList; import java.util.Collections; import java.util.List; @@ -21,7 +34,7 @@ public class CompositeItemWriterTests { // object under test - private CompositeItemWriter itemWriter = new CompositeItemWriter(); + private CompositeItemWriter itemWriter = new CompositeItemWriter<>(); /** * Regular usage scenario. All injected processors should be called. @@ -32,7 +45,7 @@ public void testProcess() throws Exception { final int NUMBER_OF_WRITERS = 10; List data = Collections.singletonList(new Object()); - List> writers = new ArrayList>(); + List> writers = new ArrayList<>(); for (int i = 0; i < NUMBER_OF_WRITERS; i++) { @SuppressWarnings("unchecked") @@ -69,7 +82,7 @@ private void doTestItemStream(boolean expectOpen) throws Exception { } writer.write(data); - List> writers = new ArrayList>(); + List> writers = new ArrayList<>(); writers.add(writer); itemWriter.setDelegates(writers); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ItemCountingItemStreamItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ItemCountingItemStreamItemReaderTests.java index 3ad9b7bf5d..82cfd9d172 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ItemCountingItemStreamItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ItemCountingItemStreamItemReaderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,6 +26,7 @@ import org.junit.Before; import org.junit.Test; import org.springframework.batch.item.ExecutionContext; +import org.springframework.lang.Nullable; /** * @author Dave Syer @@ -144,6 +145,7 @@ protected void doOpen() throws Exception { openCalled = true; } + @Nullable @Override protected String doRead() throws Exception { if (!items.hasNext()) { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/IteratorItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/IteratorItemReaderTests.java index 50ce249ea5..5991f464dd 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/IteratorItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/IteratorItemReaderTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,7 +23,7 @@ public class IteratorItemReaderTests extends TestCase{ public void testIterable() throws Exception { - IteratorItemReader reader = new IteratorItemReader(Arrays.asList(new String[] { "a", "b", "c" })); + IteratorItemReader reader = new IteratorItemReader<>(Arrays.asList(new String[]{"a", "b", "c"})); assertEquals("a", reader.read()); assertEquals("b", reader.read()); assertEquals("c", reader.read()); @@ -31,7 +31,7 @@ public void testIterable() throws Exception { } public void testIterator() throws Exception { - IteratorItemReader reader = new IteratorItemReader(Arrays.asList(new String[] { "a", "b", "c" }).iterator()); + IteratorItemReader reader = new IteratorItemReader<>(Arrays.asList(new String[] { "a", "b", "c" }).iterator()); assertEquals("a", reader.read()); assertEquals("b", reader.read()); assertEquals("c", reader.read()); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ListItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ListItemReaderTests.java index 60ca0aa2dc..6b9cb98a1d 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ListItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ListItemReaderTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,7 +26,7 @@ public class ListItemReaderTests extends TestCase { - ListItemReader reader = new ListItemReader(Arrays.asList(new String[] { "a", "b", "c" })); + ListItemReader reader = new ListItemReader<>(Arrays.asList(new String[]{"a", "b", "c"})); public void testNext() throws Exception { assertEquals("a", reader.read()); @@ -36,8 +36,8 @@ public void testNext() throws Exception { } public void testChangeList() throws Exception { - List list = new ArrayList(Arrays.asList(new String[] { "a", "b", "c" })); - reader = new ListItemReader(list); + List list = new ArrayList<>(Arrays.asList(new String[]{"a", "b", "c"})); + reader = new ListItemReader<>(list); assertEquals("a", reader.read()); list.clear(); assertEquals(0, list.size()); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ScriptItemProcessorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ScriptItemProcessorTests.java new file mode 100644 index 0000000000..36298843d6 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ScriptItemProcessorTests.java @@ -0,0 +1,206 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.support; + +import org.junit.BeforeClass; +import org.junit.Test; +import org.springframework.core.io.ClassPathResource; +import org.springframework.core.io.Resource; + +import javax.script.ScriptEngineFactory; +import javax.script.ScriptEngineManager; + +import java.util.ArrayList; +import java.util.List; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assume.assumeTrue; + +/** + *

        + * Test cases around {@link org.springframework.batch.item.support.ScriptItemProcessor}. + *

        + * + * @author Chris Schaefer + * @since 3.1 + */ +public class ScriptItemProcessorTests { + private static List availableLanguages = new ArrayList<>(); + + @BeforeClass + public static void populateAvailableEngines() { + List scriptEngineFactories = new ScriptEngineManager().getEngineFactories(); + + for (ScriptEngineFactory scriptEngineFactory : scriptEngineFactories) { + availableLanguages.addAll(scriptEngineFactory.getNames()); + } + } + + @Test + public void testJavascriptScriptSourceSimple() throws Exception { + assumeTrue(languageExists("javascript")); + + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setScriptSource("item.toUpperCase();", "javascript"); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", "SS", scriptItemProcessor.process("ss")); + } + + @Test + public void testJavascriptScriptSourceFunction() throws Exception { + assumeTrue(languageExists("javascript")); + + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setScriptSource("function process(item) { return item.toUpperCase(); } process(item);", "javascript"); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", "SS", scriptItemProcessor.process("ss")); + } + + @Test + public void testJRubyScriptSourceSimple() throws Exception { + assumeTrue(languageExists("jruby")); + + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setScriptSource("$item.upcase", "jruby"); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", "SS", scriptItemProcessor.process("ss")); + } + + @Test + public void testJRubyScriptSourceMethod() throws Exception { + assumeTrue(languageExists("jruby")); + + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setScriptSource("def process(item) $item.upcase end \n process($item)", "jruby"); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", "SS", scriptItemProcessor.process("ss")); + } + + @Test + public void testBeanShellScriptSourceSimple() throws Exception { + assumeTrue(languageExists("bsh")); + + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setScriptSource("item.toUpperCase();", "bsh"); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", "SS", scriptItemProcessor.process("ss")); + } + + @Test + public void testBeanShellScriptSourceFunction() throws Exception { + assumeTrue(languageExists("bsh")); + + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setScriptSource("String process(String item) { return item.toUpperCase(); } process(item);", "bsh"); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", "SS", scriptItemProcessor.process("ss")); + } + + @Test + public void testGroovyScriptSourceSimple() throws Exception { + assumeTrue(languageExists("groovy")); + + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setScriptSource("item.toUpperCase();", "groovy"); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", "SS", scriptItemProcessor.process("ss")); + } + + @Test + public void testGroovyScriptSourceMethod() throws Exception { + assumeTrue(languageExists("groovy")); + + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setScriptSource("def process(item) { return item.toUpperCase() } \n process(item)", "groovy"); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", "SS", scriptItemProcessor.process("ss")); + } + + @Test + public void testJavascriptScriptSimple() throws Exception { + assumeTrue(languageExists("javascript")); + + Resource resource = new ClassPathResource("org/springframework/batch/item/support/processor-test-simple.js"); + + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setScript(resource); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", "SS", scriptItemProcessor.process("ss")); + } + + @Test + public void testItemBinding() throws Exception { + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setScriptSource("foo.contains('World');", "javascript"); + scriptItemProcessor.setItemBindingVariableName("foo"); + + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", true, scriptItemProcessor.process("Hello World")); + } + + @Test(expected = IllegalStateException.class) + public void testNoScriptSet() throws Exception { + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.afterPropertiesSet(); + } + + @Test(expected = IllegalStateException.class) + public void testScriptSourceAndScriptResourceSet() throws Exception { + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setScriptSource("blah", "blah"); + scriptItemProcessor.setScript(new ClassPathResource("blah")); + scriptItemProcessor.afterPropertiesSet(); + } + + @Test(expected = IllegalStateException.class) + public void testNoScriptSetWithoutInitBean() throws Exception { + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.process("blah"); + } + + @Test(expected = IllegalArgumentException.class) + public void testScriptSourceWithNoLanguage() throws Exception { + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setScriptSource("function process(item) { return item.toUpperCase(); } process(item);", null); + scriptItemProcessor.afterPropertiesSet(); + } + + @Test + public void testItemBindingNameChange() throws Exception { + assumeTrue(languageExists("javascript")); + + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessor<>(); + scriptItemProcessor.setItemBindingVariableName("someOtherVarName"); + scriptItemProcessor.setScriptSource("function process(param) { return param.toUpperCase(); } process(someOtherVarName);", "javascript"); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", "SS", scriptItemProcessor.process("ss")); + } + + private boolean languageExists(String engineName) { + return availableLanguages.contains(engineName); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/SingleItemPeekableItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/SingleItemPeekableItemReaderTests.java index 496bb08bb4..8301d0fb8b 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/SingleItemPeekableItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/SingleItemPeekableItemReaderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2010 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ import org.junit.Test; import org.springframework.batch.item.ExecutionContext; +import org.springframework.lang.Nullable; /** * @author Dave Syer @@ -29,14 +30,14 @@ */ public class SingleItemPeekableItemReaderTests { - private SingleItemPeekableItemReader reader = new SingleItemPeekableItemReader(); + private SingleItemPeekableItemReader reader = new SingleItemPeekableItemReader<>(); /** * Test method for {@link org.springframework.batch.item.support.SingleItemPeekableItemReader#read()}. */ @Test public void testRead() throws Exception { - reader.setDelegate(new CountingListItemReader(Arrays.asList("a", "b"))); + reader.setDelegate(new CountingListItemReader<>(Arrays.asList("a", "b"))); assertEquals("a", reader.read()); assertEquals("b", reader.read()); assertEquals(null, reader.read()); @@ -47,7 +48,7 @@ public void testRead() throws Exception { */ @Test public void testPeek() throws Exception { - reader.setDelegate(new CountingListItemReader(Arrays.asList("a", "b"))); + reader.setDelegate(new CountingListItemReader<>(Arrays.asList("a", "b"))); assertEquals("a", reader.peek()); assertEquals("a", reader.read()); assertEquals("b", reader.read()); @@ -60,7 +61,7 @@ public void testPeek() throws Exception { */ @Test public void testCloseAndOpenNoPeek() throws Exception { - reader.setDelegate(new CountingListItemReader(Arrays.asList("a", "b"))); + reader.setDelegate(new CountingListItemReader<>(Arrays.asList("a", "b"))); assertEquals("a", reader.read()); ExecutionContext executionContext = new ExecutionContext(); reader.update(executionContext); @@ -74,7 +75,7 @@ public void testCloseAndOpenNoPeek() throws Exception { */ @Test public void testCloseAndOpenWithPeek() throws Exception { - reader.setDelegate(new CountingListItemReader(Arrays.asList("a", "b", "c"))); + reader.setDelegate(new CountingListItemReader<>(Arrays.asList("a", "b", "c"))); assertEquals("a", reader.read()); assertEquals("b", reader.peek()); ExecutionContext executionContext = new ExecutionContext(); @@ -87,7 +88,7 @@ public void testCloseAndOpenWithPeek() throws Exception { @Test public void testCloseAndOpenWithPeekAndRead() throws Exception { ExecutionContext executionContext = new ExecutionContext(); - reader.setDelegate(new CountingListItemReader(Arrays.asList("a", "b", "c"))); + reader.setDelegate(new CountingListItemReader<>(Arrays.asList("a", "b", "c"))); assertEquals("a", reader.read()); assertEquals("b", reader.peek()); reader.update(executionContext); @@ -122,6 +123,7 @@ protected void doOpen() throws Exception { counter = 0; } + @Nullable @Override protected T doRead() throws Exception { if (counter>=list.size()) { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/SynchronizedItemStreamReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/SynchronizedItemStreamReaderTests.java new file mode 100644 index 0000000000..73107e472e --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/SynchronizedItemStreamReaderTests.java @@ -0,0 +1,141 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.support; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +import java.util.HashSet; +import java.util.Set; + +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamReader; +import org.springframework.batch.item.NonTransientResourceException; +import org.springframework.batch.item.ParseException; +import org.springframework.lang.Nullable; + +/** + * + * @author Matthew Ouyang + * + */ +public class SynchronizedItemStreamReaderTests { + + /** + * A simple class used to test the SynchronizedItemStreamReader. It simply returns + * the number of times the read method has been called, manages some state variables + * and updates an ExecutionContext. + * + * @author Matthew Ouyang + * + */ + private class TestItemReader extends AbstractItemStreamItemReader implements ItemStreamReader { + + private int cursor = 0; + private boolean isClosed = false; + + public static final String HAS_BEEN_OPENED = "hasBeenOpened"; + public static final String UPDATE_COUNT_KEY = "updateCount"; + + @Nullable + public Integer read() throws Exception, ParseException, NonTransientResourceException { + cursor = cursor + 1; + return cursor; + } + + public void close() { + this.isClosed = true; + } + + public void open(ExecutionContext executionContext) { + this.isClosed = false; + executionContext.put(HAS_BEEN_OPENED, true); + executionContext.remove(UPDATE_COUNT_KEY); + } + + public void update(ExecutionContext executionContext) { + + if (!executionContext.containsKey(UPDATE_COUNT_KEY)) { + executionContext.putInt(UPDATE_COUNT_KEY, 0); + } + + executionContext.putInt(UPDATE_COUNT_KEY + , executionContext.getInt(UPDATE_COUNT_KEY) + 1 + ); + } + + public boolean isClosed() { + return this.isClosed; + } + } + + @Test + public void testMultipleThreads() throws Exception { + + // Initialized an ExecutionContext and a SynchronizedItemStreamReader to test. + final ExecutionContext executionContext = new ExecutionContext(); + + final TestItemReader testItemReader = new TestItemReader(); + final SynchronizedItemStreamReader synchronizedItemStreamReader = new SynchronizedItemStreamReader<>(); + synchronizedItemStreamReader.setDelegate(testItemReader); + + // Open the ItemReader and make sure it's initialized properly. + synchronizedItemStreamReader.open(executionContext); + assertEquals(true, executionContext.get(TestItemReader.HAS_BEEN_OPENED)); + assertFalse(testItemReader.isClosed()); + + /* Set up SIZE threads that read from the reader and updates the execution + * context. + */ + final Set ecSet = new HashSet<>(); + final int SIZE = 20; + Thread[] threads = new Thread[SIZE]; + for (int i = 0; i < SIZE; i++) { + threads[i] = new Thread() { + public void run() { + try { + ecSet.add(synchronizedItemStreamReader.read()); + synchronizedItemStreamReader.update(executionContext); + } catch (Exception ignore) { + ignore.printStackTrace(); + } + } + }; + } + + // Start the threads and block until all threads are done. + for (Thread thread : threads) { + thread.run(); + } + for (Thread thread : threads) { + thread.join(); + } + testItemReader.close(); + + /* Ensure cleanup happens as expected: status variable is set correctly and + * ExecutionContext variable is set properly. Lastly, the Set should + * have 1 to 20 which may not always be the case if the read is not synchronized. + */ + for (int i = 1; i <= SIZE; i++) { + assertTrue(ecSet.contains(i)); + } + assertTrue(testItemReader.isClosed()); + assertEquals(SIZE, executionContext.getInt(TestItemReader.UPDATE_COUNT_KEY)); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/TransactionAwareListItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/TransactionAwareListItemReaderTests.java index 24a299cb08..5674bf256d 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/TransactionAwareListItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/TransactionAwareListItemReaderTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -37,7 +37,7 @@ public class TransactionAwareListItemReaderTests extends TestCase { @Override protected void setUp() throws Exception { super.setUp(); - reader = new ListItemReader(TransactionAwareProxyFactory.createTransactionalList(Arrays.asList("a", "b", "c"))); + reader = new ListItemReader<>(TransactionAwareProxyFactory.createTransactionalList(Arrays.asList("a", "b", "c"))); } public void testNext() throws Exception { @@ -49,11 +49,11 @@ public void testNext() throws Exception { public void testCommit() throws Exception { PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); - final List taken = new ArrayList(); + final List taken = new ArrayList<>(); try { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { taken.add(reader.read()); return null; } @@ -77,10 +77,10 @@ public Object doInTransaction(TransactionStatus status) { public void testTransactionalExhausted() throws Exception { PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); - final List taken = new ArrayList(); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + final List taken = new ArrayList<>(); + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { Object next = reader.read(); while (next != null) { taken.add(next); @@ -95,11 +95,11 @@ public Object doInTransaction(TransactionStatus status) { public void testRollback() throws Exception { PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); - final List taken = new ArrayList(); + final List taken = new ArrayList<>(); try { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { taken.add(reader.read()); throw new RuntimeException("Rollback!"); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/ClassifierCompositeItemProcessorBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/ClassifierCompositeItemProcessorBuilderTests.java new file mode 100644 index 0000000000..ac4914eac9 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/ClassifierCompositeItemProcessorBuilderTests.java @@ -0,0 +1,65 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import java.util.HashMap; +import java.util.Map; + +import org.junit.Test; + +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.support.ClassifierCompositeItemProcessor; +import org.springframework.classify.PatternMatchingClassifier; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; + +/** + * @author Glenn Renfro + */ +public class ClassifierCompositeItemProcessorBuilderTests { + + @Test + public void testBasicClassifierCompositeItemProcessor() throws Exception { + + ItemProcessor fooProcessor = item -> "foo: " + item; + ItemProcessor defaultProcessor = item -> item; + + Map> routingConfiguration = new HashMap<>(); + routingConfiguration.put("foo", fooProcessor); + routingConfiguration.put("*", defaultProcessor); + ClassifierCompositeItemProcessor processor = new ClassifierCompositeItemProcessorBuilder() + .classifier(new PatternMatchingClassifier<>(routingConfiguration)) + .build(); + + assertEquals("bar", processor.process("bar")); + assertEquals("foo: foo", processor.process("foo")); + assertEquals("baz", processor.process("baz")); + } + + @Test + public void testNullClassifier() { + try { + new ClassifierCompositeItemProcessorBuilder().build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", + "A classifier is required.", iae.getMessage()); + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/ClassifierCompositeItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/ClassifierCompositeItemWriterBuilderTests.java new file mode 100644 index 0000000000..616f16dd2b --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/ClassifierCompositeItemWriterBuilderTests.java @@ -0,0 +1,69 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +import org.junit.Test; + +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.ClassifierCompositeItemWriter; +import org.springframework.classify.PatternMatchingClassifier; + +import static junit.framework.TestCase.fail; +import static org.junit.Assert.assertEquals; + +/** + * @author Glenn Renfro + */ +public class ClassifierCompositeItemWriterBuilderTests { + + private List defaults = new ArrayList<>(); + + private List foos = new ArrayList<>(); + + @Test + public void testWrite() throws Exception { + Map> map = new HashMap<>(); + ItemWriter fooWriter = items -> foos.addAll(items); + ItemWriter defaultWriter = items -> defaults.addAll(items); + map.put("foo", fooWriter); + map.put("*", defaultWriter); + ClassifierCompositeItemWriter writer = new ClassifierCompositeItemWriterBuilder() + .classifier(new PatternMatchingClassifier<>(map)).build(); + + writer.write(Arrays.asList("foo", "foo", "one", "two", "three")); + assertEquals("[foo, foo]", foos.toString()); + assertEquals("[one, two, three]", defaults.toString()); + } + + @Test + public void testSetNullClassifier() throws Exception { + try { + new ClassifierCompositeItemWriterBuilder<>().build(); + fail("A classifier is required."); + } + catch (IllegalArgumentException iae) { + assertEquals("Message returned from exception did not match expected result.", "A classifier is required.", + iae.getMessage()); + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/CompositeItemProcessorBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/CompositeItemProcessorBuilderTests.java new file mode 100644 index 0000000000..b5f266175d --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/CompositeItemProcessorBuilderTests.java @@ -0,0 +1,106 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import java.util.ArrayList; +import java.util.List; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.support.CompositeItemProcessor; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertSame; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.when; + +/** + * @author Glenn Renfro + * @author Drummond Dawson + */ +public class CompositeItemProcessorBuilderTests { + + @Mock + private ItemProcessor processor1; + + @Mock + private ItemProcessor processor2; + + private List> processors; + + @Before + public void setup() { + MockitoAnnotations.initMocks(this); + + this.processors = new ArrayList<>(); + this.processors.add(processor1); + this.processors.add(processor2); + } + + @Test + public void testTransform() throws Exception { + Object item = new Object(); + Object itemAfterFirstTransformation = new Object(); + Object itemAfterSecondTransformation = new Object(); + CompositeItemProcessor composite = new CompositeItemProcessorBuilder<>() + .delegates(this.processors).build(); + + when(processor1.process(item)).thenReturn(itemAfterFirstTransformation); + when(processor2.process(itemAfterFirstTransformation)).thenReturn(itemAfterSecondTransformation); + + assertSame(itemAfterSecondTransformation, composite.process(item)); + } + + @Test + public void testTransformVarargs() throws Exception { + Object item = new Object(); + Object itemAfterFirstTransformation = new Object(); + Object itemAfterSecondTransformation = new Object(); + CompositeItemProcessor composite = new CompositeItemProcessorBuilder<>() + .delegates(this.processor1, this.processor2).build(); + + when(processor1.process(item)).thenReturn(itemAfterFirstTransformation); + when(processor2.process(itemAfterFirstTransformation)).thenReturn(itemAfterSecondTransformation); + + assertSame(itemAfterSecondTransformation, composite.process(item)); + } + + @Test + public void testNullOrEmptyDelegates() throws Exception { + validateExceptionMessage(new CompositeItemProcessorBuilder<>().delegates(new ArrayList<>()), + "The delegates list must have one or more delegates."); + validateExceptionMessage(new CompositeItemProcessorBuilder<>().delegates(), + "The delegates list must have one or more delegates."); + validateExceptionMessage(new CompositeItemProcessorBuilder<>(), + "A list of delegates is required."); + } + + private void validateExceptionMessage(CompositeItemProcessorBuilder builder, String message) { + try { + builder.build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", message, + iae.getMessage()); + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/CompositeItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/CompositeItemWriterBuilderTests.java new file mode 100644 index 0000000000..9e1f168d8a --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/CompositeItemWriterBuilderTests.java @@ -0,0 +1,111 @@ +/* + * Copyright 2017-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamWriter; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.CompositeItemWriter; + +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; + +/** + * @author Glenn Renfro + * @author Drummond Dawson + */ +public class CompositeItemWriterBuilderTests { + + @Test + @SuppressWarnings("unchecked") + public void testProcess() throws Exception { + + final int NUMBER_OF_WRITERS = 10; + List data = Collections.singletonList(new Object()); + + List> writers = new ArrayList<>(); + + for (int i = 0; i < NUMBER_OF_WRITERS; i++) { + ItemWriter writer = mock(ItemWriter.class); + writers.add(writer); + } + CompositeItemWriter itemWriter = new CompositeItemWriterBuilder<>().delegates(writers).build(); + itemWriter.write(data); + + for (ItemWriter writer : writers) { + verify(writer).write(data); + } + + } + + @Test + @SuppressWarnings("unchecked") + public void testProcessVarargs() throws Exception { + + List data = Collections.singletonList(new Object()); + + List> writers = new ArrayList<>(); + + ItemWriter writer1 = mock(ItemWriter.class); + writers.add(writer1); + ItemWriter writer2 = mock(ItemWriter.class); + writers.add(writer2); + + CompositeItemWriter itemWriter = new CompositeItemWriterBuilder<>().delegates(writer1, writer2).build(); + itemWriter.write(data); + + for (ItemWriter writer : writers) { + verify(writer).write(data); + } + + } + + @Test + public void isStreamOpen() throws Exception { + ignoreItemStream(false); + ignoreItemStream(true); + } + + @SuppressWarnings("unchecked") + private void ignoreItemStream(boolean ignoreItemStream) throws Exception { + ItemStreamWriter writer = mock(ItemStreamWriter.class); + List data = Collections.singletonList(new Object()); + ExecutionContext executionContext = new ExecutionContext(); + + List> writers = new ArrayList<>(); + writers.add(writer); + CompositeItemWriter itemWriter = new CompositeItemWriterBuilder<>().delegates(writers) + .ignoreItemStream(ignoreItemStream).build(); + itemWriter.open(executionContext); + + int openCount = 0; + if (!ignoreItemStream) { + openCount = 1; + } + // If user has set ignoreItemStream to true, then it is expected that they opened the delegate writer. + verify(writer, times(openCount)).open(executionContext); + itemWriter.write(data); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/ScriptItemProcessorBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/ScriptItemProcessorBuilderTests.java new file mode 100644 index 0000000000..69a6d5ff63 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/ScriptItemProcessorBuilderTests.java @@ -0,0 +1,115 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import java.util.ArrayList; +import java.util.List; + +import javax.script.ScriptEngineFactory; +import javax.script.ScriptEngineManager; + +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; + +import org.springframework.batch.item.support.ScriptItemProcessor; +import org.springframework.core.io.ClassPathResource; +import org.springframework.core.io.Resource; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.junit.Assume.assumeTrue; + +/** + * @author Glenn Renfro + */ +public class ScriptItemProcessorBuilderTests { + private static List availableLanguages = new ArrayList<>(); + + @BeforeClass + public static void populateAvailableEngines() { + List scriptEngineFactories = new ScriptEngineManager().getEngineFactories(); + + for (ScriptEngineFactory scriptEngineFactory : scriptEngineFactories) { + availableLanguages.addAll(scriptEngineFactory.getNames()); + } + } + + @Before + public void setup() { + assumeTrue(availableLanguages.contains("javascript")); + } + + @Test + public void testScriptSource() throws Exception { + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessorBuilder() + .scriptSource("item.toUpperCase();") + .language("javascript") + .build(); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", "AA", scriptItemProcessor.process("aa")); + } + + @Test + public void testItemBinding() throws Exception { + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessorBuilder() + .scriptSource("foo.contains('World');") + .language("javascript") + .itemBindingVariableName("foo") + .build(); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", true, scriptItemProcessor.process("Hello World")); + } + + @Test + public void testScriptResource() throws Exception { + Resource resource = new ClassPathResource("org/springframework/batch/item/support/processor-test-simple.js"); + ScriptItemProcessor scriptItemProcessor = new ScriptItemProcessorBuilder() + .scriptResource(resource) + .build(); + scriptItemProcessor.afterPropertiesSet(); + + assertEquals("Incorrect transformed value", "BB", scriptItemProcessor.process("bb")); + } + + @Test + public void testNoScriptSourceNorResource() throws Exception { + validateExceptionMessage(new ScriptItemProcessorBuilder<>(), + "scriptResource or scriptSource is required."); + } + + @Test + public void testNoScriptSourceLanguage() throws Exception { + validateExceptionMessage(new ScriptItemProcessorBuilder().scriptSource("foo.contains('World');"), + "language is required when using scriptSource."); + + } + + private void validateExceptionMessage(ScriptItemProcessorBuilder builder, String message) { + try { + builder.build(); + fail("IllegalArgumentException should have been thrown"); + } + catch (IllegalArgumentException iae) { + assertEquals("IllegalArgumentException message did not match the expected result.", message, + iae.getMessage()); + } + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/SingleItemPeekableItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/SingleItemPeekableItemReaderBuilderTests.java new file mode 100644 index 0000000000..183fd02bf5 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/SingleItemPeekableItemReaderBuilderTests.java @@ -0,0 +1,68 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import java.util.Arrays; + +import org.junit.Test; + +import org.springframework.batch.item.sample.Foo; +import org.springframework.batch.item.support.ListItemReader; +import org.springframework.batch.item.support.SingleItemPeekableItemReader; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; + +/** + * @author Glenn Renfro + */ +public class SingleItemPeekableItemReaderBuilderTests { + + /** + * Test method to validate builder creates a + * {@link org.springframework.batch.item.support.SingleItemPeekableItemReader#peek()} + * with expected peek and read behavior. + */ + @Test + public void testPeek() throws Exception { + SingleItemPeekableItemReader reader = new SingleItemPeekableItemReaderBuilder() + .delegate( + new ListItemReader<>(Arrays.asList("a", "b"))) + .build(); + assertEquals("a", reader.peek()); + assertEquals("a", reader.read()); + assertEquals("b", reader.read()); + assertEquals(null, reader.peek()); + assertEquals(null, reader.read()); + } + + /** + * Test method to validate that an {@link IllegalArgumentException} is thrown if the + * delegate is not set in the builder. + */ + @Test + public void testValidation() { + try { + new SingleItemPeekableItemReaderBuilder().build(); + fail("A delegate is required"); + } + catch (IllegalArgumentException iae) { + assertEquals("A delegate is required", iae.getMessage()); + } + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/SynchronizedItemStreamReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/SynchronizedItemStreamReaderBuilderTests.java new file mode 100644 index 0000000000..5facf39931 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/builder/SynchronizedItemStreamReaderBuilderTests.java @@ -0,0 +1,148 @@ +/* + * Copyright 2017-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.support.builder; + +import java.util.HashSet; +import java.util.Set; + +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamReader; +import org.springframework.batch.item.NonTransientResourceException; +import org.springframework.batch.item.ParseException; +import org.springframework.batch.item.support.AbstractItemStreamItemReader; +import org.springframework.batch.item.support.SynchronizedItemStreamReader; +import org.springframework.lang.Nullable; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +/** + * @author Glenn Renfro + */ +public class SynchronizedItemStreamReaderBuilderTests { + + @Test + public void testMultipleThreads() throws Exception { + + // Initialized an ExecutionContext and a SynchronizedItemStreamReader to test. + final ExecutionContext executionContext = new ExecutionContext(); + + final SynchronizedItemStreamReaderBuilderTests.TestItemReader testItemReader = new SynchronizedItemStreamReaderBuilderTests.TestItemReader(); + final SynchronizedItemStreamReader synchronizedItemStreamReader = new SynchronizedItemStreamReaderBuilder() + .delegate(testItemReader).build(); + + // Open the ItemReader and make sure it's initialized properly. + synchronizedItemStreamReader.open(executionContext); + assertEquals(true, + executionContext.get(SynchronizedItemStreamReaderBuilderTests.TestItemReader.HAS_BEEN_OPENED)); + assertFalse(testItemReader.isClosed()); + + /* + * Set up SIZE threads that read from the reader and updates the execution + * context. + */ + final Set ecSet = new HashSet<>(); + final int SIZE = 20; + Thread[] threads = new Thread[SIZE]; + for (int i = 0; i < SIZE; i++) { + threads[i] = new Thread() { + public void run() { + try { + ecSet.add(synchronizedItemStreamReader.read()); + synchronizedItemStreamReader.update(executionContext); + } + catch (Exception ignore) { + ignore.printStackTrace(); + } + } + }; + } + + // Start the threads and block until all threads are done. + for (Thread thread : threads) { + thread.run(); + } + for (Thread thread : threads) { + thread.join(); + } + testItemReader.close(); + + /* + * Ensure cleanup happens as expected: status variable is set correctly and + * ExecutionContext variable is set properly. Lastly, the Set should have + * 1 to 20 which may not always be the case if the read is not synchronized. + */ + for (int i = 1; i <= SIZE; i++) { + assertTrue(ecSet.contains(i)); + } + assertTrue(testItemReader.isClosed()); + assertEquals(SIZE, + executionContext.getInt(SynchronizedItemStreamReaderBuilderTests.TestItemReader.UPDATE_COUNT_KEY)); + } + + /** + * A simple class used to test the SynchronizedItemStreamReader. It simply returns the + * number of times the read method has been called, manages some state variables and + * updates an ExecutionContext. + * + * @author Matthew Ouyang + * + */ + private class TestItemReader extends AbstractItemStreamItemReader implements ItemStreamReader { + + private int cursor = 0; + + private boolean isClosed = false; + + public static final String HAS_BEEN_OPENED = "hasBeenOpened"; + + public static final String UPDATE_COUNT_KEY = "updateCount"; + + @Nullable + public Integer read() throws Exception, ParseException, NonTransientResourceException { + cursor = cursor + 1; + return cursor; + } + + public void close() { + this.isClosed = true; + } + + public void open(ExecutionContext executionContext) { + this.isClosed = false; + executionContext.put(HAS_BEEN_OPENED, true); + executionContext.remove(UPDATE_COUNT_KEY); + } + + public void update(ExecutionContext executionContext) { + + if (!executionContext.containsKey(UPDATE_COUNT_KEY)) { + executionContext.putInt(UPDATE_COUNT_KEY, 0); + } + + executionContext.putInt(UPDATE_COUNT_KEY, executionContext.getInt(UPDATE_COUNT_KEY) + 1); + } + + public boolean isClosed() { + return this.isClosed; + } + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/util/ExecutionContextUserSupportTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/util/ExecutionContextUserSupportTests.java index 5d7ae29afa..284904ef62 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/util/ExecutionContextUserSupportTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/util/ExecutionContextUserSupportTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.util; import org.springframework.batch.item.util.ExecutionContextUserSupport; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/util/FileUtilsTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/util/FileUtilsTests.java index d0adae0927..4b048ad800 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/util/FileUtilsTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/util/FileUtilsTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.util; import java.io.BufferedWriter; @@ -5,14 +20,17 @@ import java.io.FileWriter; import java.io.IOException; -import static org.junit.Assert.*; - import org.junit.After; import org.junit.Before; import org.junit.Test; + import org.springframework.batch.item.ItemStreamException; import org.springframework.util.Assert; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + /** * Tests for {@link FileUtils} * @@ -20,7 +38,7 @@ */ public class FileUtilsTests { - private File file = new File("target/FileUtilsTests.tmp"); + private File file = new File("build/FileUtilsTests.tmp"); /** * No restart + file should not be overwritten => file is created if it does @@ -40,7 +58,7 @@ public void testNoRestart() throws Exception { } file.delete(); - Assert.state(!file.exists()); + Assert.state(!file.exists(), "Delete failed"); FileUtils.setUpOutputFile(file, false, false, true); assertTrue(file.exists()); @@ -49,7 +67,7 @@ public void testNoRestart() throws Exception { writer.write("testString"); writer.close(); long size = file.length(); - Assert.state(size > 0); + Assert.state(size > 0, "Nothing was written"); FileUtils.setUpOutputFile(file, false, false, true); long newSize = file.length(); @@ -109,10 +127,34 @@ public void testCreateDirectoryStructure() { dir1.delete(); } } + + /** + * If the directories on the file path do not exist, they should be created + * This must be true also in append mode + */ + @Test + public void testCreateDirectoryStructureAppendMode() { + File file = new File("testDirectory/testDirectory2/testFile.tmp"); + File dir1 = new File("testDirectory"); + File dir2 = new File("testDirectory/testDirectory2"); + + try { + FileUtils.setUpOutputFile(file, false, true, false); + assertTrue(file.exists()); + assertTrue(dir1.exists()); + assertTrue(dir2.exists()); + } + finally { + file.delete(); + dir2.delete(); + dir1.delete(); + } + } @Test public void testBadFile(){ + @SuppressWarnings("serial") File file = new File("new file"){ @Override public boolean createNewFile() throws IOException { @@ -132,6 +174,7 @@ public boolean createNewFile() throws IOException { @Test public void testCouldntCreateFile(){ + @SuppressWarnings("serial") File file = new File("new file"){ @Override @@ -154,7 +197,7 @@ public boolean exists() { @Before public void setUp() throws Exception { file.delete(); - Assert.state(!file.exists()); + Assert.state(!file.exists(), "File delete failed"); } @After diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/BeanValidatingItemProcessorTest.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/BeanValidatingItemProcessorTest.java new file mode 100644 index 0000000000..01f140f921 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/BeanValidatingItemProcessorTest.java @@ -0,0 +1,76 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.item.validator; + +import javax.validation.constraints.NotEmpty; + +import org.junit.Assert; +import org.junit.Test; + +/** + * @author Mahmoud Ben Hassine + */ +public class BeanValidatingItemProcessorTest { + + @Test + public void testValidObjectValidation() throws Exception { + // given + BeanValidatingItemProcessor validatingItemProcessor = new BeanValidatingItemProcessor<>(); + validatingItemProcessor.afterPropertiesSet(); + Foo foo = new Foo("foo"); + + // when + Foo processed = validatingItemProcessor.process(foo); + + // then + Assert.assertNotNull(processed); + } + + @Test(expected = ValidationException.class) + public void testInvalidObjectValidation() throws Exception { + // given + BeanValidatingItemProcessor validatingItemProcessor = new BeanValidatingItemProcessor<>(); + validatingItemProcessor.afterPropertiesSet(); + Foo foo = new Foo(""); + + // when + validatingItemProcessor.process(foo); + + // then + // expected exception + } + + private static class Foo { + + @NotEmpty + private String name; + + public Foo(String name) { + this.name = name; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/SpringValidatorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/SpringValidatorTests.java index bc09b5440f..2233a481f3 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/SpringValidatorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/SpringValidatorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,7 +30,7 @@ */ public class SpringValidatorTests { - private SpringValidator validator = new SpringValidator(); + private SpringValidator validator = new SpringValidator<>(); private Validator mockValidator; @@ -112,8 +112,7 @@ private static class MockSpringValidator implements Validator { public static final TestBean REJECT_MULTI_VALUE = new TestBean("foo", "bar"); @Override - @SuppressWarnings({ "rawtypes", "unchecked" }) - public boolean supports(Class clazz) { + public boolean supports(Class clazz) { return clazz.isAssignableFrom(TestBean.class); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/ValidatingItemProcessorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/ValidatingItemProcessorTests.java index 9b1d0e4b6f..72f19b1e81 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/ValidatingItemProcessorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/ValidatingItemProcessorTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.validator; import static org.mockito.Mockito.mock; @@ -20,7 +35,7 @@ public class ValidatingItemProcessorTests { @Test public void testSuccessfulValidation() throws Exception { - ValidatingItemProcessor tested = new ValidatingItemProcessor(validator); + ValidatingItemProcessor tested = new ValidatingItemProcessor<>(validator); validator.validate(ITEM); @@ -30,7 +45,7 @@ public void testSuccessfulValidation() throws Exception { @Test(expected = ValidationException.class) public void testFailedValidation() throws Exception { - ValidatingItemProcessor tested = new ValidatingItemProcessor(validator); + ValidatingItemProcessor tested = new ValidatingItemProcessor<>(validator); processFailedValidation(tested); } @@ -38,7 +53,7 @@ public void testFailedValidation() throws Exception { @Test public void testFailedValidation_Filter() throws Exception { - ValidatingItemProcessor tested = new ValidatingItemProcessor(validator); + ValidatingItemProcessor tested = new ValidatingItemProcessor<>(validator); tested.setFilter(true); assertNull(processFailedValidation(tested)); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/ValidationExceptionTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/ValidationExceptionTests.java index 676f10cded..1f274449ad 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/ValidationExceptionTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/validator/ValidationExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/EventHelper.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/EventHelper.java index f1b9a55bdb..414860a077 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/EventHelper.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/EventHelper.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; import javax.xml.stream.events.EndElement; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxEventItemReaderCommonTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxEventItemReaderCommonTests.java index dbd49d35eb..d3250fc189 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxEventItemReaderCommonTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxEventItemReaderCommonTests.java @@ -1,16 +1,26 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; -import static org.junit.Assert.assertTrue; - import java.io.IOException; - import javax.xml.stream.XMLEventReader; import javax.xml.stream.events.Attribute; import javax.xml.stream.events.StartElement; import javax.xml.transform.Source; -import org.junit.runners.JUnit4; -import org.junit.runner.RunWith; import org.springframework.batch.item.AbstractItemStreamItemReaderTests; import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.ItemReader; @@ -19,14 +29,15 @@ import org.springframework.oxm.Unmarshaller; import org.springframework.oxm.XmlMappingException; -@RunWith(JUnit4.class) +import static org.junit.Assert.assertTrue; + public class StaxEventItemReaderCommonTests extends AbstractItemStreamItemReaderTests { private final static String FOOS = " "; @Override protected ItemReader getItemReader() throws Exception { - StaxEventItemReader reader = new StaxEventItemReader(); + StaxEventItemReader reader = new StaxEventItemReader<>(); reader.setResource(new ByteArrayResource(FOOS.getBytes())); reader.setFragmentRootElementName("foo"); reader.setUnmarshaller(new Unmarshaller() { @@ -34,7 +45,7 @@ protected ItemReader getItemReader() throws Exception { public Object unmarshal(Source source) throws XmlMappingException, IOException { Attribute attr = null ; try { - XMLEventReader eventReader = StaxUtils.getXmlEventReader( source); + XMLEventReader eventReader = StaxTestUtils.getXmlEventReader( source); assertTrue(eventReader.nextEvent().isStartDocument()); StartElement event = eventReader.nextEvent().asStartElement(); attr = (Attribute) event.getAttributes().next(); @@ -48,8 +59,7 @@ public Object unmarshal(Source source) throws XmlMappingException, IOException { } @Override - @SuppressWarnings("rawtypes") - public boolean supports(Class clazz) { + public boolean supports(Class clazz) { return true; } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxEventItemReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxEventItemReaderTests.java index 0f2dc0ea02..8f4a9a1825 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxEventItemReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxEventItemReaderTests.java @@ -1,25 +1,22 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - -import java.io.IOException; -import java.io.InputStream; -import java.util.ArrayList; -import java.util.List; - -import javax.xml.stream.FactoryConfigurationError; -import javax.xml.stream.XMLEventReader; -import javax.xml.stream.XMLInputFactory; -import javax.xml.stream.XMLStreamException; -import javax.xml.stream.events.EndElement; -import javax.xml.stream.events.XMLEvent; -import javax.xml.transform.Source; - +import org.hamcrest.Matchers; +import org.junit.Assert; import org.junit.Before; import org.junit.Test; import org.springframework.batch.item.ExecutionContext; @@ -35,10 +32,34 @@ import org.springframework.oxm.XmlMappingException; import org.springframework.util.ClassUtils; +import javax.xml.namespace.QName; +import javax.xml.stream.FactoryConfigurationError; +import javax.xml.stream.XMLEventReader; +import javax.xml.stream.XMLStreamException; +import javax.xml.stream.events.EndElement; +import javax.xml.stream.events.StartElement; +import javax.xml.stream.events.XMLEvent; +import javax.xml.transform.Source; +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + /** * Tests for {@link StaxEventItemReader}. * * @author Robert Kasanicky + * @author Michael Minella + * @author Mahmoud Ben Hassine */ public class StaxEventItemReaderTests { @@ -48,6 +69,12 @@ public class StaxEventItemReaderTests { // test xml input private String xml = " testString "; + // test xml input + private String xmlMultiFragment = " testString testString "; + + // test xml input + private String xmlMultiFragmentNested = " nested nested testString testString "; + // test xml input private String emptyXml = ""; @@ -63,13 +90,15 @@ public class StaxEventItemReaderTests { private Unmarshaller unmarshaller = new MockFragmentUnmarshaller(); private static final String FRAGMENT_ROOT_ELEMENT = "fragment"; + + private static final String[] MULTI_FRAGMENT_ROOT_ELEMENTS = {"fragmentA", "fragmentB"}; private ExecutionContext executionContext; @Before public void setUp() throws Exception { this.executionContext = new ExecutionContext(); - source = createNewInputSouce(); + source = createNewInputSource(); } @Test @@ -80,7 +109,7 @@ public void testAfterPropertiesSet() throws Exception { @Test public void testAfterPropertesSetException() throws Exception { - source = createNewInputSouce(); + source = createNewInputSource(); source.setFragmentRootElementName(""); try { source.afterPropertiesSet(); @@ -90,7 +119,7 @@ public void testAfterPropertesSetException() throws Exception { // expected } - source = createNewInputSouce(); + source = createNewInputSource(); source.setUnmarshaller(null); try { source.afterPropertiesSet(); @@ -119,7 +148,7 @@ public void testFragmentWrapping() throws Exception { @Test public void testItemCountAwareFragment() throws Exception { - StaxEventItemReader source = createNewItemCountAwareInputSouce(); + StaxEventItemReader source = createNewItemCountAwareInputSource(); source.afterPropertiesSet(); source.open(executionContext); assertEquals(1, source.read().getItemCount()); @@ -131,13 +160,13 @@ public void testItemCountAwareFragment() throws Exception { @Test public void testItemCountAwareFragmentRestart() throws Exception { - StaxEventItemReader source = createNewItemCountAwareInputSouce(); + StaxEventItemReader source = createNewItemCountAwareInputSource(); source.afterPropertiesSet(); source.open(executionContext); assertEquals(1, source.read().getItemCount()); source.update(executionContext); source.close(); - source = createNewItemCountAwareInputSouce(); + source = createNewItemCountAwareInputSource(); source.afterPropertiesSet(); source.open(executionContext); assertEquals(2, source.read().getItemCount()); @@ -194,14 +223,117 @@ public void testFragmentInvalid() throws Exception { source.close(); } + + @Test + public void testMultiFragment() throws Exception { + + source.setResource(new ByteArrayResource(xmlMultiFragment.getBytes())); + source.setFragmentRootElementNames(MULTI_FRAGMENT_ROOT_ELEMENTS); + source.afterPropertiesSet(); + source.open(executionContext); + // see asserts in the mock unmarshaller + assertNotNull(source.read()); + assertNotNull(source.read()); + assertNotNull(source.read()); + assertNull(source.read()); // there are only three fragments + + source.close(); + } + + @Test + public void testMultiFragmentNameSpace() throws Exception { + + source.setResource(new ByteArrayResource(xmlMultiFragment.getBytes())); + source.setFragmentRootElementNames(new String[] {"{urn:org.test.bar}fragmentA", "fragmentB"}); + source.afterPropertiesSet(); + source.open(executionContext); + // see asserts in the mock unmarshaller + assertNotNull(source.read()); + assertNotNull(source.read()); + assertNull(source.read()); // there are only two fragments (one has wrong namespace) + + source.close(); + } + + @Test + public void testMultiFragmentRestart() throws Exception { + + source.setResource(new ByteArrayResource(xmlMultiFragment.getBytes())); + source.setFragmentRootElementNames(MULTI_FRAGMENT_ROOT_ELEMENTS); + source.afterPropertiesSet(); + source.open(executionContext); + // see asserts in the mock unmarshaller + assertNotNull(source.read()); + assertNotNull(source.read()); + + source.update(executionContext); + assertEquals(2, executionContext.getInt(ClassUtils.getShortName(StaxEventItemReader.class) + ".read.count")); + + source.close(); + + source = createNewInputSource(); + source.setResource(new ByteArrayResource(xmlMultiFragment.getBytes())); + source.setFragmentRootElementNames(MULTI_FRAGMENT_ROOT_ELEMENTS); + source.afterPropertiesSet(); + source.open(executionContext); + + assertNotNull(source.read()); + assertNull(source.read()); // there are only three fragments + + source.close(); + } + + @Test + public void testMultiFragmentNested() throws Exception { + + source.setResource(new ByteArrayResource(xmlMultiFragmentNested.getBytes())); + source.setFragmentRootElementNames(MULTI_FRAGMENT_ROOT_ELEMENTS); + source.afterPropertiesSet(); + source.open(executionContext); + // see asserts in the mock unmarshaller + assertNotNull(source.read()); + assertNotNull(source.read()); + assertNotNull(source.read()); + assertNull(source.read()); // there are only three fragments + + source.close(); + } + + @Test + public void testMultiFragmentNestedRestart() throws Exception { + + source.setResource(new ByteArrayResource(xmlMultiFragmentNested.getBytes())); + source.setFragmentRootElementNames(MULTI_FRAGMENT_ROOT_ELEMENTS); + source.afterPropertiesSet(); + source.open(executionContext); + // see asserts in the mock unmarshaller + assertNotNull(source.read()); + assertNotNull(source.read()); + + source.update(executionContext); + assertEquals(2, executionContext.getInt(ClassUtils.getShortName(StaxEventItemReader.class) + ".read.count")); + + source.close(); + + source = createNewInputSource(); + source.setResource(new ByteArrayResource(xmlMultiFragment.getBytes())); + source.setFragmentRootElementNames(MULTI_FRAGMENT_ROOT_ELEMENTS); + source.afterPropertiesSet(); + source.open(executionContext); + + assertNotNull(source.read()); + assertNull(source.read()); // there are only three fragments + source.close(); + } + /** * Cursor is moved before beginning of next fragment. */ @Test public void testMoveCursorToNextFragment() throws XMLStreamException, FactoryConfigurationError, IOException { Resource resource = new ByteArrayResource(xml.getBytes()); - XMLEventReader reader = XMLInputFactory.newInstance().createXMLEventReader(resource.getInputStream()); + XMLEventReader reader = StaxUtils.createXmlInputFactory().createXMLEventReader(resource.getInputStream()); final int EXPECTED_NUMBER_OF_FRAGMENTS = 2; for (int i = 0; i < EXPECTED_NUMBER_OF_FRAGMENTS; i++) { @@ -218,7 +350,7 @@ public void testMoveCursorToNextFragment() throws XMLStreamException, FactoryCon @Test public void testMoveCursorToNextFragmentOnEmpty() throws XMLStreamException, FactoryConfigurationError, IOException { Resource resource = new ByteArrayResource(emptyXml.getBytes()); - XMLEventReader reader = XMLInputFactory.newInstance().createXMLEventReader(resource.getInputStream()); + XMLEventReader reader = StaxUtils.createXmlInputFactory().createXMLEventReader(resource.getInputStream()); assertFalse(source.moveCursorToNextFragment(reader)); } @@ -229,7 +361,7 @@ public void testMoveCursorToNextFragmentOnEmpty() throws XMLStreamException, Fac @Test public void testMoveCursorToNextFragmentOnMissing() throws XMLStreamException, FactoryConfigurationError, IOException { Resource resource = new ByteArrayResource(missingXml.getBytes()); - XMLEventReader reader = XMLInputFactory.newInstance().createXMLEventReader(resource.getInputStream()); + XMLEventReader reader = StaxUtils.createXmlInputFactory().createXMLEventReader(resource.getInputStream()); assertFalse(source.moveCursorToNextFragment(reader)); } @@ -246,7 +378,7 @@ public void testRestart() throws Exception { assertEquals(1, executionContext.getInt(ClassUtils.getShortName(StaxEventItemReader.class) + ".read.count")); List expectedAfterRestart = source.read(); - source = createNewInputSouce(); + source = createNewInputSource(); source.open(executionContext); List afterRestart = source.read(); assertEquals(expectedAfterRestart.size(), afterRestart.size()); @@ -268,7 +400,7 @@ public void testRestartAtEndOfFile() throws Exception { assertEquals(3, executionContext.getInt(ClassUtils.getShortName(StaxEventItemReader.class) + ".read.count")); - source = createNewInputSouce(); + source = createNewInputSource(); source.open(executionContext); assertNull(source.read()); } @@ -386,7 +518,7 @@ public void testNonExistentResource() throws Exception { @Test public void testDirectoryResource() throws Exception { - FileSystemResource resource = new FileSystemResource("target/data"); + FileSystemResource resource = new FileSystemResource("build/data"); resource.getFile().mkdirs(); assertTrue(resource.getFile().isDirectory()); source.setResource(resource); @@ -449,6 +581,39 @@ public void exceptionDuringUnmarshalling() throws Exception { assertNull(source.read()); } + @Test + public void testDtdXml() { + String xmlWithDtd = "\n\n]>\n&entityex;"; + StaxEventItemReader reader = new StaxEventItemReader<>(); + reader.setName("foo"); + reader.setResource(new ByteArrayResource(xmlWithDtd.getBytes())); + reader.setUnmarshaller(new MockFragmentUnmarshaller() { + @Override + public Object unmarshal(Source source) throws XmlMappingException { + try { + XMLEventReader xmlEventReader = StaxTestUtils.getXmlEventReader(source); + xmlEventReader.nextEvent(); + xmlEventReader.nextEvent(); + return xmlEventReader.getElementText(); + } catch (Exception e) { + throw new RuntimeException(e); + } + } + }); + reader.setFragmentRootElementName("abc"); + + reader.open(new ExecutionContext()); + + try { + reader.read(); + fail("Should fail when XML contains DTD"); + } catch (Exception e) { + Assert.assertThat(e.getMessage(), Matchers.containsString("Undeclared general entity \"entityex\"")); + } + } + /** * Stub emulating problems during unmarshalling. */ @@ -462,17 +627,16 @@ public Object unmarshal(Source source) throws XmlMappingException, IOException { } @Override - @SuppressWarnings("rawtypes") - public boolean supports(Class clazz) { + public boolean supports(Class clazz) { return true; } } - private StaxEventItemReader> createNewInputSouce() { + private StaxEventItemReader> createNewInputSource() { Resource resource = new ByteArrayResource(xml.getBytes()); - StaxEventItemReader> newSource = new StaxEventItemReader>(); + StaxEventItemReader> newSource = new StaxEventItemReader<>(); newSource.setResource(resource); newSource.setFragmentRootElementName(FRAGMENT_ROOT_ELEMENT); @@ -482,10 +646,10 @@ private StaxEventItemReader> createNewInputSouce() { return newSource; } - private StaxEventItemReader createNewItemCountAwareInputSouce() { + private StaxEventItemReader createNewItemCountAwareInputSource() { Resource resource = new ByteArrayResource(xml.getBytes()); - StaxEventItemReader newSource = new StaxEventItemReader(); + StaxEventItemReader newSource = new StaxEventItemReader<>(); newSource.setResource(resource); newSource.setFragmentRootElementName(FRAGMENT_ROOT_ELEMENT); @@ -504,13 +668,13 @@ private static class MockFragmentUnmarshaller implements Unmarshaller { /** * Skips the XML fragment contents. */ - private List readRecordsInsideFragment(XMLEventReader eventReader) throws XMLStreamException { + private List readRecordsInsideFragment(XMLEventReader eventReader, QName fragmentName) throws XMLStreamException { XMLEvent eventInsideFragment; - List events = new ArrayList(); + List events = new ArrayList<>(); do { eventInsideFragment = eventReader.peek(); if (eventInsideFragment instanceof EndElement - && ((EndElement) eventInsideFragment).getName().getLocalPart().equals(FRAGMENT_ROOT_ELEMENT)) { + && fragmentName.equals(((EndElement) eventInsideFragment).getName())) { break; } events.add(eventReader.nextEvent()); @@ -520,8 +684,7 @@ private List readRecordsInsideFragment(XMLEventReader eventReader) thr } @Override - @SuppressWarnings("rawtypes") - public boolean supports(Class clazz) { + public boolean supports(Class clazz) { return true; } @@ -536,7 +699,7 @@ public Object unmarshal(Source source) throws XmlMappingException, IOException { List fragmentContent; try { - XMLEventReader eventReader = StaxUtils.getXmlEventReader(source); + XMLEventReader eventReader = StaxTestUtils.getXmlEventReader(source); // first event should be StartDocument XMLEvent event1 = eventReader.nextEvent(); @@ -545,15 +708,16 @@ public Object unmarshal(Source source) throws XmlMappingException, IOException { // second should be StartElement of the fragment XMLEvent event2 = eventReader.nextEvent(); assertTrue(event2.isStartElement()); - assertTrue(EventHelper.startElementName(event2).equals(FRAGMENT_ROOT_ELEMENT)); + assertTrue(isFragmentRootElement(EventHelper.startElementName(event2))); + QName fragmentName = ((StartElement) event2).getName(); // jump before the end of fragment - fragmentContent = readRecordsInsideFragment(eventReader); + fragmentContent = readRecordsInsideFragment(eventReader, fragmentName); // end of fragment XMLEvent event3 = eventReader.nextEvent(); assertTrue(event3.isEndElement()); - assertTrue(EventHelper.endElementName(event3).equals(FRAGMENT_ROOT_ELEMENT)); + assertTrue(isFragmentRootElement(EventHelper.endElementName(event3))); // EndDocument should follow the end of fragment XMLEvent event4 = eventReader.nextEvent(); @@ -561,10 +725,14 @@ public Object unmarshal(Source source) throws XmlMappingException, IOException { } catch (Exception e) { - throw new RuntimeException("Error occured in FragmentDeserializer", e); + throw new RuntimeException("Error occurred in FragmentDeserializer", e); } return fragmentContent; } + + private boolean isFragmentRootElement(String name) { + return FRAGMENT_ROOT_ELEMENT.equals(name) || Arrays.asList(MULTI_FRAGMENT_ROOT_ELEMENTS).contains(name); + } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxEventItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxEventItemWriterTests.java index df60339e19..1b0169bc16 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxEventItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxEventItemWriterTests.java @@ -1,17 +1,24 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - import java.io.File; import java.io.IOException; import java.util.Collections; import java.util.List; - import javax.xml.bind.annotation.XmlRootElement; import javax.xml.stream.XMLEventFactory; import javax.xml.stream.XMLEventWriter; @@ -21,8 +28,10 @@ import org.apache.commons.io.FileUtils; import org.junit.Before; import org.junit.Test; + import org.springframework.batch.item.ExecutionContext; import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.batch.item.WriterNotOpenException; import org.springframework.batch.support.transaction.ResourcelessTransactionManager; import org.springframework.core.io.FileSystemResource; import org.springframework.core.io.Resource; @@ -37,6 +46,13 @@ import org.springframework.util.ClassUtils; import org.springframework.util.StringUtils; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + /** * Tests for {@link StaxEventItemWriter}. */ @@ -92,7 +108,7 @@ public String toString() { @Before public void setUp() throws Exception { - File directory = new File("target/data"); + File directory = new File("build/data"); directory.mkdirs(); resource = new FileSystemResource(File.createTempFile("StaxEventWriterOutputSourceTests", ".xml", directory)); writer = createItemWriter(); @@ -100,6 +116,26 @@ public void setUp() throws Exception { jaxbMarshaller = new Jaxb2Marshaller(); jaxbMarshaller.setClassesToBeBound(JAXBItem.class); } + + /** + * Test setting writer name. + */ + @Test + public void testSetName() throws Exception { + writer.setName("test"); + writer.open(executionContext); + writer.write(items); + writer.update(executionContext); + writer.close(); + assertTrue("execution context keys should be prefixed with writer name", executionContext.containsKey("test.position")); + } + + @Test(expected = WriterNotOpenException.class) + public void testAssertWriterIsInitialized() throws Exception { + StaxEventItemWriter writer = new StaxEventItemWriter<>(); + + writer.write(Collections.singletonList("foo")); + } /** * Item is written to the output file only after flush. @@ -149,15 +185,14 @@ public void testRestart() throws Exception { } @Test - @SuppressWarnings({"unchecked", "rawtypes"}) public void testTransactionalRestart() throws Exception { writer.open(executionContext); PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { // write item writer.write(items); @@ -175,9 +210,9 @@ public Object doInTransaction(TransactionStatus status) { // create new writer from saved restart data and continue writing writer = createItemWriter(); writer.open(executionContext); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { writer.write(items); } @@ -216,9 +251,9 @@ private void testTransactionalRestartWithMultiByteCharacter(String encoding) thr PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { // write item writer.write(itemsMultiByte); @@ -237,9 +272,9 @@ public Object doInTransaction(TransactionStatus status) { writer = createItemWriter(); writer.setEncoding(encoding); writer.open(executionContext); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { writer.write(itemsMultiByte); } @@ -261,16 +296,15 @@ public Object doInTransaction(TransactionStatus status) { } @Test - @SuppressWarnings({"unchecked", "rawtypes"}) public void testTransactionalRestartFailOnFirstWrite() throws Exception { PlatformTransactionManager transactionManager = new ResourcelessTransactionManager(); writer.open(executionContext); try { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { writer.write(items); } @@ -290,9 +324,9 @@ public Object doInTransaction(TransactionStatus status) { // create new writer from saved restart data and continue writing writer = createItemWriter(); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { writer.open(executionContext); try { writer.write(items); @@ -590,14 +624,14 @@ public void write(XMLEventWriter writer) throws IOException { */ @Test public void testWriteRootTagWithNamespace() throws Exception { - writer.setRootTagName("{http://www.springframework.org/test}root"); + writer.setRootTagName("{https://www.springframework.org/test}root"); writer.afterPropertiesSet(); writer.open(executionContext); writer.write(items); writer.close(); String content = getOutputFileContent(); assertTrue("Wrong content: " + content, content - .contains((""))); + .contains((""))); assertTrue("Wrong content: " + content, content.contains(TEST_STRING)); assertTrue("Wrong content: " + content, content.contains((""))); } @@ -607,7 +641,7 @@ public void testWriteRootTagWithNamespace() throws Exception { */ @Test public void testWriteRootTagWithNamespaceAndPrefix() throws Exception { - writer.setRootTagName("{http://www.springframework.org/test}ns:root"); + writer.setRootTagName("{https://www.springframework.org/test}ns:root"); writer.afterPropertiesSet(); marshaller.setNamespace(writer.getRootTagNamespace()); marshaller.setNamespacePrefix(writer.getRootTagNamespacePrefix()); @@ -616,7 +650,7 @@ public void testWriteRootTagWithNamespaceAndPrefix() throws Exception { writer.close(); String content = getOutputFileContent(); assertTrue("Wrong content: " + content, content - .contains((""))); + .contains((""))); assertTrue("Wrong content: " + content, content.contains(NS_TEST_STRING)); assertTrue("Wrong content: " + content, content.contains((""))); assertTrue("Wrong content: " + content, content.contains((""))); assertTrue("Wrong content: " + content, content.contains(FOO_TEST_STRING)); assertTrue("Wrong content: " + content, content.contains((""))); @@ -650,7 +684,7 @@ public void testWriteRootTagWithAdditionalNamespace() throws Exception { @Test public void testRootTagWithNamespaceRestart() throws Exception { writer.setMarshaller(jaxbMarshaller); - writer.setRootTagName("{http://www.springframework.org/test}root"); + writer.setRootTagName("{https://www.springframework.org/test}root"); writer.afterPropertiesSet(); writer.open(executionContext); writer.write(jaxbItems); @@ -659,7 +693,7 @@ public void testRootTagWithNamespaceRestart() throws Exception { writer = createItemWriter(); writer.setMarshaller(jaxbMarshaller); - writer.setRootTagName("{http://www.springframework.org/test}root"); + writer.setRootTagName("{https://www.springframework.org/test}root"); writer.afterPropertiesSet(); writer.open(executionContext); writer.write(jaxbItems); @@ -668,7 +702,7 @@ public void testRootTagWithNamespaceRestart() throws Exception { String content = getOutputFileContent(); assertEquals("Wrong content: " + content, - "", content); + "", content); } /** @@ -677,7 +711,7 @@ public void testRootTagWithNamespaceRestart() throws Exception { @Test public void testRootTagWithNamespaceAndPrefixRestart() throws Exception { writer.setMarshaller(jaxbMarshaller); - writer.setRootTagName("{http://www.springframework.org/test}ns:root"); + writer.setRootTagName("{https://www.springframework.org/test}ns:root"); writer.afterPropertiesSet(); writer.open(executionContext); writer.write(jaxbItems); @@ -686,7 +720,7 @@ public void testRootTagWithNamespaceAndPrefixRestart() throws Exception { writer = createItemWriter(); writer.setMarshaller(jaxbMarshaller); - writer.setRootTagName("{http://www.springframework.org/test}ns:root"); + writer.setRootTagName("{https://www.springframework.org/test}ns:root"); writer.afterPropertiesSet(); writer.open(executionContext); writer.write(jaxbItems); @@ -695,7 +729,7 @@ public void testRootTagWithNamespaceAndPrefixRestart() throws Exception { String content = getOutputFileContent(); assertEquals("Wrong content: " + content, - "", content); + "", content); } /** @@ -705,7 +739,7 @@ public void testRootTagWithNamespaceAndPrefixRestart() throws Exception { public void testRootTagWithAdditionalNamespaceRestart() throws Exception { writer.setMarshaller(jaxbMarshaller); writer.setRootTagName("{urn:org.test.foo}foo:root"); - writer.setRootElementAttributes(Collections.singletonMap("xmlns:ns", "http://www.springframework.org/test")); + writer.setRootElementAttributes(Collections.singletonMap("xmlns:ns", "https://www.springframework.org/test")); writer.afterPropertiesSet(); writer.open(executionContext); writer.write(jaxbItems); @@ -715,7 +749,7 @@ public void testRootTagWithAdditionalNamespaceRestart() throws Exception { writer = createItemWriter(); writer.setMarshaller(jaxbMarshaller); writer.setRootTagName("{urn:org.test.foo}foo:root"); - writer.setRootElementAttributes(Collections.singletonMap("xmlns:ns", "http://www.springframework.org/test")); + writer.setRootElementAttributes(Collections.singletonMap("xmlns:ns", "https://www.springframework.org/test")); writer.afterPropertiesSet(); writer.open(executionContext); writer.write(jaxbItems); @@ -724,7 +758,7 @@ public void testRootTagWithAdditionalNamespaceRestart() throws Exception { String content = getOutputFileContent(); assertEquals("Wrong content: " + content, - "", content); + "", content); } /** @@ -738,7 +772,7 @@ public void testMarshallingClosingEventWriter() throws Exception { public void marshal(Object graph, Result result) throws XmlMappingException, IOException { super.marshal(graph, result); try { - StaxUtils.getXmlEventWriter(result).close(); + StaxTestUtils.getXmlEventWriter(result).close(); } catch (Exception e) { throw new RuntimeException("Exception while writing to output file", e); } @@ -753,6 +787,153 @@ public void marshal(Object graph, Result result) throws XmlMappingException, IOE writer.write(items); } + /** + * Test opening and closing corresponding tags in header- and footer callback. + */ + @Test + public void testOpenAndCloseTagsInCallbacks() throws Exception { + initWriterForSimpleCallbackTests(); + writer.open(executionContext); + writer.write(items); + writer.close(); + String content = getOutputFileContent(); + + assertEquals("Wrong content: " + content, + "", content); + } + + /** + * Test opening and closing corresponding tags in header- and footer callback (restart). + */ + @Test + public void testOpenAndCloseTagsInCallbacksRestart() throws Exception { + initWriterForSimpleCallbackTests(); + writer.open(executionContext); + writer.write(items); + writer.update(executionContext); + + initWriterForSimpleCallbackTests(); + + writer.open(executionContext); + writer.write(items); + writer.close(); + String content = getOutputFileContent(); + + assertEquals("Wrong content: " + content, + "" + + "", content); + } + + /** + * Test opening and closing corresponding tags in complex header- and footer callback (restart). + */ + @Test + public void testOpenAndCloseTagsInComplexCallbacksRestart() throws Exception { + initWriterForComplexCallbackTests(); + writer.open(executionContext); + writer.write(items); + writer.update(executionContext); + + initWriterForComplexCallbackTests(); + + writer.open(executionContext); + writer.write(items); + writer.close(); + String content = getOutputFileContent(); + + assertEquals("Wrong content: " + content, + "" + + "PRE-HEADERPOST-HEADER" + + "" + + "PRE-FOOTERPOST-FOOTER" + + "", content); + } + + private void initWriterForSimpleCallbackTests() throws Exception { + writer = createItemWriter(); + writer.setHeaderCallback(new StaxWriterCallback() { + + @Override + public void write(XMLEventWriter writer) throws IOException { + XMLEventFactory factory = XMLEventFactory.newInstance(); + try { + writer.add(factory.createStartElement("ns", "https://www.springframework.org/test", "group")); + } + catch (XMLStreamException e) { + throw new RuntimeException(e); + } + } + + }); + writer.setFooterCallback(new StaxWriterCallback() { + + @Override + public void write(XMLEventWriter writer) throws IOException { + XMLEventFactory factory = XMLEventFactory.newInstance(); + try { + writer.add(factory.createEndElement("ns", "https://www.springframework.org/test", "group")); + } + catch (XMLStreamException e) { + throw new RuntimeException(e); + } + + } + + }); + writer.setRootTagName("{https://www.springframework.org/test}ns:testroot"); + writer.afterPropertiesSet(); + } + + // more complex callbacks, writing element before and after the multiple corresponding header- and footer elements + private void initWriterForComplexCallbackTests() throws Exception { + writer = createItemWriter(); + writer.setHeaderCallback(new StaxWriterCallback() { + + @Override + public void write(XMLEventWriter writer) throws IOException { + XMLEventFactory factory = XMLEventFactory.newInstance(); + try { + writer.add(factory.createStartElement("", "", "preHeader")); + writer.add(factory.createCharacters("PRE-HEADER")); + writer.add(factory.createEndElement("", "", "preHeader")); + writer.add(factory.createStartElement("ns", "https://www.springframework.org/test", "group")); + writer.add(factory.createStartElement("", "", "subGroup")); + writer.add(factory.createStartElement("", "", "postHeader")); + writer.add(factory.createCharacters("POST-HEADER")); + writer.add(factory.createEndElement("", "", "postHeader")); + } + catch (XMLStreamException e) { + throw new RuntimeException(e); + } + } + + }); + writer.setFooterCallback(new StaxWriterCallback() { + + @Override + public void write(XMLEventWriter writer) throws IOException { + XMLEventFactory factory = XMLEventFactory.newInstance(); + try { + writer.add(factory.createStartElement("", "", "preFooter")); + writer.add(factory.createCharacters("PRE-FOOTER")); + writer.add(factory.createEndElement("", "", "preFooter")); + writer.add(factory.createEndElement("", "", "subGroup")); + writer.add(factory.createEndElement("ns", "https://www.springframework.org/test", "group")); + writer.add(factory.createStartElement("", "", "postFooter")); + writer.add(factory.createCharacters("POST-FOOTER")); + writer.add(factory.createEndElement("", "", "postFooter")); + } + catch (XMLStreamException e) { + throw new RuntimeException(e); + } + + } + + }); + writer.setRootTagName("{https://www.springframework.org/test}ns:testroot"); + writer.afterPropertiesSet(); + } + /** * Writes object's toString representation as XML comment. */ @@ -774,8 +955,8 @@ public void setNamespacePrefix(String namespacePrefix) { public void marshal(Object graph, Result result) throws XmlMappingException, IOException { Assert.isInstanceOf( Result.class, result); try { - StaxUtils.getXmlEventWriter( result ).add( XMLEventFactory.newInstance().createStartElement(namespacePrefix, namespace, graph.toString())); - StaxUtils.getXmlEventWriter( result ).add( XMLEventFactory.newInstance().createEndElement(namespacePrefix, namespace, graph.toString())); + StaxTestUtils.getXmlEventWriter( result ).add( XMLEventFactory.newInstance().createStartElement(namespacePrefix, namespace, graph.toString())); + StaxTestUtils.getXmlEventWriter( result ).add( XMLEventFactory.newInstance().createEndElement(namespacePrefix, namespace, graph.toString())); } catch ( Exception e) { throw new RuntimeException("Exception while writing to output file", e); @@ -783,8 +964,7 @@ public void marshal(Object graph, Result result) throws XmlMappingException, IOE } @Override - @SuppressWarnings("rawtypes") - public boolean supports(Class clazz) { + public boolean supports(Class clazz) { return true; } } @@ -812,7 +992,7 @@ private String getOutputFileContent(String encoding) throws IOException { * @return new instance of fully configured writer */ private StaxEventItemWriter createItemWriter() throws Exception { - StaxEventItemWriter source = new StaxEventItemWriter(); + StaxEventItemWriter source = new StaxEventItemWriter<>(); source.setResource(resource); marshaller = new SimpleMarshaller(); @@ -829,7 +1009,7 @@ private StaxEventItemWriter createItemWriter() throws Exception { return source; } - @XmlRootElement(name="item", namespace="http://www.springframework.org/test") + @XmlRootElement(name="item", namespace="https://www.springframework.org/test") private static class JAXBItem { } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxTestUtils.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxTestUtils.java new file mode 100644 index 0000000000..bb3ab59aa8 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/StaxTestUtils.java @@ -0,0 +1,48 @@ +/* + * Copyright 2015-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.xml; + +import java.lang.reflect.Method; + +import javax.xml.stream.XMLEventReader; +import javax.xml.stream.XMLEventWriter; +import javax.xml.transform.Result; +import javax.xml.transform.Source; + +/** + * Utility methods for StAX related tests. + */ +public final class StaxTestUtils { + + public static XMLEventWriter getXmlEventWriter(Result r) throws Exception { + Method m = r.getClass().getDeclaredMethod("getXMLEventWriter"); + boolean accessible = m.isAccessible(); + m.setAccessible(true); + Object result = m.invoke(r); + m.setAccessible(accessible); + return (XMLEventWriter) result; + } + + public static XMLEventReader getXmlEventReader(Source s) throws Exception { + Method m = s.getClass().getDeclaredMethod("getXMLEventReader"); + boolean accessible = m.isAccessible(); + m.setAccessible(true); + Object result = m.invoke(s); + m.setAccessible(accessible); + return (XMLEventReader) result; + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/TransactionalStaxEventItemWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/TransactionalStaxEventItemWriterTests.java index abc632f0b3..a1422013f9 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/TransactionalStaxEventItemWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/TransactionalStaxEventItemWriterTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml; import static org.junit.Assert.assertEquals; @@ -71,9 +86,9 @@ public void setUp() throws Exception { @Test public void testWriteAndFlush() throws Exception { writer.open(executionContext); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { writer.write(items); } @@ -111,9 +126,9 @@ public void write(XMLEventWriter writer) throws IOException { }); writer.open(executionContext); try { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { writer.write(items); } @@ -130,9 +145,9 @@ public Object doInTransaction(TransactionStatus status) { } writer.close(); writer.open(executionContext); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { writer.write(items); } @@ -170,9 +185,9 @@ public void write(XMLEventWriter writer) throws IOException { }); writer.open(executionContext); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { writer.write(items); } @@ -186,9 +201,9 @@ public Object doInTransaction(TransactionStatus status) { writer.close(); writer.open(executionContext); try { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { try { writer.write(items); } @@ -213,7 +228,7 @@ public Object doInTransaction(TransactionStatus status) { * @return output file content as String */ private String outputFileContent() throws IOException { - return FileUtils.readFileToString(resource.getFile(), null); + return FileUtils.readFileToString(resource.getFile(), (String)null); } /** @@ -223,7 +238,7 @@ private static class SimpleMarshaller implements Marshaller { @Override public void marshal(Object graph, Result result) throws XmlMappingException, IOException { try { - StaxUtils.getXmlEventWriter(result).add(XMLEventFactory.newInstance().createComment(graph.toString())); + StaxTestUtils.getXmlEventWriter(result).add(XMLEventFactory.newInstance().createComment(graph.toString())); } catch ( Exception e) { throw new RuntimeException("Exception while writing to output file", e); @@ -231,8 +246,7 @@ public void marshal(Object graph, Result result) throws XmlMappingException, IOE } @Override - @SuppressWarnings("rawtypes") - public boolean supports(Class clazz) { + public boolean supports(Class clazz) { return true; } } @@ -241,7 +255,7 @@ public boolean supports(Class clazz) { * @return new instance of fully configured writer */ private StaxEventItemWriter createItemWriter() throws Exception { - StaxEventItemWriter source = new StaxEventItemWriter(); + StaxEventItemWriter source = new StaxEventItemWriter<>(); source.setResource(resource); Marshaller marshaller = new SimpleMarshaller(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/builder/StaxEventItemReaderBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/builder/StaxEventItemReaderBuilderTests.java new file mode 100644 index 0000000000..7d2355140a --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/builder/StaxEventItemReaderBuilderTests.java @@ -0,0 +1,215 @@ +/* + * Copyright 2017-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.xml.builder; + +import javax.xml.bind.annotation.XmlRootElement; +import javax.xml.stream.XMLInputFactory; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.xml.StaxEventItemReader; +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.Resource; +import org.springframework.oxm.jaxb.Jaxb2Marshaller; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.fail; + +/** + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +public class StaxEventItemReaderBuilderTests { + + private static final String SIMPLE_XML = "1" + + "twothree4" + + "fivesix7" + + "eightnine"; + + @Mock + private Resource resource; + + @Before + public void setUp() { + MockitoAnnotations.initMocks(this); + } + + @Test + public void testValidation() { + try { + new StaxEventItemReaderBuilder().build(); + fail("Validation of the missing resource failed"); + } + catch (IllegalArgumentException ignore) { + } + + try { + new StaxEventItemReaderBuilder() + .resource(this.resource) + .build(); + fail("saveState == true should require a name"); + } + catch (IllegalStateException iae) { + assertEquals("A name is required when saveState is set to true.", + iae.getMessage()); + } + + try { + new StaxEventItemReaderBuilder() + .resource(this.resource) + .saveState(false) + .build(); + fail("No root tags have been configured"); + } + catch (IllegalArgumentException iae) { + assertEquals("At least one fragment root element is required", iae.getMessage()); + } + } + + @Test + public void testConfiguration() throws Exception { + Jaxb2Marshaller unmarshaller = new Jaxb2Marshaller(); + unmarshaller.setClassesToBeBound(Foo.class); + + StaxEventItemReader reader = new StaxEventItemReaderBuilder() + .name("fooReader") + .resource(getResource(SIMPLE_XML)) + .addFragmentRootElements("foo") + .currentItemCount(1) + .maxItemCount(2) + .unmarshaller(unmarshaller) + .xmlInputFactory(XMLInputFactory.newInstance()) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + Foo item = reader.read(); + assertNull(reader.read()); + reader.update(executionContext); + + reader.close(); + + assertEquals(4, item.getFirst()); + assertEquals("five", item.getSecond()); + assertEquals("six", item.getThird()); + assertEquals(2, executionContext.size()); + } + + @Test(expected = ItemStreamException.class) + public void testStrict() throws Exception { + Jaxb2Marshaller unmarshaller = new Jaxb2Marshaller(); + unmarshaller.setClassesToBeBound(Foo.class); + + StaxEventItemReader reader = new StaxEventItemReaderBuilder() + .name("fooReader") + .resource(this.resource) + .addFragmentRootElements("foo") + .unmarshaller(unmarshaller) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + } + + @Test + public void testSaveState() throws Exception { + Jaxb2Marshaller unmarshaller = new Jaxb2Marshaller(); + unmarshaller.setClassesToBeBound(Foo.class); + + StaxEventItemReader reader = new StaxEventItemReaderBuilder() + .name("fooReader") + .resource(getResource(SIMPLE_XML)) + .addFragmentRootElements("foo") + .unmarshaller(unmarshaller) + .saveState(false) + .build(); + + reader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + reader.open(executionContext); + Foo item = reader.read(); + item = reader.read(); + item = reader.read(); + assertNull(reader.read()); + reader.update(executionContext); + + reader.close(); + + assertEquals(7, item.getFirst()); + assertEquals("eight", item.getSecond()); + assertEquals("nine", item.getThird()); + assertEquals(0, executionContext.size()); + } + + private Resource getResource(String contents) { + return new ByteArrayResource(contents.getBytes()); + } + + @XmlRootElement(name="foo") + public static class Foo { + private int first; + private String second; + private String third; + + public Foo() {} + + public Foo(int first, String second, String third) { + this.first = first; + this.second = second; + this.third = third; + } + + public int getFirst() { + return first; + } + + public void setFirst(int first) { + this.first = first; + } + + public String getSecond() { + return second; + } + + public void setSecond(String second) { + this.second = second; + } + + public String getThird() { + return third; + } + + public void setThird(String third) { + this.third = third; + } + + @Override + public String toString() { + return String.format("{%s, %s, %s}", this.first, this.second, this.third); + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/builder/StaxEventItemWriterBuilderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/builder/StaxEventItemWriterBuilderTests.java new file mode 100644 index 0000000000..1b7ae7dcb1 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/builder/StaxEventItemWriterBuilderTests.java @@ -0,0 +1,271 @@ +/* + * Copyright 2017 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.xml.builder; + +import java.io.File; +import java.io.IOException; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import javax.xml.bind.annotation.XmlRootElement; +import javax.xml.stream.XMLEventFactory; +import javax.xml.stream.XMLStreamException; + +import org.apache.commons.io.FileUtils; +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.xml.StaxEventItemWriter; +import org.springframework.batch.support.transaction.TransactionAwareBufferedWriter; +import org.springframework.core.io.FileSystemResource; +import org.springframework.core.io.Resource; +import org.springframework.oxm.Marshaller; +import org.springframework.oxm.jaxb.Jaxb2Marshaller; +import org.springframework.test.util.ReflectionTestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +/** + * @author Michael Minella + */ +public class StaxEventItemWriterBuilderTests { + + private Resource resource; + + private List items; + + private Marshaller marshaller; + + private static final String FULL_OUTPUT = "" + + "\uFEFF" + + "1twothree\uFEFF" + + "4" + + "fivesix\uFEFF" + + "7" + + "eightnine\uFEFF\uFEFF" + + ""; + + private static final String SIMPLE_OUTPUT = "" + + "1twothree" + + "4" + + "fivesix" + + "7" + + "eightnine"; + + @Before + public void setUp() throws IOException { + File directory = new File("build/data"); + directory.mkdirs(); + this.resource = new FileSystemResource( + File.createTempFile("StaxEventItemWriterBuilderTests", ".xml", directory)); + + this.items = new ArrayList<>(3); + this.items.add(new Foo(1, "two", "three")); + this.items.add(new Foo(4, "five", "six")); + this.items.add(new Foo(7, "eight", "nine")); + + marshaller = new Jaxb2Marshaller(); + ((Jaxb2Marshaller) marshaller).setClassesToBeBound(Foo.class); + } + + @Test(expected = ItemStreamException.class) + public void testOverwriteOutput() throws Exception { + StaxEventItemWriter staxEventItemWriter = new StaxEventItemWriterBuilder() + .name("fooWriter") + .marshaller(marshaller) + .resource(this.resource) + .overwriteOutput(false) + .build(); + + staxEventItemWriter.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + staxEventItemWriter.open(executionContext); + + staxEventItemWriter.write(this.items); + + staxEventItemWriter.update(executionContext); + staxEventItemWriter.close(); + + File output = this.resource.getFile(); + + assertTrue(output.exists()); + + executionContext = new ExecutionContext(); + staxEventItemWriter.open(executionContext); + } + + @Test + public void testDeleteIfEmpty() throws Exception { + ExecutionContext executionContext = new ExecutionContext(); + + StaxEventItemWriter staxEventItemWriter = new StaxEventItemWriterBuilder() + .name("fooWriter") + .resource(this.resource) + .marshaller(this.marshaller) + .shouldDeleteIfEmpty(true) + .build(); + + staxEventItemWriter.afterPropertiesSet(); + staxEventItemWriter.open(executionContext); + staxEventItemWriter.write(Collections.emptyList()); + staxEventItemWriter.update(executionContext); + staxEventItemWriter.close(); + + File file = this.resource.getFile(); + + assertFalse(file.exists()); + } + + @Test + public void testTransactional() { + + StaxEventItemWriter staxEventItemWriter = new StaxEventItemWriterBuilder() + .name("fooWriter") + .resource(this.resource) + .marshaller(this.marshaller) + .transactional(true) + .forceSync(true) + .build(); + + ExecutionContext executionContext = new ExecutionContext(); + + staxEventItemWriter.open(executionContext); + + Object writer = ReflectionTestUtils.getField(staxEventItemWriter, "bufferedWriter"); + + assertTrue(writer instanceof TransactionAwareBufferedWriter); + + assertTrue((Boolean) ReflectionTestUtils.getField(writer, "forceSync")); + } + + @Test + public void testConfiguration() throws Exception { + Map rootElementAttributes = new HashMap<>(); + rootElementAttributes.put("baz", "quix"); + + StaxEventItemWriter staxEventItemWriter = new StaxEventItemWriterBuilder() + .name("fooWriter") + .marshaller(marshaller) + .encoding("UTF-16") + .footerCallback(writer -> { + XMLEventFactory factory = XMLEventFactory.newInstance(); + try { + writer.add(factory.createEndElement("ns", + "https://www.springframework.org/test", + "group")); + } + catch (XMLStreamException e) { + throw new RuntimeException(e); + } + }) + .headerCallback(writer -> { + XMLEventFactory factory = XMLEventFactory.newInstance(); + try { + writer.add(factory.createStartElement("ns", + "https://www.springframework.org/test", + "group")); + } + catch (XMLStreamException e) { + throw new RuntimeException(e); + } + }) + .resource(this.resource) + .rootTagName("foobarred") + .rootElementAttributes(rootElementAttributes) + .saveState(false) + .version("1.1") + .build(); + + staxEventItemWriter.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + staxEventItemWriter.open(executionContext); + + staxEventItemWriter.write(this.items); + + staxEventItemWriter.update(executionContext); + staxEventItemWriter.close(); + + assertEquals(FULL_OUTPUT, getOutputFileContent("UTF-16")); + assertEquals(0, executionContext.size()); + } + + @Test(expected = IllegalArgumentException.class) + public void testMissingMarshallerValidation() { + new StaxEventItemWriterBuilder() + .name("fooWriter") + .build(); + } + + @Test(expected = IllegalArgumentException.class) + public void testMissingNameValidation() { + new StaxEventItemWriterBuilder() + .marshaller(new Jaxb2Marshaller()) + .build(); + } + + private String getOutputFileContent(String encoding) throws IOException { + String value = FileUtils.readFileToString(resource.getFile(), encoding); + value = value.replace("", ""); + return value; + } + + @XmlRootElement(name="item", namespace="https://www.springframework.org/test") + public static class Foo { + private int first; + private String second; + private String third; + + public Foo() {} + + public Foo(int first, String second, String third) { + this.first = first; + this.second = second; + this.third = third; + } + + public int getFirst() { + return first; + } + + public void setFirst(int first) { + this.first = first; + } + + public String getSecond() { + return second; + } + + public void setSecond(String second) { + this.second = second; + } + + public String getThird() { + return third; + } + + public void setThird(String third) { + this.third = third; + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/AbstractEventReaderWrapperTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/AbstractEventReaderWrapperTests.java index e4fcf6ebc0..e1506e7558 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/AbstractEventReaderWrapperTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/AbstractEventReaderWrapperTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/AbstractEventWriterWrapperTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/AbstractEventWriterWrapperTests.java index 6d04500adf..3cdc84b02a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/AbstractEventWriterWrapperTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/AbstractEventWriterWrapperTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/DefaultFragmentEventReaderTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/DefaultFragmentEventReaderTests.java index 39193e9a89..3250477250 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/DefaultFragmentEventReaderTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/DefaultFragmentEventReaderTests.java @@ -1,15 +1,30 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml.stax; import java.util.NoSuchElementException; import javax.xml.stream.XMLEventReader; -import javax.xml.stream.XMLInputFactory; import javax.xml.stream.XMLStreamException; import javax.xml.stream.events.XMLEvent; import junit.framework.TestCase; import org.springframework.batch.item.xml.EventHelper; +import org.springframework.batch.item.xml.StaxUtils; import org.springframework.core.io.ByteArrayResource; import org.springframework.core.io.Resource; @@ -17,6 +32,7 @@ * Tests for {@link DefaultFragmentEventReader}. * * @author Robert Kasanicky + * @author Mahmoud Ben Hassine */ public class DefaultFragmentEventReaderTests extends TestCase { @@ -35,7 +51,7 @@ public class DefaultFragmentEventReaderTests extends TestCase { @Override protected void setUp() throws Exception { Resource input = new ByteArrayResource(xml.getBytes()); - eventReader = XMLInputFactory.newInstance().createXMLEventReader( + eventReader = StaxUtils.createXmlInputFactory().createXMLEventReader( input.getInputStream()); fragmentReader = new DefaultFragmentEventReader(eventReader); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/NoStartEndDocumentWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/NoStartEndDocumentWriterTests.java index 1de835ede1..14e6489112 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/NoStartEndDocumentWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/NoStartEndDocumentWriterTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.item.xml.stax; import javax.xml.stream.XMLEventFactory; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/UnclosedElementCollectingEventWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/UnclosedElementCollectingEventWriterTests.java new file mode 100644 index 0000000000..7576f88133 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/UnclosedElementCollectingEventWriterTests.java @@ -0,0 +1,100 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.xml.stax; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; + +import javax.xml.namespace.QName; +import javax.xml.stream.XMLEventFactory; +import javax.xml.stream.XMLEventWriter; +import javax.xml.stream.events.XMLEvent; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mockito; + +/** + * Tests for {@link UnclosedElementCollectingEventWriter} + * + * @author Jimmy Praet + */ +public class UnclosedElementCollectingEventWriterTests { + + private UnclosedElementCollectingEventWriter writer; + + private XMLEventWriter wrappedWriter; + + private XMLEventFactory eventFactory = XMLEventFactory.newInstance(); + + private QName elementA = new QName("elementA"); + + private QName elementB = new QName("elementB"); + + private QName elementC = new QName("elementC"); + + @Before + public void setUp() throws Exception { + wrappedWriter = mock(XMLEventWriter.class); + writer = new UnclosedElementCollectingEventWriter(wrappedWriter); + } + + @Test + public void testNoUnclosedElements() throws Exception { + writer.add(eventFactory.createStartElement(elementA, null, null)); + writer.add(eventFactory.createEndElement(elementA, null)); + + assertEquals(0, writer.getUnclosedElements().size()); + verify(wrappedWriter, Mockito.times(2)).add(Mockito.any(XMLEvent.class)); + } + + @Test + public void testSingleUnclosedElement() throws Exception { + writer.add(eventFactory.createStartElement(elementA, null, null)); + writer.add(eventFactory.createEndElement(elementA, null)); + writer.add(eventFactory.createStartElement(elementB, null, null)); + + assertEquals(1, writer.getUnclosedElements().size()); + assertEquals(elementB, writer.getUnclosedElements().get(0)); + verify(wrappedWriter, Mockito.times(3)).add(Mockito.any(XMLEvent.class)); + } + + @Test + public void testMultipleUnclosedElements() throws Exception { + writer.add(eventFactory.createStartElement(elementA, null, null)); + writer.add(eventFactory.createStartElement(elementB, null, null)); + writer.add(eventFactory.createStartElement(elementC, null, null)); + writer.add(eventFactory.createEndElement(elementC, null)); + + assertEquals(2, writer.getUnclosedElements().size()); + assertEquals(elementA, writer.getUnclosedElements().get(0)); + assertEquals(elementB, writer.getUnclosedElements().get(1)); + verify(wrappedWriter, Mockito.times(4)).add(Mockito.any(XMLEvent.class)); + } + + @Test + public void testMultipleIdenticalUnclosedElement() throws Exception { + writer.add(eventFactory.createStartElement(elementA, null, null)); + writer.add(eventFactory.createStartElement(elementA, null, null)); + + assertEquals(2, writer.getUnclosedElements().size()); + assertEquals(elementA, writer.getUnclosedElements().get(0)); + assertEquals(elementA, writer.getUnclosedElements().get(1)); + verify(wrappedWriter, Mockito.times(2)).add(Mockito.any(XMLEvent.class)); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/UnopenedElementClosingEventWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/UnopenedElementClosingEventWriterTests.java new file mode 100644 index 0000000000..6fe2acdcec --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/xml/stax/UnopenedElementClosingEventWriterTests.java @@ -0,0 +1,132 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.item.xml.stax; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; + +import java.io.IOException; +import java.io.Writer; +import java.util.LinkedList; +import java.util.List; + +import javax.xml.namespace.QName; +import javax.xml.stream.XMLEventFactory; +import javax.xml.stream.XMLEventWriter; +import javax.xml.stream.events.EndElement; +import javax.xml.stream.events.StartElement; +import javax.xml.stream.events.XMLEvent; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mockito; +import org.springframework.dao.DataAccessResourceFailureException; + +/** + * Tests for {@link UnopenedElementClosingEventWriter} + * + * @author Jimmy Praet + */ +public class UnopenedElementClosingEventWriterTests { + + private UnopenedElementClosingEventWriter writer; + + private XMLEventWriter wrappedWriter; + + private Writer ioWriter; + + private XMLEventFactory eventFactory = XMLEventFactory.newInstance(); + + private List unopenedElements = new LinkedList<>(); + + private QName unopenedA = new QName("http://test/", "unopened-a", "t"); + + private QName unopenedB = new QName("", "unopened-b", ""); + + private QName other = new QName("http://test/", "other", "t"); + + @Before + public void setUp() throws Exception { + wrappedWriter = mock(XMLEventWriter.class); + ioWriter = mock(Writer.class); + unopenedElements.add(unopenedA); + unopenedElements.add(unopenedB); + writer = new UnopenedElementClosingEventWriter(wrappedWriter, ioWriter, unopenedElements); + } + + @Test + public void testEndUnopenedElements() throws Exception { + EndElement endElementB = eventFactory.createEndElement(unopenedB, null); + writer.add(endElementB); + + EndElement endElementA = eventFactory.createEndElement(unopenedA, null); + writer.add(endElementA); + + verify(wrappedWriter, Mockito.never()).add(endElementB); + verify(wrappedWriter, Mockito.never()).add(endElementA); + verify(wrappedWriter, Mockito.times(2)).flush(); + verify(ioWriter).write(""); + verify(ioWriter).write(""); + verify(ioWriter, Mockito.times(2)).flush(); + } + + @Test + public void testEndUnopenedElementRemovesFromList() throws Exception { + EndElement endElement = eventFactory.createEndElement(unopenedB, null); + writer.add(endElement); + + verify(wrappedWriter, Mockito.never()).add(endElement); + verify(wrappedWriter).flush(); + verify(ioWriter).write(""); + verify(ioWriter).flush(); + + StartElement startElement = eventFactory.createStartElement(unopenedB, null, null); + writer.add(startElement); + endElement = eventFactory.createEndElement(unopenedB, null); + writer.add(endElement); + + verify(wrappedWriter).add(startElement); + verify(wrappedWriter).add(endElement); + + // only internal list should be modified + assertEquals(2, unopenedElements.size()); + } + + @Test + public void testOtherEndElement() throws Exception { + EndElement endElement = eventFactory.createEndElement(other, null); + writer.add(endElement); + + verify(wrappedWriter).add(endElement); + } + + @Test + public void testOtherEvent() throws Exception { + XMLEvent event = eventFactory.createCharacters("foo"); + writer.add(event); + + verify(wrappedWriter).add(event); + } + + @Test (expected = DataAccessResourceFailureException.class) + public void testIOException() throws Exception { + EndElement endElementB = eventFactory.createEndElement(unopenedB, null); + Mockito.doThrow(new IOException("Simulated IOException")).when(ioWriter).write(""); + writer.add(endElementB); + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/jsr/item/ItemProcessorAdapterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/jsr/item/ItemProcessorAdapterTests.java new file mode 100644 index 0000000000..ed4e60be13 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/jsr/item/ItemProcessorAdapterTests.java @@ -0,0 +1,55 @@ +/* + * Copyright 2013-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.jsr.item; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.when; + +import javax.batch.api.chunk.ItemProcessor; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +public class ItemProcessorAdapterTests { + + private ItemProcessorAdapter adapter; + @Mock + private ItemProcessor delegate; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + + adapter = new ItemProcessorAdapter<>(delegate); + } + + @Test(expected=IllegalArgumentException.class) + public void testCreateWithNull() { + adapter = new ItemProcessorAdapter<>(null); + } + + @Test + public void testProcess() throws Exception { + String input = "input"; + String output = "output"; + + when(delegate.processItem(input)).thenReturn(output); + + assertEquals(output, adapter.process(input)); + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/jsr/item/ItemReaderAdapterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/jsr/item/ItemReaderAdapterTests.java new file mode 100644 index 0000000000..a944c2975f --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/jsr/item/ItemReaderAdapterTests.java @@ -0,0 +1,186 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.jsr.item; + +import java.io.Serializable; +import java.util.ArrayList; +import java.util.List; +import javax.batch.api.chunk.ItemReader; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamException; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.doThrow; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +public class ItemReaderAdapterTests { + + private ItemReaderAdapter adapter; + @Mock + private ItemReader delegate; + @Mock + private ExecutionContext executionContext; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + + adapter = new ItemReaderAdapter<>(delegate); + adapter.setName("jsrReader"); + } + + @Test(expected=IllegalArgumentException.class) + public void testCreateWithNull() { + adapter = new ItemReaderAdapter<>(null); + } + + @Test + public void testOpen() throws Exception { + when(executionContext.get("jsrReader.reader.checkpoint")).thenReturn("checkpoint"); + + adapter.open(executionContext); + + verify(delegate).open("checkpoint"); + } + + @Test(expected=ItemStreamException.class) + public void testOpenException() throws Exception { + when(executionContext.get("jsrReader.reader.checkpoint")).thenReturn("checkpoint"); + + doThrow(new Exception("expected")).when(delegate).open("checkpoint"); + + adapter.open(executionContext); + } + + @Test + public void testUpdate() throws Exception { + when(delegate.checkpointInfo()).thenReturn("checkpoint"); + + adapter.update(executionContext); + + verify(executionContext).put("jsrReader.reader.checkpoint", "checkpoint"); + } + + @Test(expected=ItemStreamException.class) + public void testUpdateException() throws Exception { + doThrow(new Exception("expected")).when(delegate).checkpointInfo(); + + adapter.update(executionContext); + } + + @Test + public void testClose() throws Exception { + adapter.close(); + + verify(delegate).close(); + } + + @Test(expected=ItemStreamException.class) + public void testCloseException() throws Exception { + doThrow(new Exception("expected")).when(delegate).close(); + + adapter.close(); + } + + @Test + public void testRead() throws Exception { + when(delegate.readItem()).thenReturn("item"); + + assertEquals("item", adapter.read()); + } + + @Test + @SuppressWarnings("serial") + public void testCheckpointChange() throws Exception { + ItemReaderAdapter adapter = new ItemReaderAdapter<>(new ItemReader() { + + private CheckpointContainer container = null; + private List items = new ArrayList() {{ + add("foo"); + add("bar"); + add("baz"); + }}; + + @Override + public Object readItem() throws Exception { + int index = container.getCount(); + + if(index < items.size()) { + container.setCount(index + 1); + return items.get(index); + } else { + return null; + } + } + + @Override + public void open(Serializable checkpoint) throws Exception { + container = new CheckpointContainer(); + } + + @Override + public void close() throws Exception { + } + + @Override + public Serializable checkpointInfo() throws Exception { + return container; + } + }); + + ExecutionContext context = new ExecutionContext(); + + adapter.open(context); + adapter.read(); + adapter.read(); + adapter.update(context); + adapter.read(); + adapter.close(); + + CheckpointContainer container = (CheckpointContainer) context.get("ItemReaderAdapterTests.1.reader.checkpoint"); + assertEquals(2, container.getCount()); + + } + + public static class CheckpointContainer implements Serializable{ + private static final long serialVersionUID = 1L; + private int count; + + public CheckpointContainer() { + count = 0; + } + + public int getCount() { + return count; + } + + public void setCount(int count) { + this.count = count; + } + + @Override + public String toString() { + return "CheckpointContainer has a count of " + count; + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/jsr/item/ItemWriterAdapterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/jsr/item/ItemWriterAdapterTests.java new file mode 100644 index 0000000000..85bc6a2238 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/jsr/item/ItemWriterAdapterTests.java @@ -0,0 +1,183 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.jsr.item; + +import java.io.Serializable; +import java.util.ArrayList; +import java.util.List; +import javax.batch.api.chunk.ItemWriter; + +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamException; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.doThrow; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +public class ItemWriterAdapterTests { + + private ItemWriterAdapter adapter; + @Mock + private ItemWriter delegate; + @Mock + private ExecutionContext executionContext; + + @Before + public void setUp() throws Exception { + MockitoAnnotations.initMocks(this); + + adapter = new ItemWriterAdapter<>(delegate); + adapter.setName("jsrWriter"); + } + + @Test(expected=IllegalArgumentException.class) + public void testCreateWithNull() { + adapter = new ItemWriterAdapter<>(null); + } + + @Test + public void testOpen() throws Exception { + when(executionContext.get("jsrWriter.writer.checkpoint")).thenReturn("checkpoint"); + + adapter.open(executionContext); + + verify(delegate).open("checkpoint"); + } + + @Test(expected=ItemStreamException.class) + public void testOpenException() throws Exception { + when(executionContext.get("jsrWriter.writer.checkpoint")).thenReturn("checkpoint"); + + doThrow(new Exception("expected")).when(delegate).open("checkpoint"); + + adapter.open(executionContext); + } + + @Test + public void testUpdate() throws Exception { + when(delegate.checkpointInfo()).thenReturn("checkpoint"); + + adapter.update(executionContext); + + verify(executionContext).put("jsrWriter.writer.checkpoint", "checkpoint"); + } + + @Test(expected=ItemStreamException.class) + public void testUpdateException() throws Exception { + doThrow(new Exception("expected")).when(delegate).checkpointInfo(); + + adapter.update(executionContext); + } + + @Test + public void testClose() throws Exception { + adapter.close(); + + verify(delegate).close(); + } + + @Test(expected=ItemStreamException.class) + public void testCloseException() throws Exception { + doThrow(new Exception("expected")).when(delegate).close(); + + adapter.close(); + } + + @Test + @SuppressWarnings({"rawtypes", "unchecked"}) + public void testWrite() throws Exception { + List items = new ArrayList(); + + items.add("item1"); + items.add("item2"); + + adapter.write(items); + + verify(delegate).writeItems(items); + } + + @Test + public void testCheckpointChange() throws Exception { + ItemWriterAdapter adapter = new ItemWriterAdapter<>(new ItemWriter() { + + private CheckpointContainer container = null; + + @Override + public void open(Serializable checkpoint) throws Exception { + container = new CheckpointContainer(); + } + + @Override + public void close() throws Exception { + } + + @Override + public void writeItems(List items) throws Exception { + container.setCount(container.getCount() + items.size()); + } + + @Override + public Serializable checkpointInfo() throws Exception { + return container; + } + }); + + ExecutionContext context = new ExecutionContext(); + + List items = new ArrayList<>(); + items.add("foo"); + items.add("bar"); + items.add("baz"); + adapter.open(context); + adapter.write(items); + adapter.update(context); + adapter.write(items); + adapter.close(); + + CheckpointContainer container = (CheckpointContainer) context.get("ItemWriterAdapterTests.1.writer.checkpoint"); + assertEquals(3, container.getCount()); + + } + + public static class CheckpointContainer implements Serializable{ + private static final long serialVersionUID = 1L; + + private int count; + + public CheckpointContainer() { + count = 0; + } + + public int getCount() { + return count; + } + + public void setCount(int count) { + this.count = count; + } + + @Override + public String toString() { + return "CheckpointContainer has a count of " + count; + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/poller/DirectPollerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/poller/DirectPollerTests.java index 27f451d8f3..5470457532 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/poller/DirectPollerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/poller/DirectPollerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -33,7 +33,7 @@ */ public class DirectPollerTests { - private Set repository = new HashSet(); + private Set repository = new HashSet<>(); @Test public void testSimpleSingleThreaded() throws Exception { @@ -42,7 +42,7 @@ public void testSimpleSingleThreaded() throws Exception { @Override public String call() throws Exception { - Set executions = new HashSet(repository); + Set executions = new HashSet<>(repository); if (executions.isEmpty()) { return null; } @@ -67,7 +67,7 @@ public void testTimeUnit() throws Exception { @Override public String call() throws Exception { - Set executions = new HashSet(repository); + Set executions = new HashSet<>(repository); if (executions.isEmpty()) { return null; } @@ -92,7 +92,7 @@ public void testWithError() throws Exception { @Override public String call() throws Exception { - Set executions = new HashSet(repository); + Set executions = new HashSet<>(repository); if (executions.isEmpty()) { return null; } @@ -101,7 +101,7 @@ public String call() throws Exception { }; - Poller poller = new DirectPoller(100L); + Poller poller = new DirectPoller<>(100L); sleepAndCreateStringInBackground(500L); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/AbstractExceptionTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/AbstractExceptionTests.java index a7c8c4e9d6..b913d9a3d7 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/AbstractExceptionTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/AbstractExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/RepeatExceptionTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/RepeatExceptionTests.java index 36841c902c..c2ba952fe0 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/RepeatExceptionTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/RepeatExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/callback/NestedRepeatCallbackTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/callback/NestedRepeatCallbackTests.java index f2adb24b69..b7856f2666 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/callback/NestedRepeatCallbackTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/callback/NestedRepeatCallbackTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/context/RepeatContextCounterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/context/RepeatContextCounterTests.java index 1b3cb0b4e4..ac2792d951 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/context/RepeatContextCounterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/context/RepeatContextCounterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/context/RepeatContextSupportTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/context/RepeatContextSupportTests.java index e5db942913..152bd20928 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/context/RepeatContextSupportTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/context/RepeatContextSupportTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,7 +26,7 @@ */ public class RepeatContextSupportTests extends TestCase { - private List list = new ArrayList(); + private List list = new ArrayList<>(); /** * Test method for {@link org.springframework.batch.repeat.context.RepeatContextSupport#registerDestructionCallback(java.lang.String, java.lang.Runnable)}. diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/context/SynchronizedAttributeAccessorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/context/SynchronizedAttributeAccessorTests.java index ef08153ef1..c081a1eda6 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/context/SynchronizedAttributeAccessorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/context/SynchronizedAttributeAccessorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -76,6 +76,7 @@ public void testEqualsWrongType() { } public void testEqualsSupport() { + @SuppressWarnings("serial") AttributeAccessorSupport another = new AttributeAccessorSupport() { }; accessor.setAttribute("foo", "bar"); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/CompositeExceptionHandlerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/CompositeExceptionHandlerTests.java index 7a1946989f..f1b297c419 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/CompositeExceptionHandlerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/CompositeExceptionHandlerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -39,7 +39,7 @@ public void testNewHandler() throws Throwable { } public void testDelegation() throws Throwable { - final List list = new ArrayList(); + final List list = new ArrayList<>(); handler.setHandlers(new ExceptionHandler[] { new ExceptionHandler() { @Override diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/DefaultExceptionHandlerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/DefaultExceptionHandlerTests.java index b0ba148863..1a003d5f82 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/DefaultExceptionHandlerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/DefaultExceptionHandlerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/LogOrRethrowExceptionHandlerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/LogOrRethrowExceptionHandlerTests.java index 6f668a3186..d61ecbb124 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/LogOrRethrowExceptionHandlerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/LogOrRethrowExceptionHandlerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,9 +20,16 @@ import junit.framework.TestCase; -import org.apache.log4j.Logger; -import org.apache.log4j.SimpleLayout; -import org.apache.log4j.WriterAppender; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.core.Appender; +import org.apache.logging.log4j.core.LoggerContext; +import org.apache.logging.log4j.core.appender.WriterAppender; +import org.apache.logging.log4j.core.config.Configuration; +import org.apache.logging.log4j.core.config.LoggerConfig; +import org.apache.logging.log4j.core.layout.PatternLayout; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + import org.springframework.classify.ClassifierSupport; import org.springframework.batch.repeat.RepeatContext; import org.springframework.batch.repeat.exception.LogOrRethrowExceptionHandler.Level; @@ -38,12 +45,17 @@ public class LogOrRethrowExceptionHandlerTests extends TestCase { @Override protected void setUp() throws Exception { super.setUp(); - Logger logger = Logger.getLogger(LogOrRethrowExceptionHandler.class); - logger.setLevel(org.apache.log4j.Level.DEBUG); + Logger logger = LoggerFactory.getLogger(LogOrRethrowExceptionHandler.class); writer = new StringWriter(); - logger.removeAllAppenders(); - logger.getParent().removeAllAppenders(); - logger.addAppender(new WriterAppender(new SimpleLayout(), writer)); + LoggerContext loggerContext = (LoggerContext) LogManager.getContext(); + Configuration configuration = loggerContext.getConfiguration(); + + LoggerConfig rootLoggerConfig = configuration.getLoggerConfig(logger.getName()); + rootLoggerConfig.getAppenders().forEach((name, appender) -> { + rootLoggerConfig.removeAppender(name); + }); + Appender appender = WriterAppender.createAppender(PatternLayout.createDefaultLayout(), null, writer,"TESTWriter", false, false); + rootLoggerConfig.addAppender(appender, org.apache.logging.log4j.Level.DEBUG, null); } public void testRuntimeException() throws Throwable { @@ -66,6 +78,7 @@ public void testError() throws Throwable { } } + @SuppressWarnings("serial") public void testNotRethrownErrorLevel() throws Throwable { handler.setExceptionClassifier(new ClassifierSupport(Level.RETHROW) { @Override @@ -78,6 +91,7 @@ public Level classify(Throwable throwable) { assertNotNull(writer.toString()); } + @SuppressWarnings("serial") public void testNotRethrownWarnLevel() throws Throwable { handler.setExceptionClassifier(new ClassifierSupport(Level.RETHROW) { @Override @@ -90,6 +104,7 @@ public Level classify(Throwable throwable) { assertNotNull(writer.toString()); } + @SuppressWarnings("serial") public void testNotRethrownDebugLevel() throws Throwable { handler.setExceptionClassifier(new ClassifierSupport(Level.RETHROW) { @Override diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/RethrowOnThresholdExceptionHandlerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/RethrowOnThresholdExceptionHandlerTests.java index 9866dd7ae4..5fade13e88 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/RethrowOnThresholdExceptionHandlerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/RethrowOnThresholdExceptionHandlerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/SimpleLimitExceptionHandlerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/SimpleLimitExceptionHandlerTests.java index fe52192807..fec0a26272 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/SimpleLimitExceptionHandlerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/exception/SimpleLimitExceptionHandlerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -194,6 +194,7 @@ public void testExceptionNotThrownBelowLimit() throws Throwable { handler.setLimit(EXCEPTION_LIMIT); handler.afterPropertiesSet(); + @SuppressWarnings("serial") List throwables = new ArrayList() { { for (int i = 0; i < (EXCEPTION_LIMIT); i++) { @@ -230,6 +231,7 @@ public void testExceptionThrownAboveLimit() throws Throwable { handler.setLimit(EXCEPTION_LIMIT); handler.afterPropertiesSet(); + @SuppressWarnings("serial") List throwables = new ArrayList() { { for (int i = 0; i < (EXCEPTION_LIMIT); i++) { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/interceptor/RepeatOperationsInterceptorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/interceptor/RepeatOperationsInterceptorTests.java index c2cea59d11..aa85662f26 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/interceptor/RepeatOperationsInterceptorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/interceptor/RepeatOperationsInterceptorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -48,7 +48,7 @@ protected void setUp() throws Exception { interceptor = new RepeatOperationsInterceptor(); target = new ServiceImpl(); ProxyFactory factory = new ProxyFactory(RepeatOperations.class.getClassLoader()); - factory.setInterfaces(new Class[] { Service.class }); + factory.setInterfaces(Service.class); factory.setTarget(target); service = (Service) factory.getProxy(); } @@ -67,7 +67,7 @@ public void testCompleteOnFirstInvocation() throws Exception { } public void testSetTemplate() throws Exception { - final List calls = new ArrayList(); + final List calls = new ArrayList<>(); interceptor.setRepeatOperations(new RepeatOperations() { @Override public RepeatStatus iterate(RepeatCallback callback) { @@ -87,7 +87,7 @@ public RepeatStatus iterate(RepeatCallback callback) { } public void testCallbackNotExecuted() throws Exception { - final List calls = new ArrayList(); + final List calls = new ArrayList<>(); interceptor.setRepeatOperations(new RepeatOperations() { @Override public RepeatStatus iterate(RepeatCallback callback) { @@ -101,7 +101,7 @@ public RepeatStatus iterate(RepeatCallback callback) { fail("Expected IllegalStateException"); } catch (IllegalStateException e) { String message = e.getMessage(); - assertTrue("Wrong exception message: "+message, message.toLowerCase().indexOf("no result available")>=0); + assertTrue("Wrong exception message: "+message, message.toLowerCase().contains("no result available")); } assertEquals(1, calls.size()); } @@ -160,7 +160,7 @@ public void testCallbackWithBooleanReturningFalseFirstTime() throws Exception { public void testInterceptorChainWithRetry() throws Exception { ((Advised) service).addAdvice(interceptor); - final List list = new ArrayList(); + final List list = new ArrayList<>(); ((Advised) service).addAdvice(new MethodInterceptor() { @Override public Object invoke(MethodInvocation invocation) throws Throwable { @@ -182,7 +182,7 @@ public void testIllegalMethodInvocationType() throws Throwable { @Override public Method getMethod() { try { - return Object.class.getMethod("toString", new Class[0]); + return Object.class.getMethod("toString"); } catch (Exception e) { throw new RuntimeException(e); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/listener/CompositeRepeatListenerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/listener/CompositeRepeatListenerTests.java index a2f1fce13a..1fe21997b1 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/listener/CompositeRepeatListenerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/listener/CompositeRepeatListenerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -33,7 +33,7 @@ public class CompositeRepeatListenerTests extends TestCase { private CompositeRepeatListener listener = new CompositeRepeatListener(); private RepeatContext context = new RepeatContextSupport(null); - private List list = new ArrayList(); + private List list = new ArrayList<>(); /** * Test method for {@link CompositeRepeatListener#setListeners(RepeatListener[])}. diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/listener/RepeatListenerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/listener/RepeatListenerTests.java index 7b5275a776..1e7979978e 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/listener/RepeatListenerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/listener/RepeatListenerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,7 +35,7 @@ public class RepeatListenerTests extends TestCase { public void testBeforeInterceptors() throws Exception { RepeatTemplate template = new RepeatTemplate(); - final List calls = new ArrayList(); + final List calls = new ArrayList<>(); template.setListeners(new RepeatListener[] { new RepeatListenerSupport() { @Override public void before(RepeatContext context) { @@ -64,7 +64,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { public void testBeforeInterceptorCanVeto() throws Exception { RepeatTemplate template = new RepeatTemplate(); - final List calls = new ArrayList(); + final List calls = new ArrayList<>(); template.registerListener(new RepeatListenerSupport() { @Override public void before(RepeatContext context) { @@ -86,7 +86,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { public void testAfterInterceptors() throws Exception { RepeatTemplate template = new RepeatTemplate(); - final List calls = new ArrayList(); + final List calls = new ArrayList<>(); template.setListeners(new RepeatListener[] { new RepeatListenerSupport() { @Override public void after(RepeatContext context, RepeatStatus result) { @@ -113,7 +113,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { public void testOpenInterceptors() throws Exception { RepeatTemplate template = new RepeatTemplate(); - final List calls = new ArrayList(); + final List calls = new ArrayList<>(); template.setListeners(new RepeatListener[] { new RepeatListenerSupport() { @Override public void open(RepeatContext context) { @@ -139,7 +139,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { public void testSingleOpenInterceptor() throws Exception { RepeatTemplate template = new RepeatTemplate(); - final List calls = new ArrayList(); + final List calls = new ArrayList<>(); template.registerListener(new RepeatListenerSupport() { @Override public void open(RepeatContext context) { @@ -160,7 +160,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { public void testCloseInterceptors() throws Exception { RepeatTemplate template = new RepeatTemplate(); - final List calls = new ArrayList(); + final List calls = new ArrayList<>(); template.setListeners(new RepeatListener[] { new RepeatListenerSupport() { @Override public void close(RepeatContext context) { @@ -188,7 +188,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { public void testOnErrorInterceptors() throws Exception { RepeatTemplate template = new RepeatTemplate(); - final List calls = new ArrayList(); + final List calls = new ArrayList<>(); template.setListeners(new RepeatListener[] { new RepeatListenerSupport() { @Override public void onError(RepeatContext context, Throwable t) { @@ -218,7 +218,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { public void testOnErrorInterceptorsPrecedence() throws Exception { RepeatTemplate template = new RepeatTemplate(); - final List calls = new ArrayList(); + final List calls = new ArrayList<>(); template.setListeners(new RepeatListener[] { new RepeatListenerSupport() { @Override public void after(RepeatContext context, RepeatStatus result) { @@ -250,8 +250,8 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { public void testAsynchronousOnErrorInterceptorsPrecedence() throws Exception { TaskExecutorRepeatTemplate template = new TaskExecutorRepeatTemplate(); template.setTaskExecutor(new SimpleAsyncTaskExecutor()); - final List calls = new ArrayList(); - final List fails = new ArrayList(); + final List calls = new ArrayList<>(); + final List fails = new ArrayList<>(); template.setListeners(new RepeatListener[] { new RepeatListenerSupport() { @Override public void after(RepeatContext context, RepeatStatus result) { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/CompositeCompletionPolicyTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/CompositeCompletionPolicyTests.java index 7959f4177f..a28b60a29a 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/CompositeCompletionPolicyTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/CompositeCompletionPolicyTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/CountingCompletionPolicyTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/CountingCompletionPolicyTests.java index 44a9699eea..f639e0b7de 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/CountingCompletionPolicyTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/CountingCompletionPolicyTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,7 +29,7 @@ public void testDefaultBehaviour() throws Exception { @Override protected int getCount(RepeatContext context) { return 1; - }; + } }; RepeatContext context = policy.start(null); assertTrue(policy.isComplete(context)); @@ -40,7 +40,7 @@ public void testNullResult() throws Exception { @Override protected int getCount(RepeatContext context) { return 1; - }; + } }; policy.setMaxCount(10); RepeatContext context = policy.start(null); @@ -52,7 +52,7 @@ public void testFinishedResult() throws Exception { @Override protected int getCount(RepeatContext context) { return 1; - }; + } }; policy.setMaxCount(10); RepeatContext context = policy.start(null); @@ -66,7 +66,7 @@ public void testDefaultBehaviourWithUpdate() throws Exception { @Override protected int getCount(RepeatContext context) { return count; - }; + } @Override protected int doUpdate(RepeatContext context) { @@ -89,7 +89,7 @@ public void testUpdateNotSavedAcrossSession() throws Exception { @Override protected int getCount(RepeatContext context) { return count; - }; + } @Override protected int doUpdate(RepeatContext context) { @@ -121,7 +121,7 @@ public void testUpdateSavedAcrossSession() throws Exception { @Override protected int getCount(RepeatContext context) { return count; - }; + } @Override protected int doUpdate(RepeatContext context) { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/MockCompletionPolicySupport.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/MockCompletionPolicySupport.java index 5459f4fd6b..8c47b0a536 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/MockCompletionPolicySupport.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/MockCompletionPolicySupport.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/SimpleCompletionPolicyTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/SimpleCompletionPolicyTests.java index f922321753..28501a9989 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/SimpleCompletionPolicyTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/SimpleCompletionPolicyTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/TimeoutCompletionPolicyTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/TimeoutCompletionPolicyTests.java index 08318b1ba1..d6388a8c2b 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/TimeoutCompletionPolicyTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/policy/TimeoutCompletionPolicyTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/AbstractTradeBatchTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/AbstractTradeBatchTests.java index 13e28883c7..8ffd7b025b 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/AbstractTradeBatchTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/AbstractTradeBatchTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -56,7 +56,7 @@ protected static class TradeItemReader extends FlatFileItemReader { protected TradeItemReader(Resource resource) throws Exception { super(); setResource(resource); - DefaultLineMapper mapper = new DefaultLineMapper(); + DefaultLineMapper mapper = new DefaultLineMapper<>(); mapper.setLineTokenizer(new DelimitedLineTokenizer()); mapper.setFieldSetMapper(new TradeMapper()); setLineMapper(mapper); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ChunkedRepeatTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ChunkedRepeatTests.java index a014e5091e..234d4d0b61 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ChunkedRepeatTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ChunkedRepeatTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,11 +16,8 @@ package org.springframework.batch.repeat.support; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - import org.junit.Test; + import org.springframework.batch.item.ItemReader; import org.springframework.batch.repeat.RepeatCallback; import org.springframework.batch.repeat.RepeatContext; @@ -28,6 +25,11 @@ import org.springframework.batch.repeat.callback.NestedRepeatCallback; import org.springframework.batch.repeat.policy.SimpleCompletionPolicy; import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.lang.Nullable; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; /** * Test various approaches to chunking of a batch. Not really a unit test, but @@ -43,14 +45,12 @@ public class ChunkedRepeatTests extends AbstractTradeBatchTests { /** * Chunking using a dedicated TerminationPolicy. Transactions would be laid * on at the level of chunkTemplate.execute() or the surrounding callback. - * - * @throws Exception */ @Test public void testChunkedBatchWithTerminationPolicy() throws Exception { RepeatTemplate repeatTemplate = new RepeatTemplate(); - final RepeatCallback callback = new ItemReaderRepeatCallback(provider, processor); + final RepeatCallback callback = new ItemReaderRepeatCallback<>(provider, processor); final RepeatTemplate chunkTemplate = new RepeatTemplate(); // The policy is resettable so we only have to resolve this dependency @@ -79,14 +79,12 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { /** * Chunking with an asynchronous taskExecutor in the chunks. Transactions * have to be at the level of the business callback. - * - * @throws Exception */ @Test public void testAsynchronousChunkedBatchWithCompletionPolicy() throws Exception { RepeatTemplate repeatTemplate = new RepeatTemplate(); - final RepeatCallback callback = new ItemReaderRepeatCallback(provider, processor); + final RepeatCallback callback = new ItemReaderRepeatCallback<>(provider, processor); final TaskExecutorRepeatTemplate chunkTemplate = new TaskExecutorRepeatTemplate(); // The policy is resettable so we only have to resolve this dependency @@ -113,8 +111,6 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { /** * Explicit chunking of input data. Transactions would be laid on at the * level of template.execute(). - * - * @throws Exception */ @Test public void testChunksWithTruncatedItemProvider() throws Exception { @@ -150,7 +146,6 @@ void increment() { count++; } } - ; final Chunker chunker = new Chunker(); @@ -159,7 +154,8 @@ void increment() { ItemReader truncated = new ItemReader() { int count = 0; - @Override + @Nullable + @Override public Trade read() throws Exception { if (count++ < 2) return provider.read(); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ItemReaderRepeatCallback.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ItemReaderRepeatCallback.java index e53a0d4442..aaafcb8a68 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ItemReaderRepeatCallback.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ItemReaderRepeatCallback.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/RepeatSynchronizationManagerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/RepeatSynchronizationManagerTests.java index eaa702cc1a..1d17e355b8 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/RepeatSynchronizationManagerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/RepeatSynchronizationManagerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ResultHolderResultQueueTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ResultHolderResultQueueTests.java index 921e2b151c..67bcd38539 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ResultHolderResultQueueTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ResultHolderResultQueueTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.repeat.support; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/SimpleRepeatTemplateTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/SimpleRepeatTemplateTests.java index 6d0e04fe54..53c8765baa 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/SimpleRepeatTemplateTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/SimpleRepeatTemplateTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -58,7 +58,7 @@ public RepeatTemplate getRepeatTemplate() { @Test public void testExecute() throws Exception { - template.iterate(new ItemReaderRepeatCallback(provider, processor)); + template.iterate(new ItemReaderRepeatCallback<>(provider, processor)); assertEquals(NUMBER_OF_ITEMS, processor.count); } @@ -72,7 +72,7 @@ public void testEarlyCompletionWithPolicy() throws Exception { template.setCompletionPolicy(new SimpleCompletionPolicy(2)); - template.iterate(new ItemReaderRepeatCallback(provider, processor)); + template.iterate(new ItemReaderRepeatCallback<>(provider, processor)); assertEquals(2, processor.count); @@ -113,7 +113,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { @Test public void testContextClosedOnNormalCompletion() throws Exception { - final List list = new ArrayList(); + final List list = new ArrayList<>(); final RepeatContext context = new RepeatContextSupport(null) { @Override @@ -149,7 +149,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { @Test public void testContextClosedOnAbnormalCompletion() throws Exception { - final List list = new ArrayList(); + final List list = new ArrayList<>(); final RepeatContext context = new RepeatContextSupport(null) { @Override @@ -191,7 +191,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { @Test public void testExceptionHandlerCalledOnAbnormalCompletion() throws Exception { - final List list = new ArrayList(); + final List list = new ArrayList<>(); template.setExceptionHandler(new ExceptionHandler() { @Override @@ -358,7 +358,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { */ @Test public void testResult() throws Exception { - RepeatStatus result = template.iterate(new ItemReaderRepeatCallback(provider, processor)); + RepeatStatus result = template.iterate(new ItemReaderRepeatCallback<>(provider, processor)); assertEquals(NUMBER_OF_ITEMS, processor.count); // We are complete - do not expect to be called again assertFalse(result.isContinuable()); @@ -437,6 +437,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { @Test public void testExceptionUnwrapping() { + @SuppressWarnings("serial") class TestException extends Exception { TestException(String msg) { super(msg); diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplateAsynchronousTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplateAsynchronousTests.java index 1547e2ca1f..9e3fd57285 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplateAsynchronousTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplateAsynchronousTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -149,7 +149,7 @@ public RepeatStatus doInIteration(RepeatContext context) throws Exception { public void testMultiThreadAsynchronousExecution() throws Exception { final String threadName = Thread.currentThread().getName(); - final Set threadNames = new HashSet(); + final Set threadNames = new HashSet<>(); final RepeatCallback callback = new RepeatCallback() { @Override @@ -184,8 +184,8 @@ public void testThrottleLimit() throws Exception { template.setThrottleLimit(throttleLimit); final String threadName = Thread.currentThread().getName(); - final Set threadNames = new HashSet(); - final List items = new ArrayList(); + final Set threadNames = new HashSet<>(); + final List items = new ArrayList<>(); final RepeatCallback callback = new RepeatCallback() { @Override @@ -234,7 +234,7 @@ public void testSingleThreadAsynchronousExecution() throws Exception { jobTemplate.setTaskExecutor(taskExecutor); final String threadName = Thread.currentThread().getName(); - final Set threadNames = new HashSet(); + final Set threadNames = new HashSet<>(); final RepeatCallback stepCallback = new ItemReaderRepeatCallback(provider, processor) { @Override diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplateTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplateTests.java index a0b58f723d..355b174a28 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplateTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/TaskExecutorRepeatTemplateTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ThrottleLimitResultQueueTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ThrottleLimitResultQueueTests.java index 527063a12f..ef45e3ed53 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ThrottleLimitResultQueueTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/ThrottleLimitResultQueueTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,7 +31,7 @@ */ public class ThrottleLimitResultQueueTests { - private ThrottleLimitResultQueue queue = new ThrottleLimitResultQueue(1); + private ThrottleLimitResultQueue queue = new ThrottleLimitResultQueue<>(1); @Test public void testPutTake() throws Exception { diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/Trade.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/Trade.java index e749868e7d..1f8e8f0bad 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/Trade.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/repeat/support/Trade.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/AbstractExceptionTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/AbstractExceptionTests.java index e6343a9cb9..7afd46a3fd 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/AbstractExceptionTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/AbstractExceptionTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/AnnotationMethodResolverTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/AnnotationMethodResolverTests.java index 2bbeab744e..b9efef4032 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/AnnotationMethodResolverTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/AnnotationMethodResolverTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -74,7 +74,6 @@ public String lowerCase(String s) { } - @SuppressWarnings("unused") private static class MultipleAnnotationTestBean { @TestAnnotation diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DatabaseTypeIntegrationTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DatabaseTypeIntegrationTests.java index 5c6baec0db..6de1279dca 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DatabaseTypeIntegrationTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DatabaseTypeIntegrationTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2009 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,11 +16,11 @@ package org.springframework.batch.support; -import static org.junit.Assert.assertEquals; +import org.junit.Test; import javax.sql.DataSource; -import org.junit.Test; +import static org.junit.Assert.assertEquals; /** * @author Dave Syer @@ -31,7 +31,7 @@ public class DatabaseTypeIntegrationTests { @Test public void testH2() throws Exception { DataSource dataSource = DatabaseTypeTestUtils.getDataSource(org.h2.Driver.class, - "jdbc:h2:file:target/data/sample"); + "jdbc:h2:file:./build/data/sample"); assertEquals(DatabaseType.H2, DatabaseType.fromMetaData(dataSource)); dataSource.getConnection(); } @@ -39,7 +39,7 @@ public void testH2() throws Exception { @Test public void testDerby() throws Exception { DataSource dataSource = DatabaseTypeTestUtils.getDataSource(org.apache.derby.jdbc.EmbeddedDriver.class, - "jdbc:derby:derby-home/test;create=true", "sa", ""); + "jdbc:derby:./build/derby-home/test;create=true", "sa", ""); assertEquals(DatabaseType.DERBY, DatabaseType.fromMetaData(dataSource)); dataSource.getConnection(); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DatabaseTypeTestUtils.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DatabaseTypeTestUtils.java index 5f866cbbdd..c78e7915d3 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DatabaseTypeTestUtils.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DatabaseTypeTestUtils.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,15 +15,14 @@ */ package org.springframework.batch.support; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - import java.sql.Connection; import java.sql.DatabaseMetaData; - import javax.sql.DataSource; -import org.apache.commons.dbcp.BasicDataSource; +import org.apache.commons.dbcp2.BasicDataSource; + +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; /** * @author Dave Syer diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DatabaseTypeTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DatabaseTypeTests.java index e7e1ae5edb..2184260851 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DatabaseTypeTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DatabaseTypeTests.java @@ -1,22 +1,40 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.support; +import org.junit.Test; +import org.springframework.jdbc.support.MetaDataAccessException; + +import javax.sql.DataSource; + import static org.junit.Assert.assertEquals; import static org.springframework.batch.support.DatabaseType.DB2; +import static org.springframework.batch.support.DatabaseType.DB2VSE; import static org.springframework.batch.support.DatabaseType.DB2ZOS; +import static org.springframework.batch.support.DatabaseType.DB2AS400; import static org.springframework.batch.support.DatabaseType.DERBY; import static org.springframework.batch.support.DatabaseType.HSQL; import static org.springframework.batch.support.DatabaseType.MYSQL; import static org.springframework.batch.support.DatabaseType.ORACLE; import static org.springframework.batch.support.DatabaseType.POSTGRES; +import static org.springframework.batch.support.DatabaseType.SQLITE; import static org.springframework.batch.support.DatabaseType.SQLSERVER; import static org.springframework.batch.support.DatabaseType.SYBASE; import static org.springframework.batch.support.DatabaseType.fromProductName; -import javax.sql.DataSource; - -import org.junit.Test; -import org.springframework.jdbc.support.MetaDataAccessException; - /** * * @author Lucas Ward @@ -29,13 +47,16 @@ public class DatabaseTypeTests { public void testFromProductName() { assertEquals(DERBY, fromProductName("Apache Derby")); assertEquals(DB2, fromProductName("DB2")); + assertEquals(DB2VSE, fromProductName("DB2VSE")); assertEquals(DB2ZOS, fromProductName("DB2ZOS")); + assertEquals(DB2AS400, fromProductName("DB2AS400")); assertEquals(HSQL, fromProductName("HSQL Database Engine")); assertEquals(SQLSERVER, fromProductName("Microsoft SQL Server")); assertEquals(MYSQL, fromProductName("MySQL")); assertEquals(ORACLE, fromProductName("Oracle")); assertEquals(POSTGRES, fromProductName("PostgreSQL")); assertEquals(SYBASE, fromProductName("Sybase")); + assertEquals(SQLITE, fromProductName("SQLite")); } @Test(expected = IllegalArgumentException.class) @@ -52,14 +73,38 @@ public void testFromMetaDataForDerby() throws Exception { @Test public void testFromMetaDataForDB2() throws Exception { - DataSource ds = DatabaseTypeTestUtils.getMockDataSource("DB2/Linux"); - assertEquals(DB2, DatabaseType.fromMetaData(ds)); + DataSource oldDs = DatabaseTypeTestUtils.getMockDataSource("DB2/Linux", "SQL0901"); + assertEquals(DB2, DatabaseType.fromMetaData(oldDs)); + + DataSource newDs = DatabaseTypeTestUtils.getMockDataSource("DB2/NT", "SQL0901"); + assertEquals(DB2, DatabaseType.fromMetaData(newDs)); + } + + @Test + public void testFromMetaDataForDB2VSE() throws Exception { + DataSource ds = DatabaseTypeTestUtils.getMockDataSource("DB2 for DB2 for z/OS VUE", "ARI08015"); + assertEquals(DB2VSE, DatabaseType.fromMetaData(ds)); } @Test public void testFromMetaDataForDB2ZOS() throws Exception { - DataSource ds = DatabaseTypeTestUtils.getMockDataSource("DB2", "DSN08015"); - assertEquals(DB2ZOS, DatabaseType.fromMetaData(ds)); + DataSource oldDs = DatabaseTypeTestUtils.getMockDataSource("DB2", "DSN08015"); + assertEquals(DB2ZOS, DatabaseType.fromMetaData(oldDs)); + + DataSource newDs = DatabaseTypeTestUtils.getMockDataSource("DB2 for DB2 UDB for z/OS", "DSN08015"); + assertEquals(DB2ZOS, DatabaseType.fromMetaData(newDs)); + } + + @Test + public void testFromMetaDataForDB2AS400() throws Exception { + DataSource toolboxDs = DatabaseTypeTestUtils.getMockDataSource("DB2 UDB for AS/400", "07.01.0000 V7R1m0"); + assertEquals(DB2AS400, DatabaseType.fromMetaData(toolboxDs)); + + DataSource nativeDs = DatabaseTypeTestUtils.getMockDataSource("DB2 UDB for AS/400", "V7R1M0"); + assertEquals(DB2AS400, DatabaseType.fromMetaData(nativeDs)); + + DataSource prdidDs = DatabaseTypeTestUtils.getMockDataSource("DB2 UDB for AS/400", "QSQ07010"); + assertEquals(DB2AS400, DatabaseType.fromMetaData(prdidDs)); } @Test diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DefaultPropertEditorRegistrarTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DefaultPropertEditorRegistrarTests.java deleted file mode 100644 index 299ec80503..0000000000 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DefaultPropertEditorRegistrarTests.java +++ /dev/null @@ -1,89 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.support; - -import static org.junit.Assert.assertEquals; - -import java.util.Collections; -import java.util.Properties; - -import org.junit.Test; -import org.springframework.beans.BeanWrapperImpl; -import org.springframework.beans.MutablePropertyValues; -import org.springframework.beans.propertyeditors.CustomNumberEditor; -import org.springframework.beans.propertyeditors.PropertiesEditor; - -public class DefaultPropertEditorRegistrarTests { - - @Test - public void testIntArray() throws Exception { - DefaultPropertyEditorRegistrar mapper = new DefaultPropertyEditorRegistrar(); - BeanWithIntArray result = new BeanWithIntArray(); - mapper.setCustomEditors(Collections.singletonMap(int[].class, new IntArrayPropertyEditor())); - BeanWrapperImpl wrapper = new BeanWrapperImpl(result); - mapper.registerCustomEditors(wrapper); - PropertiesEditor editor = new PropertiesEditor(); - editor.setAsText("numbers=1,2,3, 4"); - Properties props = (Properties) editor.getValue(); - wrapper.setPropertyValues(props); - assertEquals(4, result.numbers[3]); - } - - @Test(expected = IllegalArgumentException.class) - public void testSetCustomEditorsWithInvalidTypeName() throws Exception { - - DefaultPropertyEditorRegistrar mapper = new DefaultPropertyEditorRegistrar(); - mapper.setCustomEditors(Collections.singletonMap("FOO", new CustomNumberEditor(Long.class, true))); - } - - @Test - public void testSetCustomEditorsWithStringTypeName() throws Exception { - - DefaultPropertyEditorRegistrar mapper = new DefaultPropertyEditorRegistrar(); - mapper.setCustomEditors(Collections.singletonMap("java.lang.Long", new CustomNumberEditor(Long.class, true))); - BeanWithIntArray result = new BeanWithIntArray(); - BeanWrapperImpl wrapper = new BeanWrapperImpl(result); - mapper.registerCustomEditors(wrapper); - wrapper.setPropertyValues(new MutablePropertyValues(Collections.singletonMap("number", "123"))); - assertEquals(123L, result.number); - - } - - @Test(expected = IllegalArgumentException.class) - public void testSetCustomEditorsWithInvalidType() throws Exception { - - DefaultPropertyEditorRegistrar mapper = new DefaultPropertyEditorRegistrar(); - mapper.setCustomEditors(Collections.singletonMap(new Object(), new CustomNumberEditor(Long.class, true))); - } - - @SuppressWarnings("unused") - private static class BeanWithIntArray { - private int[] numbers; - - private long number; - - - public void setNumbers(int[] numbers) { - this.numbers = numbers; - } - - public void setNumber(long number) { - this.number = number; - } - } - -} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DefaultPropertyEditorRegistrarTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DefaultPropertyEditorRegistrarTests.java new file mode 100644 index 0000000000..29420a88c0 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/DefaultPropertyEditorRegistrarTests.java @@ -0,0 +1,89 @@ +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.support; + +import static org.junit.Assert.assertEquals; + +import java.util.Collections; +import java.util.Properties; + +import org.junit.Test; +import org.springframework.beans.BeanWrapperImpl; +import org.springframework.beans.MutablePropertyValues; +import org.springframework.beans.propertyeditors.CustomNumberEditor; +import org.springframework.beans.propertyeditors.PropertiesEditor; + +public class DefaultPropertyEditorRegistrarTests { + + @Test + public void testIntArray() throws Exception { + DefaultPropertyEditorRegistrar mapper = new DefaultPropertyEditorRegistrar(); + BeanWithIntArray result = new BeanWithIntArray(); + mapper.setCustomEditors(Collections.singletonMap(int[].class, new IntArrayPropertyEditor())); + BeanWrapperImpl wrapper = new BeanWrapperImpl(result); + mapper.registerCustomEditors(wrapper); + PropertiesEditor editor = new PropertiesEditor(); + editor.setAsText("numbers=1,2,3, 4"); + Properties props = (Properties) editor.getValue(); + wrapper.setPropertyValues(props); + assertEquals(4, result.numbers[3]); + } + + @Test(expected = IllegalArgumentException.class) + public void testSetCustomEditorsWithInvalidTypeName() throws Exception { + + DefaultPropertyEditorRegistrar mapper = new DefaultPropertyEditorRegistrar(); + mapper.setCustomEditors(Collections.singletonMap("FOO", new CustomNumberEditor(Long.class, true))); + } + + @Test + public void testSetCustomEditorsWithStringTypeName() throws Exception { + + DefaultPropertyEditorRegistrar mapper = new DefaultPropertyEditorRegistrar(); + mapper.setCustomEditors(Collections.singletonMap("java.lang.Long", new CustomNumberEditor(Long.class, true))); + BeanWithIntArray result = new BeanWithIntArray(); + BeanWrapperImpl wrapper = new BeanWrapperImpl(result); + mapper.registerCustomEditors(wrapper); + wrapper.setPropertyValues(new MutablePropertyValues(Collections.singletonMap("number", "123"))); + assertEquals(123L, result.number); + + } + + @Test(expected = IllegalArgumentException.class) + public void testSetCustomEditorsWithInvalidType() throws Exception { + + DefaultPropertyEditorRegistrar mapper = new DefaultPropertyEditorRegistrar(); + mapper.setCustomEditors(Collections.singletonMap(new Object(), new CustomNumberEditor(Long.class, true))); + } + + @SuppressWarnings("unused") + private static class BeanWithIntArray { + private int[] numbers; + + private long number; + + + public void setNumbers(int[] numbers) { + this.numbers = numbers; + } + + public void setNumber(long number) { + this.number = number; + } + } + +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/LastModifiedResourceComparatorTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/LastModifiedResourceComparatorTests.java index 194027be41..63a809f25d 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/LastModifiedResourceComparatorTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/LastModifiedResourceComparatorTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,15 +15,15 @@ */ package org.springframework.batch.support; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; +import org.apache.commons.io.FileUtils; +import org.junit.Test; +import org.springframework.core.io.FileSystemResource; import java.io.File; import java.io.IOException; -import org.apache.commons.io.FileUtils; -import org.junit.Test; -import org.springframework.core.io.FileSystemResource; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; /** * @author Dave Syer @@ -31,6 +31,8 @@ */ public class LastModifiedResourceComparatorTests { + public static final String FILE_PATH = "src/test/resources/org/springframework/batch/support/existing.txt"; + private LastModifiedResourceComparator comparator = new LastModifiedResourceComparator(); @Test(expected = IllegalArgumentException.class) @@ -40,12 +42,12 @@ public void testCompareTwoNonExistent() { @Test(expected = IllegalArgumentException.class) public void testCompareOneNonExistent() { - comparator.compare(new FileSystemResource("pom.xml"), new FileSystemResource("crap")); + comparator.compare(new FileSystemResource(FILE_PATH), new FileSystemResource("crap")); } @Test public void testCompareSame() { - assertEquals(0, comparator.compare(new FileSystemResource("pom.xml"), new FileSystemResource("pom.xml"))); + assertEquals(0, comparator.compare(new FileSystemResource(FILE_PATH), new FileSystemResource(FILE_PATH))); } @Test @@ -53,24 +55,25 @@ public void testCompareNewWithOld() throws IOException { File temp = File.createTempFile(getClass().getSimpleName(), ".txt"); temp.deleteOnExit(); assertTrue(temp.exists()); - assertEquals(1, comparator.compare(new FileSystemResource(temp), new FileSystemResource("pom.xml"))); + assertEquals(1, comparator.compare(new FileSystemResource(temp), new FileSystemResource(FILE_PATH))); } @Test public void testCompareNewWithOldAfterCopy() throws Exception { - File temp1 = new File("target/temp1.txt"); - File temp2 = new File("target/temp2.txt"); - if (temp1.exists()) temp1.delete(); - if (temp2.exists()) temp2.delete(); - temp1.getParentFile().mkdirs(); - temp2.createNewFile(); + File temp1 = new File("build/temp1.txt"); + File temp2 = new File("build/temp2.txt"); + if (temp1.exists()) temp1.delete(); + if (temp2.exists()) temp2.delete(); + temp1.getParentFile().mkdirs(); + temp2.createNewFile(); assertTrue(!temp1.exists() && temp2.exists()); - // For Linux sleep here otherwise files show same - // modified date - Thread.sleep(1000); + // For Linux sleep here otherwise files show same + // modified date + Thread.sleep(1000); // Need to explicitly ask not to preserve the last modified date when we // copy... - FileUtils.copyFile(new File("pom.xml"), temp1, false); + + FileUtils.copyFile(new FileSystemResource(FILE_PATH).getFile(), temp1, false); assertEquals(1, comparator.compare(new FileSystemResource(temp1), new FileSystemResource(temp2))); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/MapSerializationUtilsTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/MapSerializationUtilsTests.java deleted file mode 100644 index e9d6e161ce..0000000000 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/MapSerializationUtilsTests.java +++ /dev/null @@ -1,54 +0,0 @@ -/* - * Copyright 2006-2010 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.support; - -import static org.junit.Assert.assertEquals; - -import java.util.Map; -import java.util.concurrent.ConcurrentHashMap; - -import org.junit.Test; - -/** - * @author Dave Syer - * - */ -public class MapSerializationUtilsTests { - - private Map map = new ConcurrentHashMap(); - - @Test - public void testCycle() throws Exception { - map.put("foo.bar.spam", 123); - Map result = getCopy(map); - assertEquals(map, result); - } - - @Test - public void testMultipleCycles() throws Exception { - map.put("foo.bar.spam", 123); - for (int i = 0; i < 1000; i++) { - Map result = getCopy(map); - assertEquals(map, result); - } - } - - @SuppressWarnings("unchecked") - private Map getCopy(Map map) { - return (Map) SerializationUtils.deserialize(SerializationUtils.serialize(map)); - } - -} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/PatternMatcherTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/PatternMatcherTests.java index 7debe99e65..f6fc338d18 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/PatternMatcherTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/PatternMatcherTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,9 +15,9 @@ */ package org.springframework.batch.support; -import static junit.framework.Assert.assertEquals; -import static junit.framework.Assert.assertFalse; -import static junit.framework.Assert.assertTrue; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; import java.util.HashMap; import java.util.Map; @@ -30,14 +30,14 @@ */ public class PatternMatcherTests { - private static Map map = new HashMap(); + private static Map map = new HashMap<>(); static { map.put("an*", 3); map.put("a*", 2); map.put("big*", 4); } - private static Map defaultMap = new HashMap(); + private static Map defaultMap = new HashMap<>(); static { defaultMap.put("an", 3); defaultMap.put("a", 2); @@ -103,36 +103,36 @@ public void testMatchStarNo() { @Test public void testMatchPrefixSubsumed() { - assertEquals(2, new PatternMatcher(map).match("apple").intValue()); + assertEquals(2, new PatternMatcher<>(map).match("apple").intValue()); } @Test public void testMatchPrefixSubsuming() { - assertEquals(3, new PatternMatcher(map).match("animal").intValue()); + assertEquals(3, new PatternMatcher<>(map).match("animal").intValue()); } @Test public void testMatchPrefixUnrelated() { - assertEquals(4, new PatternMatcher(map).match("biggest").intValue()); + assertEquals(4, new PatternMatcher<>(map).match("biggest").intValue()); } @Test(expected = IllegalStateException.class) public void testMatchPrefixNoMatch() { - new PatternMatcher(map).match("bat"); + new PatternMatcher<>(map).match("bat"); } @Test public void testMatchPrefixDefaultValueUnrelated() { - assertEquals(5, new PatternMatcher(defaultMap).match("biggest").intValue()); + assertEquals(5, new PatternMatcher<>(defaultMap).match("biggest").intValue()); } @Test public void testMatchPrefixDefaultValueEmptyString() { - assertEquals(1, new PatternMatcher(defaultMap).match("").intValue()); + assertEquals(1, new PatternMatcher<>(defaultMap).match("").intValue()); } @Test public void testMatchPrefixDefaultValueNoMatch() { - assertEquals(1, new PatternMatcher(defaultMap).match("bat").intValue()); + assertEquals(1, new PatternMatcher<>(defaultMap).match("bat").intValue()); } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/PropertiesConverterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/PropertiesConverterTests.java index 26743d44fc..ecd638cbb0 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/PropertiesConverterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/PropertiesConverterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/ReflectionUtilsTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/ReflectionUtilsTests.java new file mode 100644 index 0000000000..89e84db629 --- /dev/null +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/ReflectionUtilsTests.java @@ -0,0 +1,96 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.support; + +import org.junit.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.transaction.annotation.Transactional; + +import java.lang.reflect.Method; +import java.util.Iterator; +import java.util.Set; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +/** + * @author Michael Minella + * @since 2.2.6 + */ +public class ReflectionUtilsTests { + + @Test + public void testFindAnnotatedMethod() { + Set methods = ReflectionUtils.findMethod(AnnotatedClass.class, Transactional.class); + assertEquals(1, methods.size()); + assertEquals("toString", methods.iterator().next().getName()); + } + + @Test + public void testFindNoAnnotatedMethod() { + Set methods = ReflectionUtils.findMethod(AnnotatedClass.class, Autowired.class); + assertEquals(0, methods.size()); + } + + @Test + public void testFindAnnotatedMethodHierarchy() { + Set methods = ReflectionUtils.findMethod(AnnotatedSubClass.class, Transactional.class); + assertEquals(2, methods.size()); + + boolean toStringFound = false; + boolean methodOneFound = false; + + Iterator iterator = methods.iterator(); + + String name = iterator.next().getName(); + + if(name.equals("toString")) { + toStringFound = true; + } else if(name.equals("methodOne")) { + methodOneFound = true; + } + + name = iterator.next().getName(); + + if(name.equals("toString")) { + toStringFound = true; + } else if(name.equals("methodOne")) { + methodOneFound = true; + } + + assertTrue(toStringFound && methodOneFound); + } + + public static class AnnotatedClass { + + public void methodOne() { + System.err.println("This is method 1"); + } + + @Transactional + public String toString() { + return "AnnotatedClass"; + } + } + + public static class AnnotatedSubClass extends AnnotatedClass { + + @Transactional + public void methodOne() { + System.err.println("This is method 1 in the sub class"); + } + } +} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/SerializationUtilsTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/SerializationUtilsTests.java deleted file mode 100644 index 2fd8da7352..0000000000 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/SerializationUtilsTests.java +++ /dev/null @@ -1,69 +0,0 @@ -/* - * Copyright 2006-2010 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.support; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertNull; - -import java.math.BigInteger; - -import org.junit.Test; - -/** - * Static utility to help with serialization. - * - * @author Dave Syer - * - */ -public class SerializationUtilsTests { - - private static BigInteger FOO = new BigInteger( - "-97029424235490125267223648383278313796609415534328015655051436753861088839708112925637575585166033560096810615697574744209306031461371833798723505120163874786203211176873686513374052845353833564048"); - - @Test - public void testSerializeCycleSunnyDay() throws Exception { - assertEquals("foo", SerializationUtils.deserialize(SerializationUtils.serialize("foo"))); - } - - @Test(expected = IllegalStateException.class) - public void testDeserializeUndefined() throws Exception { - byte[] bytes = FOO.toByteArray(); - Object foo = SerializationUtils.deserialize(bytes); - assertNotNull(foo); - } - - @Test(expected = IllegalArgumentException.class) - public void testSerializeNonSerializable() throws Exception { - SerializationUtils.serialize(new Object()); - } - - @Test(expected = IllegalArgumentException.class) - public void testDeserializeNonSerializable() throws Exception { - SerializationUtils.deserialize("foo".getBytes()); - } - - @Test - public void testSerializeNull() throws Exception { - assertNull(SerializationUtils.serialize(null)); - } - - @Test - public void testDeserializeNull() throws Exception { - assertNull(SerializationUtils.deserialize(null)); - } - -} diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/SimpleMethodInvokerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/SimpleMethodInvokerTests.java index 39ed0d1245..f298784747 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/SimpleMethodInvokerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/SimpleMethodInvokerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,7 +21,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,16 +31,17 @@ */ package org.springframework.batch.support; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - import java.lang.reflect.Method; import org.junit.Before; import org.junit.Test; + import org.springframework.util.Assert; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + /** * @author Lucas Ward * @@ -137,7 +138,7 @@ public void beforeWithTooManyArguments(String value, int someInt){ } public void argumentTest(Object object){ - Assert.notNull(object); + Assert.notNull(object, "Object must not be null"); argumentTestCalled = true; } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/SystemPropertyInitializerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/SystemPropertyInitializerTests.java index 81234545af..2a0c2cd23c 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/SystemPropertyInitializerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/SystemPropertyInitializerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/ConcurrentTransactionAwareProxyTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/ConcurrentTransactionAwareProxyTests.java index 485b271f7e..9ebc41cb8c 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/ConcurrentTransactionAwareProxyTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/ConcurrentTransactionAwareProxyTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -66,7 +66,7 @@ public class ConcurrentTransactionAwareProxyTests { @Before public void init() { executor = Executors.newFixedThreadPool(outerMax); - completionService = new ExecutorCompletionService>(executor); + completionService = new ExecutorCompletionService<>(executor); } @After @@ -125,9 +125,9 @@ public void testConcurrentTransactionalMap() throws Exception { @Test public void testTransactionalContains() throws Exception { final Map> map = TransactionAwareProxyFactory.createAppendOnlyTransactionalMap(); - boolean result = (Boolean) new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + boolean result = new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Boolean doInTransaction(TransactionStatus status) { return map.containsKey("foo"); } }); @@ -142,7 +142,7 @@ private void testSet(final Set set) throws Exception { completionService.submit(new Callable>() { @Override public List call() throws Exception { - List list = new ArrayList(); + List list = new ArrayList<>(); for (int i = 0; i < innerMax; i++) { String value = count + "bar" + i; saveInSetAndAssert(set, value); @@ -170,7 +170,7 @@ private void testList(final List list, final boolean mutate) throws Exce completionService.submit(new Callable>() { @Override public List call() throws Exception { - List result = new ArrayList(); + List result = new ArrayList<>(); for (int i = 0; i < innerMax; i++) { String value = "bar" + i; saveInListAndAssert(list, value); @@ -210,7 +210,7 @@ private void testMap(final Map> map) throws Exception completionService.submit(new Callable>() { @Override public List call() throws Exception { - List list = new ArrayList(); + List list = new ArrayList<>(); for (int i = 0; i < innerMax; i++) { String value = "bar" + i; list.add(saveInMapAndAssert(map, id, value).get("foo")); @@ -230,9 +230,9 @@ public List call() throws Exception { private String saveInSetAndAssert(final Set set, final String value) { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { set.add(value); return null; } @@ -246,9 +246,9 @@ public Object doInTransaction(TransactionStatus status) { private String saveInListAndAssert(final List list, final String value) { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { list.add(value); return null; } @@ -263,11 +263,11 @@ public Object doInTransaction(TransactionStatus status) { private Map saveInMapAndAssert(final Map> map, final Long id, final String value) { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { if (!map.containsKey(id)) { - map.put(id, new HashMap()); + map.put(id, new HashMap<>()); } map.get(id).put("foo", value); return null; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/ResourcelessTransactionManagerTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/ResourcelessTransactionManagerTests.java index d9ebdbb55b..2405689870 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/ResourcelessTransactionManagerTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/ResourcelessTransactionManagerTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -37,9 +37,9 @@ public class ResourcelessTransactionManagerTests { @Test public void testCommit() throws Exception { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() { @Override public void afterCompletion(int status) { @@ -57,9 +57,9 @@ public void afterCompletion(int status) { public void testCommitTwice() throws Exception { testCommit(); txStatus = -1; - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() { @Override public void afterCompletion(int status) { @@ -76,9 +76,9 @@ public void afterCompletion(int status) { @Test public void testCommitNested() throws Exception { final TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager); - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() { @Override public void afterCompletion(int status) { @@ -87,9 +87,9 @@ public void afterCompletion(int status) { count++; } }); - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { assertEquals(0, count); count++; return null; @@ -109,9 +109,9 @@ public void testCommitNestedTwice() throws Exception { count = 0; txStatus = -1; final TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager); - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() { @Override public void afterCompletion(int status) { @@ -120,9 +120,9 @@ public void afterCompletion(int status) { count++; } }); - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { assertEquals(0, count); count++; return null; @@ -139,9 +139,9 @@ public Object doInTransaction(TransactionStatus status) { @Test public void testRollback() throws Exception { try { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() { @Override public void afterCompletion(int status) { @@ -164,9 +164,9 @@ public void afterCompletion(int status) { public void testRollbackNestedInner() throws Exception { final TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager); try { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() { @Override public void afterCompletion(int status) { @@ -175,9 +175,9 @@ public void afterCompletion(int status) { count++; } }); - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { assertEquals(0, count); count++; throw new RuntimeException("Rollback!"); @@ -200,9 +200,9 @@ public Object doInTransaction(TransactionStatus status) { public void testRollbackNestedOuter() throws Exception { final TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager); try { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() { @Override public void afterCompletion(int status) { @@ -211,9 +211,9 @@ public void afterCompletion(int status) { count++; } }); - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { assertEquals(0, count); count++; return null; diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareBufferedWriterTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareBufferedWriterTests.java index 857532d849..2a588fd7b5 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareBufferedWriterTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareBufferedWriterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,15 +15,6 @@ */ package org.springframework.batch.support.transaction; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.fail; -import static org.mockito.Matchers.anyObject; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.never; -import static org.mockito.Mockito.times; -import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.when; - import java.io.IOException; import java.nio.ByteBuffer; import java.nio.channels.FileChannel; @@ -31,13 +22,20 @@ import org.junit.Before; import org.junit.Test; import org.mockito.ArgumentCaptor; -import org.mockito.invocation.InvocationOnMock; -import org.mockito.stubbing.Answer; + import org.springframework.transaction.PlatformTransactionManager; -import org.springframework.transaction.TransactionStatus; import org.springframework.transaction.support.TransactionCallback; import org.springframework.transaction.support.TransactionTemplate; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.Mockito.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + /** * @author Dave Syer * @author Michael Minella @@ -54,16 +52,13 @@ public class TransactionAwareBufferedWriterTests { public void init() { fileChannel = mock(FileChannel.class); - writer = new TransactionAwareBufferedWriter(fileChannel, new Runnable() { - @Override - public void run() { - try { - ByteBuffer bb = ByteBuffer.wrap("c".getBytes()); - fileChannel.write(bb); - } - catch (IOException e) { - throw new IllegalStateException(e); - } + writer = new TransactionAwareBufferedWriter(fileChannel, () -> { + try { + ByteBuffer bb = ByteBuffer.wrap("c".getBytes()); + fileChannel.write(bb); + } + catch (IOException e) { + throw new IllegalStateException(e); } }); @@ -76,7 +71,6 @@ public void run() { * Test method for * {@link org.springframework.batch.support.transaction.TransactionAwareBufferedWriter#write(java.lang.String)} * . - * @throws Exception */ @Test public void testWriteOutsideTransaction() throws Exception { @@ -125,12 +119,7 @@ public void testBufferSizeOutsideTransaction() throws Exception { public void testCloseOutsideTransaction() throws Exception { ArgumentCaptor byteBufferCaptor = ArgumentCaptor.forClass(ByteBuffer.class); - when(fileChannel.write(byteBufferCaptor.capture())).thenAnswer(new Answer() { - @Override - public Integer answer(InvocationOnMock invocation) throws Throwable { - return ((ByteBuffer) invocation.getArguments()[0]).remaining(); - } - }); + when(fileChannel.write(byteBufferCaptor.capture())).thenAnswer(invocation -> ((ByteBuffer) invocation.getArguments()[0]).remaining()); writer.write("foo"); writer.close(); @@ -140,124 +129,103 @@ public Integer answer(InvocationOnMock invocation) throws Throwable { } @Test - @SuppressWarnings({"unchecked", "rawtypes"}) public void testFlushInTransaction() throws Exception { - when(fileChannel.write((ByteBuffer)anyObject())).thenReturn(3); + when(fileChannel.write(any(ByteBuffer.class))).thenReturn(3); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - @Override - public Object doInTransaction(TransactionStatus status) { - try { - writer.write("foo"); - writer.flush(); - } - catch (IOException e) { - throw new IllegalStateException("Unexpected IOException", e); - } - assertEquals(3, writer.getBufferSize()); - return null; + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + writer.write("foo"); + writer.flush(); + } + catch (IOException e) { + throw new IllegalStateException("Unexpected IOException", e); } + assertEquals(3, writer.getBufferSize()); + return null; }); verify(fileChannel, never()).force(false); } @Test - @SuppressWarnings({"unchecked", "rawtypes"}) public void testFlushInTransactionForceSync() throws Exception { writer.setForceSync(true); - when(fileChannel.write((ByteBuffer)anyObject())).thenReturn(3); + when(fileChannel.write(any(ByteBuffer.class))).thenReturn(3); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - @Override - public Object doInTransaction(TransactionStatus status) { - try { - writer.write("foo"); - writer.flush(); - } - catch (IOException e) { - throw new IllegalStateException("Unexpected IOException", e); - } - assertEquals(3, writer.getBufferSize()); - return null; + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + writer.write("foo"); + writer.flush(); + } + catch (IOException e) { + throw new IllegalStateException("Unexpected IOException", e); } + assertEquals(3, writer.getBufferSize()); + return null; }); verify(fileChannel, times(1)).force(false); } @Test - @SuppressWarnings({"unchecked", "rawtypes"}) public void testWriteWithCommit() throws Exception { ArgumentCaptor bb = ArgumentCaptor.forClass(ByteBuffer.class); when(fileChannel.write(bb.capture())).thenReturn(3); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - @Override - public Object doInTransaction(TransactionStatus status) { - try { - writer.write("foo"); - } - catch (IOException e) { - throw new IllegalStateException("Unexpected IOException", e); - } - assertEquals(3, writer.getBufferSize()); - return null; + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + writer.write("foo"); + } + catch (IOException e) { + throw new IllegalStateException("Unexpected IOException", e); } + assertEquals(3, writer.getBufferSize()); + return null; }); assertEquals(0, writer.getBufferSize()); } @Test - @SuppressWarnings({"unchecked", "rawtypes"}) public void testBufferSizeInTransaction() throws Exception { ArgumentCaptor bb = ArgumentCaptor.forClass(ByteBuffer.class); when(fileChannel.write(bb.capture())).thenReturn(3); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - @Override - public Object doInTransaction(TransactionStatus status) { - try { - writer.write("foo"); - } - catch (IOException e) { - throw new IllegalStateException("Unexpected IOException", e); - } - assertEquals(3, writer.getBufferSize()); - return null; + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + writer.write("foo"); } + catch (IOException e) { + throw new IllegalStateException("Unexpected IOException", e); + } + assertEquals(3, writer.getBufferSize()); + return null; }); assertEquals(0, writer.getBufferSize()); } @Test - @SuppressWarnings({"unchecked", "rawtypes"}) // BATCH-1959 public void testBufferSizeInTransactionWithMultiByteCharacterUTF8() throws Exception { ArgumentCaptor bb = ArgumentCaptor.forClass(ByteBuffer.class); when(fileChannel.write(bb.capture())).thenReturn(5); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - @Override - public Object doInTransaction(TransactionStatus status) { - try { - writer.write("fóó"); - } - catch (IOException e) { - throw new IllegalStateException("Unexpected IOException", e); - } - assertEquals(5, writer.getBufferSize()); - return null; + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + writer.write("fóó"); + } + catch (IOException e) { + throw new IllegalStateException("Unexpected IOException", e); } + assertEquals(5, writer.getBufferSize()); + return null; }); assertEquals(0, writer.getBufferSize()); } @Test - @SuppressWarnings({"unchecked", "rawtypes"}) // BATCH-1959 public void testBufferSizeInTransactionWithMultiByteCharacterUTF16BE() throws Exception { writer.setEncoding("UTF-16BE"); @@ -265,38 +233,31 @@ public void testBufferSizeInTransactionWithMultiByteCharacterUTF16BE() throws Ex ArgumentCaptor bb = ArgumentCaptor.forClass(ByteBuffer.class); when(fileChannel.write(bb.capture())).thenReturn(6); - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - @Override - public Object doInTransaction(TransactionStatus status) { - try { - writer.write("fóó"); - } - catch (IOException e) { - throw new IllegalStateException("Unexpected IOException", e); - } - assertEquals(6, writer.getBufferSize()); - return null; + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + writer.write("fóó"); } + catch (IOException e) { + throw new IllegalStateException("Unexpected IOException", e); + } + assertEquals(6, writer.getBufferSize()); + return null; }); assertEquals(0, writer.getBufferSize()); } @Test - @SuppressWarnings({"unchecked", "rawtypes"}) public void testWriteWithRollback() throws Exception { try { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - @Override - public Object doInTransaction(TransactionStatus status) { - try { - writer.write("foo"); - } - catch (IOException e) { - throw new IllegalStateException("Unexpected IOException", e); - } - throw new RuntimeException("Planned failure"); + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + writer.write("foo"); + } + catch (IOException e) { + throw new IllegalStateException("Unexpected IOException", e); } + throw new RuntimeException("Planned failure"); }); fail("Exception was not thrown"); } @@ -315,26 +276,19 @@ public void testCleanUpAfterRollback() throws Exception { } @Test - @SuppressWarnings({"unchecked", "rawtypes"}) public void testExceptionOnFlush() throws Exception { - writer = new TransactionAwareBufferedWriter(fileChannel, new Runnable() { - @Override - public void run() { - } + writer = new TransactionAwareBufferedWriter(fileChannel, () -> { }); try { - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - @Override - public Object doInTransaction(TransactionStatus status) { - try { - writer.write("foo"); - } - catch (IOException e) { - throw new IllegalStateException("Unexpected IOException", e); - } - return null; + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + writer.write("foo"); + } + catch (IOException e) { + throw new IllegalStateException("Unexpected IOException", e); } + return null; }); fail("Exception was not thrown"); @@ -342,11 +296,50 @@ public Object doInTransaction(TransactionStatus status) { assertEquals("Could not write to output buffer", ffe.getMessage()); } } + + // BATCH-2018 + @Test + public void testResourceKeyCollision() throws Exception { + final int limit = 5000; + final TransactionAwareBufferedWriter[] writers = new TransactionAwareBufferedWriter[limit]; + final String[] results = new String[limit]; + for(int i = 0; i< limit; i++) { + final int index = i; + @SuppressWarnings("resource") + FileChannel fileChannel = mock(FileChannel.class); + when(fileChannel.write(any(ByteBuffer.class))).thenAnswer(invocation -> { + ByteBuffer buffer = (ByteBuffer) invocation.getArguments()[0]; + String val = new String(buffer.array(), "UTF-8"); + if(results[index] == null) { + results[index] = val; + } else { + results[index] += val; + } + return buffer.limit(); + }); + writers[i] = new TransactionAwareBufferedWriter(fileChannel, null); + } + + new TransactionTemplate(transactionManager).execute((TransactionCallback) status -> { + try { + for(int i=0; i< limit; i++) { + writers[i].write(String.valueOf(i)); + } + } + catch (IOException e) { + throw new IllegalStateException("Unexpected IOException", e); + } + return null; + }); + + for(int i=0; i< limit; i++) { + assertEquals(String.valueOf(i), results[i]); + } + } private String getStringFromByteBuffer(ByteBuffer bb) { byte[] bytearr = new byte[bb.remaining()]; bb.get(bytearr); - String s = new String(bytearr); - return s; + return new String(bytearr); } } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareListFactoryTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareListFactoryTests.java index f65562b6a3..0d34edee0f 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareListFactoryTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareListFactoryTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -65,9 +65,9 @@ public void testClear() { @Test public void testTransactionalAdd() throws Exception { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testAdd(); return null; } @@ -77,9 +77,9 @@ public Object doInTransaction(TransactionStatus status) { @Test public void testTransactionalRemove() throws Exception { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testRemove(); return null; } @@ -89,9 +89,9 @@ public Object doInTransaction(TransactionStatus status) { @Test public void testTransactionalClear() throws Exception { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testClear(); return null; } @@ -102,9 +102,9 @@ public Object doInTransaction(TransactionStatus status) { @Test public void testTransactionalAddWithRollback() throws Exception { try { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testAdd(); throw new RuntimeException("Rollback!"); } @@ -120,9 +120,9 @@ public Object doInTransaction(TransactionStatus status) { @Test public void testTransactionalRemoveWithRollback() throws Exception { try { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testRemove(); throw new RuntimeException("Rollback!"); } @@ -138,9 +138,9 @@ public Object doInTransaction(TransactionStatus status) { @Test public void testTransactionalClearWithRollback() throws Exception { try { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testClear(); throw new RuntimeException("Rollback!"); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareMapFactoryTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareMapFactoryTests.java index 789953de4e..fa0e109327 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareMapFactoryTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareMapFactoryTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -33,7 +33,7 @@ public class TransactionAwareMapFactoryTests extends TestCase { @Override protected void setUp() throws Exception { - Map seed = new HashMap(); + Map seed = new HashMap<>(); seed.put("foo", "oof"); seed.put("bar", "bar"); seed.put("spam", "maps"); @@ -72,9 +72,9 @@ public void testClear() { } public void testTransactionalAdd() throws Exception { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testAdd(); return null; } @@ -83,9 +83,9 @@ public Object doInTransaction(TransactionStatus status) { } public void testTransactionalEmpty() throws Exception { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testEmpty(); return null; } @@ -94,9 +94,9 @@ public Object doInTransaction(TransactionStatus status) { } public void testTransactionalValues() throws Exception { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testValues(); return null; } @@ -105,9 +105,9 @@ public Object doInTransaction(TransactionStatus status) { } public void testTransactionalRemove() throws Exception { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testRemove(); return null; } @@ -116,9 +116,9 @@ public Object doInTransaction(TransactionStatus status) { } public void testTransactionalClear() throws Exception { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testClear(); return null; } @@ -128,9 +128,9 @@ public Object doInTransaction(TransactionStatus status) { public void testTransactionalAddWithRollback() throws Exception { try { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testAdd(); throw new RuntimeException("Rollback!"); } @@ -145,9 +145,9 @@ public Object doInTransaction(TransactionStatus status) { public void testTransactionalRemoveWithRollback() throws Exception { try { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testRemove(); throw new RuntimeException("Rollback!"); } @@ -162,9 +162,9 @@ public Object doInTransaction(TransactionStatus status) { public void testTransactionalClearWithRollback() throws Exception { try { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testClear(); throw new RuntimeException("Rollback!"); } diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareProxyFactoryTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareProxyFactoryTests.java index a928157fce..97b230a993 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareProxyFactoryTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareProxyFactoryTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareSetFactoryTests.java b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareSetFactoryTests.java index 515a332fda..d19adb279c 100644 --- a/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareSetFactoryTests.java +++ b/spring-batch-infrastructure/src/test/java/org/springframework/batch/support/transaction/TransactionAwareSetFactoryTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -39,7 +39,7 @@ public class TransactionAwareSetFactoryTests { @Before public void setUp() throws Exception { - set = TransactionAwareProxyFactory.createTransactionalSet(new HashSet(Arrays.asList("foo", "bar", "spam"))); + set = TransactionAwareProxyFactory.createTransactionalSet(new HashSet<>(Arrays.asList("foo", "bar", "spam"))); } @Test @@ -66,9 +66,9 @@ public void testClear() { @Test public void testTransactionalAdd() throws Exception { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testAdd(); return null; } @@ -78,9 +78,9 @@ public Object doInTransaction(TransactionStatus status) { @Test public void testTransactionalRemove() throws Exception { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testRemove(); return null; } @@ -90,9 +90,9 @@ public Object doInTransaction(TransactionStatus status) { @Test public void testTransactionalClear() throws Exception { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testClear(); return null; } @@ -103,9 +103,9 @@ public Object doInTransaction(TransactionStatus status) { @Test public void testTransactionalAddWithRollback() throws Exception { try { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testAdd(); throw new RuntimeException("Rollback!"); } @@ -121,9 +121,9 @@ public Object doInTransaction(TransactionStatus status) { @Test public void testTransactionalRemoveWithRollback() throws Exception { try { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testRemove(); throw new RuntimeException("Rollback!"); } @@ -139,9 +139,9 @@ public Object doInTransaction(TransactionStatus status) { @Test public void testTransactionalClearWithRollback() throws Exception { try { - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { testClear(); throw new RuntimeException("Rollback!"); } diff --git a/spring-batch-infrastructure/src/test/java/test/jdbc/datasource/DataSourceInitializer.java b/spring-batch-infrastructure/src/test/java/test/jdbc/datasource/DataSourceInitializer.java index a13153ba6b..53658f9bc4 100644 --- a/spring-batch-infrastructure/src/test/java/test/jdbc/datasource/DataSourceInitializer.java +++ b/spring-batch-infrastructure/src/test/java/test/jdbc/datasource/DataSourceInitializer.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,12 +18,12 @@ import java.io.IOException; import java.util.List; - import javax.sql.DataSource; import org.apache.commons.io.IOUtils; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.beans.factory.BeanInitializationException; import org.springframework.beans.factory.DisposableBean; import org.springframework.beans.factory.InitializingBean; @@ -41,7 +41,7 @@ /** * Wrapper for a {@link DataSource} that can run scripts on start up and shut - * down. Us as a bean definition

        + * down. Us as a bean definition

        * * Run this class to initialize a database in a running server process. * Make sure the server is running first by launching the "hsql-server" from the @@ -71,6 +71,7 @@ public class DataSourceInitializer implements InitializingBean, DisposableBean { * * @param args */ + @SuppressWarnings("resource") public static void main(String... args) { new ClassPathXmlApplicationContext(ClassUtils.addResourcePathToPackagePath(DataSourceInitializer.class, DataSourceInitializer.class.getSimpleName() + "-context.xml")); @@ -113,7 +114,7 @@ public void doDestroy() { @Override public void afterPropertiesSet() throws Exception { - Assert.notNull(dataSource); + Assert.notNull(dataSource, "A DataSource is required"); initialize(); } @@ -136,15 +137,15 @@ private void doExecuteScript(final Resource scriptResource) { final JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); TransactionTemplate transactionTemplate = new TransactionTemplate(new DataSourceTransactionManager(dataSource)); - transactionTemplate.execute(new TransactionCallback() { + transactionTemplate.execute(new TransactionCallback() { @Override @SuppressWarnings("unchecked") - public Object doInTransaction(TransactionStatus status) { + public Void doInTransaction(TransactionStatus status) { String[] scripts; try { scripts = StringUtils.delimitedListToStringArray(stripComments(IOUtils.readLines(scriptResource - .getInputStream())), ";"); + .getInputStream(), "UTF-8")), ";"); } catch (IOException e) { throw new BeanInitializationException("Cannot load script from [" + scriptResource + "]", e); @@ -173,10 +174,10 @@ public Object doInTransaction(TransactionStatus status) { } private String stripComments(List list) { - StringBuffer buffer = new StringBuffer(); + StringBuilder buffer = new StringBuilder(); for (String line : list) { if (!line.startsWith("//") && !line.startsWith("--")) { - buffer.append(line + "\n"); + buffer.append(line).append("\n"); } } return buffer.toString(); diff --git a/spring-batch-infrastructure/src/test/java/test/jdbc/datasource/DerbyDataSourceFactoryBean.java b/spring-batch-infrastructure/src/test/java/test/jdbc/datasource/DerbyDataSourceFactoryBean.java index b9cfd1a3eb..803647a7d3 100644 --- a/spring-batch-infrastructure/src/test/java/test/jdbc/datasource/DerbyDataSourceFactoryBean.java +++ b/spring-batch-infrastructure/src/test/java/test/jdbc/datasource/DerbyDataSourceFactoryBean.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package test.jdbc.datasource; import java.io.File; @@ -9,25 +24,25 @@ import org.apache.derby.jdbc.EmbeddedDataSource; import org.springframework.beans.factory.config.AbstractFactoryBean; -public class DerbyDataSourceFactoryBean extends AbstractFactoryBean { +public class DerbyDataSourceFactoryBean extends AbstractFactoryBean { private static Log logger = LogFactory.getLog(DerbyDataSourceFactoryBean.class); - private String dataDirectory = "derby-home"; + private String dataDirectory = "build/derby-home"; public void setDataDirectory(String dataDirectory) { this.dataDirectory = dataDirectory; } @Override - protected Object createInstance() throws Exception { + protected DataSource createInstance() throws Exception { File directory = new File(dataDirectory); System.setProperty("derby.system.home", directory.getCanonicalPath()); System.setProperty("derby.storage.fileSyncTransactionLog", "true"); System.setProperty("derby.storage.pageCacheSize", "100"); final EmbeddedDataSource ds = new EmbeddedDataSource(); - ds.setDatabaseName("derbydb"); + ds.setDatabaseName("build/derbydb"); ds.setCreateDatabase("create"); logger.info("Created instance of " + ds.toString()); diff --git a/spring-batch-infrastructure/src/test/java/test/jdbc/datasource/DerbyShutdownBean.java b/spring-batch-infrastructure/src/test/java/test/jdbc/datasource/DerbyShutdownBean.java index 0c551f8207..2e40826d1e 100644 --- a/spring-batch-infrastructure/src/test/java/test/jdbc/datasource/DerbyShutdownBean.java +++ b/spring-batch-infrastructure/src/test/java/test/jdbc/datasource/DerbyShutdownBean.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package test.jdbc.datasource; import java.sql.SQLException; diff --git a/spring-batch-infrastructure/src/test/java/test/jdbc/proc/derby/TestProcedures.java b/spring-batch-infrastructure/src/test/java/test/jdbc/proc/derby/TestProcedures.java index e3832d246d..2c3b2e6ae9 100644 --- a/spring-batch-infrastructure/src/test/java/test/jdbc/proc/derby/TestProcedures.java +++ b/spring-batch-infrastructure/src/test/java/test/jdbc/proc/derby/TestProcedures.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package test.jdbc.proc.derby; import java.sql.*; diff --git a/spring-batch-infrastructure/src/test/resources/META-INF/persistence.xml b/spring-batch-infrastructure/src/test/resources/META-INF/persistence.xml index f2ab28139d..c5f634cbf6 100644 --- a/spring-batch-infrastructure/src/test/resources/META-INF/persistence.xml +++ b/spring-batch-infrastructure/src/test/resources/META-INF/persistence.xml @@ -1,6 +1,6 @@ diff --git a/spring-batch-infrastructure/src/test/resources/log4j.properties b/spring-batch-infrastructure/src/test/resources/log4j.properties index 7168892cda..a53c07b34d 100644 --- a/spring-batch-infrastructure/src/test/resources/log4j.properties +++ b/spring-batch-infrastructure/src/test/resources/log4j.properties @@ -1,7 +1,7 @@ log4j.rootCategory=INFO, stdout -log4j.appender.stdout=org.apache.log4j.ConsoleAppender -log4j.appender.stdout.layout=org.apache.log4j.PatternLayout +log4j.appender.stdout=org.apache.logging.log4j.core.appender.ConsoleAppender +log4j.appender.stdout.layout=org.apache.logging.log4j.core.layout.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%d %p %t [%c] - <%m>%n #log4j.category.org.springframework.batch=DEBUG diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/delegating-item-processor.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/delegating-item-processor.xml index 5a19743dfa..99a96622e4 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/delegating-item-processor.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/delegating-item-processor.xml @@ -1,6 +1,6 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/delegating-item-provider.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/delegating-item-provider.xml index 66e98f8d18..e0006edbda 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/delegating-item-provider.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/delegating-item-provider.xml @@ -1,6 +1,6 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/delegating-item-writer.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/delegating-item-writer.xml index 9a414ec76b..63f6d26fa3 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/delegating-item-writer.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/delegating-item-writer.xml @@ -1,6 +1,6 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/pe-delegating-item-writer.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/pe-delegating-item-writer.xml index 685479b06e..443bbef036 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/pe-delegating-item-writer.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/adapter/pe-delegating-item-writer.xml @@ -1,6 +1,6 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/avro/plain-old-user-data-no-schema.avro b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/avro/plain-old-user-data-no-schema.avro new file mode 100644 index 0000000000..104e8581c9 --- /dev/null +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/avro/plain-old-user-data-no-schema.avro @@ -0,0 +1,3 @@ + +David(blueSuered +Alana yellowJoepink \ No newline at end of file diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/avro/user-data-no-schema.avro b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/avro/user-data-no-schema.avro new file mode 100644 index 0000000000..1b51bdec07 Binary files /dev/null and b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/avro/user-data-no-schema.avro differ diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/avro/user-data.avro b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/avro/user-data.avro new file mode 100644 index 0000000000..90cb9858c5 Binary files /dev/null and b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/avro/user-data.avro differ diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/avro/user-schema.json b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/avro/user-schema.json new file mode 100644 index 0000000000..51117abe58 --- /dev/null +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/avro/user-schema.json @@ -0,0 +1,9 @@ +{"namespace": "org.springframework.batch.item.avro.example", + "type": "record", + "name": "User", + "fields": [ + {"name": "name", "type": "string"}, + {"name": "favorite_number", "type": ["int", "null"]}, + {"name": "favorite_color", "type": ["string", "null"]} + ] +} \ No newline at end of file diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/Foo-write.hbm.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/Foo-write.hbm.xml deleted file mode 100644 index 612a29924f..0000000000 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/Foo-write.hbm.xml +++ /dev/null @@ -1,15 +0,0 @@ - - - - - - - - - - - - - \ No newline at end of file diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/Foo.hbm.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/Foo.hbm.xml index 4c4b7bbbb0..7f05cc9ed6 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/Foo.hbm.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/Foo.hbm.xml @@ -1,7 +1,7 @@ + "https://hibernate.org/dtd/hibernate-mapping-3.0.dtd"> diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests-context.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests-context.xml index 01a5f53e02..9749efef08 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests-context.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderCommonTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd+http://www.springframework.org/schema/util%20https://www.springframework.org/schema/util/spring-util-3.1.xsd"> diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderConfigTests-context.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderConfigTests-context.xml index 7028205192..415806f252 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderConfigTests-context.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderConfigTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd+http://www.springframework.org/schema/util%20https://www.springframework.org/schema/util/spring-util-3.1.xsd"> diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderParameterTests-context.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderParameterTests-context.xml index 01a5f53e02..9749efef08 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderParameterTests-context.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JdbcPagingItemReaderParameterTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd+http://www.springframework.org/schema/util%20https://www.springframework.org/schema/util/spring-util-3.1.xsd"> diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests-context.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests-context.xml index c4860041b3..8c6caafc33 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests-context.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderCommonTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd+http://www.springframework.org/schema/util%20https://www.springframework.org/schema/util/spring-util-3.1.xsd"> diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderParameterTests-context.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderParameterTests-context.xml index f87fbafccb..1c08d9f66a 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderParameterTests-context.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/JpaPagingItemReaderParameterTests-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd+http://www.springframework.org/schema/util%20https://www.springframework.org/schema/util/spring-util-3.1.xsd"> diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/data-source-context.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/data-source-context.xml index ed0891d50a..46c32cdf9f 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/data-source-context.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/data-source-context.xml @@ -1,9 +1,9 @@ - + diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/ibatis-config.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/ibatis-config.xml deleted file mode 100644 index f23807357e..0000000000 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/ibatis-config.xml +++ /dev/null @@ -1,12 +0,0 @@ - - - - - - - - - - \ No newline at end of file diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/ibatis-foo.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/ibatis-foo.xml deleted file mode 100644 index d418da0819..0000000000 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/ibatis-foo.xml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - insert INTO T_WRITE_FOOS (ID, NAME, VALUE) VALUES (#id#, #name#, #value#) - - - - update T_WRITE_FOOS set NAME = #name#, VALUE = #value# where ID = #id# - - - - delete from T_WRITE_FOOS where ID = #id# - - - \ No newline at end of file diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/stored-procedure-context.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/stored-procedure-context.xml index abb7d264dd..62d9d7d556 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/stored-procedure-context.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/database/stored-procedure-context.xml @@ -1,8 +1,7 @@ - + @@ -15,7 +14,7 @@ - + diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/file/mapping/bean-wrapper.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/file/mapping/bean-wrapper.xml index 66ef6d888c..6c5bb4d3d3 100644 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/file/mapping/bean-wrapper.xml +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/file/mapping/bean-wrapper.xml @@ -2,7 +2,7 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/support/processor-test-simple.js b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/support/processor-test-simple.js new file mode 100644 index 0000000000..3a9b9db1dd --- /dev/null +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/item/support/processor-test-simple.js @@ -0,0 +1 @@ +item.toUpperCase(); diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/retry/interceptor/retry-transaction-test.xml b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/retry/interceptor/retry-transaction-test.xml deleted file mode 100644 index 3cad105f4f..0000000000 --- a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/retry/interceptor/retry-transaction-test.xml +++ /dev/null @@ -1,35 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - diff --git a/spring-batch-infrastructure/src/test/resources/org/springframework/batch/support/existing.txt b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/support/existing.txt new file mode 100644 index 0000000000..5608f4c833 --- /dev/null +++ b/spring-batch-infrastructure/src/test/resources/org/springframework/batch/support/existing.txt @@ -0,0 +1 @@ +This file is pre-existing \ No newline at end of file diff --git a/spring-batch-infrastructure/template.mf b/spring-batch-infrastructure/template.mf deleted file mode 100644 index b33f95ca5d..0000000000 --- a/spring-batch-infrastructure/template.mf +++ /dev/null @@ -1,48 +0,0 @@ -Manifest-Version: 1.0 -Bundle-SymbolicName: org.springframework.batch.infrastructure -Bundle-Name: Spring Batch Infrastructure -Bundle-Vendor: Spring -Bundle-Version: ${version} -Bundle-ManifestVersion: 2 -Import-Template: - com.thoughtworks.xstream.*;version="[1.3,1.4)";resolution:=optional, - org.codehaus.jettison.*;version="[1.0,1.1)";resolution:=optional, - org.codehaus.jackson.*;version="[1.0.1,1.1)";resolution:=optional, - org.aopalliance.*;version="[1.0.0,2.0.0)", - org.aspectj.*;version="[1.5.2,1.7.0)", - org.hibernate.*;version="[3.2.5.ga,3.3.0.ga)";resolution:=optional, - org.apache.commons.lang.*;version="[2.1,3.0)";resolution:=optional, - org.apache.commons.logging.*;version="[1.1.0,2.0.0)", - com.ibatis.sqlmap.*;version="[2.3.0,3.0.0)";resolution:=optional, - org.springframework.batch.*;version="[2.0.0,3.0.0)", - org.springframework.aop.*;version="[3.1.2,4.0.0)", - org.springframework.beans.*;version="[3.1.2,4.0.0)", - org.springframework.context.*;version="[3.1.2,4.0.0)", - org.springframework.core.*;version="[3.1.2,4.0.0)", - org.springframework.dao.*;version="[3.1.2,4.0.0)";resolution:=optional, - org.springframework.jdbc.*;version="[3.1.2,4.0.0)";resolution:=optional, - org.springframework.jms.*;version="[3.1.2,4.0.0)";resolution:=optional, - org.springframework.mail.*;version="[3.1.2,4.0.0)";resolution:=optional, - org.springframework.orm.*;version="[3.1.2,4.0.0)";resolution:=optional, - org.springframework.oxm.*;version="[3.1.2,4.0.0)";resolution:=optional, - org.springframework.xml.*;version="[3.1.2,4.0.0)";resolution:=optional, - org.springframework.validation.*;version="[3.1.2,4.0.0)";resolution:=optional, - org.springframework.transaction.*;version="[3.1.2,4.0.0)";resolution:=optional, - org.springframework.util.*;version="[3.1.2,4.0.0)";resolution:=optional, - org.springframework.osgi.*;version="[1.1.0,2.0.0)";resolution:=optional, - org.springframework.amqp.*;version="[1.1.0,2.0.0)";resolution:=optional, - org.springframework.retry.*;version="[1.0.0,2.0.0)";resolution:=optional, - org.springframework.classify.*;version="[1.0.0,2.0.0)";resolution:=optional, - org.springframework.data.domain.*;version="[1.5.0,2.0.0)";resolution:=optional, - org.springframework.data.gemfire.*;version="[1.3.0,2.0.0)";resolution:=optional, - org.springframework.data.mongodb.*;version="[1.1.0,2.0.0)";resolution:=optional, - org.springframework.data.neo4j.*;version="[2.2.0,3.0.0)";resolution:=optional, - org.springframework.data.repository.*;version="[1.5.0,2.0.0)";resolution:=optional, - com.mongodb.*;version="[2.1.9,3.0.0)";resolution:=optional, - javax.sql.*;version="0";resolution:=optional, - javax.jms;version="0";resolution:=optional, - javax.persistence;version="0";resolution:=optional, - javax.xml.*;version="0";resolution:=optional, - javax.mail.*;version="0";resolution:=optional -Import-Package: - org.springframework.batch.core;version="[2.0.0,3.0.0)" diff --git a/spring-batch-integration/.springBeans b/spring-batch-integration/.springBeans new file mode 100644 index 0000000000..71e07211f9 --- /dev/null +++ b/spring-batch-integration/.springBeans @@ -0,0 +1,32 @@ + + + 1 + + + + + + + src/test/resources/job-execution-context.xml + src/test/resources/simple-job-launcher-context.xml + src/test/resources/org/springframework/batch/integration/chunk/ChunkMessageItemWriterIntegrationTests-context.xml + src/test/resources/org/springframework/batch/integration/launch/JobLaunchingMessageHandlerIntegrationTests-context.xml + src/test/resources/org/springframework/batch/integration/file/ResourceSplitterIntegrationTests-context.xml + src/test/resources/org/springframework/batch/integration/retry/TransactionalPollingIntegrationTests-context.xml + src/test/resources/org/springframework/batch/integration/retry/RepeatTransactionalPollingIntegrationTests-context.xml + src/test/resources/org/springframework/batch/integration/retry/RetryRepeatTransactionalPollingIntegrationTests-context.xml + src/test/resources/org/springframework/batch/integration/SmokeTests-context.xml + src/test/resources/org/springframework/batch/integration/retry/RetryTransactionalPollingIntegrationTests-context.xml + src/test/resources/org/springframework/batch/integration/item/MessagingGatewayIntegrationTests-context.xml + src/test/resources/org/springframework/batch/integration/partition/VanillaIntegrationTests-context.xml + src/test/resources/org/springframework/batch/integration/async/AsyncItemProcessorMessagingGatewayTests-context.xml + src/test/resources/org/springframework/batch/integration/file/FileToMessagesJobIntegrationTests-context.xml + src/test/resources/org/springframework/batch/integration/step/StepGatewayIntegrationTests-context.xml + src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepIntegrationTests-context.xml + src/test/resources/jms-context.xml + src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJmsIntegrationTests-context.xml + src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkStepIntegrationTests-context.xml + + + + diff --git a/spring-batch-integration/src/main/java/META-INF/MANIFEST.MF b/spring-batch-integration/src/main/java/META-INF/MANIFEST.MF new file mode 100644 index 0000000000..5e9495128c --- /dev/null +++ b/spring-batch-integration/src/main/java/META-INF/MANIFEST.MF @@ -0,0 +1,3 @@ +Manifest-Version: 1.0 +Class-Path: + diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/async/AsyncItemProcessor.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/async/AsyncItemProcessor.java new file mode 100644 index 0000000000..cd742a6cb9 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/async/AsyncItemProcessor.java @@ -0,0 +1,129 @@ +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.async; + +import java.util.concurrent.Callable; +import java.util.concurrent.Future; +import java.util.concurrent.FutureTask; + +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.scope.context.StepContext; +import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemWriter; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.core.task.SyncTaskExecutor; +import org.springframework.core.task.TaskExecutor; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; + +/** + * An {@link ItemProcessor} that delegates to a nested processor and in the + * background. To allow for background processing the return value from the + * processor is a {@link Future} which needs to be unpacked before the item can + * be used by a client. + * + * Because the {@link Future} is typically unwrapped in the {@link ItemWriter}, + * there are lifecycle and stats limitations (since the framework doesn't know + * what the result of the processor is). While not an exhaustive list, things like + * {@link StepExecution#filterCount} will not reflect the number of filtered items + * and {@link org.springframework.batch.core.ItemProcessListener#onProcessError(Object, Exception)} + * will not be called. + * + * @author Dave Syer + * + * @param the input object type + * @param the output object type (will be wrapped in a Future) + * @see AsyncItemWriter + */ +public class AsyncItemProcessor implements ItemProcessor>, InitializingBean { + + private ItemProcessor delegate; + + private TaskExecutor taskExecutor = new SyncTaskExecutor(); + + /** + * Check mandatory properties (the {@link #setDelegate(ItemProcessor)}). + * + * @see InitializingBean#afterPropertiesSet() + */ + public void afterPropertiesSet() throws Exception { + Assert.notNull(delegate, "The delegate must be set."); + } + + /** + * The {@link ItemProcessor} to use to delegate processing to in a + * background thread. + * + * @param delegate the {@link ItemProcessor} to use as a delegate + */ + public void setDelegate(ItemProcessor delegate) { + this.delegate = delegate; + } + + /** + * The {@link TaskExecutor} to use to allow the item processing to proceed + * in the background. Defaults to a {@link SyncTaskExecutor} so no threads + * are created unless this is overridden. + * + * @param taskExecutor a {@link TaskExecutor} + */ + public void setTaskExecutor(TaskExecutor taskExecutor) { + this.taskExecutor = taskExecutor; + } + + /** + * Transform the input by delegating to the provided item processor. The + * return value is wrapped in a {@link Future} so that clients can unpack it + * later. + * + * @see ItemProcessor#process(Object) + */ + @Nullable + public Future process(final I item) throws Exception { + final StepExecution stepExecution = getStepExecution(); + FutureTask task = new FutureTask<>(new Callable() { + public O call() throws Exception { + if (stepExecution != null) { + StepSynchronizationManager.register(stepExecution); + } + try { + return delegate.process(item); + } + finally { + if (stepExecution != null) { + StepSynchronizationManager.close(); + } + } + } + }); + taskExecutor.execute(task); + return task; + } + + /** + * @return the current step execution if there is one + */ + private StepExecution getStepExecution() { + StepContext context = StepSynchronizationManager.getContext(); + if (context==null) { + return null; + } + StepExecution stepExecution = context.getStepExecution(); + return stepExecution; + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/async/AsyncItemWriter.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/async/AsyncItemWriter.java new file mode 100644 index 0000000000..1df06bff1c --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/async/AsyncItemWriter.java @@ -0,0 +1,108 @@ +/* + * Copyright 2006-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.async; + +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.Future; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStream; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.ItemStreamWriter; +import org.springframework.batch.item.ItemWriter; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.util.Assert; + +public class AsyncItemWriter implements ItemStreamWriter>, InitializingBean { + + private static final Log logger = LogFactory.getLog(AsyncItemWriter.class); + + private ItemWriter delegate; + + public void afterPropertiesSet() throws Exception { + Assert.notNull(delegate, "A delegate ItemWriter must be provided."); + } + + /** + * @param delegate ItemWriter that does the actual writing of the Future results + */ + public void setDelegate(ItemWriter delegate) { + this.delegate = delegate; + } + + /** + * In the processing of the {@link java.util.concurrent.Future}s passed, nulls are not passed to the + * delegate since they are considered filtered out by the {@link org.springframework.batch.integration.async.AsyncItemProcessor}'s + * delegated {@link org.springframework.batch.item.ItemProcessor}. If the unwrapping + * of the {@link Future} results in an {@link ExecutionException}, that will be + * unwrapped and the cause will be thrown. + * + * @param items {@link java.util.concurrent.Future}s to be unwrapped and passed to the delegate + * @throws Exception The exception returned by the Future if one was thrown + */ + public void write(List> items) throws Exception { + List list = new ArrayList<>(); + for (Future future : items) { + try { + T item = future.get(); + + if(item != null) { + list.add(future.get()); + } + } + catch (ExecutionException e) { + Throwable cause = e.getCause(); + + if(cause != null && cause instanceof Exception) { + logger.debug("An exception was thrown while processing an item", e); + + throw (Exception) cause; + } + else { + throw e; + } + } + } + + delegate.write(list); + } + + @Override + public void open(ExecutionContext executionContext) throws ItemStreamException { + if (delegate instanceof ItemStream) { + ((ItemStream) delegate).open(executionContext); + } + } + + @Override + public void update(ExecutionContext executionContext) throws ItemStreamException { + if (delegate instanceof ItemStream) { + ((ItemStream) delegate).update(executionContext); + } + } + + @Override + public void close() throws ItemStreamException { + if (delegate instanceof ItemStream) { + ((ItemStream) delegate).close(); + } + } +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/async/StepExecutionInterceptor.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/async/StepExecutionInterceptor.java new file mode 100644 index 0000000000..560cae1291 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/async/StepExecutionInterceptor.java @@ -0,0 +1,52 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.async; + +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.scope.context.StepContext; +import org.springframework.batch.core.scope.context.StepSynchronizationManager; +import org.springframework.integration.support.MessageBuilder; +import org.springframework.messaging.Message; +import org.springframework.messaging.MessageChannel; +import org.springframework.messaging.support.ChannelInterceptor; + +/** + * A {@link ChannelInterceptor} that adds the current {@link StepExecution} (if + * there is one) as a header to the message. Downstream asynchronous handlers + * can then take advantage of the step context without needing to be step + * scoped, which is a problem for handlers executing in another thread because + * the scope context is not available. + * + * @author Dave Syer + * + */ +public class StepExecutionInterceptor implements ChannelInterceptor { + + /** + * The name of the header + */ + public static final String STEP_EXECUTION = "stepExecution"; + + @Override + public Message preSend(Message message, MessageChannel channel) { + StepContext context = StepSynchronizationManager.getContext(); + if (context == null) { + return message; + } + return MessageBuilder.fromMessage(message).setHeader(STEP_EXECUTION, context.getStepExecution()).build(); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/async/package-info.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/async/package-info.java new file mode 100644 index 0000000000..807d25ae49 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/async/package-info.java @@ -0,0 +1,10 @@ +/** + * Components for executing item processing asynchronously and writing the results when processing is complete. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.integration.async; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/AsynchronousFailureException.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/AsynchronousFailureException.java new file mode 100644 index 0000000000..02a94cc577 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/AsynchronousFailureException.java @@ -0,0 +1,55 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.chunk; + +import org.springframework.batch.item.ItemWriterException; + +/** + * Exception indicating that a failure or early completion condition was + * detected in a remote worker. + * + * @author Dave Syer + * + */ +public class AsynchronousFailureException extends ItemWriterException { + + private static final long serialVersionUID = 1L; + + /** + * Create a new {@link AsynchronousFailureException} based on a message and + * another exception. + * + * @param message + * the message for this exception + * @param cause + * the other exception + */ + public AsynchronousFailureException(String message, Throwable cause) { + super(message, cause); + } + + /** + * Create a new {@link AsynchronousFailureException} based on a message. + * + * @param message + * the message for this exception + */ + public AsynchronousFailureException(String message) { + super(message); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkHandler.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkHandler.java new file mode 100644 index 0000000000..c6bbdcabe9 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkHandler.java @@ -0,0 +1,44 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.chunk; + + +/** + * Interface for a remote worker in the Remote Chunking pattern. A request comes from a manager process containing some + * items to be processed. Once the items are done with a response needs to be generated containing a summary of the + * result. + * + * @author Dave Syer + * + * @param the type of the items to be processed (it is recommended to use a Memento like a primary key) + */ +public interface ChunkHandler { + + /** + * Handle the chunk, processing all the items and returning a response summarising the result. If the result is a + * failure then the response should say so. The handler only throws an exception if it needs to roll back a + * transaction and knows that the request will be re-delivered (if not to the same handler then to one processing + * the same Step). + * + * @param chunk a request containing the chunk to process + * @return a response summarising the result + * + * @throws Exception if the handler needs to roll back a transaction and have the chunk re-delivered + */ + ChunkResponse handleChunk(ChunkRequest chunk) throws Exception; + +} \ No newline at end of file diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkMessageChannelItemWriter.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkMessageChannelItemWriter.java new file mode 100644 index 0000000000..93922fda4a --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkMessageChannelItemWriter.java @@ -0,0 +1,342 @@ +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.chunk; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; +import java.util.Queue; +import java.util.concurrent.LinkedBlockingQueue; +import java.util.concurrent.atomic.AtomicInteger; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.listener.StepExecutionListenerSupport; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStream; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.ItemWriter; +import org.springframework.integration.core.MessagingTemplate; +import org.springframework.lang.Nullable; +import org.springframework.messaging.Message; +import org.springframework.messaging.PollableChannel; +import org.springframework.messaging.support.GenericMessage; +import org.springframework.util.Assert; + +public class ChunkMessageChannelItemWriter extends StepExecutionListenerSupport implements ItemWriter, + ItemStream, StepContributionSource { + + private static final Log logger = LogFactory.getLog(ChunkMessageChannelItemWriter.class); + + static final String ACTUAL = ChunkMessageChannelItemWriter.class.getName() + ".ACTUAL"; + + static final String EXPECTED = ChunkMessageChannelItemWriter.class.getName() + ".EXPECTED"; + + private static final long DEFAULT_THROTTLE_LIMIT = 6; + + private MessagingTemplate messagingGateway; + + private final LocalState localState = new LocalState(); + + private long throttleLimit = DEFAULT_THROTTLE_LIMIT; + + private final int DEFAULT_MAX_WAIT_TIMEOUTS = 40; + + private int maxWaitTimeouts = DEFAULT_MAX_WAIT_TIMEOUTS; + + private PollableChannel replyChannel; + + /** + * The maximum number of times to wait at the end of a step for a non-null result from the remote workers. This is a + * multiplier on the receive timeout set separately on the gateway. The ideal value is a compromise between allowing + * slow workers time to finish, and responsiveness if there is a dead worker. Defaults to 40. + * + * @param maxWaitTimeouts the maximum number of wait timeouts + */ + public void setMaxWaitTimeouts(int maxWaitTimeouts) { + this.maxWaitTimeouts = maxWaitTimeouts; + } + + /** + * Public setter for the throttle limit. This limits the number of pending requests for chunk processing to avoid + * overwhelming the receivers. + * @param throttleLimit the throttle limit to set + */ + public void setThrottleLimit(long throttleLimit) { + this.throttleLimit = throttleLimit; + } + + public void setMessagingOperations(MessagingTemplate messagingGateway) { + this.messagingGateway = messagingGateway; + } + + public void setReplyChannel(PollableChannel replyChannel) { + this.replyChannel = replyChannel; + } + + public void write(List items) throws Exception { + + // Block until expecting <= throttle limit + while (localState.getExpecting() > throttleLimit) { + getNextResult(); + } + + if (!items.isEmpty()) { + + ChunkRequest request = localState.getRequest(items); + if (logger.isDebugEnabled()) { + logger.debug("Dispatching chunk: " + request); + } + messagingGateway.send(new GenericMessage<>(request)); + localState.incrementExpected(); + + } + + } + + @Override + public void beforeStep(StepExecution stepExecution) { + localState.setStepExecution(stepExecution); + } + + @Nullable + @Override + public ExitStatus afterStep(StepExecution stepExecution) { + if (!(stepExecution.getStatus() == BatchStatus.COMPLETED)) { + return ExitStatus.EXECUTING; + } + long expecting = localState.getExpecting(); + boolean timedOut; + try { + logger.debug("Waiting for results in step listener..."); + timedOut = !waitForResults(); + logger.debug("Finished waiting for results in step listener."); + } + catch (RuntimeException e) { + logger.debug("Detected failure waiting for results in step listener.", e); + stepExecution.setStatus(BatchStatus.FAILED); + return ExitStatus.FAILED.addExitDescription(e.getClass().getName() + ": " + e.getMessage()); + } + finally { + + if (logger.isDebugEnabled()) { + logger.debug("Finished waiting for results in step listener. Still expecting: " + + localState.getExpecting()); + } + + for (StepContribution contribution : getStepContributions()) { + stepExecution.apply(contribution); + } + } + if (timedOut) { + stepExecution.setStatus(BatchStatus.FAILED); + return ExitStatus.FAILED.addExitDescription("Timed out waiting for " + localState.getExpecting() + + " backlog at end of step"); + } + return ExitStatus.COMPLETED.addExitDescription("Waited for " + expecting + " results."); + } + + public void close() throws ItemStreamException { + localState.reset(); + } + + public void open(ExecutionContext executionContext) throws ItemStreamException { + if (executionContext.containsKey(EXPECTED)) { + localState.open(executionContext.getInt(EXPECTED), executionContext.getInt(ACTUAL)); + if (!waitForResults()) { + throw new ItemStreamException("Timed out waiting for back log on open"); + } + } + } + + public void update(ExecutionContext executionContext) throws ItemStreamException { + executionContext.putInt(EXPECTED, localState.expected.intValue()); + executionContext.putInt(ACTUAL, localState.actual.intValue()); + } + + public Collection getStepContributions() { + List contributions = new ArrayList<>(); + for (ChunkResponse response : localState.pollChunkResponses()) { + StepContribution contribution = response.getStepContribution(); + if (logger.isDebugEnabled()) { + logger.debug("Applying: " + response); + } + contributions.add(contribution); + } + return contributions; + } + + /** + * Wait until all the results that are in the pipeline come back to the reply channel. + * + * @return true if successfully received a result, false if timed out + */ + private boolean waitForResults() throws AsynchronousFailureException { + int count = 0; + int maxCount = maxWaitTimeouts; + Throwable failure = null; + logger.info("Waiting for " + localState.getExpecting() + " results"); + while (localState.getExpecting() > 0 && count++ < maxCount) { + try { + getNextResult(); + } + catch (Throwable t) { + logger.error("Detected error in remote result. Trying to recover " + localState.getExpecting() + + " outstanding results before completing.", t); + failure = t; + } + } + if (failure != null) { + throw wrapIfNecessary(failure); + } + return count < maxCount; + } + + /** + * Get the next result if it is available (within the timeout specified in the gateway), otherwise do nothing. + * + * @throws AsynchronousFailureException If there is a response and it contains a failed chunk response. + * + * @throws IllegalStateException if the result contains the wrong job instance id (maybe we are sharing a channel + * and we shouldn't be) + */ + @SuppressWarnings("unchecked") + private void getNextResult() throws AsynchronousFailureException { + Message message = (Message) messagingGateway.receive(replyChannel); + if (message != null) { + ChunkResponse payload = message.getPayload(); + if (logger.isDebugEnabled()) { + logger.debug("Found result: " + payload); + } + Long jobInstanceId = payload.getJobId(); + Assert.state(jobInstanceId != null, "Message did not contain job instance id."); + Assert.state(jobInstanceId.equals(localState.getJobId()), "Message contained wrong job instance id [" + + jobInstanceId + "] should have been [" + localState.getJobId() + "]."); + if (payload.isRedelivered()) { + logger + .warn("Redelivered result detected, which may indicate stale state. In the best case, we just picked up a timed out message " + + "from a previous failed execution. In the worst case (and if this is not a restart), " + + "the step may now timeout. In that case if you believe that all messages " + + "from workers have been sent, the business state " + + "is probably inconsistent, and the step will fail."); + localState.incrementRedelivered(); + } + localState.pushResponse(payload); + localState.incrementActual(); + if (!payload.isSuccessful()) { + throw new AsynchronousFailureException("Failure or interrupt detected in handler: " + + payload.getMessage()); + } + } + } + + /** + * Re-throws the original throwable if it is unchecked, wraps checked exceptions into + * {@link AsynchronousFailureException}. + */ + private static AsynchronousFailureException wrapIfNecessary(Throwable throwable) { + if (throwable instanceof Error) { + throw (Error) throwable; + } + else if (throwable instanceof AsynchronousFailureException) { + return (AsynchronousFailureException) throwable; + } + else { + return new AsynchronousFailureException("Exception in remote process", throwable); + } + } + + private static class LocalState { + + private final AtomicInteger current = new AtomicInteger(-1); + + private final AtomicInteger actual = new AtomicInteger(); + + private final AtomicInteger expected = new AtomicInteger(); + + private final AtomicInteger redelivered = new AtomicInteger(); + + private StepExecution stepExecution; + + private final Queue contributions = new LinkedBlockingQueue<>(); + + public int getExpecting() { + return expected.get() - actual.get(); + } + + public ChunkRequest getRequest(List items) { + return new ChunkRequest<>(current.incrementAndGet(), items, getJobId(), createStepContribution()); + } + + public void open(int expectedValue, int actualValue) { + actual.set(actualValue); + expected.set(expectedValue); + } + + public Collection pollChunkResponses() { + Collection set = new ArrayList<>(); + synchronized (contributions) { + ChunkResponse item = contributions.poll(); + while (item != null) { + set.add(item); + item = contributions.poll(); + } + } + return set; + } + + public void pushResponse(ChunkResponse stepContribution) { + synchronized (contributions) { + contributions.add(stepContribution); + } + } + + public void incrementRedelivered() { + redelivered.incrementAndGet(); + } + + public void incrementActual() { + actual.incrementAndGet(); + } + + public void incrementExpected() { + expected.incrementAndGet(); + } + + public StepContribution createStepContribution() { + return stepExecution.createStepContribution(); + } + + public Long getJobId() { + return stepExecution.getJobExecution().getJobId(); + } + + public void setStepExecution(StepExecution stepExecution) { + this.stepExecution = stepExecution; + } + + public void reset() { + expected.set(0); + actual.set(0); + } + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkProcessorChunkHandler.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkProcessorChunkHandler.java new file mode 100644 index 0000000000..cc39c8a7c4 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkProcessorChunkHandler.java @@ -0,0 +1,138 @@ +/* + * Copyright 2006-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.chunk; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.core.JobInterruptedException; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.step.item.Chunk; +import org.springframework.batch.core.step.item.ChunkProcessor; +import org.springframework.batch.core.step.item.FaultTolerantChunkProcessor; +import org.springframework.batch.core.step.skip.NonSkippableReadException; +import org.springframework.batch.core.step.skip.SkipLimitExceededException; +import org.springframework.batch.core.step.skip.SkipListenerFailedException; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.integration.annotation.MessageEndpoint; +import org.springframework.integration.annotation.ServiceActivator; +import org.springframework.retry.RetryException; +import org.springframework.util.Assert; + +/** + * A {@link ChunkHandler} based on a {@link ChunkProcessor}. Knows how to distinguish between a processor that is fault + * tolerant, and one that is not. If the processor is fault tolerant then exceptions can be propagated on the assumption + * that there will be a roll back and the request will be re-delivered. + * + * @author Dave Syer + * @author Michael Minella + * + * @param the type of the items in the chunk to be handled + */ +@MessageEndpoint +public class ChunkProcessorChunkHandler implements ChunkHandler, InitializingBean { + + private static final Log logger = LogFactory.getLog(ChunkProcessorChunkHandler.class); + + private ChunkProcessor chunkProcessor; + + /* + * (non-Javadoc) + * + * @see org.springframework.beans.factory.InitializingBean#afterPropertiesSet() + */ + public void afterPropertiesSet() throws Exception { + Assert.notNull(chunkProcessor, "A ChunkProcessor must be provided"); + } + + /** + * Public setter for the {@link ChunkProcessor}. + * + * @param chunkProcessor the chunkProcessor to set + */ + public void setChunkProcessor(ChunkProcessor chunkProcessor) { + this.chunkProcessor = chunkProcessor; + } + + /** + * + * @see ChunkHandler#handleChunk(ChunkRequest) + */ + @ServiceActivator + public ChunkResponse handleChunk(ChunkRequest chunkRequest) throws Exception { + + if (logger.isDebugEnabled()) { + logger.debug("Handling chunk: " + chunkRequest); + } + + StepContribution stepContribution = chunkRequest.getStepContribution(); + + Throwable failure = process(chunkRequest, stepContribution); + if (failure != null) { + logger.debug("Failed chunk", failure); + return new ChunkResponse(false, chunkRequest.getSequence(), chunkRequest.getJobId(), stepContribution, failure.getClass().getName() + + ": " + failure.getMessage()); + } + + if (logger.isDebugEnabled()) { + logger.debug("Completed chunk handling with " + stepContribution); + } + return new ChunkResponse(true, chunkRequest.getSequence(), chunkRequest.getJobId(), stepContribution); + + } + + /** + * @param chunkRequest the current request + * @param stepContribution the step contribution to update + * @throws Exception if there is a fatal exception + */ + private Throwable process(ChunkRequest chunkRequest, StepContribution stepContribution) throws Exception { + + Chunk chunk = new Chunk<>(chunkRequest.getItems()); + Throwable failure = null; + try { + chunkProcessor.process(stepContribution, chunk); + } + catch (SkipLimitExceededException e) { + failure = e; + } + catch (NonSkippableReadException e) { + failure = e; + } + catch (SkipListenerFailedException e) { + failure = e; + } + catch (RetryException e) { + failure = e; + } + catch (JobInterruptedException e) { + failure = e; + } + catch (Exception e) { + if (chunkProcessor instanceof FaultTolerantChunkProcessor) { + // try again... + throw e; + } + else { + failure = e; + } + } + + return failure; + + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkRequest.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkRequest.java new file mode 100644 index 0000000000..c43c8ff58a --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkRequest.java @@ -0,0 +1,79 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.chunk; + +import java.io.Serializable; +import java.util.Collection; + +import org.springframework.batch.core.StepContribution; + +/** + * Encapsulation of a chunk of items to be processed remotely as part of a step + * execution. + * + * @author Dave Syer + * + * @param the type of the items to process + */ +public class ChunkRequest implements Serializable { + + private static final long serialVersionUID = 1L; + + private final long jobId; + + private final Collection items; + + private final StepContribution stepContribution; + + private final int sequence; + + public ChunkRequest(int sequence, Collection items, long jobId, StepContribution stepContribution) { + this.sequence = sequence; + this.items = items; + this.jobId = jobId; + this.stepContribution = stepContribution; + } + + public long getJobId() { + return jobId; + } + + public Collection getItems() { + return items; + } + + public int getSequence() { + return sequence; + } + + /** + * @return the {@link StepContribution} for this chunk + */ + public StepContribution getStepContribution() { + return stepContribution; + } + + /** + * @see java.lang.Object#toString() + */ + @Override + public String toString() { + return getClass().getSimpleName() + ": jobId=" + jobId + ", sequence=" + sequence + ", contribution=" + + stepContribution + ", item count=" + items.size(); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkResponse.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkResponse.java new file mode 100644 index 0000000000..a8faaf430d --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/ChunkResponse.java @@ -0,0 +1,105 @@ +/* + * Copyright 2006-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.chunk; + +import java.io.Serializable; + +import org.springframework.batch.core.StepContribution; +import org.springframework.lang.Nullable; + +/** + * Encapsulates a response to processing a chunk of items, summarising the result as a {@link StepContribution}. + * + * @author Dave Syer + * @author Mahmoud Ben Hassine + * + */ +public class ChunkResponse implements Serializable { + + private static final long serialVersionUID = 1L; + + private final StepContribution stepContribution; + + private final Long jobId; + + private final boolean status; + + private final String message; + + private final boolean redelivered; + + private final int sequence; + + public ChunkResponse(int sequence, Long jobId, StepContribution stepContribution) { + this(true, sequence, jobId, stepContribution, null); + } + + public ChunkResponse(boolean status, int sequence, Long jobId, StepContribution stepContribution) { + this(status, sequence, jobId, stepContribution, null); + } + + public ChunkResponse(boolean status, int sequence, Long jobId, StepContribution stepContribution, @Nullable String message) { + this(status, sequence, jobId, stepContribution, message, false); + } + + public ChunkResponse(ChunkResponse input, boolean redelivered) { + this(input.status, input.sequence, input.jobId, input.stepContribution, input.message, redelivered); + } + + public ChunkResponse(boolean status, int sequence, Long jobId, StepContribution stepContribution, @Nullable String message, boolean redelivered) { + this.status = status; + this.sequence = sequence; + this.jobId = jobId; + this.stepContribution = stepContribution; + this.message = message; + this.redelivered = redelivered; + } + + public StepContribution getStepContribution() { + return stepContribution; + } + + public Long getJobId() { + return jobId; + } + + public int getSequence() { + return sequence; + } + + public boolean isSuccessful() { + return status; + } + + public boolean isRedelivered() { + return redelivered; + } + + public String getMessage() { + return message; + } + + /** + * @see java.lang.Object#toString() + */ + @Override + public String toString() { + return getClass().getSimpleName() + ": jobId=" + jobId + ", sequence=" + sequence + ", stepContribution=" + stepContribution + + ", successful=" + status; + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/JmsRedeliveredExtractor.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/JmsRedeliveredExtractor.java new file mode 100644 index 0000000000..952ef17495 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/JmsRedeliveredExtractor.java @@ -0,0 +1,38 @@ +/* + * Copyright 2009-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.chunk; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.messaging.handler.annotation.Header; +import org.springframework.jms.support.JmsHeaders; + +/** + * @author Dave Syer + * + */ +public class JmsRedeliveredExtractor { + + private static final Log logger = LogFactory.getLog(JmsRedeliveredExtractor.class); + + public ChunkResponse extract(ChunkResponse input, @Header(JmsHeaders.REDELIVERED) boolean redelivered) { + if (logger.isDebugEnabled()) { + logger.debug("Extracted redelivered flag for response, value="+redelivered); + } + return new ChunkResponse(input, redelivered); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/MessageSourcePollerInterceptor.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/MessageSourcePollerInterceptor.java new file mode 100644 index 0000000000..ecebd838a8 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/MessageSourcePollerInterceptor.java @@ -0,0 +1,90 @@ +package org.springframework.batch.integration.chunk; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.integration.core.MessageSource; +import org.springframework.messaging.Message; +import org.springframework.messaging.MessageChannel; +import org.springframework.messaging.support.ChannelInterceptor; +import org.springframework.util.Assert; + + +/** + * A {@link ChannelInterceptor} that turns a pollable channel into a "pass-thru channel": if a client calls + * receive() on the channel it will delegate to a {@link MessageSource} to pull the message directly from + * an external source. This is particularly useful in combination with a message channel in thread scope, in which case + * the receive() can join a transaction which was started by the caller. + * + * @author Dave Syer + * + */ +public class MessageSourcePollerInterceptor implements ChannelInterceptor, InitializingBean { + + private static Log logger = LogFactory.getLog(MessageSourcePollerInterceptor.class); + + private MessageSource source; + + private MessageChannel channel; + + /** + * Convenient default constructor for configuration purposes. + */ + public MessageSourcePollerInterceptor() { + } + + /** + * @param source a message source to poll for messages on receive. + */ + public MessageSourcePollerInterceptor(MessageSource source) { + this.source = source; + } + + /** + * Optional MessageChannel for injecting the message received from the source (defaults to the channel intercepted + * in {@link #preReceive(MessageChannel)}). + * + * @param channel the channel to set + */ + public void setChannel(MessageChannel channel) { + this.channel = channel; + } + + /** + * Asserts that mandatory properties are set. + * @see InitializingBean#afterPropertiesSet() + */ + public void afterPropertiesSet() throws Exception { + Assert.state(source != null, "A MessageSource must be provided"); + } + + /** + * @param source a message source to poll for messages on receive. + */ + public void setMessageSource(MessageSource source) { + this.source = source; + } + + /** + * Receive from the {@link MessageSource} and send immediately to the input channel, so that the call that we are + * intercepting always a message to receive. + * + * @see ChannelInterceptor#preReceive(MessageChannel) + */ + @Override + public boolean preReceive(MessageChannel channel) { + Message message = source.receive(); + if (message != null) { + if (this.channel != null) { + channel = this.channel; + } + channel.send(message); + if (logger.isDebugEnabled()) { + logger.debug("Sent " + message + " to channel " + channel); + } + return true; + } + return true; + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkHandlerFactoryBean.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkHandlerFactoryBean.java new file mode 100644 index 0000000000..bce7a19fa1 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkHandlerFactoryBean.java @@ -0,0 +1,266 @@ +/* + * Copyright 2006-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.chunk; + +import java.lang.reflect.Field; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.StepExecutionListener; +import org.springframework.batch.core.step.item.Chunk; +import org.springframework.batch.core.step.item.ChunkOrientedTasklet; +import org.springframework.batch.core.step.item.ChunkProcessor; +import org.springframework.batch.core.step.item.FaultTolerantChunkProcessor; +import org.springframework.batch.core.step.item.SimpleChunkProcessor; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.PassThroughItemProcessor; +import org.springframework.beans.factory.FactoryBean; +import org.springframework.util.Assert; +import org.springframework.util.ReflectionUtils; + +/** + * Convenient factory bean for a chunk handler that also converts an existing chunk-oriented step into a remote chunk + * master. The idea is to lift the existing chunk processor out of a Step that works locally, and replace it with a one + * that writes chunks into a message channel. The existing step hands its business chunk processing responsibility over + * to the handler produced by the factory, which then needs to be set up as a worker on the other end of the channel the + * chunks are being sent to. Once this chunk handler is installed the application is playing the role of both the manager + * and the worker listeners in the Remote Chunking pattern for the Step in question. + * + * @author Dave Syer + * @author Mahmoud Ben Hassine + * + */ +public class RemoteChunkHandlerFactoryBean implements FactoryBean> { + + private static Log logger = LogFactory.getLog(RemoteChunkHandlerFactoryBean.class); + + private TaskletStep step; + + private ItemWriter chunkWriter; + + private StepContributionSource stepContributionSource; + + /** + * The local step that is to be converted to a remote chunk master. + * + * @param step the step to set + */ + public void setStep(TaskletStep step) { + this.step = step; + } + + /** + * The item writer to be injected into the step. Its responsibility is to send chunks of items to remote workers. + * Usually in practice it will be a {@link ChunkMessageChannelItemWriter}. + * + * @param chunkWriter the chunk writer to set + */ + public void setChunkWriter(ItemWriter chunkWriter) { + this.chunkWriter = chunkWriter; + } + + /** + * A source of {@link StepContribution} instances coming back from remote workers. + * + * @param stepContributionSource the step contribution source to set (defaults to the chunk writer) + */ + public void setStepContributionSource(StepContributionSource stepContributionSource) { + this.stepContributionSource = stepContributionSource; + } + + /** + * The type of object created by this factory. Returns {@link ChunkHandler} class. + * + * @see FactoryBean#getObjectType() + */ + public Class getObjectType() { + return ChunkHandler.class; + } + + /** + * Optimization for the bean factory (always returns true). + * + * @see FactoryBean#isSingleton() + */ + public boolean isSingleton() { + return true; + } + + /** + * Builds a {@link ChunkHandler} from the {@link ChunkProcessor} extracted from the {@link #setStep(TaskletStep) + * step} provided. Also modifies the step to send chunks to the chunk handler via the + * {@link #setChunkWriter(ItemWriter) chunk writer}. + * + * @see FactoryBean#getObject() + */ + public ChunkHandler getObject() throws Exception { + + if (stepContributionSource == null) { + Assert.state(chunkWriter instanceof StepContributionSource, + "The chunk writer must be a StepContributionSource or else the source must be provided explicitly"); + stepContributionSource = (StepContributionSource) chunkWriter; + } + + Assert.state(step instanceof TaskletStep, "Step [" + step.getName() + "] must be a TaskletStep"); + if (logger.isDebugEnabled()) { + logger.debug("Converting TaskletStep with name=" + step.getName()); + } + + Tasklet tasklet = getTasklet(step); + Assert.state(tasklet instanceof ChunkOrientedTasklet, "Tasklet must be ChunkOrientedTasklet in step=" + + step.getName()); + + ChunkProcessor chunkProcessor = getChunkProcessor((ChunkOrientedTasklet) tasklet); + Assert.state(chunkProcessor != null, "ChunkProcessor must be accessible in Tasklet in step=" + step.getName()); + + ItemWriter itemWriter = getItemWriter(chunkProcessor); + Assert.state(!(itemWriter instanceof ChunkMessageChannelItemWriter), "Cannot adapt step [" + step.getName() + + "] because it already has a remote chunk writer. Use a local writer in the step."); + + replaceChunkProcessor((ChunkOrientedTasklet) tasklet, chunkWriter, stepContributionSource); + if (chunkWriter instanceof StepExecutionListener) { + step.registerStepExecutionListener((StepExecutionListener) chunkWriter); + } + + ChunkProcessorChunkHandler handler = new ChunkProcessorChunkHandler<>(); + setNonBuffering(chunkProcessor); + handler.setChunkProcessor(chunkProcessor); + // TODO: create step context for the processor in case it has + // scope="step" dependencies + handler.afterPropertiesSet(); + + return handler; + + } + + /** + * Overrides the buffering settings in the chunk processor if it is fault tolerant. + * @param chunkProcessor the chunk processor that is going to be used in the workers + */ + private void setNonBuffering(ChunkProcessor chunkProcessor) { + if (chunkProcessor instanceof FaultTolerantChunkProcessor) { + ((FaultTolerantChunkProcessor) chunkProcessor).setBuffering(false); + } + } + + /** + * Replace the chunk processor in the tasklet provided with one that can act as a master in the Remote Chunking + * pattern. + * + * @param tasklet a ChunkOrientedTasklet + * @param chunkWriter an ItemWriter that can send the chunks to remote workers + * @param stepContributionSource a StepContributionSource used to gather results from the workers + */ + private void replaceChunkProcessor(ChunkOrientedTasklet tasklet, ItemWriter chunkWriter, + final StepContributionSource stepContributionSource) { + setField(tasklet, "chunkProcessor", new SimpleChunkProcessor(new PassThroughItemProcessor<>(), + chunkWriter) { + @Override + protected void write(StepContribution contribution, Chunk inputs, Chunk outputs) throws Exception { + doWrite(outputs.getItems()); + // Do not update the step contribution until the chunks are + // actually processed + updateStepContribution(contribution, stepContributionSource); + } + }); + } + + /** + * Update a StepContribution with all the data from a StepContributionSource. The filter and write counts plus the + * exit status will be updated to reflect the data in the source. + * + * @param contribution the current contribution + * @param stepContributionSource a source of StepContributions + */ + protected void updateStepContribution(StepContribution contribution, StepContributionSource stepContributionSource) { + for (StepContribution result : stepContributionSource.getStepContributions()) { + contribution.incrementFilterCount(result.getFilterCount()); + contribution.incrementWriteCount(result.getWriteCount()); + for (int i = 0; i < result.getProcessSkipCount(); i++) { + contribution.incrementProcessSkipCount(); + } + for (int i = 0; i < result.getWriteSkipCount(); i++) { + contribution.incrementWriteSkipCount(); + } + contribution.setExitStatus(contribution.getExitStatus().and(result.getExitStatus())); + } + } + + /** + * Pull out an item writer from a ChunkProcessor + * @param chunkProcessor a ChunkProcessor + * @return its ItemWriter + */ + @SuppressWarnings("unchecked") + private ItemWriter getItemWriter(ChunkProcessor chunkProcessor) { + return (ItemWriter) getField(chunkProcessor, "itemWriter"); + } + + /** + * Pull the ChunkProcessor out of a tasklet. + * @param tasklet a ChunkOrientedTasklet + * @return the ChunkProcessor + */ + @SuppressWarnings("unchecked") + private ChunkProcessor getChunkProcessor(ChunkOrientedTasklet tasklet) { + return (ChunkProcessor) getField(tasklet, "chunkProcessor"); + } + + /** + * Pull a Tasklet out of a step. + * @param step a TaskletStep + * @return the Tasklet + */ + private Tasklet getTasklet(TaskletStep step) { + return (Tasklet) getField(step, "tasklet"); + } + + private static Object getField(Object target, String name) { + Assert.notNull(target, "Target object must not be null"); + Field field = ReflectionUtils.findField(target.getClass(), name); + if (field == null) { + if (logger.isDebugEnabled()) { + logger.debug("Could not find field [" + name + "] on target [" + target + "]"); + } + return null; + } + + if (logger.isDebugEnabled()) { + logger.debug("Getting field [" + name + "] from target [" + target + "]"); + } + ReflectionUtils.makeAccessible(field); + return ReflectionUtils.getField(field, target); + } + + private static void setField(Object target, String name, Object value) { + Assert.notNull(target, "Target object must not be null"); + Field field = ReflectionUtils.findField(target.getClass(), name); + if (field == null) { + throw new IllegalStateException("Could not find field [" + name + "] on target [" + target + "]"); + } + + if (logger.isDebugEnabled()) { + logger.debug("Getting field [" + name + "] from target [" + target + "]"); + } + ReflectionUtils.makeAccessible(field); + ReflectionUtils.setField(field, target, value); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingManagerStepBuilder.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingManagerStepBuilder.java new file mode 100644 index 0000000000..c3c35e9dd7 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingManagerStepBuilder.java @@ -0,0 +1,420 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.chunk; + +import org.springframework.batch.core.ChunkListener; +import org.springframework.batch.core.ItemReadListener; +import org.springframework.batch.core.ItemWriteListener; +import org.springframework.batch.core.SkipListener; +import org.springframework.batch.core.StepExecutionListener; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.step.builder.FaultTolerantStepBuilder; +import org.springframework.batch.core.step.builder.StepBuilder; +import org.springframework.batch.core.step.item.KeyGenerator; +import org.springframework.batch.core.step.skip.SkipPolicy; +import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemStream; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.repeat.CompletionPolicy; +import org.springframework.batch.repeat.RepeatOperations; +import org.springframework.batch.repeat.exception.ExceptionHandler; +import org.springframework.integration.core.MessagingTemplate; +import org.springframework.messaging.MessageChannel; +import org.springframework.messaging.PollableChannel; +import org.springframework.retry.RetryPolicy; +import org.springframework.retry.backoff.BackOffPolicy; +import org.springframework.retry.policy.RetryContextCache; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.transaction.interceptor.TransactionAttribute; +import org.springframework.util.Assert; + +/** + * Builder for a manager step in a remote chunking setup. This builder creates and + * sets a {@link ChunkMessageChannelItemWriter} on the manager step. + * + *

        If no {@code messagingTemplate} is provided through + * {@link RemoteChunkingManagerStepBuilder#messagingTemplate(MessagingTemplate)}, + * this builder will create one and set its default channel to the {@code outputChannel} + * provided through {@link RemoteChunkingManagerStepBuilder#outputChannel(MessageChannel)}.

        + * + *

        If a {@code messagingTemplate} is provided, it is assumed that it is fully configured + * and that its default channel is set to an output channel on which requests to workers + * will be sent.

        + * + * @param type of input items + * @param type of output items + * + * @since 4.2 + * @author Mahmoud Ben Hassine + */ +public class RemoteChunkingManagerStepBuilder extends FaultTolerantStepBuilder { + + private MessagingTemplate messagingTemplate; + private PollableChannel inputChannel; + private MessageChannel outputChannel; + + private final int DEFAULT_MAX_WAIT_TIMEOUTS = 40; + private static final long DEFAULT_THROTTLE_LIMIT = 6; + private int maxWaitTimeouts = DEFAULT_MAX_WAIT_TIMEOUTS; + private long throttleLimit = DEFAULT_THROTTLE_LIMIT; + + /** + * Create a new {@link RemoteChunkingManagerStepBuilder}. + * + * @param stepName name of the manager step + */ + public RemoteChunkingManagerStepBuilder(String stepName) { + super(new StepBuilder(stepName)); + } + + /** + * Set the input channel on which replies from workers will be received. + * The provided input channel will be set as a reply channel on the + * {@link ChunkMessageChannelItemWriter} created by this builder. + * + * @param inputChannel the input channel + * @return this builder instance for fluent chaining + * + * @see ChunkMessageChannelItemWriter#setReplyChannel + */ + public RemoteChunkingManagerStepBuilder inputChannel(PollableChannel inputChannel) { + Assert.notNull(inputChannel, "inputChannel must not be null"); + this.inputChannel = inputChannel; + return this; + } + + /** + * Set the output channel on which requests to workers will be sent. By using + * this setter, a default messaging template will be created and the output + * channel will be set as its default channel. + *

        Use either this setter or {@link RemoteChunkingManagerStepBuilder#messagingTemplate(MessagingTemplate)} + * to provide a fully configured messaging template.

        + * + * @param outputChannel the output channel. + * @return this builder instance for fluent chaining + * + * @see RemoteChunkingManagerStepBuilder#messagingTemplate(MessagingTemplate) + */ + public RemoteChunkingManagerStepBuilder outputChannel(MessageChannel outputChannel) { + Assert.notNull(outputChannel, "outputChannel must not be null"); + this.outputChannel = outputChannel; + return this; + } + + /** + * Set the {@link MessagingTemplate} to use to send data to workers. + * The default channel of the messaging template must be set. + *

        Use either this setter to provide a fully configured messaging template or + * provide an output channel through {@link RemoteChunkingManagerStepBuilder#outputChannel(MessageChannel)} + * and a default messaging template will be created.

        + * + * @param messagingTemplate the messaging template to use + * @return this builder instance for fluent chaining + * @see RemoteChunkingManagerStepBuilder#outputChannel(MessageChannel) + */ + public RemoteChunkingManagerStepBuilder messagingTemplate(MessagingTemplate messagingTemplate) { + Assert.notNull(messagingTemplate, "messagingTemplate must not be null"); + this.messagingTemplate = messagingTemplate; + return this; + } + + /** + * The maximum number of times to wait at the end of a step for a non-null result from the remote workers. This is a + * multiplier on the receive timeout set separately on the gateway. The ideal value is a compromise between allowing + * slow workers time to finish, and responsiveness if there is a dead worker. Defaults to 40. + * + * @param maxWaitTimeouts the maximum number of wait timeouts + * @return this builder instance for fluent chaining + * @see ChunkMessageChannelItemWriter#setMaxWaitTimeouts(int) + */ + public RemoteChunkingManagerStepBuilder maxWaitTimeouts(int maxWaitTimeouts) { + Assert.isTrue(maxWaitTimeouts > 0, "maxWaitTimeouts must be greater than zero"); + this.maxWaitTimeouts = maxWaitTimeouts; + return this; + } + + /** + * Public setter for the throttle limit. This limits the number of pending requests for chunk processing to avoid + * overwhelming the receivers. + * + * @param throttleLimit the throttle limit to set + * @return this builder instance for fluent chaining + * @see ChunkMessageChannelItemWriter#setThrottleLimit(long) + */ + public RemoteChunkingManagerStepBuilder throttleLimit(long throttleLimit) { + Assert.isTrue(throttleLimit > 0, "throttleLimit must be greater than zero"); + this.throttleLimit = throttleLimit; + return this; + } + + /** + * Build a manager {@link TaskletStep}. + * + * @return the configured manager step + * @see RemoteChunkHandlerFactoryBean + */ + public TaskletStep build() { + Assert.notNull(this.inputChannel, "An InputChannel must be provided"); + Assert.state(this.outputChannel == null || this.messagingTemplate == null, + "You must specify either an outputChannel or a messagingTemplate but not both."); + + // configure messaging template + if (this.messagingTemplate == null) { + this.messagingTemplate = new MessagingTemplate(); + this.messagingTemplate.setDefaultChannel(this.outputChannel); + if (this.logger.isDebugEnabled()) { + this.logger.debug("No messagingTemplate was provided, using a default one"); + } + } + + // configure item writer + ChunkMessageChannelItemWriter chunkMessageChannelItemWriter = new ChunkMessageChannelItemWriter<>(); + chunkMessageChannelItemWriter.setMessagingOperations(this.messagingTemplate); + chunkMessageChannelItemWriter.setMaxWaitTimeouts(this.maxWaitTimeouts); + chunkMessageChannelItemWriter.setThrottleLimit(this.throttleLimit); + chunkMessageChannelItemWriter.setReplyChannel(this.inputChannel); + super.writer(chunkMessageChannelItemWriter); + + return super.build(); + } + + /* + * The following methods override those from parent builders and return + * the current builder type. + * FIXME: Change parent builders to be generic and return current builder + * type in each method. + */ + + @Override + public RemoteChunkingManagerStepBuilder reader(ItemReader reader) { + super.reader(reader); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder repository(JobRepository jobRepository) { + super.repository(jobRepository); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder transactionManager(PlatformTransactionManager transactionManager) { + super.transactionManager(transactionManager); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder listener(Object listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder listener(SkipListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder listener(ChunkListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder transactionAttribute(TransactionAttribute transactionAttribute) { + super.transactionAttribute(transactionAttribute); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder listener(org.springframework.retry.RetryListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder keyGenerator(KeyGenerator keyGenerator) { + super.keyGenerator(keyGenerator); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder retryLimit(int retryLimit) { + super.retryLimit(retryLimit); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder retryPolicy(RetryPolicy retryPolicy) { + super.retryPolicy(retryPolicy); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder backOffPolicy(BackOffPolicy backOffPolicy) { + super.backOffPolicy(backOffPolicy); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder retryContextCache(RetryContextCache retryContextCache) { + super.retryContextCache(retryContextCache); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder skipLimit(int skipLimit) { + super.skipLimit(skipLimit); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder noSkip(Class type) { + super.noSkip(type); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder skip(Class type) { + super.skip(type); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder skipPolicy(SkipPolicy skipPolicy) { + super.skipPolicy(skipPolicy); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder noRollback(Class type) { + super.noRollback(type); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder noRetry(Class type) { + super.noRetry(type); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder retry(Class type) { + super.retry(type); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder stream(ItemStream stream) { + super.stream(stream); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder chunk(int chunkSize) { + super.chunk(chunkSize); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder chunk(CompletionPolicy completionPolicy) { + super.chunk(completionPolicy); + return this; + } + + /** + * This method will throw a {@link UnsupportedOperationException} since + * the item writer of the manager step in a remote chunking setup will be + * automatically set to an instance of {@link ChunkMessageChannelItemWriter}. + * + * When building a manager step for remote chunking, no item writer must be + * provided. + * + * @throws UnsupportedOperationException if an item writer is provided + * @see ChunkMessageChannelItemWriter + * @see RemoteChunkHandlerFactoryBean#setChunkWriter(ItemWriter) + */ + @Override + public RemoteChunkingManagerStepBuilder writer(ItemWriter writer) throws UnsupportedOperationException { + throw new UnsupportedOperationException("When configuring a manager step " + + "for remote chunking, the item writer will be automatically set " + + "to an instance of ChunkMessageChannelItemWriter. The item writer " + + "must not be provided in this case."); + } + + @Override + public RemoteChunkingManagerStepBuilder readerIsTransactionalQueue() { + super.readerIsTransactionalQueue(); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder listener(ItemReadListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder listener(ItemWriteListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder chunkOperations(RepeatOperations repeatTemplate) { + super.chunkOperations(repeatTemplate); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder exceptionHandler(ExceptionHandler exceptionHandler) { + super.exceptionHandler(exceptionHandler); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder stepOperations(RepeatOperations repeatTemplate) { + super.stepOperations(repeatTemplate); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder startLimit(int startLimit) { + super.startLimit(startLimit); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder listener(StepExecutionListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder allowStartIfComplete(boolean allowStartIfComplete) { + super.allowStartIfComplete(allowStartIfComplete); + return this; + } + + @Override + public RemoteChunkingManagerStepBuilder processor(ItemProcessor itemProcessor) { + super.processor(itemProcessor); + return this; + } +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingManagerStepBuilderFactory.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingManagerStepBuilderFactory.java new file mode 100644 index 0000000000..f831495126 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingManagerStepBuilderFactory.java @@ -0,0 +1,63 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.chunk; + +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.transaction.PlatformTransactionManager; + +/** + * Convenient factory for a {@link RemoteChunkingManagerStepBuilder} which sets + * the {@link JobRepository} and {@link PlatformTransactionManager} automatically. + * + * @since 4.2 + * @author Mahmoud Ben Hassine + */ +public class RemoteChunkingManagerStepBuilderFactory { + + private JobRepository jobRepository; + + private PlatformTransactionManager transactionManager; + + /** + * Create a new {@link RemoteChunkingManagerStepBuilderFactory}. + * + * @param jobRepository the job repository to use + * @param transactionManager the transaction manager to use + */ + public RemoteChunkingManagerStepBuilderFactory( + JobRepository jobRepository, + PlatformTransactionManager transactionManager) { + + this.jobRepository = jobRepository; + this.transactionManager = transactionManager; + } + + /** + * Creates a {@link RemoteChunkingManagerStepBuilder} and initializes its job + * repository and transaction manager. + * + * @param name the name of the step + * @param type of input items + * @param type of output items + * @return a {@link RemoteChunkingManagerStepBuilder} + */ + public RemoteChunkingManagerStepBuilder get(String name) { + return new RemoteChunkingManagerStepBuilder(name) + .repository(this.jobRepository) + .transactionManager(this.transactionManager); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingMasterStepBuilder.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingMasterStepBuilder.java new file mode 100644 index 0000000000..8d88f0ea3d --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingMasterStepBuilder.java @@ -0,0 +1,423 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.chunk; + +import org.springframework.batch.core.ChunkListener; +import org.springframework.batch.core.ItemReadListener; +import org.springframework.batch.core.ItemWriteListener; +import org.springframework.batch.core.SkipListener; +import org.springframework.batch.core.StepExecutionListener; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.step.builder.FaultTolerantStepBuilder; +import org.springframework.batch.core.step.builder.StepBuilder; +import org.springframework.batch.core.step.item.KeyGenerator; +import org.springframework.batch.core.step.skip.SkipPolicy; +import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemStream; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.repeat.CompletionPolicy; +import org.springframework.batch.repeat.RepeatOperations; +import org.springframework.batch.repeat.exception.ExceptionHandler; +import org.springframework.integration.core.MessagingTemplate; +import org.springframework.messaging.MessageChannel; +import org.springframework.messaging.PollableChannel; +import org.springframework.retry.RetryPolicy; +import org.springframework.retry.backoff.BackOffPolicy; +import org.springframework.retry.policy.RetryContextCache; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.transaction.interceptor.TransactionAttribute; +import org.springframework.util.Assert; + +/** + * Builder for a master step in a remote chunking setup. This builder creates and + * sets a {@link ChunkMessageChannelItemWriter} on the master step. + * + *

        If no {@code messagingTemplate} is provided through + * {@link RemoteChunkingMasterStepBuilder#messagingTemplate(MessagingTemplate)}, + * this builder will create one and set its default channel to the {@code outputChannel} + * provided through {@link RemoteChunkingMasterStepBuilder#outputChannel(MessageChannel)}.

        + * + *

        If a {@code messagingTemplate} is provided, it is assumed that it is fully configured + * and that its default channel is set to an output channel on which requests to workers + * will be sent.

        + * + * @param type of input items + * @param type of output items + * + * @deprecated Use {@link RemoteChunkingManagerStepBuilder} instead. + * + * @since 4.1 + * @author Mahmoud Ben Hassine + */ +@Deprecated +public class RemoteChunkingMasterStepBuilder extends FaultTolerantStepBuilder { + + private MessagingTemplate messagingTemplate; + private PollableChannel inputChannel; + private MessageChannel outputChannel; + + private final int DEFAULT_MAX_WAIT_TIMEOUTS = 40; + private static final long DEFAULT_THROTTLE_LIMIT = 6; + private int maxWaitTimeouts = DEFAULT_MAX_WAIT_TIMEOUTS; + private long throttleLimit = DEFAULT_THROTTLE_LIMIT; + + /** + * Create a new {@link RemoteChunkingMasterStepBuilder}. + * + * @param stepName name of the master step + */ + public RemoteChunkingMasterStepBuilder(String stepName) { + super(new StepBuilder(stepName)); + } + + /** + * Set the input channel on which replies from workers will be received. + * The provided input channel will be set as a reply channel on the + * {@link ChunkMessageChannelItemWriter} created by this builder. + * + * @param inputChannel the input channel + * @return this builder instance for fluent chaining + * + * @see ChunkMessageChannelItemWriter#setReplyChannel + */ + public RemoteChunkingMasterStepBuilder inputChannel(PollableChannel inputChannel) { + Assert.notNull(inputChannel, "inputChannel must not be null"); + this.inputChannel = inputChannel; + return this; + } + + /** + * Set the output channel on which requests to workers will be sent. By using + * this setter, a default messaging template will be created and the output + * channel will be set as its default channel. + *

        Use either this setter or {@link RemoteChunkingMasterStepBuilder#messagingTemplate(MessagingTemplate)} + * to provide a fully configured messaging template.

        + * + * @param outputChannel the output channel. + * @return this builder instance for fluent chaining + * + * @see RemoteChunkingMasterStepBuilder#messagingTemplate(MessagingTemplate) + */ + public RemoteChunkingMasterStepBuilder outputChannel(MessageChannel outputChannel) { + Assert.notNull(outputChannel, "outputChannel must not be null"); + this.outputChannel = outputChannel; + return this; + } + + /** + * Set the {@link MessagingTemplate} to use to send data to workers. + * The default channel of the messaging template must be set. + *

        Use either this setter to provide a fully configured messaging template or + * provide an output channel through {@link RemoteChunkingMasterStepBuilder#outputChannel(MessageChannel)} + * and a default messaging template will be created.

        + * + * @param messagingTemplate the messaging template to use + * @return this builder instance for fluent chaining + * @see RemoteChunkingMasterStepBuilder#outputChannel(MessageChannel) + */ + public RemoteChunkingMasterStepBuilder messagingTemplate(MessagingTemplate messagingTemplate) { + Assert.notNull(messagingTemplate, "messagingTemplate must not be null"); + this.messagingTemplate = messagingTemplate; + return this; + } + + /** + * The maximum number of times to wait at the end of a step for a non-null result from the remote workers. This is a + * multiplier on the receive timeout set separately on the gateway. The ideal value is a compromise between allowing + * slow workers time to finish, and responsiveness if there is a dead worker. Defaults to 40. + * + * @param maxWaitTimeouts the maximum number of wait timeouts + * @return this builder instance for fluent chaining + * @see ChunkMessageChannelItemWriter#setMaxWaitTimeouts(int) + */ + public RemoteChunkingMasterStepBuilder maxWaitTimeouts(int maxWaitTimeouts) { + Assert.isTrue(maxWaitTimeouts > 0, "maxWaitTimeouts must be greater than zero"); + this.maxWaitTimeouts = maxWaitTimeouts; + return this; + } + + /** + * Public setter for the throttle limit. This limits the number of pending requests for chunk processing to avoid + * overwhelming the receivers. + * + * @param throttleLimit the throttle limit to set + * @return this builder instance for fluent chaining + * @see ChunkMessageChannelItemWriter#setThrottleLimit(long) + */ + public RemoteChunkingMasterStepBuilder throttleLimit(long throttleLimit) { + Assert.isTrue(throttleLimit > 0, "throttleLimit must be greater than zero"); + this.throttleLimit = throttleLimit; + return this; + } + + /** + * Build a master {@link TaskletStep}. + * + * @return the configured master step + * @see RemoteChunkHandlerFactoryBean + */ + public TaskletStep build() { + Assert.notNull(this.inputChannel, "An InputChannel must be provided"); + Assert.state(this.outputChannel == null || this.messagingTemplate == null, + "You must specify either an outputChannel or a messagingTemplate but not both."); + + // configure messaging template + if (this.messagingTemplate == null) { + this.messagingTemplate = new MessagingTemplate(); + this.messagingTemplate.setDefaultChannel(this.outputChannel); + if (this.logger.isDebugEnabled()) { + this.logger.debug("No messagingTemplate was provided, using a default one"); + } + } + + // configure item writer + ChunkMessageChannelItemWriter chunkMessageChannelItemWriter = new ChunkMessageChannelItemWriter<>(); + chunkMessageChannelItemWriter.setMessagingOperations(this.messagingTemplate); + chunkMessageChannelItemWriter.setMaxWaitTimeouts(this.maxWaitTimeouts); + chunkMessageChannelItemWriter.setThrottleLimit(this.throttleLimit); + chunkMessageChannelItemWriter.setReplyChannel(this.inputChannel); + super.writer(chunkMessageChannelItemWriter); + + return super.build(); + } + + /* + * The following methods override those from parent builders and return + * the current builder type. + * FIXME: Change parent builders to be generic and return current builder + * type in each method. + */ + + @Override + public RemoteChunkingMasterStepBuilder reader(ItemReader reader) { + super.reader(reader); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder repository(JobRepository jobRepository) { + super.repository(jobRepository); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder transactionManager(PlatformTransactionManager transactionManager) { + super.transactionManager(transactionManager); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder listener(Object listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder listener(SkipListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder listener(ChunkListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder transactionAttribute(TransactionAttribute transactionAttribute) { + super.transactionAttribute(transactionAttribute); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder listener(org.springframework.retry.RetryListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder keyGenerator(KeyGenerator keyGenerator) { + super.keyGenerator(keyGenerator); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder retryLimit(int retryLimit) { + super.retryLimit(retryLimit); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder retryPolicy(RetryPolicy retryPolicy) { + super.retryPolicy(retryPolicy); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder backOffPolicy(BackOffPolicy backOffPolicy) { + super.backOffPolicy(backOffPolicy); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder retryContextCache(RetryContextCache retryContextCache) { + super.retryContextCache(retryContextCache); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder skipLimit(int skipLimit) { + super.skipLimit(skipLimit); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder noSkip(Class type) { + super.noSkip(type); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder skip(Class type) { + super.skip(type); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder skipPolicy(SkipPolicy skipPolicy) { + super.skipPolicy(skipPolicy); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder noRollback(Class type) { + super.noRollback(type); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder noRetry(Class type) { + super.noRetry(type); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder retry(Class type) { + super.retry(type); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder stream(ItemStream stream) { + super.stream(stream); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder chunk(int chunkSize) { + super.chunk(chunkSize); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder chunk(CompletionPolicy completionPolicy) { + super.chunk(completionPolicy); + return this; + } + + /** + * This method will throw a {@link UnsupportedOperationException} since + * the item writer of the master step in a remote chunking setup will be + * automatically set to an instance of {@link ChunkMessageChannelItemWriter}. + * + * When building a master step for remote chunking, no item writer must be + * provided. + * + * @throws UnsupportedOperationException if an item writer is provided + * @see ChunkMessageChannelItemWriter + * @see RemoteChunkHandlerFactoryBean#setChunkWriter(ItemWriter) + */ + @Override + public RemoteChunkingMasterStepBuilder writer(ItemWriter writer) throws UnsupportedOperationException { + throw new UnsupportedOperationException("When configuring a master step " + + "for remote chunking, the item writer will be automatically set " + + "to an instance of ChunkMessageChannelItemWriter. The item writer " + + "must not be provided in this case."); + } + + @Override + public RemoteChunkingMasterStepBuilder readerIsTransactionalQueue() { + super.readerIsTransactionalQueue(); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder listener(ItemReadListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder listener(ItemWriteListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder chunkOperations(RepeatOperations repeatTemplate) { + super.chunkOperations(repeatTemplate); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder exceptionHandler(ExceptionHandler exceptionHandler) { + super.exceptionHandler(exceptionHandler); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder stepOperations(RepeatOperations repeatTemplate) { + super.stepOperations(repeatTemplate); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder startLimit(int startLimit) { + super.startLimit(startLimit); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder listener(StepExecutionListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder allowStartIfComplete(boolean allowStartIfComplete) { + super.allowStartIfComplete(allowStartIfComplete); + return this; + } + + @Override + public RemoteChunkingMasterStepBuilder processor(ItemProcessor itemProcessor) { + super.processor(itemProcessor); + return this; + } +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingMasterStepBuilderFactory.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingMasterStepBuilderFactory.java new file mode 100644 index 0000000000..44405a2167 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingMasterStepBuilderFactory.java @@ -0,0 +1,66 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.chunk; + +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.transaction.PlatformTransactionManager; + +/** + * Convenient factory for a {@link RemoteChunkingMasterStepBuilder} which sets + * the {@link JobRepository} and {@link PlatformTransactionManager} automatically. + * + * @deprecated Use {@link RemoteChunkingManagerStepBuilderFactory} instead. + * + * @since 4.1 + * @author Mahmoud Ben Hassine + */ +@Deprecated +public class RemoteChunkingMasterStepBuilderFactory { + + private JobRepository jobRepository; + + private PlatformTransactionManager transactionManager; + + /** + * Create a new {@link RemoteChunkingMasterStepBuilderFactory}. + * + * @param jobRepository the job repository to use + * @param transactionManager the transaction manager to use + */ + public RemoteChunkingMasterStepBuilderFactory( + JobRepository jobRepository, + PlatformTransactionManager transactionManager) { + + this.jobRepository = jobRepository; + this.transactionManager = transactionManager; + } + + /** + * Creates a {@link RemoteChunkingMasterStepBuilder} and initializes its job + * repository and transaction manager. + * + * @param name the name of the step + * @param type of input items + * @param type of output items + * @return a {@link RemoteChunkingMasterStepBuilder} + */ + public RemoteChunkingMasterStepBuilder get(String name) { + return new RemoteChunkingMasterStepBuilder(name) + .repository(this.jobRepository) + .transactionManager(this.transactionManager); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingWorkerBuilder.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingWorkerBuilder.java new file mode 100644 index 0000000000..9ef9bee4db --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/RemoteChunkingWorkerBuilder.java @@ -0,0 +1,132 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.chunk; + +import org.springframework.batch.core.step.item.SimpleChunkProcessor; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.PassThroughItemProcessor; +import org.springframework.integration.dsl.IntegrationFlow; +import org.springframework.integration.dsl.IntegrationFlows; +import org.springframework.messaging.MessageChannel; +import org.springframework.util.Assert; + +/** + * Builder for a worker in a remote chunking setup. This builder: + * + *
          + *
        • creates a {@link ChunkProcessorChunkHandler} with the provided + * item processor and writer. If no item processor is provided, a + * {@link PassThroughItemProcessor} will be used
        • + *
        • creates an {@link IntegrationFlow} with the + * {@link ChunkProcessorChunkHandler} as a service activator which listens + * to incoming requests on inputChannel and sends replies + * on outputChannel
        • + *
        + * + * @param type of input items + * @param type of output items + * + * @since 4.1 + * @author Mahmoud Ben Hassine + */ +public class RemoteChunkingWorkerBuilder { + + private static final String SERVICE_ACTIVATOR_METHOD_NAME = "handleChunk"; + + private ItemProcessor itemProcessor; + private ItemWriter itemWriter; + private MessageChannel inputChannel; + private MessageChannel outputChannel; + + /** + * Set the {@link ItemProcessor} to use to process items sent by the master + * step. + * + * @param itemProcessor to use + * @return this builder instance for fluent chaining + */ + public RemoteChunkingWorkerBuilder itemProcessor(ItemProcessor itemProcessor) { + Assert.notNull(itemProcessor, "itemProcessor must not be null"); + this.itemProcessor = itemProcessor; + return this; + } + + /** + * Set the {@link ItemWriter} to use to write items sent by the master step. + * + * @param itemWriter to use + * @return this builder instance for fluent chaining + */ + public RemoteChunkingWorkerBuilder itemWriter(ItemWriter itemWriter) { + Assert.notNull(itemWriter, "itemWriter must not be null"); + this.itemWriter = itemWriter; + return this; + } + + /** + * Set the input channel on which items sent by the master are received. + * + * @param inputChannel the input channel + * @return this builder instance for fluent chaining + */ + public RemoteChunkingWorkerBuilder inputChannel(MessageChannel inputChannel) { + Assert.notNull(inputChannel, "inputChannel must not be null"); + this.inputChannel = inputChannel; + return this; + } + + /** + * Set the output channel on which replies will be sent to the master step. + * + * @param outputChannel the output channel + * @return this builder instance for fluent chaining + */ + public RemoteChunkingWorkerBuilder outputChannel(MessageChannel outputChannel) { + Assert.notNull(outputChannel, "outputChannel must not be null"); + this.outputChannel = outputChannel; + return this; + } + + /** + * Create an {@link IntegrationFlow} with a {@link ChunkProcessorChunkHandler} + * configured as a service activator listening to the input channel and replying + * on the output channel. + * + * @return the integration flow + */ + @SuppressWarnings({"unchecked", "rawtypes"}) + public IntegrationFlow build() { + Assert.notNull(this.itemWriter, "An ItemWriter must be provided"); + Assert.notNull(this.inputChannel, "An InputChannel must be provided"); + Assert.notNull(this.outputChannel, "An OutputChannel must be provided"); + + if(this.itemProcessor == null) { + this.itemProcessor = new PassThroughItemProcessor(); + } + SimpleChunkProcessor chunkProcessor = new SimpleChunkProcessor<>(this.itemProcessor, this.itemWriter); + + ChunkProcessorChunkHandler chunkProcessorChunkHandler = new ChunkProcessorChunkHandler<>(); + chunkProcessorChunkHandler.setChunkProcessor(chunkProcessor); + + return IntegrationFlows + .from(this.inputChannel) + .handle(chunkProcessorChunkHandler, SERVICE_ACTIVATOR_METHOD_NAME) + .channel(this.outputChannel) + .get(); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/StepContributionSource.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/StepContributionSource.java new file mode 100644 index 0000000000..9b6997f677 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/StepContributionSource.java @@ -0,0 +1,41 @@ +/* + * Copyright 2006-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.chunk; + +import java.util.Collection; + +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.StepExecution; + +/** + * A source of {@link StepContribution} instances that can be aggregated and used to update an ongoing + * {@link StepExecution}. + * + * @author Dave Syer + * + */ +public interface StepContributionSource { + + /** + * Get the currently available contributions and drain the source. The next call would return an empty collection, + * unless new contributions have arrived. + * + * @return a collection of {@link StepContribution} instances + */ + Collection getStepContributions(); + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/package-info.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/package-info.java new file mode 100644 index 0000000000..e375ea5649 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/chunk/package-info.java @@ -0,0 +1,10 @@ +/** + * Components for remote chunking. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.integration.chunk; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/annotation/BatchIntegrationConfiguration.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/annotation/BatchIntegrationConfiguration.java new file mode 100644 index 0000000000..d377da9428 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/annotation/BatchIntegrationConfiguration.java @@ -0,0 +1,94 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.config.annotation; + +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.integration.chunk.RemoteChunkingMasterStepBuilderFactory; +import org.springframework.batch.integration.chunk.RemoteChunkingManagerStepBuilderFactory; +import org.springframework.batch.integration.chunk.RemoteChunkingWorkerBuilder; +import org.springframework.batch.integration.partition.RemotePartitioningMasterStepBuilderFactory; +import org.springframework.batch.integration.partition.RemotePartitioningManagerStepBuilderFactory; +import org.springframework.batch.integration.partition.RemotePartitioningWorkerStepBuilderFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.transaction.PlatformTransactionManager; + +/** + * Base configuration class for Spring Batch Integration factory beans. + * + * @since 4.1 + * @author Mahmoud Ben Hassine + */ +@Configuration +public class BatchIntegrationConfiguration { + + private JobExplorer jobExplorer; + + private JobRepository jobRepository; + + private PlatformTransactionManager transactionManager; + + @Autowired + public BatchIntegrationConfiguration( + JobRepository jobRepository, + JobExplorer jobExplorer, + PlatformTransactionManager transactionManager) { + + this.jobRepository = jobRepository; + this.jobExplorer = jobExplorer; + this.transactionManager = transactionManager; + } + + @Deprecated + @Bean + public RemoteChunkingMasterStepBuilderFactory remoteChunkingMasterStepBuilderFactory() { + return new RemoteChunkingMasterStepBuilderFactory(this.jobRepository, + this.transactionManager); + } + + @Bean + public RemoteChunkingManagerStepBuilderFactory remoteChunkingManagerStepBuilderFactory() { + return new RemoteChunkingManagerStepBuilderFactory(this.jobRepository, + this.transactionManager); + } + + @Bean + public RemoteChunkingWorkerBuilder remoteChunkingWorkerBuilder() { + return new RemoteChunkingWorkerBuilder<>(); + } + + @Deprecated + @Bean + public RemotePartitioningMasterStepBuilderFactory remotePartitioningMasterStepBuilderFactory() { + return new RemotePartitioningMasterStepBuilderFactory(this.jobRepository, + this.jobExplorer, this.transactionManager); + } + + @Bean + public RemotePartitioningManagerStepBuilderFactory remotePartitioningManagerStepBuilderFactory() { + return new RemotePartitioningManagerStepBuilderFactory(this.jobRepository, + this.jobExplorer, this.transactionManager); + } + + @Bean + public RemotePartitioningWorkerStepBuilderFactory remotePartitioningWorkerStepBuilderFactory() { + return new RemotePartitioningWorkerStepBuilderFactory(this.jobRepository, + this.jobExplorer, this.transactionManager); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/annotation/EnableBatchIntegration.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/annotation/EnableBatchIntegration.java new file mode 100644 index 0000000000..8d5897ec47 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/annotation/EnableBatchIntegration.java @@ -0,0 +1,143 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.config.annotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +import org.springframework.batch.integration.chunk.RemoteChunkingWorkerBuilder; +import org.springframework.batch.integration.chunk.RemoteChunkingManagerStepBuilderFactory; +import org.springframework.batch.integration.partition.RemotePartitioningManagerStepBuilderFactory; +import org.springframework.batch.integration.partition.RemotePartitioningWorkerStepBuilderFactory; +import org.springframework.context.annotation.Import; +import org.springframework.integration.config.EnableIntegration; + +/** + * Enable Spring Batch Integration features and provide a base configuration for + * setting up remote chunking or partitioning infrastructure beans. + * + * By adding this annotation on a {@link org.springframework.context.annotation.Configuration} + * class, it will be possible to autowire the following beans: + * + *
          + *
        • {@link RemoteChunkingManagerStepBuilderFactory}: + * used to create a manager step of a remote chunking setup by automatically + * setting the job repository and transaction manager.
        • + *
        • {@link RemoteChunkingWorkerBuilder}: used to create the integration + * flow on the worker side of a remote chunking setup.
        • + *
        • {@link RemotePartitioningManagerStepBuilderFactory}: used to create + * a manager step of a remote partitioning setup by automatically setting + * the job repository, job explorer, bean factory and transaction manager.
        • + *
        • {@link RemotePartitioningWorkerStepBuilderFactory}: used to create + * a worker step of a remote partitioning setup by automatically setting + * the job repository, job explorer, bean factory and transaction manager.
        • + *
        + * + * For remote chunking, an example of a configuration class would be: + * + *
        + * @Configuration
        + * @EnableBatchIntegration
        + * @EnableBatchProcessing
        + * public class RemoteChunkingAppConfig {
        + *
        + * 	@Autowired
        + * 	private RemoteChunkingManagerStepBuilderFactory managerStepBuilderFactory;
        + *
        + * 	@Autowired
        + * 	private RemoteChunkingWorkerBuilder workerBuilder;
        + *
        + * 	@Bean
        + * 	public TaskletStep managerStep() {
        + *       	 return this.managerStepBuilderFactory
        + *       		.get("managerStep")
        + *       		.chunk(100)
        + *       		.reader(itemReader())
        + *       		.outputChannel(outgoingRequestsToWorkers())
        + *       		.inputChannel(incomingRepliesFromWorkers())
        + *       		.build();
        + * 	}
        + *
        + * 	@Bean
        + * 	public IntegrationFlow worker() {
        + *       	 return this.workerBuilder
        + *       		.itemProcessor(itemProcessor())
        + *       		.itemWriter(itemWriter())
        + *       		.inputChannel(incomingRequestsFromManager())
        + *       		.outputChannel(outgoingRepliesToManager())
        + *       		.build();
        + * 	}
        + *
        + * 	// Middleware beans omitted
        + *
        + * }
        + * 
        + * + * For remote partitioning, an example of a configuration class would be: + * + *
        + * @Configuration
        + * @EnableBatchIntegration
        + * @EnableBatchProcessing
        + * public class RemotePartitioningAppConfig {
        + *
        + * 	@Autowired
        + * 	private RemotePartitioningManagerStepBuilderFactory managerStepBuilderFactory;
        + *
        + * 	@Autowired
        + * 	private RemotePartitioningWorkerStepBuilderFactory workerStepBuilderFactory;
        + *
        + * 	@Bean
        + * 	public Step managerStep() {
        + *       	 return this.managerStepBuilderFactory
        + *       		.get("managerStep")
        + *       		.partitioner("workerStep", partitioner())
        + *       		.gridSize(10)
        + *       		.outputChannel(outgoingRequestsToWorkers())
        + *       		.inputChannel(incomingRepliesFromWorkers())
        + *       		.build();
        + * 	}
        + *
        + * 	@Bean
        + * 	public Step workerStep() {
        + *       	 return this.workerStepBuilderFactory
        + *       		.get("workerStep")
        + *       		.inputChannel(incomingRequestsFromManager())
        + *       		.outputChannel(outgoingRepliesToManager())
        + *       		.chunk(100)
        + *       		.reader(itemReader())
        + *       		.processor(itemProcessor())
        + *       		.writer(itemWriter())
        + *       		.build();
        + * 	}
        + *
        + * 	// Middleware beans omitted
        + *
        + * }
        + * 
        + * @since 4.1 + * @author Mahmoud Ben Hassine + */ +@Target(ElementType.TYPE) +@Retention(RetentionPolicy.RUNTIME) +@Documented +@EnableIntegration +@Import(BatchIntegrationConfiguration.class) +public @interface EnableBatchIntegration { +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/annotation/package-info.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/annotation/package-info.java new file mode 100644 index 0000000000..40a894455a --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/annotation/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * APIs for the configuration of Spring Integration components through annotations. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.integration.config.annotation; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/BatchIntegrationNamespaceHandler.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/BatchIntegrationNamespaceHandler.java new file mode 100644 index 0000000000..e870b693c2 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/BatchIntegrationNamespaceHandler.java @@ -0,0 +1,43 @@ +/* + * Copyright 2002-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.config.xml; + +import org.springframework.integration.config.xml.AbstractIntegrationNamespaceHandler; + +/** + * The namespace handler for the Spring Batch Integration namespace. + * + * @author Gunnar Hillert + * @author Chris Schaefer + * @author Mahmoud Ben Hassine + * @since 1.3 + */ +public class BatchIntegrationNamespaceHandler extends AbstractIntegrationNamespaceHandler { + /* (non-Javadoc) + * @see org.springframework.beans.factory.xml.NamespaceHandler#init() + */ + public void init() { + this.registerBeanDefinitionParser("job-launching-gateway", new JobLaunchingGatewayParser()); + RemoteChunkingManagerParser remoteChunkingManagerParser = new RemoteChunkingManagerParser(); + this.registerBeanDefinitionParser("remote-chunking-manager", remoteChunkingManagerParser); + RemoteChunkingWorkerParser remoteChunkingWorkerParser = new RemoteChunkingWorkerParser(); + this.registerBeanDefinitionParser("remote-chunking-worker", remoteChunkingWorkerParser); + + // TODO remove the following when related deprecated APIs are removed + this.registerBeanDefinitionParser("remote-chunking-master", remoteChunkingManagerParser); + this.registerBeanDefinitionParser("remote-chunking-slave", remoteChunkingWorkerParser); + } +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParser.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParser.java new file mode 100644 index 0000000000..7bb3f769d8 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParser.java @@ -0,0 +1,78 @@ +/* + * Copyright 2002-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.config.xml; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.integration.launch.JobLaunchingGateway; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.integration.config.xml.AbstractConsumerEndpointParser; +import org.springframework.integration.config.xml.IntegrationNamespaceUtils; +import org.springframework.util.StringUtils; +import org.w3c.dom.Element; + +/** + * The parser for the Job-Launching Gateway, which will instantiate a + * {@link JobLaunchingGatewayParser}. If no {@link JobLauncher} reference has + * been provided, this parse will use the use the globally registered bean + * 'jobLauncher'. + * + * @author Gunnar Hillert + * @since 1.3 + * + */ +public class JobLaunchingGatewayParser extends AbstractConsumerEndpointParser { + + private static final Log logger = LogFactory.getLog(JobLaunchingGatewayParser.class); + + @Override + protected String getInputChannelAttributeName() { + return "request-channel"; + } + + @Override + protected BeanDefinitionBuilder parseHandler(Element element, ParserContext parserContext) { + + final BeanDefinitionBuilder jobLaunchingGatewayBuilder = + BeanDefinitionBuilder.genericBeanDefinition(JobLaunchingGateway.class); + + final String jobLauncher = element.getAttribute("job-launcher"); + + if (StringUtils.hasText(jobLauncher)) { + jobLaunchingGatewayBuilder.addConstructorArgReference(jobLauncher); + } + else { + if (logger.isDebugEnabled()) { + logger.debug("No jobLauncher specified, using default 'jobLauncher' reference instead."); + } + jobLaunchingGatewayBuilder.addConstructorArgReference("jobLauncher"); + } + + IntegrationNamespaceUtils.setValueIfAttributeDefined(jobLaunchingGatewayBuilder, element, "reply-timeout", "sendTimeout"); + + final String replyChannel = element.getAttribute("reply-channel"); + + if (StringUtils.hasText(replyChannel)) { + jobLaunchingGatewayBuilder.addPropertyReference("outputChannel", replyChannel); + } + + return jobLaunchingGatewayBuilder; + + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParser.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParser.java new file mode 100644 index 0000000000..0959fa4e0b --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParser.java @@ -0,0 +1,84 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.config.xml; + +import org.springframework.batch.integration.chunk.ChunkMessageChannelItemWriter; +import org.springframework.batch.integration.chunk.RemoteChunkHandlerFactoryBean; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.support.BeanDefinitionRegistry; +import org.springframework.beans.factory.xml.AbstractBeanDefinitionParser; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.util.Assert; +import org.w3c.dom.Element; + +/** + *

        + * Parser for the remote-chunking-manager namespace element. + *

        + * + * @author Chris Schaefer + * @author Mahmoud Ben Hassine + * @since 3.1 + */ +public class RemoteChunkingManagerParser extends AbstractBeanDefinitionParser { + private static final String MESSAGE_TEMPLATE_ATTRIBUTE = "message-template"; + private static final String STEP_ATTRIBUTE = "step"; + private static final String REPLY_CHANNEL_ATTRIBUTE = "reply-channel"; + private static final String MESSAGING_OPERATIONS_PROPERTY = "messagingOperations"; + private static final String REPLY_CHANNEL_PROPERTY = "replyChannel"; + private static final String CHUNK_WRITER_PROPERTY = "chunkWriter"; + private static final String STEP_PROPERTY = "step"; + private static final String CHUNK_HANDLER_BEAN_NAME_PREFIX = "remoteChunkHandlerFactoryBean_"; + + @Override + public AbstractBeanDefinition parseInternal(Element element, ParserContext parserContext) { + String id = element.getAttribute(ID_ATTRIBUTE); + Assert.hasText(id, "The id attribute must be specified"); + + String messageTemplate = element.getAttribute(MESSAGE_TEMPLATE_ATTRIBUTE); + Assert.hasText(messageTemplate, "The message-template attribute must be specified"); + + String step = element.getAttribute(STEP_ATTRIBUTE); + Assert.hasText(step, "The step attribute must be specified"); + + String replyChannel = element.getAttribute(REPLY_CHANNEL_ATTRIBUTE); + Assert.hasText(replyChannel, "The reply-channel attribute must be specified"); + + BeanDefinitionRegistry beanDefinitionRegistry = parserContext.getRegistry(); + + BeanDefinition chunkMessageChannelItemWriter = + BeanDefinitionBuilder + .genericBeanDefinition(ChunkMessageChannelItemWriter.class) + .addPropertyReference(MESSAGING_OPERATIONS_PROPERTY, messageTemplate) + .addPropertyReference(REPLY_CHANNEL_PROPERTY, replyChannel) + .getBeanDefinition(); + + beanDefinitionRegistry.registerBeanDefinition(id, chunkMessageChannelItemWriter); + + BeanDefinition remoteChunkHandlerFactoryBean = + BeanDefinitionBuilder + .genericBeanDefinition(RemoteChunkHandlerFactoryBean.class) + .addPropertyValue(CHUNK_WRITER_PROPERTY, chunkMessageChannelItemWriter) + .addPropertyValue(STEP_PROPERTY, step) + .getBeanDefinition(); + + beanDefinitionRegistry.registerBeanDefinition(CHUNK_HANDLER_BEAN_NAME_PREFIX + step, remoteChunkHandlerFactoryBean); + + return null; + } +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParser.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParser.java new file mode 100644 index 0000000000..f0052f4adb --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParser.java @@ -0,0 +1,119 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.config.xml; + +import org.w3c.dom.Element; + +import org.springframework.batch.core.step.item.SimpleChunkProcessor; +import org.springframework.batch.integration.chunk.ChunkProcessorChunkHandler; +import org.springframework.batch.item.support.PassThroughItemProcessor; +import org.springframework.beans.factory.config.BeanDefinition; +import org.springframework.beans.factory.config.RuntimeBeanReference; +import org.springframework.beans.factory.support.AbstractBeanDefinition; +import org.springframework.beans.factory.support.BeanDefinitionBuilder; +import org.springframework.beans.factory.support.BeanDefinitionRegistry; +import org.springframework.beans.factory.xml.AbstractBeanDefinitionParser; +import org.springframework.beans.factory.xml.ParserContext; +import org.springframework.integration.config.ServiceActivatorFactoryBean; +import org.springframework.integration.config.xml.AbstractConsumerEndpointParser; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + *

        + * Parser for the remote-chunking-worker namespace element. If an + * {@link org.springframework.batch.item.ItemProcessor} is not provided, an + * {@link org.springframework.batch.item.support.PassThroughItemProcessor} will be + * configured. + *

        + * + * @author Chris Schaefer + * @author Mahmoud Ben Hassine + * @since 3.1 + */ +public class RemoteChunkingWorkerParser extends AbstractBeanDefinitionParser { + private static final String INPUT_CHANNEL_ATTRIBUTE = "input-channel"; + private static final String OUTPUT_CHANNEL_ATTRIBUTE = "output-channel"; + private static final String ITEM_PROCESSOR_ATTRIBUTE = "item-processor"; + private static final String ITEM_WRITER_ATTRIBUTE = "item-writer"; + private static final String ITEM_PROCESSOR_PROPERTY_NAME = "itemProcessor"; + private static final String ITEM_WRITER_PROPERTY_NAME = "itemWriter"; + private static final String CHUNK_PROCESSOR_PROPERTY_NAME = "chunkProcessor"; + private static final String CHUNK_PROCESSOR_CHUNK_HANDLER_BEAN_NAME_PREFIX = "chunkProcessorChunkHandler_"; + + @Override + protected AbstractBeanDefinition parseInternal(Element element, ParserContext parserContext) { + String id = element.getAttribute(ID_ATTRIBUTE); + Assert.hasText(id, "The id attribute must be specified"); + + String inputChannel = element.getAttribute(INPUT_CHANNEL_ATTRIBUTE); + Assert.hasText(inputChannel, "The input-channel attribute must be specified"); + + String outputChannel = element.getAttribute(OUTPUT_CHANNEL_ATTRIBUTE); + Assert.hasText(outputChannel, "The output-channel attribute must be specified"); + + String itemProcessor = element.getAttribute(ITEM_PROCESSOR_ATTRIBUTE); + + String itemWriter = element.getAttribute(ITEM_WRITER_ATTRIBUTE); + Assert.hasText(itemWriter, "The item-writer attribute must be specified"); + + BeanDefinitionRegistry beanDefinitionRegistry = parserContext.getRegistry(); + + BeanDefinitionBuilder chunkProcessorBuilder = + BeanDefinitionBuilder + .genericBeanDefinition(SimpleChunkProcessor.class) + .addPropertyReference(ITEM_WRITER_PROPERTY_NAME, itemWriter); + + if(StringUtils.hasText(itemProcessor)) { + chunkProcessorBuilder.addPropertyReference(ITEM_PROCESSOR_PROPERTY_NAME, itemProcessor); + } else { + chunkProcessorBuilder.addPropertyValue(ITEM_PROCESSOR_PROPERTY_NAME, new PassThroughItemProcessor<>()); + } + + BeanDefinition chunkProcessorChunkHandler = + BeanDefinitionBuilder + .genericBeanDefinition(ChunkProcessorChunkHandler.class) + .addPropertyValue(CHUNK_PROCESSOR_PROPERTY_NAME, chunkProcessorBuilder.getBeanDefinition()) + .getBeanDefinition(); + + beanDefinitionRegistry.registerBeanDefinition(CHUNK_PROCESSOR_CHUNK_HANDLER_BEAN_NAME_PREFIX + id, chunkProcessorChunkHandler); + + new ServiceActivatorParser(id).parse(element, parserContext); + + return null; + } + + private static class ServiceActivatorParser extends AbstractConsumerEndpointParser { + private static final String TARGET_METHOD_NAME_PROPERTY_NAME = "targetMethodName"; + private static final String TARGET_OBJECT_PROPERTY_NAME = "targetObject"; + private static final String HANDLE_CHUNK_METHOD_NAME = "handleChunk"; + private static final String CHUNK_PROCESSOR_CHUNK_HANDLER_BEAN_NAME_PREFIX = "chunkProcessorChunkHandler_"; + + private String id; + + public ServiceActivatorParser(String id) { + this.id = id; + } + + @Override + protected BeanDefinitionBuilder parseHandler(Element element, ParserContext parserContext) { + BeanDefinitionBuilder builder = BeanDefinitionBuilder.genericBeanDefinition(ServiceActivatorFactoryBean.class); + builder.addPropertyValue(TARGET_METHOD_NAME_PROPERTY_NAME, HANDLE_CHUNK_METHOD_NAME); + builder.addPropertyValue(TARGET_OBJECT_PROPERTY_NAME, new RuntimeBeanReference(CHUNK_PROCESSOR_CHUNK_HANDLER_BEAN_NAME_PREFIX + id)); + return builder; + } + } +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/package-info.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/package-info.java new file mode 100644 index 0000000000..aaf3e6a87e --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/config/xml/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * APIs for the configuration of Spring Integration components through XML. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.integration.config.xml; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/JobLaunchRequest.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/JobLaunchRequest.java new file mode 100644 index 0000000000..116b4975c6 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/JobLaunchRequest.java @@ -0,0 +1,62 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.launch; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobParameters; + +/** + * Encapsulation of a {@link Job} and its {@link JobParameters} forming a request for a job to be launched. + * + * @author Dave Syer + * + */ +public class JobLaunchRequest { + + private final Job job; + + private final JobParameters jobParameters; + + /** + * @param job job to be launched + * @param jobParameters parameters to run the job with + */ + public JobLaunchRequest(Job job, JobParameters jobParameters) { + super(); + this.job = job; + this.jobParameters = jobParameters; + } + + /** + * @return the {@link Job} to be executed + */ + public Job getJob() { + return this.job; + } + + /** + * @return the {@link JobParameters} for this request + */ + public JobParameters getJobParameters() { + return this.jobParameters; + } + + @Override + public String toString() { + return "JobLaunchRequest: " + job.getName() + ", parameters=" + jobParameters; + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/JobLaunchRequestHandler.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/JobLaunchRequestHandler.java new file mode 100644 index 0000000000..20f18afadb --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/JobLaunchRequestHandler.java @@ -0,0 +1,31 @@ +/* + * Copyright 2006-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.launch; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobExecutionException; + +/** + * Interface for handling a {@link JobLaunchRequest} and returning a {@link JobExecution}. + * @author Dave Syer + * + */ +public interface JobLaunchRequestHandler { + + JobExecution launch(JobLaunchRequest request) throws JobExecutionException; + +} \ No newline at end of file diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/JobLaunchingGateway.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/JobLaunchingGateway.java new file mode 100644 index 0000000000..c1cb65b20e --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/JobLaunchingGateway.java @@ -0,0 +1,85 @@ +/* + * Copyright 2002-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.launch; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobExecutionException; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.integration.handler.AbstractReplyProducingMessageHandler; +import org.springframework.messaging.Message; +import org.springframework.messaging.MessageHandlingException; +import org.springframework.util.Assert; + +/** + * The {@link JobLaunchingGateway} is used to launch Batch Jobs. Internally it + * delegates to a {@link JobLaunchingMessageHandler}. + * + * @author Gunnar Hillert + * + * @since 1.3 + */ +public class JobLaunchingGateway extends AbstractReplyProducingMessageHandler { + + private final JobLaunchingMessageHandler jobLaunchingMessageHandler; + + /** + * Constructor taking a {@link JobLauncher} as parameter. + * + * @param jobLauncher Must not be null. + * + */ + public JobLaunchingGateway(JobLauncher jobLauncher) { + Assert.notNull(jobLauncher, "jobLauncher must not be null."); + this.jobLaunchingMessageHandler = new JobLaunchingMessageHandler(jobLauncher); + } + + /** + * Launches a Batch Job using the provided request {@link Message}. The payload + * of the {@link Message} must be an instance of {@link JobLaunchRequest}. + * + * @param requestMessage must not be null. + * @return Generally a {@link JobExecution} will always be returned. An + * exception ({@link MessageHandlingException}) will only be thrown if there + * is a failure to start the job. The cause of the exception will be a + * {@link JobExecutionException}. + * + * @throws MessageHandlingException when a job cannot be launched + */ + @Override + protected Object handleRequestMessage(Message requestMessage) { + + Assert.notNull(requestMessage, "The provided requestMessage must not be null."); + + final Object payload = requestMessage.getPayload(); + + Assert.isInstanceOf(JobLaunchRequest.class, payload, "The payload must be of type JobLaunchRequest."); + + final JobLaunchRequest jobLaunchRequest = (JobLaunchRequest) payload; + + final JobExecution jobExecution; + + try { + jobExecution = this.jobLaunchingMessageHandler.launch(jobLaunchRequest); + } catch (JobExecutionException e) { + throw new MessageHandlingException(requestMessage, e); + } + + return jobExecution; + + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/JobLaunchingMessageHandler.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/JobLaunchingMessageHandler.java new file mode 100644 index 0000000000..e905151bad --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/JobLaunchingMessageHandler.java @@ -0,0 +1,53 @@ +/* + * Copyright 2006-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.launch; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobExecutionException; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.integration.annotation.ServiceActivator; + +/** + * Message handler which uses strategies to convert a Message into a job and a set of job parameters + * @author Jonas Partner + * @author Dave Syer + * @author Gunnar Hillert + * + */ +public class JobLaunchingMessageHandler implements JobLaunchRequestHandler { + + private final JobLauncher jobLauncher; + + /** + * @param jobLauncher {@link org.springframework.batch.core.launch.JobLauncher} used to execute Spring Batch jobs + */ + public JobLaunchingMessageHandler(JobLauncher jobLauncher) { + super(); + this.jobLauncher = jobLauncher; + } + + @ServiceActivator + public JobExecution launch(JobLaunchRequest request) throws JobExecutionException { + Job job = request.getJob(); + JobParameters jobParameters = request.getJobParameters(); + + return jobLauncher.run(job, jobParameters); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/package-info.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/package-info.java new file mode 100644 index 0000000000..0c5d640f42 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/launch/package-info.java @@ -0,0 +1,10 @@ +/** + * Message based job launching components. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.integration.launch; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/BeanFactoryStepLocator.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/BeanFactoryStepLocator.java new file mode 100644 index 0000000000..d034d84d4f --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/BeanFactoryStepLocator.java @@ -0,0 +1,48 @@ +package org.springframework.batch.integration.partition; + +import java.util.Arrays; +import java.util.Collection; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.step.StepLocator; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.beans.factory.BeanFactoryAware; +import org.springframework.beans.factory.ListableBeanFactory; +import org.springframework.util.Assert; + +/** + * A {@link StepLocator} implementation that just looks in its enclosing bean + * factory for components of type {@link Step}. + * + * @author Dave Syer + * @author Mahmoud Ben Hassine + * + */ +public class BeanFactoryStepLocator implements StepLocator, BeanFactoryAware { + + private BeanFactory beanFactory; + + public void setBeanFactory(BeanFactory beanFactory) throws BeansException { + this.beanFactory = beanFactory; + } + + /** + * Look up a bean with the provided name of type {@link Step}. + * @see StepLocator#getStep(String) + */ + public Step getStep(String stepName) { + return beanFactory.getBean(stepName, Step.class); + } + + /** + * Look in the bean factory for all beans of type {@link Step}. + * @throws IllegalStateException if the {@link BeanFactory} is not listable + * @see StepLocator#getStepNames() + */ + public Collection getStepNames() { + Assert.state(beanFactory instanceof ListableBeanFactory, "BeanFactory is not listable."); + return Arrays.asList(((ListableBeanFactory) beanFactory).getBeanNamesForType(Step.class)); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/MessageChannelPartitionHandler.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/MessageChannelPartitionHandler.java new file mode 100644 index 0000000000..bac0462b61 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/MessageChannelPartitionHandler.java @@ -0,0 +1,304 @@ +package org.springframework.batch.integration.partition; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.Iterator; +import java.util.List; +import java.util.Set; +import java.util.concurrent.Callable; +import java.util.concurrent.Future; +import java.util.concurrent.TimeUnit; + +import javax.sql.DataSource; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.explore.support.JobExplorerFactoryBean; +import org.springframework.batch.core.partition.PartitionHandler; +import org.springframework.batch.core.partition.StepExecutionSplitter; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.poller.DirectPoller; +import org.springframework.batch.poller.Poller; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.integration.MessageTimeoutException; +import org.springframework.integration.annotation.Aggregator; +import org.springframework.integration.annotation.MessageEndpoint; +import org.springframework.integration.annotation.Payloads; +import org.springframework.integration.channel.QueueChannel; +import org.springframework.integration.core.MessagingTemplate; +import org.springframework.integration.support.MessageBuilder; +import org.springframework.messaging.Message; +import org.springframework.messaging.MessageChannel; +import org.springframework.messaging.PollableChannel; +import org.springframework.util.Assert; +import org.springframework.util.CollectionUtils; + +/** + * A {@link PartitionHandler} that uses {@link MessageChannel} instances to send instructions to remote workers and + * receive their responses. The {@link MessageChannel} provides a nice abstraction so that the location of the workers + * and the transport used to communicate with them can be changed at run time. The communication with the remote workers + * does not need to be transactional or have guaranteed delivery, so a local thread pool based implementation works as + * well as a remote web service or JMS implementation. If a remote worker fails, the job will fail and can be restarted + * to pick up missing messages and processing. The remote workers need access to the Spring Batch {@link JobRepository} + * so that the shared state across those restarts can be managed centrally. + * + * While a {@link org.springframework.messaging.MessageChannel} is used for sending the requests to the workers, the + * worker's responses can be obtained in one of two ways: + *
          + *
        • A reply channel - Workers will respond with messages that will be aggregated via this component.
        • + *
        • Polling the job repository - Since the state of each worker is maintained independently within the job + * repository, we can poll the store to determine the state without the need of the workers to formally respond.
        • + *
        + * + * Note: The reply channel for this is instance based. Sharing this component across + * multiple step instances may result in the crossing of messages. It's recommended that + * this component be step or job scoped. + * + * @author Dave Syer + * @author Will Schipp + * @author Michael Minella + * @author Mahmoud Ben Hassine + * + */ +@MessageEndpoint +public class MessageChannelPartitionHandler implements PartitionHandler, InitializingBean { + + private static Log logger = LogFactory.getLog(MessageChannelPartitionHandler.class); + + private int gridSize = 1; + + private MessagingTemplate messagingGateway; + + private String stepName; + + private long pollInterval = 10000; + + private JobExplorer jobExplorer; + + private boolean pollRepositoryForResults = false; + + private long timeout = -1; + + private DataSource dataSource; + + /** + * pollable channel for the replies + */ + private PollableChannel replyChannel; + + @Override + public void afterPropertiesSet() throws Exception { + Assert.notNull(stepName, "A step name must be provided for the remote workers."); + Assert.state(messagingGateway != null, "The MessagingOperations must be set"); + + pollRepositoryForResults = !(dataSource == null && jobExplorer == null); + + if(pollRepositoryForResults) { + logger.debug("MessageChannelPartitionHandler is configured to poll the job repository for worker results"); + } + + if(dataSource != null && jobExplorer == null) { + JobExplorerFactoryBean jobExplorerFactoryBean = new JobExplorerFactoryBean(); + jobExplorerFactoryBean.setDataSource(dataSource); + jobExplorerFactoryBean.afterPropertiesSet(); + jobExplorer = jobExplorerFactoryBean.getObject(); + } + + if (!pollRepositoryForResults && replyChannel == null) { + replyChannel = new QueueChannel(); + }//end if + + } + + /** + * When using job repository polling, the time limit to wait. + * + * @param timeout milliseconds to wait, defaults to -1 (no timeout). + */ + public void setTimeout(long timeout) { + this.timeout = timeout; + } + + /** + * {@link org.springframework.batch.core.explore.JobExplorer} to use to query the job repository. Either this or + * a {@link javax.sql.DataSource} is required when using job repository polling. + * + * @param jobExplorer {@link org.springframework.batch.core.explore.JobExplorer} to use for lookups + */ + public void setJobExplorer(JobExplorer jobExplorer) { + this.jobExplorer = jobExplorer; + } + + /** + * How often to poll the job repository for the status of the workers. + * + * @param pollInterval milliseconds between polls, defaults to 10000 (10 seconds). + */ + public void setPollInterval(long pollInterval) { + this.pollInterval = pollInterval; + } + + /** + * {@link javax.sql.DataSource} pointing to the job repository + * + * @param dataSource {@link javax.sql.DataSource} that points to the job repository's store + */ + public void setDataSource(DataSource dataSource) { + this.dataSource = dataSource; + } + + /** + * A pre-configured gateway for sending and receiving messages to the remote workers. Using this property allows a + * large degree of control over the timeouts and other properties of the send. It should have channels set up + * internally:
        • request channel capable of accepting {@link StepExecutionRequest} payloads
        • reply + * channel that returns a list of {@link StepExecution} results
        The timeout for the reply should be set + * sufficiently long that the remote steps have time to complete. + * + * @param messagingGateway the {@link org.springframework.integration.core.MessagingTemplate} to set + */ + public void setMessagingOperations(MessagingTemplate messagingGateway) { + this.messagingGateway = messagingGateway; + } + + /** + * Passed to the {@link StepExecutionSplitter} in the {@link #handle(StepExecutionSplitter, StepExecution)} method, + * instructing it how many {@link StepExecution} instances are required, ideally. The {@link StepExecutionSplitter} + * is allowed to ignore the grid size in the case of a restart, since the input data partitions must be preserved. + * + * @param gridSize the number of step executions that will be created + */ + public void setGridSize(int gridSize) { + this.gridSize = gridSize; + } + + /** + * The name of the {@link Step} that will be used to execute the partitioned {@link StepExecution}. This is a + * regular Spring Batch step, with all the business logic required to complete an execution based on the input + * parameters in its {@link StepExecution} context. The name will be translated into a {@link Step} instance by the + * remote worker. + * + * @param stepName the name of the {@link Step} instance to execute business logic + */ + public void setStepName(String stepName) { + this.stepName = stepName; + } + + /** + * @param messages the messages to be aggregated + * @return the list as it was passed in + */ + @Aggregator(sendPartialResultsOnExpiry = "true") + public List aggregate(@Payloads List messages) { + return messages; + } + + public void setReplyChannel(PollableChannel replyChannel) { + this.replyChannel = replyChannel; + } + + /** + * Sends {@link StepExecutionRequest} objects to the request channel of the {@link MessagingTemplate}, and then + * receives the result back as a list of {@link StepExecution} on a reply channel. Use the {@link #aggregate(List)} + * method as an aggregator of the individual remote replies. The receive timeout needs to be set realistically in + * the {@link MessagingTemplate} and the aggregator, so that there is a good chance of all work being done. + * + * @see PartitionHandler#handle(StepExecutionSplitter, StepExecution) + */ + public Collection handle(StepExecutionSplitter stepExecutionSplitter, + final StepExecution masterStepExecution) throws Exception { + + final Set split = stepExecutionSplitter.split(masterStepExecution, gridSize); + + if(CollectionUtils.isEmpty(split)) { + return split; + } + + int count = 0; + + for (StepExecution stepExecution : split) { + Message request = createMessage(count++, split.size(), new StepExecutionRequest( + stepName, stepExecution.getJobExecutionId(), stepExecution.getId()), replyChannel); + if (logger.isDebugEnabled()) { + logger.debug("Sending request: " + request); + } + messagingGateway.send(request); + } + + if(!pollRepositoryForResults) { + return receiveReplies(replyChannel); + } + else { + return pollReplies(masterStepExecution, split); + } + } + + private Collection pollReplies(final StepExecution masterStepExecution, final Set split) throws Exception { + final Collection result = new ArrayList<>(split.size()); + + Callable> callback = new Callable>() { + @Override + public Collection call() throws Exception { + + for(Iterator stepExecutionIterator = split.iterator(); stepExecutionIterator.hasNext(); ) { + StepExecution curStepExecution = stepExecutionIterator.next(); + + if(!result.contains(curStepExecution)) { + StepExecution partitionStepExecution = + jobExplorer.getStepExecution(masterStepExecution.getJobExecutionId(), curStepExecution.getId()); + + if(!partitionStepExecution.getStatus().isRunning()) { + result.add(partitionStepExecution); + } + } + } + + if(logger.isDebugEnabled()) { + logger.debug(String.format("Currently waiting on %s partitions to finish", split.size())); + } + + if(result.size() == split.size()) { + return result; + } + else { + return null; + } + } + }; + + Poller> poller = new DirectPoller<>(pollInterval); + Future> resultsFuture = poller.poll(callback); + + if(timeout >= 0) { + return resultsFuture.get(timeout, TimeUnit.MILLISECONDS); + } + else { + return resultsFuture.get(); + } + } + + private Collection receiveReplies(PollableChannel currentReplyChannel) { + @SuppressWarnings("unchecked") + Message> message = (Message>) messagingGateway.receive(currentReplyChannel); + + if(message == null) { + throw new MessageTimeoutException("Timeout occurred before all partitions returned"); + } else if (logger.isDebugEnabled()) { + logger.debug("Received replies: " + message); + } + + return message.getPayload(); + } + + private Message createMessage(int sequenceNumber, int sequenceSize, + StepExecutionRequest stepExecutionRequest, PollableChannel replyChannel) { + return MessageBuilder.withPayload(stepExecutionRequest).setSequenceNumber(sequenceNumber) + .setSequenceSize(sequenceSize) + .setCorrelationId(stepExecutionRequest.getJobExecutionId() + ":" + stepExecutionRequest.getStepName()) + .setReplyChannel(replyChannel) + .build(); + } +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningManagerStepBuilder.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningManagerStepBuilder.java new file mode 100644 index 0000000000..56af8c13db --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningManagerStepBuilder.java @@ -0,0 +1,305 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.partition; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecutionListener; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.partition.PartitionHandler; +import org.springframework.batch.core.partition.StepExecutionSplitter; +import org.springframework.batch.core.partition.support.Partitioner; +import org.springframework.batch.core.partition.support.StepExecutionAggregator; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.step.builder.PartitionStepBuilder; +import org.springframework.batch.core.step.builder.StepBuilder; +import org.springframework.beans.factory.BeanCreationException; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.integration.channel.QueueChannel; +import org.springframework.integration.core.MessagingTemplate; +import org.springframework.integration.dsl.IntegrationFlows; +import org.springframework.integration.dsl.StandardIntegrationFlow; +import org.springframework.integration.dsl.context.IntegrationFlowContext; +import org.springframework.messaging.MessageChannel; +import org.springframework.messaging.PollableChannel; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.util.Assert; + +/** + * Builder for a manager step in a remote partitioning setup. This builder creates and + * sets a {@link MessageChannelPartitionHandler} on the manager step. + * + *

        If no {@code messagingTemplate} is provided through + * {@link RemotePartitioningManagerStepBuilder#messagingTemplate(MessagingTemplate)}, + * this builder will create one and set its default channel to the {@code outputChannel} + * provided through {@link RemotePartitioningManagerStepBuilder#outputChannel(MessageChannel)}.

        + * + *

        If a {@code messagingTemplate} is provided, it is assumed that it is fully configured + * and that its default channel is set to an output channel on which requests to workers + * will be sent.

        + * + * @since 4.2 + * @author Mahmoud Ben Hassine + */ +public class RemotePartitioningManagerStepBuilder extends PartitionStepBuilder { + + private static final long DEFAULT_POLL_INTERVAL = 10000L; + private static final long DEFAULT_TIMEOUT = -1L; + + private MessagingTemplate messagingTemplate; + private MessageChannel inputChannel; + private MessageChannel outputChannel; + private JobExplorer jobExplorer; + private BeanFactory beanFactory; + private long pollInterval = DEFAULT_POLL_INTERVAL; + private long timeout = DEFAULT_TIMEOUT; + + /** + * Create a new {@link RemotePartitioningManagerStepBuilder}. + * @param stepName name of the manager step + */ + public RemotePartitioningManagerStepBuilder(String stepName) { + super(new StepBuilder(stepName)); + } + + /** + * Set the input channel on which replies from workers will be received. + * @param inputChannel the input channel + * @return this builder instance for fluent chaining + */ + public RemotePartitioningManagerStepBuilder inputChannel(MessageChannel inputChannel) { + Assert.notNull(inputChannel, "inputChannel must not be null"); + this.inputChannel = inputChannel; + return this; + } + + /** + * Set the output channel on which requests to workers will be sent. By using + * this setter, a default messaging template will be created and the output + * channel will be set as its default channel. + *

        Use either this setter or {@link RemotePartitioningManagerStepBuilder#messagingTemplate(MessagingTemplate)} + * to provide a fully configured messaging template.

        + * + * @param outputChannel the output channel. + * @return this builder instance for fluent chaining + * @see RemotePartitioningManagerStepBuilder#messagingTemplate(MessagingTemplate) + */ + public RemotePartitioningManagerStepBuilder outputChannel(MessageChannel outputChannel) { + Assert.notNull(outputChannel, "outputChannel must not be null"); + this.outputChannel = outputChannel; + return this; + } + + /** + * Set the {@link MessagingTemplate} to use to send data to workers. + * The default channel of the messaging template must be set. + *

        Use either this setter to provide a fully configured messaging template or + * provide an output channel through {@link RemotePartitioningManagerStepBuilder#outputChannel(MessageChannel)} + * and a default messaging template will be created.

        + * + * @param messagingTemplate the messaging template to use + * @return this builder instance for fluent chaining + * @see RemotePartitioningManagerStepBuilder#outputChannel(MessageChannel) + */ + public RemotePartitioningManagerStepBuilder messagingTemplate(MessagingTemplate messagingTemplate) { + Assert.notNull(messagingTemplate, "messagingTemplate must not be null"); + this.messagingTemplate = messagingTemplate; + return this; + } + + /** + * Set the job explorer. + * @param jobExplorer the job explorer to use. + * @return this builder instance for fluent chaining + */ + public RemotePartitioningManagerStepBuilder jobExplorer(JobExplorer jobExplorer) { + Assert.notNull(jobExplorer, "jobExplorer must not be null"); + this.jobExplorer = jobExplorer; + return this; + } + + /** + * How often to poll the job repository for the status of the workers. Defaults to 10 seconds. + * @param pollInterval the poll interval value in milliseconds + * @return this builder instance for fluent chaining + */ + public RemotePartitioningManagerStepBuilder pollInterval(long pollInterval) { + Assert.isTrue(pollInterval > 0, "The poll interval must be greater than zero"); + this.pollInterval = pollInterval; + return this; + } + + /** + * When using job repository polling, the time limit to wait. Defaults to -1 (no timeout). + * @param timeout the timeout value in milliseconds + * @return this builder instance for fluent chaining + */ + public RemotePartitioningManagerStepBuilder timeout(long timeout) { + this.timeout = timeout; + return this; + } + + /** + * Set the bean factory. + * @param beanFactory the bean factory to use + * @return this builder instance for fluent chaining + */ + public RemotePartitioningManagerStepBuilder beanFactory(BeanFactory beanFactory) { + this.beanFactory = beanFactory; + return this; + } + + public Step build() { + Assert.state(this.outputChannel == null || this.messagingTemplate == null, + "You must specify either an outputChannel or a messagingTemplate but not both."); + + // configure messaging template + if (this.messagingTemplate == null) { + this.messagingTemplate = new MessagingTemplate(); + this.messagingTemplate.setDefaultChannel(this.outputChannel); + if (this.logger.isDebugEnabled()) { + this.logger.debug("No messagingTemplate was provided, using a default one"); + } + } + + // Configure the partition handler + final MessageChannelPartitionHandler partitionHandler = new MessageChannelPartitionHandler(); + partitionHandler.setStepName(getStepName()); + partitionHandler.setGridSize(getGridSize()); + partitionHandler.setMessagingOperations(this.messagingTemplate); + + if (isPolling()) { + partitionHandler.setJobExplorer(this.jobExplorer); + partitionHandler.setPollInterval(this.pollInterval); + partitionHandler.setTimeout(this.timeout); + } + else { + PollableChannel replies = new QueueChannel(); + partitionHandler.setReplyChannel(replies); + StandardIntegrationFlow standardIntegrationFlow = IntegrationFlows + .from(this.inputChannel) + .aggregate(aggregatorSpec -> aggregatorSpec.processor(partitionHandler)) + .channel(replies) + .get(); + IntegrationFlowContext integrationFlowContext = this.beanFactory.getBean(IntegrationFlowContext.class); + integrationFlowContext.registration(standardIntegrationFlow) + .autoStartup(false) + .register(); + } + + try { + partitionHandler.afterPropertiesSet(); + super.partitionHandler(partitionHandler); + } + catch (Exception e) { + throw new BeanCreationException("Unable to create a manager step for remote partitioning", e); + } + + return super.build(); + } + + private boolean isPolling() { + return this.inputChannel == null; + } + + @Override + public RemotePartitioningManagerStepBuilder repository(JobRepository jobRepository) { + super.repository(jobRepository); + return this; + } + + @Override + public RemotePartitioningManagerStepBuilder transactionManager(PlatformTransactionManager transactionManager) { + super.transactionManager(transactionManager); + return this; + } + + @Override + public RemotePartitioningManagerStepBuilder partitioner(String workerStepName, Partitioner partitioner) { + super.partitioner(workerStepName, partitioner); + return this; + } + + @Override + public RemotePartitioningManagerStepBuilder gridSize(int gridSize) { + super.gridSize(gridSize); + return this; + } + + @Override + public RemotePartitioningManagerStepBuilder step(Step step) { + super.step(step); + return this; + } + + @Override + public RemotePartitioningManagerStepBuilder splitter(StepExecutionSplitter splitter) { + super.splitter(splitter); + return this; + } + + @Override + public RemotePartitioningManagerStepBuilder aggregator(StepExecutionAggregator aggregator) { + super.aggregator(aggregator); + return this; + } + + @Override + public RemotePartitioningManagerStepBuilder startLimit(int startLimit) { + super.startLimit(startLimit); + return this; + } + + @Override + public RemotePartitioningManagerStepBuilder listener(Object listener) { + super.listener(listener); + return this; + } + + @Override + public RemotePartitioningManagerStepBuilder listener(StepExecutionListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemotePartitioningManagerStepBuilder allowStartIfComplete(boolean allowStartIfComplete) { + super.allowStartIfComplete(allowStartIfComplete); + return this; + } + + /** + * This method will throw a {@link UnsupportedOperationException} since + * the partition handler of the manager step will be automatically set to an + * instance of {@link MessageChannelPartitionHandler}. + * + * When building a manager step for remote partitioning using this builder, + * no partition handler must be provided. + * + * @param partitionHandler a partition handler + * @return this builder instance for fluent chaining + * @throws UnsupportedOperationException if a partition handler is provided + */ + @Override + public RemotePartitioningManagerStepBuilder partitionHandler(PartitionHandler partitionHandler) throws UnsupportedOperationException { + throw new UnsupportedOperationException("When configuring a manager step " + + "for remote partitioning using the RemotePartitioningManagerStepBuilder, " + + "the partition handler will be automatically set to an instance " + + "of MessageChannelPartitionHandler. The partition handler must " + + "not be provided in this case."); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningManagerStepBuilderFactory.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningManagerStepBuilderFactory.java new file mode 100644 index 0000000000..7697eeca45 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningManagerStepBuilderFactory.java @@ -0,0 +1,75 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.partition; + +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.beans.factory.BeanFactoryAware; +import org.springframework.transaction.PlatformTransactionManager; + +/** + * Convenient factory for a {@link RemotePartitioningManagerStepBuilder} which sets + * the {@link JobRepository}, {@link JobExplorer}, {@link BeanFactory} and + * {@link PlatformTransactionManager} automatically. + * + * @since 4.2 + * @author Mahmoud Ben Hassine + */ +public class RemotePartitioningManagerStepBuilderFactory implements BeanFactoryAware { + + private BeanFactory beanFactory; + final private JobExplorer jobExplorer; + final private JobRepository jobRepository; + final private PlatformTransactionManager transactionManager; + + + /** + * Create a new {@link RemotePartitioningManagerStepBuilderFactory}. + * @param jobRepository the job repository to use + * @param jobExplorer the job explorer to use + * @param transactionManager the transaction manager to use + */ + public RemotePartitioningManagerStepBuilderFactory(JobRepository jobRepository, + JobExplorer jobExplorer, PlatformTransactionManager transactionManager) { + + this.jobRepository = jobRepository; + this.jobExplorer = jobExplorer; + this.transactionManager = transactionManager; + } + + @Override + public void setBeanFactory(BeanFactory beanFactory) throws BeansException { + this.beanFactory = beanFactory; + } + + /** + * Creates a {@link RemotePartitioningManagerStepBuilder} and initializes its job + * repository, job explorer, bean factory and transaction manager. + * @param name the name of the step + * @return a {@link RemotePartitioningManagerStepBuilder} + */ + public RemotePartitioningManagerStepBuilder get(String name) { + return new RemotePartitioningManagerStepBuilder(name) + .repository(this.jobRepository) + .jobExplorer(this.jobExplorer) + .beanFactory(this.beanFactory) + .transactionManager(this.transactionManager); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningMasterStepBuilder.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningMasterStepBuilder.java new file mode 100644 index 0000000000..ae3397b5d6 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningMasterStepBuilder.java @@ -0,0 +1,308 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.partition; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecutionListener; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.partition.PartitionHandler; +import org.springframework.batch.core.partition.StepExecutionSplitter; +import org.springframework.batch.core.partition.support.Partitioner; +import org.springframework.batch.core.partition.support.StepExecutionAggregator; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.step.builder.PartitionStepBuilder; +import org.springframework.batch.core.step.builder.StepBuilder; +import org.springframework.beans.factory.BeanCreationException; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.integration.channel.QueueChannel; +import org.springframework.integration.core.MessagingTemplate; +import org.springframework.integration.dsl.IntegrationFlows; +import org.springframework.integration.dsl.StandardIntegrationFlow; +import org.springframework.integration.dsl.context.IntegrationFlowContext; +import org.springframework.messaging.MessageChannel; +import org.springframework.messaging.PollableChannel; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.util.Assert; + +/** + * Builder for a master step in a remote partitioning setup. This builder creates and + * sets a {@link MessageChannelPartitionHandler} on the master step. + * + *

        If no {@code messagingTemplate} is provided through + * {@link RemotePartitioningMasterStepBuilder#messagingTemplate(MessagingTemplate)}, + * this builder will create one and set its default channel to the {@code outputChannel} + * provided through {@link RemotePartitioningMasterStepBuilder#outputChannel(MessageChannel)}.

        + * + *

        If a {@code messagingTemplate} is provided, it is assumed that it is fully configured + * and that its default channel is set to an output channel on which requests to workers + * will be sent.

        + * + * @deprecated Use {@link RemotePartitioningManagerStepBuilder} instead. + * + * @since 4.1 + * @author Mahmoud Ben Hassine + */ +@Deprecated +public class RemotePartitioningMasterStepBuilder extends PartitionStepBuilder { + + private static final long DEFAULT_POLL_INTERVAL = 10000L; + private static final long DEFAULT_TIMEOUT = -1L; + + private MessagingTemplate messagingTemplate; + private MessageChannel inputChannel; + private MessageChannel outputChannel; + private JobExplorer jobExplorer; + private BeanFactory beanFactory; + private long pollInterval = DEFAULT_POLL_INTERVAL; + private long timeout = DEFAULT_TIMEOUT; + + /** + * Create a new {@link RemotePartitioningMasterStepBuilder}. + * @param stepName name of the master step + */ + public RemotePartitioningMasterStepBuilder(String stepName) { + super(new StepBuilder(stepName)); + } + + /** + * Set the input channel on which replies from workers will be received. + * @param inputChannel the input channel + * @return this builder instance for fluent chaining + */ + public RemotePartitioningMasterStepBuilder inputChannel(MessageChannel inputChannel) { + Assert.notNull(inputChannel, "inputChannel must not be null"); + this.inputChannel = inputChannel; + return this; + } + + /** + * Set the output channel on which requests to workers will be sent. By using + * this setter, a default messaging template will be created and the output + * channel will be set as its default channel. + *

        Use either this setter or {@link RemotePartitioningMasterStepBuilder#messagingTemplate(MessagingTemplate)} + * to provide a fully configured messaging template.

        + * + * @param outputChannel the output channel. + * @return this builder instance for fluent chaining + * @see RemotePartitioningMasterStepBuilder#messagingTemplate(MessagingTemplate) + */ + public RemotePartitioningMasterStepBuilder outputChannel(MessageChannel outputChannel) { + Assert.notNull(outputChannel, "outputChannel must not be null"); + this.outputChannel = outputChannel; + return this; + } + + /** + * Set the {@link MessagingTemplate} to use to send data to workers. + * The default channel of the messaging template must be set. + *

        Use either this setter to provide a fully configured messaging template or + * provide an output channel through {@link RemotePartitioningMasterStepBuilder#outputChannel(MessageChannel)} + * and a default messaging template will be created.

        + * + * @param messagingTemplate the messaging template to use + * @return this builder instance for fluent chaining + * @see RemotePartitioningMasterStepBuilder#outputChannel(MessageChannel) + */ + public RemotePartitioningMasterStepBuilder messagingTemplate(MessagingTemplate messagingTemplate) { + Assert.notNull(messagingTemplate, "messagingTemplate must not be null"); + this.messagingTemplate = messagingTemplate; + return this; + } + + /** + * Set the job explorer. + * @param jobExplorer the job explorer to use. + * @return this builder instance for fluent chaining + */ + public RemotePartitioningMasterStepBuilder jobExplorer(JobExplorer jobExplorer) { + Assert.notNull(jobExplorer, "jobExplorer must not be null"); + this.jobExplorer = jobExplorer; + return this; + } + + /** + * How often to poll the job repository for the status of the workers. Defaults to 10 seconds. + * @param pollInterval the poll interval value in milliseconds + * @return this builder instance for fluent chaining + */ + public RemotePartitioningMasterStepBuilder pollInterval(long pollInterval) { + Assert.isTrue(pollInterval > 0, "The poll interval must be greater than zero"); + this.pollInterval = pollInterval; + return this; + } + + /** + * When using job repository polling, the time limit to wait. Defaults to -1 (no timeout). + * @param timeout the timeout value in milliseconds + * @return this builder instance for fluent chaining + */ + public RemotePartitioningMasterStepBuilder timeout(long timeout) { + this.timeout = timeout; + return this; + } + + /** + * Set the bean factory. + * @param beanFactory the bean factory to use + * @return this builder instance for fluent chaining + */ + public RemotePartitioningMasterStepBuilder beanFactory(BeanFactory beanFactory) { + this.beanFactory = beanFactory; + return this; + } + + public Step build() { + Assert.state(this.outputChannel == null || this.messagingTemplate == null, + "You must specify either an outputChannel or a messagingTemplate but not both."); + + // configure messaging template + if (this.messagingTemplate == null) { + this.messagingTemplate = new MessagingTemplate(); + this.messagingTemplate.setDefaultChannel(this.outputChannel); + if (this.logger.isDebugEnabled()) { + this.logger.debug("No messagingTemplate was provided, using a default one"); + } + } + + // Configure the partition handler + final MessageChannelPartitionHandler partitionHandler = new MessageChannelPartitionHandler(); + partitionHandler.setStepName(getStepName()); + partitionHandler.setGridSize(getGridSize()); + partitionHandler.setMessagingOperations(this.messagingTemplate); + + if (isPolling()) { + partitionHandler.setJobExplorer(this.jobExplorer); + partitionHandler.setPollInterval(this.pollInterval); + partitionHandler.setTimeout(this.timeout); + } + else { + PollableChannel replies = new QueueChannel(); + partitionHandler.setReplyChannel(replies); + StandardIntegrationFlow standardIntegrationFlow = IntegrationFlows + .from(this.inputChannel) + .aggregate(aggregatorSpec -> aggregatorSpec.processor(partitionHandler)) + .channel(replies) + .get(); + IntegrationFlowContext integrationFlowContext = this.beanFactory.getBean(IntegrationFlowContext.class); + integrationFlowContext.registration(standardIntegrationFlow) + .autoStartup(false) + .register(); + } + + try { + partitionHandler.afterPropertiesSet(); + super.partitionHandler(partitionHandler); + } + catch (Exception e) { + throw new BeanCreationException("Unable to create a master step for remote partitioning", e); + } + + return super.build(); + } + + private boolean isPolling() { + return this.inputChannel == null; + } + + @Override + public RemotePartitioningMasterStepBuilder repository(JobRepository jobRepository) { + super.repository(jobRepository); + return this; + } + + @Override + public RemotePartitioningMasterStepBuilder transactionManager(PlatformTransactionManager transactionManager) { + super.transactionManager(transactionManager); + return this; + } + + @Override + public RemotePartitioningMasterStepBuilder partitioner(String slaveStepName, Partitioner partitioner) { + super.partitioner(slaveStepName, partitioner); + return this; + } + + @Override + public RemotePartitioningMasterStepBuilder gridSize(int gridSize) { + super.gridSize(gridSize); + return this; + } + + @Override + public RemotePartitioningMasterStepBuilder step(Step step) { + super.step(step); + return this; + } + + @Override + public RemotePartitioningMasterStepBuilder splitter(StepExecutionSplitter splitter) { + super.splitter(splitter); + return this; + } + + @Override + public RemotePartitioningMasterStepBuilder aggregator(StepExecutionAggregator aggregator) { + super.aggregator(aggregator); + return this; + } + + @Override + public RemotePartitioningMasterStepBuilder startLimit(int startLimit) { + super.startLimit(startLimit); + return this; + } + + @Override + public RemotePartitioningMasterStepBuilder listener(Object listener) { + super.listener(listener); + return this; + } + + @Override + public RemotePartitioningMasterStepBuilder listener(StepExecutionListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemotePartitioningMasterStepBuilder allowStartIfComplete(boolean allowStartIfComplete) { + super.allowStartIfComplete(allowStartIfComplete); + return this; + } + + /** + * This method will throw a {@link UnsupportedOperationException} since + * the partition handler of the master step will be automatically set to an + * instance of {@link MessageChannelPartitionHandler}. + * + * When building a master step for remote partitioning using this builder, + * no partition handler must be provided. + * + * @param partitionHandler a partition handler + * @return this builder instance for fluent chaining + * @throws UnsupportedOperationException if a partition handler is provided + */ + @Override + public RemotePartitioningMasterStepBuilder partitionHandler(PartitionHandler partitionHandler) throws UnsupportedOperationException { + throw new UnsupportedOperationException("When configuring a master step " + + "for remote partitioning using the RemotePartitioningMasterStepBuilder, " + + "the partition handler will be automatically set to an instance " + + "of MessageChannelPartitionHandler. The partition handler must " + + "not be provided in this case."); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningMasterStepBuilderFactory.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningMasterStepBuilderFactory.java new file mode 100644 index 0000000000..7d8249c822 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningMasterStepBuilderFactory.java @@ -0,0 +1,78 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.partition; + +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.beans.factory.BeanFactoryAware; +import org.springframework.transaction.PlatformTransactionManager; + +/** + * Convenient factory for a {@link RemotePartitioningMasterStepBuilder} which sets + * the {@link JobRepository}, {@link JobExplorer}, {@link BeanFactory} and + * {@link PlatformTransactionManager} automatically. + * + * @deprecated Use {@link RemotePartitioningManagerStepBuilderFactory} instead + * + * @since 4.1 + * @author Mahmoud Ben Hassine + */ +@Deprecated +public class RemotePartitioningMasterStepBuilderFactory implements BeanFactoryAware { + + private BeanFactory beanFactory; + final private JobExplorer jobExplorer; + final private JobRepository jobRepository; + final private PlatformTransactionManager transactionManager; + + + /** + * Create a new {@link RemotePartitioningMasterStepBuilderFactory}. + * @param jobRepository the job repository to use + * @param jobExplorer the job explorer to use + * @param transactionManager the transaction manager to use + */ + public RemotePartitioningMasterStepBuilderFactory(JobRepository jobRepository, + JobExplorer jobExplorer, PlatformTransactionManager transactionManager) { + + this.jobRepository = jobRepository; + this.jobExplorer = jobExplorer; + this.transactionManager = transactionManager; + } + + @Override + public void setBeanFactory(BeanFactory beanFactory) throws BeansException { + this.beanFactory = beanFactory; + } + + /** + * Creates a {@link RemotePartitioningMasterStepBuilder} and initializes its job + * repository, job explorer, bean factory and transaction manager. + * @param name the name of the step + * @return a {@link RemotePartitioningMasterStepBuilder} + */ + public RemotePartitioningMasterStepBuilder get(String name) { + return new RemotePartitioningMasterStepBuilder(name) + .repository(this.jobRepository) + .jobExplorer(this.jobExplorer) + .beanFactory(this.beanFactory) + .transactionManager(this.transactionManager); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningWorkerStepBuilder.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningWorkerStepBuilder.java new file mode 100644 index 0000000000..4a8936104b --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningWorkerStepBuilder.java @@ -0,0 +1,258 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.partition; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecutionListener; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.job.flow.Flow; +import org.springframework.batch.core.partition.support.Partitioner; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.step.StepLocator; +import org.springframework.batch.core.step.builder.FlowStepBuilder; +import org.springframework.batch.core.step.builder.JobStepBuilder; +import org.springframework.batch.core.step.builder.PartitionStepBuilder; +import org.springframework.batch.core.step.builder.SimpleStepBuilder; +import org.springframework.batch.core.step.builder.StepBuilder; +import org.springframework.batch.core.step.builder.TaskletStepBuilder; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.repeat.CompletionPolicy; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.integration.channel.NullChannel; +import org.springframework.integration.dsl.IntegrationFlow; +import org.springframework.integration.dsl.IntegrationFlows; +import org.springframework.integration.dsl.StandardIntegrationFlow; +import org.springframework.integration.dsl.context.IntegrationFlowContext; +import org.springframework.messaging.MessageChannel; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.util.Assert; + +/** + * Builder for a worker step in a remote partitioning setup. This builder + * creates an {@link IntegrationFlow} that: + * + *
          + *
        • listens to {@link StepExecutionRequest}s coming from the master + * on the input channel
        • + *
        • invokes the {@link StepExecutionRequestHandler} to execute the worker + * step for each incoming request. The worker step is located using the provided + * {@link StepLocator}. If no {@link StepLocator} is provided, a {@link BeanFactoryStepLocator} + * configured with the current {@link BeanFactory} will be used + *
        • replies to the master on the output channel (when the master step is + * configured to aggregate replies from workers). If no output channel + * is provided, a {@link NullChannel} will be used (assuming the master side + * is configured to poll the job repository for workers status)
        • + *
        + * + * @since 4.1 + * @author Mahmoud Ben Hassine + */ +public class RemotePartitioningWorkerStepBuilder extends StepBuilder { + + private static final String SERVICE_ACTIVATOR_METHOD_NAME = "handle"; + private static final Log logger = LogFactory.getLog(RemotePartitioningWorkerStepBuilder.class); + + private MessageChannel inputChannel; + private MessageChannel outputChannel; + private JobExplorer jobExplorer; + private StepLocator stepLocator; + private BeanFactory beanFactory; + + /** + * Initialize a step builder for a step with the given name. + * @param name the name of the step + */ + public RemotePartitioningWorkerStepBuilder(String name) { + super(name); + } + + /** + * Set the input channel on which step execution requests sent by the master + * are received. + * @param inputChannel the input channel + * @return this builder instance for fluent chaining + */ + public RemotePartitioningWorkerStepBuilder inputChannel(MessageChannel inputChannel) { + Assert.notNull(inputChannel, "inputChannel must not be null"); + this.inputChannel = inputChannel; + return this; + } + + /** + * Set the output channel on which replies will be sent to the master step. + * @param outputChannel the input channel + * @return this builder instance for fluent chaining + */ + public RemotePartitioningWorkerStepBuilder outputChannel(MessageChannel outputChannel) { + Assert.notNull(outputChannel, "outputChannel must not be null"); + this.outputChannel = outputChannel; + return this; + } + + /** + * Set the job explorer. + * @param jobExplorer the job explorer to use + * @return this builder instance for fluent chaining + */ + public RemotePartitioningWorkerStepBuilder jobExplorer(JobExplorer jobExplorer) { + Assert.notNull(jobExplorer, "jobExplorer must not be null"); + this.jobExplorer = jobExplorer; + return this; + } + + /** + * Set the step locator used to locate the worker step to execute. + * @param stepLocator the step locator to use + * @return this builder instance for fluent chaining + */ + public RemotePartitioningWorkerStepBuilder stepLocator(StepLocator stepLocator) { + Assert.notNull(stepLocator, "stepLocator must not be null"); + this.stepLocator = stepLocator; + return this; + } + + /** + * Set the bean factory. + * @param beanFactory the bean factory + * @return this builder instance for fluent chaining + */ + public RemotePartitioningWorkerStepBuilder beanFactory(BeanFactory beanFactory) { + Assert.notNull(beanFactory, "beanFactory must not be null"); + this.beanFactory = beanFactory; + return this; + } + + @Override + public RemotePartitioningWorkerStepBuilder repository(JobRepository jobRepository) { + super.repository(jobRepository); + return this; + } + + @Override + public RemotePartitioningWorkerStepBuilder transactionManager(PlatformTransactionManager transactionManager) { + super.transactionManager(transactionManager); + return this; + } + + @Override + public RemotePartitioningWorkerStepBuilder startLimit(int startLimit) { + super.startLimit(startLimit); + return this; + } + + @Override + public RemotePartitioningWorkerStepBuilder listener(Object listener) { + super.listener(listener); + return this; + } + + @Override + public RemotePartitioningWorkerStepBuilder listener(StepExecutionListener listener) { + super.listener(listener); + return this; + } + + @Override + public RemotePartitioningWorkerStepBuilder allowStartIfComplete(boolean allowStartIfComplete) { + super.allowStartIfComplete(allowStartIfComplete); + return this; + } + + @Override + public TaskletStepBuilder tasklet(Tasklet tasklet) { + configureWorkerIntegrationFlow(); + return super.tasklet(tasklet); + } + + @Override + public SimpleStepBuilder chunk(int chunkSize) { + configureWorkerIntegrationFlow(); + return super.chunk(chunkSize); + } + + @Override + public SimpleStepBuilder chunk(CompletionPolicy completionPolicy) { + configureWorkerIntegrationFlow(); + return super.chunk(completionPolicy); + } + + @Override + public PartitionStepBuilder partitioner(String stepName, Partitioner partitioner) { + configureWorkerIntegrationFlow(); + return super.partitioner(stepName, partitioner); + } + + @Override + public PartitionStepBuilder partitioner(Step step) { + configureWorkerIntegrationFlow(); + return super.partitioner(step); + } + + @Override + public JobStepBuilder job(Job job) { + configureWorkerIntegrationFlow(); + return super.job(job); + } + + @Override + public FlowStepBuilder flow(Flow flow) { + configureWorkerIntegrationFlow(); + return super.flow(flow); + } + + /** + * Create an {@link IntegrationFlow} with a {@link StepExecutionRequestHandler} + * configured as a service activator listening to the input channel and replying + * on the output channel. + */ + private void configureWorkerIntegrationFlow() { + Assert.notNull(this.inputChannel, "An InputChannel must be provided"); + Assert.notNull(this.jobExplorer, "A JobExplorer must be provided"); + + if (this.stepLocator == null) { + BeanFactoryStepLocator beanFactoryStepLocator = new BeanFactoryStepLocator(); + beanFactoryStepLocator.setBeanFactory(this.beanFactory); + this.stepLocator = beanFactoryStepLocator; + } + if (this.outputChannel == null) { + if (logger.isDebugEnabled()) { + logger.debug("The output channel is set to a NullChannel. " + + "The master step must poll the job repository for workers status."); + } + this.outputChannel = new NullChannel(); + } + + StepExecutionRequestHandler stepExecutionRequestHandler = new StepExecutionRequestHandler(); + stepExecutionRequestHandler.setJobExplorer(this.jobExplorer); + stepExecutionRequestHandler.setStepLocator(this.stepLocator); + + StandardIntegrationFlow standardIntegrationFlow = IntegrationFlows + .from(this.inputChannel) + .handle(stepExecutionRequestHandler, SERVICE_ACTIVATOR_METHOD_NAME) + .channel(this.outputChannel) + .get(); + IntegrationFlowContext integrationFlowContext = this.beanFactory.getBean(IntegrationFlowContext.class); + integrationFlowContext.registration(standardIntegrationFlow) + .autoStartup(false) + .register(); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningWorkerStepBuilderFactory.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningWorkerStepBuilderFactory.java new file mode 100644 index 0000000000..674d6c5df8 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/RemotePartitioningWorkerStepBuilderFactory.java @@ -0,0 +1,76 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.partition; + +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.beans.factory.BeanFactoryAware; +import org.springframework.transaction.PlatformTransactionManager; + +/** + * Convenient factory for a {@link RemotePartitioningWorkerStepBuilder} which sets + * the {@link JobRepository}, {@link JobExplorer}, {@link BeanFactory} and + * {@link PlatformTransactionManager} automatically. + * + * @since 4.1 + * @author Mahmoud Ben Hassine + */ +public class RemotePartitioningWorkerStepBuilderFactory implements BeanFactoryAware { + + private BeanFactory beanFactory; + final private JobExplorer jobExplorer; + final private JobRepository jobRepository; + final private PlatformTransactionManager transactionManager; + + + /** + * Create a new {@link RemotePartitioningWorkerStepBuilderFactory}. + * @param jobRepository the job repository to use + * @param jobExplorer the job explorer to use + * @param transactionManager the transaction manager to use + */ + public RemotePartitioningWorkerStepBuilderFactory(JobRepository jobRepository, + JobExplorer jobExplorer, + PlatformTransactionManager transactionManager) { + + this.jobExplorer = jobExplorer; + this.jobRepository = jobRepository; + this.transactionManager = transactionManager; + } + + @Override + public void setBeanFactory(BeanFactory beanFactory) throws BeansException { + this.beanFactory = beanFactory; + } + + /** + * Creates a {@link RemotePartitioningWorkerStepBuilder} and initializes its job + * repository, job explorer, bean factory and transaction manager. + * @param name the name of the step + * @return a {@link RemotePartitioningWorkerStepBuilder} + */ + public RemotePartitioningWorkerStepBuilder get(String name) { + return new RemotePartitioningWorkerStepBuilder(name) + .repository(this.jobRepository) + .jobExplorer(this.jobExplorer) + .beanFactory(this.beanFactory) + .transactionManager(this.transactionManager); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/StepExecutionRequest.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/StepExecutionRequest.java new file mode 100644 index 0000000000..89132967b5 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/StepExecutionRequest.java @@ -0,0 +1,71 @@ +/* + * Copyright 2009-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.partition; + +import java.io.Serializable; + +/** + * Class encapsulating information required to request a step execution in + * a remote partitioning setup. + * + * @author Dave Syer + * @author Mahmoud Ben Hassine + */ +public class StepExecutionRequest implements Serializable { + + private static final long serialVersionUID = 1L; + + private Long stepExecutionId; + + private String stepName; + + private Long jobExecutionId; + + private StepExecutionRequest() { + //For Jackson deserialization + } + + /** + * Create a new {@link StepExecutionRequest} instance. + * @param stepName the name of the step to execute + * @param jobExecutionId the id of the job execution + * @param stepExecutionId the id of the step execution + */ + public StepExecutionRequest(String stepName, Long jobExecutionId, Long stepExecutionId) { + this.stepName = stepName; + this.jobExecutionId = jobExecutionId; + this.stepExecutionId = stepExecutionId; + } + + public Long getJobExecutionId() { + return jobExecutionId; + } + + public Long getStepExecutionId() { + return stepExecutionId; + } + + public String getStepName() { + return stepName; + } + + @Override + public String toString() { + return String.format("StepExecutionRequest: [jobExecutionId=%d, stepExecutionId=%d, stepName=%s]", + jobExecutionId, stepExecutionId, stepName); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/StepExecutionRequestHandler.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/StepExecutionRequestHandler.java new file mode 100644 index 0000000000..258b2404ae --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/StepExecutionRequestHandler.java @@ -0,0 +1,80 @@ +package org.springframework.batch.integration.partition; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.JobInterruptedException; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.step.NoSuchStepException; +import org.springframework.batch.core.step.StepLocator; +import org.springframework.integration.annotation.MessageEndpoint; +import org.springframework.integration.annotation.ServiceActivator; + +/** + * A {@link MessageEndpoint} that can handle a {@link StepExecutionRequest} and + * return a {@link StepExecution} as the result. Typically these need to be + * aggregated into a response to a partition handler. + * + * @author Dave Syer + * + */ +@MessageEndpoint +public class StepExecutionRequestHandler { + + private JobExplorer jobExplorer; + + private StepLocator stepLocator; + + /** + * Used to locate a {@link Step} to execute for each request. + * @param stepLocator a {@link StepLocator} + */ + public void setStepLocator(StepLocator stepLocator) { + this.stepLocator = stepLocator; + } + + /** + * An explorer that should be used to check for {@link StepExecution} + * completion. + * + * @param jobExplorer a {@link JobExplorer} that is linked to the shared + * repository used by all remote workers. + */ + public void setJobExplorer(JobExplorer jobExplorer) { + this.jobExplorer = jobExplorer; + } + + @ServiceActivator + public StepExecution handle(StepExecutionRequest request) { + + Long jobExecutionId = request.getJobExecutionId(); + Long stepExecutionId = request.getStepExecutionId(); + StepExecution stepExecution = jobExplorer.getStepExecution(jobExecutionId, stepExecutionId); + if (stepExecution == null) { + throw new NoSuchStepException("No StepExecution could be located for this request: " + request); + } + + String stepName = request.getStepName(); + Step step = stepLocator.getStep(stepName); + if (step == null) { + throw new NoSuchStepException(String.format("No Step with name [%s] could be located.", stepName)); + } + + try { + step.execute(stepExecution); + } + catch (JobInterruptedException e) { + stepExecution.setStatus(BatchStatus.STOPPED); + // The receiver should update the stepExecution in repository + } + catch (Throwable e) { + stepExecution.addFailureException(e); + stepExecution.setStatus(BatchStatus.FAILED); + // The receiver should update the stepExecution in repository + } + + return stepExecution; + + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/package-info.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/package-info.java new file mode 100644 index 0000000000..e8b25768cb --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/partition/package-info.java @@ -0,0 +1,10 @@ +/** + * Remote partitioning components. + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.integration.partition; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/step/DelegateStep.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/step/DelegateStep.java new file mode 100644 index 0000000000..3a70d90ba9 --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/step/DelegateStep.java @@ -0,0 +1,56 @@ +/* + * Copyright 2006-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.step; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.step.AbstractStep; +import org.springframework.util.Assert; + +/** + * Provides a wrapper for an existing {@link Step}, delegating execution to it, + * but serving all other operations locally. + * + * @author Dave Syer + * + */ +public class DelegateStep extends AbstractStep { + + private Step delegate; + + /** + * @param delegate the delegate to set + */ + public void setDelegate(Step delegate) { + this.delegate = delegate; + } + + /** + * Check mandatory properties (delegate). + */ + @Override + public void afterPropertiesSet() throws Exception { + Assert.state(delegate!=null, "A delegate Step must be provided"); + super.afterPropertiesSet(); + } + + @Override + protected void doExecute(StepExecution stepExecution) throws Exception { + delegate.execute(stepExecution); + } + +} diff --git a/spring-batch-integration/src/main/java/org/springframework/batch/integration/step/package-info.java b/spring-batch-integration/src/main/java/org/springframework/batch/integration/step/package-info.java new file mode 100644 index 0000000000..82e7319a2e --- /dev/null +++ b/spring-batch-integration/src/main/java/org/springframework/batch/integration/step/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Support classes related to steps when used with Spring Integration. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.integration.step; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-integration/src/main/resources/META-INF/spring.handlers b/spring-batch-integration/src/main/resources/META-INF/spring.handlers new file mode 100644 index 0000000000..84912a6f7b --- /dev/null +++ b/spring-batch-integration/src/main/resources/META-INF/spring.handlers @@ -0,0 +1 @@ +http\://www.springframework.org/schema/batch-integration=org.springframework.batch.integration.config.xml.BatchIntegrationNamespaceHandler diff --git a/spring-batch-integration/src/main/resources/META-INF/spring.schemas b/spring-batch-integration/src/main/resources/META-INF/spring.schemas new file mode 100644 index 0000000000..3c30126140 --- /dev/null +++ b/spring-batch-integration/src/main/resources/META-INF/spring.schemas @@ -0,0 +1,4 @@ +http\://www.springframework.org/schema/batch-integration/spring-batch-integration-1.3.xsd=org/springframework/batch/integration/config/xml/spring-batch-integration-1.3.xsd +http\://www.springframework.org/schema/batch-integration/spring-batch-integration-3.1.xsd=org/springframework/batch/integration/config/xml/spring-batch-integration-3.1.xsd +http\://www.springframework.org/schema/batch-integration/spring-batch-integration-4.2.xsd=org/springframework/batch/integration/config/xml/spring-batch-integration-4.2.xsd +http\://www.springframework.org/schema/batch-integration/spring-batch-integration.xsd=org/springframework/batch/integration/config/xml/spring-batch-integration-4.2.xsd diff --git a/spring-batch-integration/src/main/resources/META-INF/spring.tooling b/spring-batch-integration/src/main/resources/META-INF/spring.tooling new file mode 100644 index 0000000000..017292a7c4 --- /dev/null +++ b/spring-batch-integration/src/main/resources/META-INF/spring.tooling @@ -0,0 +1,4 @@ +# Tooling related information for the Spring Batch Integration namespace +http\://www.springframework.org/schema/integration/spring-batch-integration@name=Spring Batch Integration Namespace +http\://www.springframework.org/schema/integration/spring-batch-integration@prefix=batch-int +http\://www.springframework.org/schema/integration/spring-batch-integration@icon=org/springframework/batch/integration/config/xml/spring-batch-integration.gif diff --git a/spring-batch-integration/src/main/resources/org/springframework/batch/integration/config/xml/spring-batch-integration-1.3.xsd b/spring-batch-integration/src/main/resources/org/springframework/batch/integration/config/xml/spring-batch-integration-1.3.xsd new file mode 100644 index 0000000000..176e2a18af --- /dev/null +++ b/spring-batch-integration/src/main/resources/org/springframework/batch/integration/config/xml/spring-batch-integration-1.3.xsd @@ -0,0 +1,135 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/main/resources/org/springframework/batch/integration/config/xml/spring-batch-integration-3.1.xsd b/spring-batch-integration/src/main/resources/org/springframework/batch/integration/config/xml/spring-batch-integration-3.1.xsd new file mode 100644 index 0000000000..f1474b8a5e --- /dev/null +++ b/spring-batch-integration/src/main/resources/org/springframework/batch/integration/config/xml/spring-batch-integration-3.1.xsd @@ -0,0 +1,248 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/main/resources/org/springframework/batch/integration/config/xml/spring-batch-integration-4.2.xsd b/spring-batch-integration/src/main/resources/org/springframework/batch/integration/config/xml/spring-batch-integration-4.2.xsd new file mode 100644 index 0000000000..0794e1129b --- /dev/null +++ b/spring-batch-integration/src/main/resources/org/springframework/batch/integration/config/xml/spring-batch-integration-4.2.xsd @@ -0,0 +1,244 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/main/resources/org/springframework/batch/integration/config/xml/spring-batch-integration.gif b/spring-batch-integration/src/main/resources/org/springframework/batch/integration/config/xml/spring-batch-integration.gif new file mode 100644 index 0000000000..210e0764fa Binary files /dev/null and b/spring-batch-integration/src/main/resources/org/springframework/batch/integration/config/xml/spring-batch-integration.gif differ diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/IgnoredTestSuite.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/IgnoredTestSuite.java new file mode 100644 index 0000000000..1b893146d4 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/IgnoredTestSuite.java @@ -0,0 +1,38 @@ +package org.springframework.batch.integration; + +import org.junit.Ignore; +import org.junit.runner.RunWith; +import org.junit.runners.Suite; +import org.junit.runners.Suite.SuiteClasses; +import org.springframework.batch.integration.chunk.RemoteChunkFaultTolerantStepJmsIntegrationTests; +import org.springframework.batch.integration.partition.JmsIntegrationTests; + +/* + * Copyright 2009-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * A test suite that is ignored, but can be resurrected to help debug ordering issues in tests. + * + * @author Dave Syer + * + */ +@RunWith(Suite.class) +@SuiteClasses(value = { JmsIntegrationTests.class, + RemoteChunkFaultTolerantStepJmsIntegrationTests.class }) +@Ignore +public class IgnoredTestSuite { + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/JobRepositorySupport.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/JobRepositorySupport.java new file mode 100644 index 0000000000..79f23a92d4 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/JobRepositorySupport.java @@ -0,0 +1,115 @@ +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration; + +import java.util.Collection; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; +import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.lang.Nullable; + +/** + * @author Dave Syer + * + */ +public class JobRepositorySupport implements JobRepository { + + /* (non-Javadoc) + * @see org.springframework.batch.core.repository.JobRepository#createJobExecution(org.springframework.batch.core.Job, org.springframework.batch.core.JobParameters) + */ + public JobExecution createJobExecution(String jobName, JobParameters jobParameters) + throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException { + return new JobExecution(new JobInstance(0L, jobName), jobParameters); + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.repository.JobRepository#getLastStepExecution(org.springframework.batch.core.JobInstance, org.springframework.batch.core.Step) + */ + @Nullable + public StepExecution getLastStepExecution(JobInstance jobInstance, String stepName) { + return null; + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.repository.JobRepository#getStepExecutionCount(org.springframework.batch.core.JobInstance, org.springframework.batch.core.Step) + */ + public int getStepExecutionCount(JobInstance jobInstance, String stepName) { + return 0; + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.repository.JobRepository#saveOrUpdate(org.springframework.batch.core.JobExecution) + */ + public void update(JobExecution jobExecution) { + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.repository.JobRepository#saveOrUpdate(org.springframework.batch.core.StepExecution) + */ + public void saveOrUpdate(StepExecution stepExecution) { + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.repository.JobRepository#saveOrUpdateExecutionContext(org.springframework.batch.core.StepExecution) + */ + public void updateExecutionContext(StepExecution stepExecution) { + } + + public void updateExecutionContext(JobExecution jobExecution) { + } + + public void add(StepExecution stepExecution) { + } + + public void update(StepExecution stepExecution) { + } + + public boolean isJobInstanceExists(String jobName, JobParameters jobParameters) { + return false; + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.repository.JobRepository#getLastJobExecution(java.lang.String, org.springframework.batch.core.JobParameters) + */ + @Nullable + public JobExecution getLastJobExecution(String jobName, JobParameters jobParameters) { + return null; + } + + public void addAll(Collection stepExecutions) { + if(stepExecutions != null) { + for (StepExecution stepExecution : stepExecutions) { + add(stepExecution); + } + } + } + + public JobInstance createJobInstance(String jobName, + JobParameters jobParameters) { + return null; + } + + public JobExecution createJobExecution(JobInstance jobInstance, + JobParameters jobParameters, String jobConfigurationLocation) { + return null; + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/JobSupport.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/JobSupport.java new file mode 100644 index 0000000000..c480dd562c --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/JobSupport.java @@ -0,0 +1,38 @@ +package org.springframework.batch.integration; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersIncrementer; +import org.springframework.batch.core.JobParametersValidator; +import org.springframework.batch.core.job.DefaultJobParametersValidator; +import org.springframework.lang.Nullable; + +public class JobSupport implements Job { + + String name; + + public JobSupport(String name){ + this.name = name; + } + + public void execute(JobExecution execution) { + } + + public String getName() { + return name; + } + + public boolean isRestartable() { + return false; + } + + @Nullable + public JobParametersIncrementer getJobParametersIncrementer() { + return null; + } + + public JobParametersValidator getJobParametersValidator() { + return new DefaultJobParametersValidator(); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/SmokeTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/SmokeTests.java new file mode 100644 index 0000000000..5e2a2c649f --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/SmokeTests.java @@ -0,0 +1,61 @@ +package org.springframework.batch.integration; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.integration.annotation.MessageEndpoint; +import org.springframework.integration.annotation.ServiceActivator; +import org.springframework.messaging.Message; +import org.springframework.messaging.MessageChannel; +import org.springframework.messaging.PollableChannel; +import org.springframework.messaging.support.GenericMessage; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class SmokeTests { + + @Autowired + private MessageChannel smokein; + + @Autowired + private PollableChannel smokeout; + + + @Test + public void testDummyWithSimpleAssert() throws Exception { + assertTrue(true); + } + + @Test + public void testVanillaSendAndReceive() throws Exception { + smokein.send(new GenericMessage<>("foo")); + @SuppressWarnings("unchecked") + Message message = (Message) smokeout.receive(100); + String result = message == null ? null : message.getPayload(); + assertEquals("foo: 1", result); + assertEquals(1, AnnotatedEndpoint.count); + } + + @MessageEndpoint + static class AnnotatedEndpoint { + + // This has to be static because Spring Integration registers the handler + // more than once (every time a test instance is created), but only one of + // them will get the message. + private volatile static int count = 0; + + @ServiceActivator(inputChannel = "smokein", outputChannel = "smokeout") + public String process(String message) { + count++; + String result = message + ": " + count; + return result; + } + + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/StepSupport.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/StepSupport.java new file mode 100644 index 0000000000..355d147c73 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/StepSupport.java @@ -0,0 +1,74 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration; + +import org.springframework.batch.core.JobInterruptedException; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; + +/** + * @author Dave Syer + * + */ +public class StepSupport implements Step { + + private String name; + private int startLimit = 1; + + /** + * @param name + */ + public StepSupport(String name) { + super(); + this.name = name; + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.Step#execute(org.springframework.batch.core.StepExecution) + */ + public void execute(StepExecution stepExecution) throws JobInterruptedException { + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.Step#getName() + */ + public String getName() { + return name; + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.Step#getStartLimit() + */ + public int getStartLimit() { + return startLimit; + } + + /* (non-Javadoc) + * @see org.springframework.batch.core.Step#isAllowStartIfComplete() + */ + public boolean isAllowStartIfComplete() { + return false; + } + + /** + * Public setter for the startLimit. + * @param startLimit the startLimit to set + */ + public void setStartLimit(int startLimit) { + this.startLimit = startLimit; + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/async/AsyncItemProcessorMessagingGatewayTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/async/AsyncItemProcessorMessagingGatewayTests.java new file mode 100644 index 0000000000..def5280628 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/async/AsyncItemProcessorMessagingGatewayTests.java @@ -0,0 +1,120 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.async; + +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.Callable; +import java.util.concurrent.Future; + +import org.junit.Ignore; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.MethodRule; +import org.junit.runner.RunWith; +import org.junit.runners.model.FrameworkMethod; +import org.junit.runners.model.Statement; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.test.MetaDataInstanceFactory; +import org.springframework.batch.test.StepScopeTestUtils; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.integration.annotation.MessageEndpoint; +import org.springframework.integration.annotation.ServiceActivator; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration +public class AsyncItemProcessorMessagingGatewayTests { + + private final AsyncItemProcessor processor = new AsyncItemProcessor<>(); + + private final StepExecution stepExecution = MetaDataInstanceFactory.createStepExecution(new JobParametersBuilder().addLong("factor", 2L).toJobParameters());; + + @Rule + public MethodRule rule = new MethodRule() { + public Statement apply(final Statement base, FrameworkMethod method, Object target) { + return new Statement() { + @Override + public void evaluate() throws Throwable { + StepScopeTestUtils.doInStepScope(stepExecution, new Callable() { + public Void call() throws Exception { + try { + base.evaluate(); + } + catch (Exception e) { + throw e; + } + catch (Throwable e) { + throw new Error(e); + } + return null; + } + }); + }; + }; + } + }; + + @Autowired + private ItemProcessor delegate; + + @Test @Ignore // TODO: Need to figure out why the Rule doesn't work with Spring 4 + public void testMultiExecution() throws Exception { + processor.setDelegate(delegate); + processor.setTaskExecutor(new SimpleAsyncTaskExecutor()); + List> list = new ArrayList<>(); + for (int count = 0; count < 10; count++) { + list.add(processor.process("foo" + count)); + } + for (Future future : list) { + String value = future.get(); + /** + * This delegate is a Spring Integration MessagingGateway. It can + * easily return null because of a timeout, but that will be treated + * by Batch as a filtered item, whereas it is really more like a + * skip. So we have to throw an exception in the processor if an + * unexpected null value comes back. + */ + assertNotNull(value); + assertTrue(value.matches("foo.*foo.*")); + } + } + + @MessageEndpoint + public static class Doubler { + private int factor = 1; + + public void setFactor(int factor) { + this.factor = factor; + } + + @ServiceActivator + public String cat(String value) { + for (int i=1; i processor = new AsyncItemProcessor<>(); + + private ItemProcessor delegate = new ItemProcessor() { + @Nullable + public String process(String item) throws Exception { + return item + item; + }; + }; + + @Test(expected = IllegalArgumentException.class) + public void testNoDelegate() throws Exception { + processor.afterPropertiesSet(); + } + + @Test + public void testExecution() throws Exception { + processor.setDelegate(delegate); + Future result = processor.process("foo"); + assertEquals("foofoo", result.get()); + } + + @Test + public void testExecutionInStepScope() throws Exception { + delegate = new ItemProcessor() { + @Nullable + public String process(String item) throws Exception { + StepContext context = StepSynchronizationManager.getContext(); + assertTrue(context != null && context.getStepExecution() != null); + return item + item; + }; + }; + processor.setDelegate(delegate); + Future result = StepScopeTestUtils.doInStepScope(MetaDataInstanceFactory.createStepExecution(), new Callable>() { + public Future call() throws Exception { + return processor.process("foo"); + } + }); + assertEquals("foofoo", result.get()); + } + + @Test + public void testMultiExecution() throws Exception { + processor.setDelegate(delegate); + processor.setTaskExecutor(new SimpleAsyncTaskExecutor()); + List> list = new ArrayList<>(); + for (int count = 0; count < 10; count++) { + list.add(processor.process("foo" + count)); + } + for (Future future : list) { + assertTrue(future.get().matches("foo.*foo.*")); + } + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/async/AsyncItemWriterTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/async/AsyncItemWriterTests.java new file mode 100644 index 0000000000..e1475ac296 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/async/AsyncItemWriterTests.java @@ -0,0 +1,275 @@ +/* + * Copyright 2014-2015 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.async; + +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.Callable; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.Future; +import java.util.concurrent.FutureTask; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; + +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.batch.item.ItemStreamWriter; +import org.springframework.batch.item.ItemWriter; +import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.core.task.TaskExecutor; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +/** + * @author mminella + */ +public class AsyncItemWriterTests { + + private AsyncItemWriter writer; + private List writtenItems; + private TaskExecutor taskExecutor; + + @Before + public void setup() { + taskExecutor = new SimpleAsyncTaskExecutor(); + writtenItems = new ArrayList<>(); + writer = new AsyncItemWriter<>(); + } + + @Test + public void testRoseyScenario() throws Exception { + writer.setDelegate(new ListItemWriter(writtenItems)); + List> processedItems = new ArrayList<>(); + + processedItems.add(new FutureTask<>(new Callable() { + @Override + public String call() throws Exception { + return "foo"; + } + })); + + processedItems.add(new FutureTask<>(new Callable() { + @Override + public String call() throws Exception { + return "bar"; + } + })); + + for (FutureTask processedItem : processedItems) { + taskExecutor.execute(processedItem); + } + + writer.write(processedItems); + + assertEquals(2, writtenItems.size()); + assertTrue(writtenItems.contains("foo")); + assertTrue(writtenItems.contains("bar")); + } + + @Test + public void testFilteredItem() throws Exception { + writer.setDelegate(new ListItemWriter(writtenItems)); + List> processedItems = new ArrayList<>(); + + processedItems.add(new FutureTask<>(new Callable() { + @Override + public String call() throws Exception { + return "foo"; + } + })); + + processedItems.add(new FutureTask<>(new Callable() { + @Override + public String call() throws Exception { + return null; + } + })); + + for (FutureTask processedItem : processedItems) { + taskExecutor.execute(processedItem); + } + + writer.write(processedItems); + + assertEquals(1, writtenItems.size()); + assertTrue(writtenItems.contains("foo")); + } + + @Test + public void testException() throws Exception { + writer.setDelegate(new ListItemWriter(writtenItems)); + List> processedItems = new ArrayList<>(); + + processedItems.add(new FutureTask<>(new Callable() { + @Override + public String call() throws Exception { + return "foo"; + } + })); + + processedItems.add(new FutureTask<>(new Callable() { + @Override + public String call() throws Exception { + throw new RuntimeException("This was expected"); + } + })); + + for (FutureTask processedItem : processedItems) { + taskExecutor.execute(processedItem); + } + + try { + writer.write(processedItems); + } + catch (Exception e) { + assertTrue(e instanceof RuntimeException); + assertEquals("This was expected", e.getMessage()); + } + } + + @Test + public void testExecutionException() { + ListItemWriter delegate = new ListItemWriter(writtenItems); + writer.setDelegate(delegate); + List> processedItems = new ArrayList<>(); + + processedItems.add(new Future() { + + @Override + public boolean cancel(boolean mayInterruptIfRunning) { + return false; + } + + @Override + public boolean isCancelled() { + return false; + } + + @Override + public boolean isDone() { + return false; + } + + @Override + public String get() throws InterruptedException, ExecutionException { + throw new InterruptedException("expected"); + } + + @Override + public String get(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException, TimeoutException { + return null; + } + }); + + try { + writer.write(processedItems); + } + catch (Exception e) { + assertFalse(e instanceof ExecutionException); + } + + assertEquals(0, writtenItems.size()); + } + + @Test + public void testStreamDelegate() throws Exception { + ListItemStreamWriter itemWriter = new ListItemStreamWriter(writtenItems); + writer.setDelegate(itemWriter); + + List> processedItems = new ArrayList<>(); + + ExecutionContext executionContext = new ExecutionContext(); + writer.open(executionContext); + writer.write(processedItems); + writer.update(executionContext); + writer.close(); + + assertTrue(itemWriter.isOpened); + assertTrue(itemWriter.isUpdated); + assertTrue(itemWriter.isClosed); + } + + @Test + public void testNonStreamDelegate() throws Exception { + ListItemWriter itemWriter = new ListItemWriter(writtenItems); + writer.setDelegate(itemWriter); + + List> processedItems = new ArrayList<>(); + + ExecutionContext executionContext = new ExecutionContext(); + writer.open(executionContext); + writer.write(processedItems); + writer.update(executionContext); + writer.close(); + + assertFalse(itemWriter.isOpened); + assertFalse(itemWriter.isUpdated); + assertFalse(itemWriter.isClosed); + } + + private class ListItemWriter implements ItemWriter { + + protected List items; + public boolean isOpened = false; + public boolean isUpdated = false; + public boolean isClosed = false; + + public ListItemWriter(List items) { + this.items = items; + } + + @Override + public void write(List items) throws Exception { + this.items.addAll(items); + } + } + + private class ListItemStreamWriter implements ItemStreamWriter { + public boolean isOpened = false; + public boolean isUpdated = false; + public boolean isClosed = false; + protected List items; + + public ListItemStreamWriter(List items) { + this.items = items; + } + + @Override + public void write(List items) throws Exception { + this.items.addAll(items); + } + + @Override + public void open(ExecutionContext executionContext) throws ItemStreamException { + isOpened = true; + } + + @Override + public void update(ExecutionContext executionContext) throws ItemStreamException { + isUpdated = true; + } + + @Override + public void close() throws ItemStreamException { + isClosed = true; + } + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/async/PollingAsyncItemProcessorMessagingGatewayTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/async/PollingAsyncItemProcessorMessagingGatewayTests.java new file mode 100644 index 0000000000..520c7f373d --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/async/PollingAsyncItemProcessorMessagingGatewayTests.java @@ -0,0 +1,118 @@ +/* + * Copyright 2006-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.async; + +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.Callable; +import java.util.concurrent.Future; + +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.MethodRule; +import org.junit.runner.RunWith; +import org.junit.runners.model.FrameworkMethod; +import org.junit.runners.model.Statement; + +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.test.MetaDataInstanceFactory; +import org.springframework.batch.test.StepScopeTestUtils; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.integration.annotation.MessageEndpoint; +import org.springframework.integration.annotation.ServiceActivator; +import org.springframework.messaging.handler.annotation.Header; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration +public class PollingAsyncItemProcessorMessagingGatewayTests { + + private AsyncItemProcessor processor = new AsyncItemProcessor<>(); + + private StepExecution stepExecution = MetaDataInstanceFactory.createStepExecution(new JobParametersBuilder().addLong("factor", 2L).toJobParameters());; + + @Rule + public MethodRule rule = new MethodRule() { + public Statement apply(final Statement base, FrameworkMethod method, Object target) { + return new Statement() { + @Override + public void evaluate() throws Throwable { + StepScopeTestUtils.doInStepScope(stepExecution, new Callable() { + public Void call() throws Exception { + try { + base.evaluate(); + } + catch (Exception e) { + throw e; + } + catch (Throwable e) { + throw new Error(e); + } + return null; + } + }); + }; + }; + } + }; + + @Autowired + private ItemProcessor delegate; + + @Test + public void testMultiExecution() throws Exception { + processor.setDelegate(delegate); + processor.setTaskExecutor(new SimpleAsyncTaskExecutor()); + List> list = new ArrayList<>(); + for (int count = 0; count < 10; count++) { + list.add(processor.process("foo" + count)); + } + for (Future future : list) { + String value = future.get(); + /** + * This delegate is a Spring Integration MessagingGateway. It can + * easily return null because of a timeout, but that will be treated + * by Batch as a filtered item, whereas it is really more like a + * skip. So we have to throw an exception in the processor if an + * unexpected null value comes back. + */ + assertNotNull(value); + assertTrue(value.matches("foo.*foo.*")); + } + } + + @MessageEndpoint + public static class Doubler { + + @ServiceActivator + public String cat(String value, @Header(value="stepExecution.jobExecution.jobParameters.getLong('factor')", required=false) Integer input) { + long factor = input==null ? 1 : input; + for (int i=1; i writer = new ChunkMessageChannelItemWriter<>(); + + @Autowired + @Qualifier("requests") + private MessageChannel requests; + + @Autowired + @Qualifier("replies") + private PollableChannel replies; + + private final SimpleStepFactoryBean factory = new SimpleStepFactoryBean<>(); + + private SimpleJobRepository jobRepository; + + private static long jobCounter; + + @Before + public void setUp() { + + jobRepository = new SimpleJobRepository(new MapJobInstanceDao(), new MapJobExecutionDao(), + new MapStepExecutionDao(), new MapExecutionContextDao()); + factory.setJobRepository(jobRepository); + factory.setTransactionManager(new ResourcelessTransactionManager()); + factory.setBeanName("step"); + factory.setItemWriter(writer); + factory.setCommitInterval(4); + + MessagingTemplate gateway = new MessagingTemplate(); + writer.setMessagingOperations(gateway); + + gateway.setDefaultChannel(requests); + writer.setReplyChannel(replies); + gateway.setReceiveTimeout(100); + + TestItemWriter.count = 0; + + // Drain queues + Message message = replies.receive(10); + while (message != null) { + System.err.println(message); + message = replies.receive(10); + } + + } + + @After + public void tearDown() { + while (replies.receive(10L) != null) { + } + } + + @Test + public void testOpenWithNoState() throws Exception { + writer.open(new ExecutionContext()); + } + + @Test + public void testUpdateAndOpenWithState() throws Exception { + ExecutionContext executionContext = new ExecutionContext(); + writer.update(executionContext); + writer.open(executionContext); + assertEquals(0, executionContext.getInt(ChunkMessageChannelItemWriter.EXPECTED)); + assertEquals(0, executionContext.getInt(ChunkMessageChannelItemWriter.ACTUAL)); + } + + @Test + public void testVanillaIteration() throws Exception { + + factory.setItemReader(new ListItemReader<>(Arrays.asList(StringUtils + .commaDelimitedListToStringArray("1,2,3,4,5,6")))); + + Step step = factory.getObject(); + + StepExecution stepExecution = getStepExecution(step); + step.execute(stepExecution); + + waitForResults(6, 10); + + assertEquals(6, TestItemWriter.count); + assertEquals(6, stepExecution.getReadCount()); + + } + + @Test + public void testSimulatedRestart() throws Exception { + + factory.setItemReader(new ListItemReader<>(Arrays.asList(StringUtils + .commaDelimitedListToStringArray("1,2,3,4,5,6")))); + + Step step = factory.getObject(); + + StepExecution stepExecution = getStepExecution(step); + + // Set up context with two messages (chunks) in the backlog + stepExecution.getExecutionContext().putInt(ChunkMessageChannelItemWriter.EXPECTED, 6); + stepExecution.getExecutionContext().putInt(ChunkMessageChannelItemWriter.ACTUAL, 4); + // And make the back log real + requests.send(getSimpleMessage("foo", stepExecution.getJobExecution().getJobId())); + requests.send(getSimpleMessage("bar", stepExecution.getJobExecution().getJobId())); + step.execute(stepExecution); + + waitForResults(8, 10); + + assertEquals(8, TestItemWriter.count); + assertEquals(6, stepExecution.getReadCount()); + + } + + @Test + public void testSimulatedRestartWithBadMessagesFromAnotherJob() throws Exception { + + factory.setItemReader(new ListItemReader<>(Arrays.asList(StringUtils + .commaDelimitedListToStringArray("1,2,3,4,5,6")))); + + Step step = factory.getObject(); + + StepExecution stepExecution = getStepExecution(step); + + // Set up context with two messages (chunks) in the backlog + stepExecution.getExecutionContext().putInt(ChunkMessageChannelItemWriter.EXPECTED, 3); + stepExecution.getExecutionContext().putInt(ChunkMessageChannelItemWriter.ACTUAL, 2); + + // Speed up the eventual failure + writer.setMaxWaitTimeouts(2); + + // And make the back log real + requests.send(getSimpleMessage("foo", 4321L)); + step.execute(stepExecution); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals(ExitStatus.FAILED.getExitCode(), stepExecution.getExitStatus().getExitCode()); + String message = stepExecution.getExitStatus().getExitDescription(); + assertTrue("Message does not contain 'wrong job': " + message, message.contains("wrong job")); + + waitForResults(1, 10); + + assertEquals(1, TestItemWriter.count); + assertEquals(0, stepExecution.getReadCount()); + + } + + @SuppressWarnings({"unchecked", "rawtypes"}) + private GenericMessage getSimpleMessage(String string, Long jobId) { + StepContribution stepContribution = new JobExecution(new JobInstance(0L, "job"), new JobParameters()) + .createStepExecution("step").createStepContribution(); + ChunkRequest chunk = new ChunkRequest(0, StringUtils.commaDelimitedListToSet(string), jobId, stepContribution); + GenericMessage message = new GenericMessage<>(chunk); + return message; + } + + @Test + public void testEarlyCompletionSignalledInHandler() throws Exception { + + factory.setItemReader(new ListItemReader<>(Arrays.asList(StringUtils + .commaDelimitedListToStringArray("1,fail,3,4,5,6")))); + factory.setCommitInterval(2); + + Step step = factory.getObject(); + + StepExecution stepExecution = getStepExecution(step); + step.execute(stepExecution); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals(ExitStatus.FAILED.getExitCode(), stepExecution.getExitStatus().getExitCode()); + String message = stepExecution.getExitStatus().getExitDescription(); + assertTrue("Message does not contain 'fail': " + message, message.contains("fail")); + + waitForResults(2, 10); + + // The number of items processed is actually between 1 and 6, because + // the one that failed might have been processed out of order. + assertTrue(1 <= TestItemWriter.count); + assertTrue(6 >= TestItemWriter.count); + // But it should fail the step in any case + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + + } + + @Test + public void testSimulatedRestartWithNoBacklog() throws Exception { + + factory.setItemReader(new ListItemReader<>(Arrays.asList(StringUtils + .commaDelimitedListToStringArray("1,2,3,4,5,6")))); + + Step step = factory.getObject(); + + StepExecution stepExecution = getStepExecution(step); + + // Set up expectation of three messages (chunks) in the backlog + stepExecution.getExecutionContext().putInt(ChunkMessageChannelItemWriter.EXPECTED, 6); + stepExecution.getExecutionContext().putInt(ChunkMessageChannelItemWriter.ACTUAL, 3); + + writer.setMaxWaitTimeouts(2); + + /* + * With no backlog we process all the items, but the listener can't + * reconcile the expected number of items with the actual. An infinite + * loop would be bad, so the best we can do is fail as fast as possible. + */ + step.execute(stepExecution); + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals(ExitStatus.FAILED.getExitCode(), stepExecution.getExitStatus().getExitCode()); + String message = stepExecution.getExitStatus().getExitDescription(); + assertTrue("Message did not contain 'timed out': " + message, message.toLowerCase().contains("timed out")); + + assertEquals(0, TestItemWriter.count); + assertEquals(0, stepExecution.getReadCount()); + + } + + /** + * This one is flakey - we try to force it to wait until after the step to + * finish processing just by waiting for long enough. + */ + @Test + public void testFailureInStepListener() throws Exception { + + factory.setItemReader(new ListItemReader<>(Arrays.asList(StringUtils + .commaDelimitedListToStringArray("wait,fail,3,4,5,6")))); + + Step step = factory.getObject(); + + StepExecution stepExecution = getStepExecution(step); + step.execute(stepExecution); + + waitForResults(2, 10); + + // The number of items processed is actually between 1 and 6, because + // the one that failed might have been processed out of order. + assertTrue(1 <= TestItemWriter.count); + assertTrue(6 >= TestItemWriter.count); + + assertEquals(BatchStatus.FAILED, stepExecution.getStatus()); + assertEquals(ExitStatus.FAILED.getExitCode(), stepExecution.getExitStatus().getExitCode()); + + String exitDescription = stepExecution.getExitStatus().getExitDescription(); + assertTrue("Exit description does not contain exception type name: " + exitDescription, exitDescription + .contains(AsynchronousFailureException.class.getName())); + + } + + // TODO : test non-dispatch of empty chunk + + private void waitForResults(int expected, int maxWait) throws InterruptedException { + int count = 0; + while (TestItemWriter.count < expected && count < maxWait) { + count++; + Thread.sleep(10); + } + } + + private StepExecution getStepExecution(Step step) throws JobExecutionAlreadyRunningException, JobRestartException, + JobInstanceAlreadyCompleteException { + SimpleJob job = new SimpleJob(); + job.setName("job"); + JobExecution jobExecution = jobRepository.createJobExecution(job.getName(), new JobParametersBuilder().addLong( + "job.counter", jobCounter++).toJobParameters()); + StepExecution stepExecution = jobExecution.createStepExecution(step.getName()); + jobRepository.add(stepExecution); + return stepExecution; + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/ChunkProcessorChunkHandlerTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/ChunkProcessorChunkHandlerTests.java new file mode 100644 index 0000000000..2b729ee7ab --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/ChunkProcessorChunkHandlerTests.java @@ -0,0 +1,35 @@ +package org.springframework.batch.integration.chunk; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +import org.junit.Test; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.step.item.Chunk; +import org.springframework.batch.core.step.item.ChunkProcessor; +import org.springframework.batch.test.MetaDataInstanceFactory; +import org.springframework.util.StringUtils; + +public class ChunkProcessorChunkHandlerTests { + + private ChunkProcessorChunkHandler handler = new ChunkProcessorChunkHandler<>(); + + protected int count = 0; + + @Test + public void testVanillaHandleChunk() throws Exception { + handler.setChunkProcessor(new ChunkProcessor() { + public void process(StepContribution contribution, Chunk chunk) throws Exception { + count += chunk.size(); + } + }); + StepContribution stepContribution = MetaDataInstanceFactory.createStepExecution().createStepContribution(); + ChunkResponse response = handler.handleChunk(new ChunkRequest<>(0, StringUtils + .commaDelimitedListToSet("foo,bar"), 12L, stepContribution)); + assertEquals(stepContribution, response.getStepContribution()); + assertEquals(12, response.getJobId().longValue()); + assertTrue(response.isSuccessful()); + assertEquals(2, count); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/ChunkRequestTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/ChunkRequestTests.java new file mode 100644 index 0000000000..f51c51cf2b --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/ChunkRequestTests.java @@ -0,0 +1,66 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.chunk; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import java.util.Arrays; + +import org.junit.Test; +import org.springframework.batch.test.MetaDataInstanceFactory; +import org.springframework.util.SerializationUtils; + +/** + * @author Dave Syer + * + */ +public class ChunkRequestTests { + + private ChunkRequest request = new ChunkRequest<>(0, Arrays.asList("foo", "bar"), + 111L, MetaDataInstanceFactory.createStepExecution().createStepContribution()); + + @Test + public void testGetJobId() { + assertEquals(111L, request.getJobId()); + } + + @Test + public void testGetItems() { + assertEquals(2, request.getItems().size()); + } + + @Test + public void testGetStepContribution() { + assertNotNull(request.getStepContribution()); + } + + @Test + public void testToString() { + System.err.println(request.toString()); + } + + @Test + public void testSerializable() throws Exception { + @SuppressWarnings("unchecked") + ChunkRequest result = (ChunkRequest) SerializationUtils.deserialize(SerializationUtils + .serialize(request)); + assertNotNull(result.getStepContribution()); + assertEquals(111L, result.getJobId()); + assertEquals(2, result.getItems().size()); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/ChunkResponseTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/ChunkResponseTests.java new file mode 100644 index 0000000000..0cd39c3ee1 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/ChunkResponseTests.java @@ -0,0 +1,56 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.chunk; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import org.junit.Test; +import org.springframework.batch.test.MetaDataInstanceFactory; +import org.springframework.util.SerializationUtils; + +/** + * @author Dave Syer + * + */ +public class ChunkResponseTests { + + private ChunkResponse response = new ChunkResponse(0, 111L, MetaDataInstanceFactory.createStepExecution() + .createStepContribution()); + + @Test + public void testGetJobId() { + assertEquals(new Long(111L), response.getJobId()); + } + + @Test + public void testGetStepContribution() { + assertNotNull(response.getStepContribution()); + } + + @Test + public void testToString() { + System.err.println(response.toString()); + } + + @Test + public void testSerializable() throws Exception { + ChunkResponse result = (ChunkResponse) SerializationUtils.deserialize(SerializationUtils.serialize(response)); + assertNotNull(result.getStepContribution()); + assertEquals(new Long(111L), result.getJobId()); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/MessageSourcePollerInterceptorTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/MessageSourcePollerInterceptorTests.java new file mode 100644 index 0000000000..a2d5a167b6 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/MessageSourcePollerInterceptorTests.java @@ -0,0 +1,55 @@ +package org.springframework.batch.integration.chunk; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +import org.junit.Test; +import org.springframework.integration.channel.QueueChannel; +import org.springframework.integration.core.MessageSource; +import org.springframework.messaging.Message; +import org.springframework.messaging.support.GenericMessage; + +public class MessageSourcePollerInterceptorTests { + + @Test(expected = IllegalStateException.class) + public void testMandatoryPropertiesUnset() throws Exception { + MessageSourcePollerInterceptor interceptor = new MessageSourcePollerInterceptor(); + interceptor.afterPropertiesSet(); + } + + @Test + public void testMandatoryPropertiesSetViaConstructor() throws Exception { + MessageSourcePollerInterceptor interceptor = new MessageSourcePollerInterceptor(new TestMessageSource("foo")); + interceptor.afterPropertiesSet(); + } + + @Test + public void testMandatoryPropertiesSet() throws Exception { + MessageSourcePollerInterceptor interceptor = new MessageSourcePollerInterceptor(); + interceptor.setMessageSource(new TestMessageSource("foo")); + interceptor.afterPropertiesSet(); + } + + @Test + public void testPreReceive() throws Exception { + MessageSourcePollerInterceptor interceptor = new MessageSourcePollerInterceptor(new TestMessageSource("foo")); + QueueChannel channel = new QueueChannel(); + assertTrue(interceptor.preReceive(channel)); + assertEquals("foo", channel.receive(10L).getPayload()); + } + + private static class TestMessageSource implements MessageSource { + + private final String payload; + + public TestMessageSource(String payload) { + super(); + this.payload = payload; + } + + public Message receive() { + return new GenericMessage<>(payload); + } + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepIntegrationTests.java new file mode 100644 index 0000000000..ebc2b45e63 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepIntegrationTests.java @@ -0,0 +1,89 @@ +package org.springframework.batch.integration.chunk; + +import static org.junit.Assert.assertEquals; + +import java.util.Collections; + +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameter; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.messaging.Message; +import org.springframework.messaging.PollableChannel; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class RemoteChunkFaultTolerantStepIntegrationTests { + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private Job job; + + @Autowired + private PollableChannel replies; + + @Before + public void drain() { + Message message = replies.receive(100L); + while (message!=null) { + // System.err.println(message); + message = replies.receive(100L); + } + } + + @Test + public void testFailedStep() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParameters(Collections.singletonMap("item.three", + new JobParameter("unsupported")))); + assertEquals(BatchStatus.FAILED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + // In principle the write count could be more than 2 and less than 9... + assertEquals(7, stepExecution.getWriteCount()); + } + + @Test + public void testFailedStepOnError() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParameters(Collections.singletonMap("item.three", + new JobParameter("error")))); + assertEquals(BatchStatus.FAILED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + // In principle the write count could be more than 2 and less than 9... + assertEquals(7, stepExecution.getWriteCount()); + } + + @Test + public void testSunnyDayFaultTolerant() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParameters(Collections.singletonMap("item.three", + new JobParameter("3")))); + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + assertEquals(9, stepExecution.getWriteCount()); + } + + @Test + public void testSkipsInWriter() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParametersBuilder().addString("item.three", "fail") + .addLong("run.id", 1L).toJobParameters()); + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + assertEquals(7, stepExecution.getWriteCount()); + // The whole chunk gets skipped... + assertEquals(2, stepExecution.getWriteSkipCount()); + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJdbcIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJdbcIntegrationTests.java new file mode 100644 index 0000000000..373d8a1934 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJdbcIntegrationTests.java @@ -0,0 +1,94 @@ +package org.springframework.batch.integration.chunk; + +import static org.junit.Assert.assertEquals; + +import java.util.Collections; + +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameter; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.messaging.Message; +import org.springframework.messaging.PollableChannel; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class RemoteChunkFaultTolerantStepJdbcIntegrationTests { + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private Job job; + + @Autowired + private PollableChannel replies; + + @Before + public void drain() { + Message message = replies.receive(100L); + while (message!=null) { + message = replies.receive(100L); + } + } + + @Test + @DirtiesContext + public void testFailedStep() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParameters(Collections.singletonMap("item.three", + new JobParameter("unsupported")))); + assertEquals(BatchStatus.FAILED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + // In principle the write count could be more than 2 and less than 9... + assertEquals(7, stepExecution.getWriteCount()); + } + + @Test + @DirtiesContext + public void testFailedStepOnError() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParameters(Collections.singletonMap("item.three", + new JobParameter("error")))); + assertEquals(BatchStatus.FAILED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + // In principle the write count could be more than 2 and less than 9... + assertEquals(7, stepExecution.getWriteCount()); + } + + @Test + @DirtiesContext + public void testSunnyDayFaultTolerant() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParameters(Collections.singletonMap("item.three", + new JobParameter("3")))); + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + assertEquals(9, stepExecution.getWriteCount()); + } + + @Test + @DirtiesContext + public void testSkipsInWriter() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParametersBuilder().addString("item.three", "fail") + .addLong("run.id", 1L).toJobParameters()); + // System.err.println(new SimpleJdbcTemplate(dataSource).queryForList("SELECT * FROM INT_MESSAGE_GROUP")); + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + assertEquals(7, stepExecution.getWriteCount()); + // The whole chunk gets skipped... + assertEquals(2, stepExecution.getWriteSkipCount()); + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJmsIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJmsIntegrationTests.java new file mode 100644 index 0000000000..6391c1af81 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJmsIntegrationTests.java @@ -0,0 +1,83 @@ +package org.springframework.batch.integration.chunk; + +import static org.junit.Assert.assertEquals; + +import java.io.File; +import java.util.Collections; + +import org.junit.BeforeClass; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameter; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.util.FileSystemUtils; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +@DirtiesContext +public class RemoteChunkFaultTolerantStepJmsIntegrationTests { + + @BeforeClass + public static void clear() { + FileSystemUtils.deleteRecursively(new File("activemq-data")); + } + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private Job job; + + @Test + public void testFailedStep() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParameters(Collections.singletonMap("item.three", + new JobParameter("unsupported")))); + assertEquals(BatchStatus.FAILED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + // In principle the write count could be more than 2 and less than 9... + assertEquals(7, stepExecution.getWriteCount()); + } + + @Test + public void testFailedStepOnError() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParameters(Collections.singletonMap("item.three", + new JobParameter("error")))); + assertEquals(BatchStatus.FAILED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + // In principle the write count could be more than 2 and less than 9... + assertEquals(7, stepExecution.getWriteCount()); + } + + @Test + public void testSunnyDayFaultTolerant() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParameters(Collections.singletonMap("item.three", + new JobParameter("3")))); + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + assertEquals(9, stepExecution.getWriteCount()); + } + + @Test + public void testSkipsInWriter() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParametersBuilder().addString("item.three", "fail") + .addLong("run.id", 1L).toJobParameters()); + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + assertEquals(7, stepExecution.getWriteCount()); + assertEquals(2, stepExecution.getWriteSkipCount()); + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkStepIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkStepIntegrationTests.java new file mode 100644 index 0000000000..0086e16724 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkStepIntegrationTests.java @@ -0,0 +1,51 @@ +package org.springframework.batch.integration.chunk; + +import static org.junit.Assert.assertEquals; + +import java.util.Collections; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameter; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class RemoteChunkStepIntegrationTests { + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private Job job; + + @Test + public void testSunnyDaySimpleStep() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParameters(Collections.singletonMap("item.three", + new JobParameter("3")))); + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + assertEquals(9, stepExecution.getWriteCount()); + } + + @Test + public void testFailedStep() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParameters(Collections.singletonMap("item.three", + new JobParameter("fail")))); + assertEquals(BatchStatus.FAILED, jobExecution.getStatus()); + StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); + assertEquals(9, stepExecution.getReadCount()); + // In principle the write count could be more than 2 and less than 9... + assertEquals(7, stepExecution.getWriteCount()); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkingManagerStepBuilderTest.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkingManagerStepBuilderTest.java new file mode 100644 index 0000000000..0304cbe262 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkingManagerStepBuilderTest.java @@ -0,0 +1,379 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.chunk; + +import java.io.IOException; +import java.util.Arrays; +import java.util.List; + +import org.junit.Assert; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; +import org.junit.runner.RunWith; + +import org.springframework.batch.core.ChunkListener; +import org.springframework.batch.core.ItemReadListener; +import org.springframework.batch.core.ItemWriteListener; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.SkipListener; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.StepExecutionListener; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.batch.core.step.item.ChunkOrientedTasklet; +import org.springframework.batch.core.step.item.SimpleChunkProcessor; +import org.springframework.batch.core.step.item.SimpleChunkProvider; +import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemStreamSupport; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.CompositeItemStream; +import org.springframework.batch.item.support.ListItemReader; +import org.springframework.batch.repeat.support.RepeatTemplate; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.annotation.Configuration; +import org.springframework.integration.channel.DirectChannel; +import org.springframework.integration.channel.QueueChannel; +import org.springframework.integration.core.MessagingTemplate; +import org.springframework.lang.Nullable; +import org.springframework.messaging.PollableChannel; +import org.springframework.retry.RetryListener; +import org.springframework.retry.backoff.NoBackOffPolicy; +import org.springframework.retry.policy.MapRetryContextCache; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringRunner; +import org.springframework.test.util.ReflectionTestUtils; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.transaction.interceptor.DefaultTransactionAttribute; + +import static org.mockito.Mockito.any; +import static org.mockito.Mockito.atLeastOnce; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +/** + * @author Mahmoud Ben Hassine + */ +@RunWith(SpringRunner.class) +@ContextConfiguration(classes = {RemoteChunkingManagerStepBuilderTest.BatchConfiguration.class}) +public class RemoteChunkingManagerStepBuilderTest { + + @Rule + public ExpectedException expectedException = ExpectedException.none(); + + @Autowired + private JobRepository jobRepository; + @Autowired + private PlatformTransactionManager transactionManager; + + private PollableChannel inputChannel = new QueueChannel(); + private DirectChannel outputChannel = new DirectChannel(); + private ItemReader itemReader = new ListItemReader<>(Arrays.asList("a", "b", "c")); + + @Test + public void inputChannelMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("inputChannel must not be null"); + + // when + TaskletStep step = new RemoteChunkingManagerStepBuilder("step") + .inputChannel(null) + .build(); + + // then + // expected exception + } + + @Test + public void outputChannelMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("outputChannel must not be null"); + + // when + TaskletStep step = new RemoteChunkingManagerStepBuilder("step") + .outputChannel(null) + .build(); + + // then + // expected exception + } + + @Test + public void messagingTemplateMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("messagingTemplate must not be null"); + + // when + TaskletStep step = new RemoteChunkingManagerStepBuilder("step") + .messagingTemplate(null) + .build(); + + // then + // expected exception + } + + @Test + public void maxWaitTimeoutsMustBeGreaterThanZero() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("maxWaitTimeouts must be greater than zero"); + + // when + TaskletStep step = new RemoteChunkingManagerStepBuilder("step") + .maxWaitTimeouts(-1) + .build(); + + // then + // expected exception + } + + @Test + public void throttleLimitMustNotBeGreaterThanZero() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("throttleLimit must be greater than zero"); + + // when + TaskletStep step = new RemoteChunkingManagerStepBuilder("step") + .throttleLimit(-1L) + .build(); + + // then + // expected exception + } + + @Test + public void testMandatoryInputChannel() { + // given + RemoteChunkingManagerStepBuilder builder = new RemoteChunkingManagerStepBuilder<>("step"); + + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("An InputChannel must be provided"); + + // when + TaskletStep step = builder.build(); + + // then + // expected exception + } + + @Test + public void eitherOutputChannelOrMessagingTemplateMustBeProvided() { + // given + RemoteChunkingManagerStepBuilder builder = new RemoteChunkingManagerStepBuilder("step") + .inputChannel(this.inputChannel) + .outputChannel(new DirectChannel()) + .messagingTemplate(new MessagingTemplate()); + + this.expectedException.expect(IllegalStateException.class); + this.expectedException.expectMessage("You must specify either an outputChannel or a messagingTemplate but not both."); + + // when + TaskletStep step = builder.build(); + + // then + // expected exception + } + + @Test + public void testUnsupportedOperationExceptionWhenSpecifyingAnItemWriter() { + // given + this.expectedException.expect(UnsupportedOperationException.class); + this.expectedException.expectMessage("When configuring a manager " + + "step for remote chunking, the item writer will be automatically " + + "set to an instance of ChunkMessageChannelItemWriter. " + + "The item writer must not be provided in this case."); + + // when + TaskletStep step = new RemoteChunkingManagerStepBuilder("step") + .reader(this.itemReader) + .writer(items -> { }) + .repository(this.jobRepository) + .transactionManager(this.transactionManager) + .inputChannel(this.inputChannel) + .outputChannel(this.outputChannel) + .build(); + + // then + // expected exception + } + + @Test + public void testManagerStepCreation() { + // when + TaskletStep taskletStep = new RemoteChunkingManagerStepBuilder("step") + .reader(this.itemReader) + .repository(this.jobRepository) + .transactionManager(this.transactionManager) + .inputChannel(this.inputChannel) + .outputChannel(this.outputChannel) + .build(); + + // then + Assert.assertNotNull(taskletStep); + } + + /* + * The following test is to cover setters that override those from parent builders. + */ + @Test + @SuppressWarnings({"unchecked", "rawtypes"}) + public void testSetters() throws Exception { + // when + DefaultTransactionAttribute transactionAttribute = new DefaultTransactionAttribute(); + + Object annotatedListener = new Object(); + MapRetryContextCache retryCache = new MapRetryContextCache(); + RepeatTemplate stepOperations = new RepeatTemplate(); + NoBackOffPolicy backOffPolicy = new NoBackOffPolicy(); + ItemStreamSupport stream = new ItemStreamSupport() { + }; + StepExecutionListener stepExecutionListener = mock(StepExecutionListener.class); + ItemReadListener itemReadListener = mock(ItemReadListener.class); + ItemWriteListener itemWriteListener = mock(ItemWriteListener.class); + ChunkListener chunkListener = mock(ChunkListener.class); + SkipListener skipListener = mock(SkipListener.class); + RetryListener retryListener = mock(RetryListener.class); + + when(retryListener.open(any(), any())).thenReturn(true); + + ItemProcessor itemProcessor = item -> { + System.out.println("processing item " + item); + if(item.equals("b")) { + throw new Exception("b was found"); + } + else { + return item; + } + }; + + ItemReader itemReader = new ItemReader() { + + int count = 0; + List items = Arrays.asList("a", "b", "c", "d", "d", "e", "f", "g", "h", "i"); + + @Nullable + @Override + public String read() throws Exception { + System.out.println(">> count == " + count); + if(count == 6) { + count++; + throw new IOException("6th item"); + } + else if(count == 7) { + count++; + throw new RuntimeException("7th item"); + } + else if(count < items.size()){ + String item = items.get(count++); + System.out.println(">> item read was " + item); + return item; + } + else { + return null; + } + } + }; + + TaskletStep taskletStep = new RemoteChunkingManagerStepBuilder("step") + .reader(itemReader) + .readerIsTransactionalQueue() + .processor(itemProcessor) + .repository(this.jobRepository) + .transactionManager(this.transactionManager) + .transactionAttribute(transactionAttribute) + .inputChannel(this.inputChannel) + .outputChannel(this.outputChannel) + .listener(annotatedListener) + .listener(skipListener) + .listener(chunkListener) + .listener(stepExecutionListener) + .listener(itemReadListener) + .listener(itemWriteListener) + .listener(retryListener) + .skip(Exception.class) + .noSkip(RuntimeException.class) + .skipLimit(10) + .retry(IOException.class) + .noRetry(RuntimeException.class) + .retryLimit(10) + .retryContextCache(retryCache) + .noRollback(Exception.class) + .startLimit(3) + .allowStartIfComplete(true) + .stepOperations(stepOperations) + .chunk(3) + .backOffPolicy(backOffPolicy) + .stream(stream) + .keyGenerator(Object::hashCode) + .build(); + + JobExecution jobExecution = this.jobRepository.createJobExecution("job1", new JobParameters()); + StepExecution stepExecution = new StepExecution("step1", jobExecution); + this.jobRepository.add(stepExecution); + + taskletStep.execute(stepExecution); + + // then + Assert.assertNotNull(taskletStep); + ChunkOrientedTasklet tasklet = (ChunkOrientedTasklet) ReflectionTestUtils.getField(taskletStep, "tasklet"); + SimpleChunkProvider provider = (SimpleChunkProvider) ReflectionTestUtils.getField(tasklet, "chunkProvider"); + SimpleChunkProcessor processor = (SimpleChunkProcessor) ReflectionTestUtils.getField(tasklet, "chunkProcessor"); + ItemWriter itemWriter = (ItemWriter) ReflectionTestUtils.getField(processor, "itemWriter"); + MessagingTemplate messagingTemplate = (MessagingTemplate) ReflectionTestUtils.getField(itemWriter, "messagingGateway"); + CompositeItemStream compositeItemStream = (CompositeItemStream) ReflectionTestUtils.getField(taskletStep, "stream"); + + Assert.assertEquals(ReflectionTestUtils.getField(provider, "itemReader"), itemReader); + Assert.assertFalse((Boolean) ReflectionTestUtils.getField(tasklet, "buffering")); + Assert.assertEquals(ReflectionTestUtils.getField(taskletStep, "jobRepository"), this.jobRepository); + Assert.assertEquals(ReflectionTestUtils.getField(taskletStep, "transactionManager"), this.transactionManager); + Assert.assertEquals(ReflectionTestUtils.getField(taskletStep, "transactionAttribute"), transactionAttribute); + Assert.assertEquals(ReflectionTestUtils.getField(itemWriter, "replyChannel"), this.inputChannel); + Assert.assertEquals(ReflectionTestUtils.getField(messagingTemplate, "defaultDestination"), this.outputChannel); + Assert.assertEquals(ReflectionTestUtils.getField(processor, "itemProcessor"), itemProcessor); + + Assert.assertEquals((int) ReflectionTestUtils.getField(taskletStep, "startLimit"), 3); + Assert.assertTrue((Boolean) ReflectionTestUtils.getField(taskletStep, "allowStartIfComplete")); + Object stepOperationsUsed = ReflectionTestUtils.getField(taskletStep, "stepOperations"); + Assert.assertEquals(stepOperationsUsed, stepOperations); + + Assert.assertEquals(((List)ReflectionTestUtils.getField(compositeItemStream, "streams")).size(), 2); + Assert.assertNotNull(ReflectionTestUtils.getField(processor, "keyGenerator")); + + verify(skipListener, atLeastOnce()).onSkipInProcess(any(), any()); + verify(retryListener, atLeastOnce()).open(any(), any()); + verify(stepExecutionListener, atLeastOnce()).beforeStep(any()); + verify(chunkListener, atLeastOnce()).beforeChunk(any()); + verify(itemReadListener, atLeastOnce()).beforeRead(); + verify(itemWriteListener, atLeastOnce()).beforeWrite(any()); + + Assert.assertEquals(stepExecution.getSkipCount(), 2); + Assert.assertEquals(stepExecution.getRollbackCount(), 3); + } + + @Configuration + @EnableBatchProcessing + public static class BatchConfiguration { + + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkingWorkerBuilderTest.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkingWorkerBuilderTest.java new file mode 100644 index 0000000000..e21fee18c7 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/RemoteChunkingWorkerBuilderTest.java @@ -0,0 +1,166 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.chunk; + +import org.junit.Assert; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; + +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.PassThroughItemProcessor; +import org.springframework.integration.channel.DirectChannel; +import org.springframework.integration.dsl.IntegrationFlow; + +/** + * @author Mahmoud Ben Hassine + */ +public class RemoteChunkingWorkerBuilderTest { + + @Rule + public ExpectedException expectedException = ExpectedException.none(); + + private ItemProcessor itemProcessor = new PassThroughItemProcessor<>(); + private ItemWriter itemWriter = items -> { }; + + @Test + public void itemProcessorMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("itemProcessor must not be null"); + + // when + IntegrationFlow integrationFlow = new RemoteChunkingWorkerBuilder() + .itemProcessor(null) + .build(); + + // then + // expected exception + } + + @Test + public void itemWriterMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("itemWriter must not be null"); + + // when + IntegrationFlow integrationFlow = new RemoteChunkingWorkerBuilder() + .itemWriter(null) + .build(); + + // then + // expected exception + } + + @Test + public void inputChannelMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("inputChannel must not be null"); + + // when + IntegrationFlow integrationFlow = new RemoteChunkingWorkerBuilder() + .inputChannel(null) + .build(); + + // then + // expected exception + } + + @Test + public void outputChannelMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("outputChannel must not be null"); + + // when + IntegrationFlow integrationFlow = new RemoteChunkingWorkerBuilder() + .outputChannel(null) + .build(); + + // then + // expected exception + } + + @Test + public void testMandatoryItemWriter() { + // given + RemoteChunkingWorkerBuilder builder = new RemoteChunkingWorkerBuilder<>(); + + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("An ItemWriter must be provided"); + + // when + builder.build(); + + // then + // expected exception + } + + @Test + public void testMandatoryInputChannel() { + // given + RemoteChunkingWorkerBuilder builder = new RemoteChunkingWorkerBuilder() + .itemWriter(items -> { }); + + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("An InputChannel must be provided"); + + // when + builder.build(); + + // then + // expected exception + } + + @Test + public void testMandatoryOutputChannel() { + // given + RemoteChunkingWorkerBuilder builder = new RemoteChunkingWorkerBuilder() + .itemWriter(items -> { }) + .inputChannel(new DirectChannel()); + + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("An OutputChannel must be provided"); + + // when + builder.build(); + + // then + // expected exception + } + + @Test + public void testIntegrationFlowCreation() { + // given + DirectChannel inputChannel = new DirectChannel(); + DirectChannel outputChannel = new DirectChannel(); + RemoteChunkingWorkerBuilder builder = new RemoteChunkingWorkerBuilder() + .itemProcessor(this.itemProcessor) + .itemWriter(this.itemWriter) + .inputChannel(inputChannel) + .outputChannel(outputChannel); + + // when + IntegrationFlow integrationFlow = builder.build(); + + // then + Assert.assertNotNull(integrationFlow); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/TestItemReader.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/TestItemReader.java new file mode 100644 index 0000000000..e3c0be7e65 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/TestItemReader.java @@ -0,0 +1,72 @@ +package org.springframework.batch.integration.chunk; + +import java.util.ArrayList; +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ParseException; +import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.lang.Nullable; +import org.springframework.stereotype.Component; + +@Component +public class TestItemReader implements ItemReader { + + private static final Log logger = LogFactory.getLog(TestItemReader.class); + + /** + * Counts the number of chunks processed in the handler. + */ + public volatile int count = 0; + + /** + * Item that causes failure in handler. + */ + public final static String FAIL_ON = "bad"; + + /** + * Item that causes handler to wait to simulate delayed processing. + */ + public static final String WAIT_ON = "wait"; + + private List items = new ArrayList<>(); + + /** + * @param items the items to set + */ + public void setItems(List items) { + this.items = items; + } + + @Nullable + public T read() throws Exception, UnexpectedInputException, ParseException { + + if (count>=items.size()) { + return null; + } + + T item = items.get(count++); + + logger.debug("Reading "+item); + + if (item.equals(WAIT_ON)) { + try { + Thread.sleep(200); + } + catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw new RuntimeException("Unexpected interruption.", e); + } + } + + if (item.equals(FAIL_ON)) { + throw new IllegalStateException("Planned failure on: " + FAIL_ON); + } + + return item; + + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/TestItemWriter.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/TestItemWriter.java new file mode 100644 index 0000000000..e4519ca6e3 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/chunk/TestItemWriter.java @@ -0,0 +1,74 @@ +package org.springframework.batch.integration.chunk; + +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.item.ItemWriter; +import org.springframework.stereotype.Component; + +@Component +public class TestItemWriter implements ItemWriter { + + private static final Log logger = LogFactory.getLog(TestItemWriter.class); + + /** + * Counts the number of chunks processed in the handler. + */ + public volatile static int count = 0; + + /** + * Item that causes failure in handler. + */ + public final static String FAIL_ON = "fail"; + + /** + * Item that causes error in handler. + */ + public final static String UNSUPPORTED_ON = "unsupported"; + + /** + * Item that causes error in handler. + */ + public final static String ERROR_ON = "error"; + + /** + * Item that causes handler to wait to simulate delayed processing. + */ + public static final String WAIT_ON = "wait"; + + public void write(List items) throws Exception { + + for (T item : items) { + + count++; + + logger.debug("Writing: " + item); + + if (item.equals(WAIT_ON)) { + try { + Thread.sleep(200); + } + catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw new RuntimeException("Unexpected interruption.", e); + } + } + + if (item.equals(FAIL_ON)) { + throw new IllegalStateException("Planned failure on: " + FAIL_ON); + } + + if (item.equals(UNSUPPORTED_ON)) { + throw new UnsupportedOperationException("Planned failure on: " + UNSUPPORTED_ON); + } + + if (item.equals(ERROR_ON)) { + throw new Error("Planned failure on: " + ERROR_ON); + } + + } + + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/config/xml/JobLauncherParserTestsConfiguration.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/config/xml/JobLauncherParserTestsConfiguration.java new file mode 100644 index 0000000000..2cde85f30d --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/config/xml/JobLauncherParserTestsConfiguration.java @@ -0,0 +1,28 @@ +/* + * Copyright 2002-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + */ +package org.springframework.batch.integration.config.xml; + +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.context.annotation.Configuration; + +/** + * + * @author Gunnar Hillert + * @since 1.3 + * + */ +@Configuration +@EnableBatchProcessing +public class JobLauncherParserTestsConfiguration { + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTests.java new file mode 100644 index 0000000000..7106c72bd3 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTests.java @@ -0,0 +1,111 @@ +/* + * Copyright 2002-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + */ +package org.springframework.batch.integration.config.xml; + +import org.junit.After; +import org.junit.Test; + +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.integration.launch.JobLaunchingMessageHandler; +import org.springframework.beans.factory.BeanCreationException; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.support.ClassPathXmlApplicationContext; +import org.springframework.integration.channel.AbstractMessageChannel; +import org.springframework.integration.core.MessagingTemplate; +import org.springframework.integration.endpoint.EventDrivenConsumer; +import org.springframework.integration.test.util.TestUtils; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * + * @author Gunnar Hillert + * @since 1.3 + * + */ +public class JobLaunchingGatewayParserTests { + + private ConfigurableApplicationContext context; + + private EventDrivenConsumer consumer; + + @Test + public void testGatewayParser() throws Exception { + setUp("JobLaunchingGatewayParserTests-context.xml", getClass()); + + final AbstractMessageChannel inputChannel = TestUtils.getPropertyValue(this.consumer, "inputChannel", AbstractMessageChannel.class); + assertEquals("requestChannel", inputChannel.getComponentName()); + + final JobLaunchingMessageHandler jobLaunchingMessageHandler = TestUtils.getPropertyValue(this.consumer, "handler.jobLaunchingMessageHandler", JobLaunchingMessageHandler.class); + + assertNotNull(jobLaunchingMessageHandler); + + final MessagingTemplate messagingTemplate = TestUtils.getPropertyValue(this.consumer, "handler.messagingTemplate", MessagingTemplate.class); + final Long sendTimeout = TestUtils.getPropertyValue(messagingTemplate, "sendTimeout", Long.class); + + assertEquals("Wrong sendTimeout", Long.valueOf(123L), sendTimeout); + assertFalse(this.consumer.isRunning()); + } + + @Test + public void testJobLaunchingGatewayIsRunning() throws Exception { + setUp("JobLaunchingGatewayParserTestsRunning-context.xml", getClass()); + assertTrue(this.consumer.isRunning()); + + final MessagingTemplate messagingTemplate = TestUtils.getPropertyValue(this.consumer, "handler.messagingTemplate", MessagingTemplate.class); + final Long sendTimeout = TestUtils.getPropertyValue(messagingTemplate, "sendTimeout", Long.class); + + assertEquals("Wrong sendTimeout", Long.valueOf(-1L), sendTimeout); + } + + @Test + public void testJobLaunchingGatewayNoJobLauncher() throws Exception { + try { + setUp("JobLaunchingGatewayParserTestsNoJobLauncher-context.xml", getClass()); + } + catch(BeanCreationException e) { + assertEquals("No bean named 'jobLauncher' available", e.getCause().getMessage()); + return; + } + fail("Expected a NoSuchBeanDefinitionException to be thrown."); + } + + @Test + public void testJobLaunchingGatewayWithEnableBatchProcessing() throws Exception { + + setUp("JobLaunchingGatewayParserTestsWithEnableBatchProcessing-context.xml", getClass()); + final JobLaunchingMessageHandler jobLaunchingMessageHandler = TestUtils.getPropertyValue(this.consumer, "handler.jobLaunchingMessageHandler", JobLaunchingMessageHandler.class); + assertNotNull(jobLaunchingMessageHandler); + + final JobLauncher jobLauncher = TestUtils.getPropertyValue(jobLaunchingMessageHandler, "jobLauncher", JobLauncher.class); + assertNotNull(jobLauncher); + + } + + @After + public void tearDown(){ + if(context != null){ + context.close(); + } + } + + public void setUp(String name, Class cls){ + context = new ClassPathXmlApplicationContext(name, cls); + consumer = this.context.getBean("batchjobExecutor", EventDrivenConsumer.class); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/config/xml/RemoteChunkingParserTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/config/xml/RemoteChunkingParserTests.java new file mode 100644 index 0000000000..8f59844dc9 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/config/xml/RemoteChunkingParserTests.java @@ -0,0 +1,507 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.config.xml; + +import java.util.List; + +import org.junit.Test; + +import org.springframework.batch.core.step.item.ChunkProcessor; +import org.springframework.batch.core.step.item.SimpleChunkProcessor; +import org.springframework.batch.integration.chunk.ChunkHandler; +import org.springframework.batch.integration.chunk.ChunkMessageChannelItemWriter; +import org.springframework.batch.integration.chunk.ChunkProcessorChunkHandler; +import org.springframework.batch.integration.chunk.RemoteChunkHandlerFactoryBean; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.PassThroughItemProcessor; +import org.springframework.beans.factory.BeanDefinitionStoreException; +import org.springframework.beans.factory.FactoryBean; +import org.springframework.context.ApplicationContext; +import org.springframework.context.support.ClassPathXmlApplicationContext; +import org.springframework.integration.config.ServiceActivatorFactoryBean; +import org.springframework.integration.test.util.TestUtils; +import org.springframework.lang.Nullable; +import org.springframework.messaging.MessageChannel; + +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + *

        + * Test cases for the {@link RemoteChunkingWorkerParser} + * and {@link RemoteChunkingManagerParser}. + *

        + * + * @author Chris Schaefer + * @author Mahmoud Ben Hassine + * @since 3.1 + */ +@SuppressWarnings("unchecked") +public class RemoteChunkingParserTests { + + /* TODO delete the following deprecated tests when related APIs are removed + * /!\ Deliberately not using parametrized tests as it will be easier to delete + * the following tests afterwards + */ + @SuppressWarnings("rawtypes") + @Test + public void testRemoteChunkingSlaveParserWithProcessorDefined() { + ApplicationContext applicationContext = + new ClassPathXmlApplicationContext("/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserTests.xml"); + + ChunkHandler chunkHandler = applicationContext.getBean(ChunkProcessorChunkHandler.class); + ChunkProcessor chunkProcessor = (SimpleChunkProcessor) TestUtils.getPropertyValue(chunkHandler, "chunkProcessor"); + assertNotNull("ChunkProcessor must not be null", chunkProcessor); + + ItemWriter itemWriter = (ItemWriter) TestUtils.getPropertyValue(chunkProcessor, "itemWriter"); + assertNotNull("ChunkProcessor ItemWriter must not be null", itemWriter); + assertTrue("Got wrong instance of ItemWriter", itemWriter instanceof Writer); + + ItemProcessor itemProcessor = (ItemProcessor) TestUtils.getPropertyValue(chunkProcessor, "itemProcessor"); + assertNotNull("ChunkProcessor ItemWriter must not be null", itemProcessor); + assertTrue("Got wrong instance of ItemProcessor", itemProcessor instanceof Processor); + + FactoryBean serviceActivatorFactoryBean = applicationContext.getBean(ServiceActivatorFactoryBean.class); + assertNotNull("ServiceActivatorFactoryBean must not be null", serviceActivatorFactoryBean); + assertNotNull("Output channel name must not be null", TestUtils.getPropertyValue(serviceActivatorFactoryBean, "outputChannelName")); + + MessageChannel inputChannel = applicationContext.getBean("requests", MessageChannel.class); + assertNotNull("Input channel must not be null", inputChannel); + + String targetMethodName = (String) TestUtils.getPropertyValue(serviceActivatorFactoryBean, "targetMethodName"); + assertNotNull("Target method name must not be null", targetMethodName); + assertTrue("Target method name must be handleChunk, got: " + targetMethodName, "handleChunk".equals(targetMethodName)); + + ChunkHandler targetObject = (ChunkHandler) TestUtils.getPropertyValue(serviceActivatorFactoryBean, "targetObject"); + assertNotNull("Target object must not be null", targetObject); + } + + @SuppressWarnings("rawtypes") + @Test + public void testRemoteChunkingSlaveParserWithProcessorNotDefined() { + ApplicationContext applicationContext = + new ClassPathXmlApplicationContext("/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserNoProcessorTests.xml"); + + ChunkHandler chunkHandler = applicationContext.getBean(ChunkProcessorChunkHandler.class); + ChunkProcessor chunkProcessor = (SimpleChunkProcessor) TestUtils.getPropertyValue(chunkHandler, "chunkProcessor"); + assertNotNull("ChunkProcessor must not be null", chunkProcessor); + + ItemProcessor itemProcessor = (ItemProcessor) TestUtils.getPropertyValue(chunkProcessor, "itemProcessor"); + assertNotNull("ChunkProcessor ItemWriter must not be null", itemProcessor); + assertTrue("Got wrong instance of ItemProcessor", itemProcessor instanceof PassThroughItemProcessor); + } + + @SuppressWarnings("rawtypes") + @Test + public void testRemoteChunkingMasterParser() { + ApplicationContext applicationContext = + new ClassPathXmlApplicationContext("/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserTests.xml"); + + ItemWriter itemWriter = applicationContext.getBean("itemWriter", ChunkMessageChannelItemWriter.class); + assertNotNull("Messaging template must not be null", TestUtils.getPropertyValue(itemWriter, "messagingGateway")); + assertNotNull("Reply channel must not be null", TestUtils.getPropertyValue(itemWriter, "replyChannel")); + + FactoryBean remoteChunkingHandlerFactoryBean = applicationContext.getBean(RemoteChunkHandlerFactoryBean.class); + assertNotNull("Chunk writer must not be null", TestUtils.getPropertyValue(remoteChunkingHandlerFactoryBean, "chunkWriter")); + assertNotNull("Step must not be null", TestUtils.getPropertyValue(remoteChunkingHandlerFactoryBean, "step")); + } + + @Test + public void testRemoteChunkingMasterIdAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingIdAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The id attribute must be specified" + " but got: " + iae.getMessage(), + "The id attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingMasterMessageTemplateAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingMessageTemplateAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The message-template attribute must be specified" + " but got: " + iae.getMessage(), + "The message-template attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingMasterStepAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingStepAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The step attribute must be specified" + " but got: " + iae.getMessage(), + "The step attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingMasterReplyChannelAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingReplyChannelAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The reply-channel attribute must be specified" + " but got: " + iae.getMessage(), + "The reply-channel attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingSlaveIdAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingIdAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The id attribute must be specified" + " but got: " + iae.getMessage(), + "The id attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingSlaveInputChannelAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingInputChannelAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The input-channel attribute must be specified" + " but got: " + iae.getMessage(), + "The input-channel attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingSlaveItemWriterAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingItemWriterAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The item-writer attribute must be specified" + " but got: " + iae.getMessage(), + "The item-writer attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingSlaveOutputChannelAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingOutputChannelAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The output-channel attribute must be specified" + " but got: " + iae.getMessage(), + "The output-channel attribute must be specified".equals(iae.getMessage())); + } + } + + /* TODO end of deprecated tests to remove */ + + @SuppressWarnings("rawtypes") + @Test + public void testRemoteChunkingWorkerParserWithProcessorDefined() { + ApplicationContext applicationContext = + new ClassPathXmlApplicationContext("/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserTests.xml"); + + ChunkHandler chunkHandler = applicationContext.getBean(ChunkProcessorChunkHandler.class); + ChunkProcessor chunkProcessor = (SimpleChunkProcessor) TestUtils.getPropertyValue(chunkHandler, "chunkProcessor"); + assertNotNull("ChunkProcessor must not be null", chunkProcessor); + + ItemWriter itemWriter = (ItemWriter) TestUtils.getPropertyValue(chunkProcessor, "itemWriter"); + assertNotNull("ChunkProcessor ItemWriter must not be null", itemWriter); + assertTrue("Got wrong instance of ItemWriter", itemWriter instanceof Writer); + + ItemProcessor itemProcessor = (ItemProcessor) TestUtils.getPropertyValue(chunkProcessor, "itemProcessor"); + assertNotNull("ChunkProcessor ItemWriter must not be null", itemProcessor); + assertTrue("Got wrong instance of ItemProcessor", itemProcessor instanceof Processor); + + FactoryBean serviceActivatorFactoryBean = applicationContext.getBean(ServiceActivatorFactoryBean.class); + assertNotNull("ServiceActivatorFactoryBean must not be null", serviceActivatorFactoryBean); + assertNotNull("Output channel name must not be null", TestUtils.getPropertyValue(serviceActivatorFactoryBean, "outputChannelName")); + + MessageChannel inputChannel = applicationContext.getBean("requests", MessageChannel.class); + assertNotNull("Input channel must not be null", inputChannel); + + String targetMethodName = (String) TestUtils.getPropertyValue(serviceActivatorFactoryBean, "targetMethodName"); + assertNotNull("Target method name must not be null", targetMethodName); + assertTrue("Target method name must be handleChunk, got: " + targetMethodName, "handleChunk".equals(targetMethodName)); + + ChunkHandler targetObject = (ChunkHandler) TestUtils.getPropertyValue(serviceActivatorFactoryBean, "targetObject"); + assertNotNull("Target object must not be null", targetObject); + } + + @SuppressWarnings("rawtypes") + @Test + public void testRemoteChunkingWorkerParserWithProcessorNotDefined() { + ApplicationContext applicationContext = + new ClassPathXmlApplicationContext("/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserNoProcessorTests.xml"); + + ChunkHandler chunkHandler = applicationContext.getBean(ChunkProcessorChunkHandler.class); + ChunkProcessor chunkProcessor = (SimpleChunkProcessor) TestUtils.getPropertyValue(chunkHandler, "chunkProcessor"); + assertNotNull("ChunkProcessor must not be null", chunkProcessor); + + ItemProcessor itemProcessor = (ItemProcessor) TestUtils.getPropertyValue(chunkProcessor, "itemProcessor"); + assertNotNull("ChunkProcessor ItemWriter must not be null", itemProcessor); + assertTrue("Got wrong instance of ItemProcessor", itemProcessor instanceof PassThroughItemProcessor); + } + + @SuppressWarnings("rawtypes") + @Test + public void testRemoteChunkingManagerParser() { + ApplicationContext applicationContext = + new ClassPathXmlApplicationContext("/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserTests.xml"); + + ItemWriter itemWriter = applicationContext.getBean("itemWriter", ChunkMessageChannelItemWriter.class); + assertNotNull("Messaging template must not be null", TestUtils.getPropertyValue(itemWriter, "messagingGateway")); + assertNotNull("Reply channel must not be null", TestUtils.getPropertyValue(itemWriter, "replyChannel")); + + FactoryBean remoteChunkingHandlerFactoryBean = applicationContext.getBean(RemoteChunkHandlerFactoryBean.class); + assertNotNull("Chunk writer must not be null", TestUtils.getPropertyValue(remoteChunkingHandlerFactoryBean, "chunkWriter")); + assertNotNull("Step must not be null", TestUtils.getPropertyValue(remoteChunkingHandlerFactoryBean, "step")); + } + + @Test + public void testRemoteChunkingManagerIdAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingIdAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The id attribute must be specified" + " but got: " + iae.getMessage(), + "The id attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingManagerMessageTemplateAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingMessageTemplateAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The message-template attribute must be specified" + " but got: " + iae.getMessage(), + "The message-template attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingManagerStepAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingStepAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The step attribute must be specified" + " but got: " + iae.getMessage(), + "The step attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingManagerReplyChannelAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingReplyChannelAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The reply-channel attribute must be specified" + " but got: " + iae.getMessage(), + "The reply-channel attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingWorkerIdAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingIdAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The id attribute must be specified" + " but got: " + iae.getMessage(), + "The id attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingWorkerInputChannelAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingInputChannelAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The input-channel attribute must be specified" + " but got: " + iae.getMessage(), + "The input-channel attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingWorkerItemWriterAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingItemWriterAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The item-writer attribute must be specified" + " but got: " + iae.getMessage(), + "The item-writer attribute must be specified".equals(iae.getMessage())); + } + } + + @Test + public void testRemoteChunkingWorkerOutputChannelAttrAssert() throws Exception { + ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext(); + applicationContext.setValidating(false); + applicationContext.setConfigLocation("/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingOutputChannelAttrTests.xml"); + + try { + applicationContext.refresh(); + fail(); + } catch (BeanDefinitionStoreException e) { + assertTrue("Nested exception must be of type IllegalArgumentException", e.getCause() instanceof IllegalArgumentException); + + IllegalArgumentException iae = (IllegalArgumentException) e.getCause(); + + assertTrue("Expected: " + "The output-channel attribute must be specified" + " but got: " + iae.getMessage(), + "The output-channel attribute must be specified".equals(iae.getMessage())); + } + } + + private static class Writer implements ItemWriter { + @Override + public void write(List items) throws Exception { + // + } + } + + private static class Processor implements ItemProcessor { + @Nullable + @Override + public String process(String item) throws Exception { + return item; + } + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/file/FileToMessagesJobIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/file/FileToMessagesJobIntegrationTests.java new file mode 100644 index 0000000000..624e5ab54d --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/file/FileToMessagesJobIntegrationTests.java @@ -0,0 +1,75 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.file; + +import static org.junit.Assert.assertEquals; + +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.messaging.Message; +import org.springframework.messaging.MessageHandler; +import org.springframework.messaging.SubscribableChannel; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +/** + * @author Dave Syer + * + */ +@ContextConfiguration() +@RunWith(SpringJUnit4ClassRunner.class) +public class FileToMessagesJobIntegrationTests implements MessageHandler { + + @Autowired + @Qualifier("requests") + private SubscribableChannel requests; + + @Autowired + private Job job; + + @Autowired + private JobLauncher jobLauncher; + + int count = 0; + + public void handleMessage(Message message) { + count++; + } + + @Before + public void setUp() { + requests.subscribe(this); + } + + @Test + public void testFileSent() throws Exception { + + JobExecution execution = jobLauncher.run(job, new JobParametersBuilder().addLong("time.stamp", + System.currentTimeMillis()).toJobParameters()); + assertEquals(BatchStatus.COMPLETED, execution.getStatus()); + // 2 chunks sent to channel (5 items and commit-interval=3) + assertEquals(2, count); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/file/ResourceSplitterIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/file/ResourceSplitterIntegrationTests.java new file mode 100644 index 0000000000..06deb93fc6 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/file/ResourceSplitterIntegrationTests.java @@ -0,0 +1,80 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.file; + +import static org.junit.Assert.assertNotNull; + +import java.util.Arrays; +import java.util.List; + +import org.junit.Ignore; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.core.io.Resource; +import org.springframework.integration.annotation.MessageEndpoint; +import org.springframework.integration.annotation.Splitter; +import org.springframework.messaging.Message; +import org.springframework.messaging.MessageChannel; +import org.springframework.messaging.PollableChannel; +import org.springframework.messaging.support.GenericMessage; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +/** + * @author Dave Syer + * + */ +@ContextConfiguration() +@RunWith(SpringJUnit4ClassRunner.class) +@MessageEndpoint +public class ResourceSplitterIntegrationTests { + + @Autowired + @Qualifier("resources") + private MessageChannel resources; + + @Autowired + @Qualifier("requests") + private PollableChannel requests; + + /* + * This is so cool (but see INT-190)... + * + * The incoming message is a Resource pattern, and it is converted to the + * correct payload type with Spring's default strategy + */ + @Splitter(inputChannel = "resources", outputChannel = "requests") + public Resource[] handle(Resource[] message) { + List list = Arrays.asList(message); + System.err.println(list); + return message; + } + + @SuppressWarnings("unchecked") + @Test + @Ignore //FIXME + // This broke with Integration 2.0 in a milestone, so watch out when upgrading... + public void testVanillaConversion() throws Exception { + resources.send(new GenericMessage<>("classpath:*-context.xml")); + Message message = (Message) requests.receive(200L); + assertNotNull(message); + message = (Message) requests.receive(100L); + assertNotNull(message); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/item/MessagingGatewayIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/item/MessagingGatewayIntegrationTests.java new file mode 100644 index 0000000000..5b1f2d4d43 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/item/MessagingGatewayIntegrationTests.java @@ -0,0 +1,143 @@ +/* + * Copyright 2006-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.item; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; + +import java.util.Arrays; +import java.util.List; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemWriter; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.integration.annotation.MessageEndpoint; +import org.springframework.integration.annotation.ServiceActivator; +import org.springframework.integration.annotation.Splitter; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +/** + * Test case showing the use of a MessagingGateway to provide an ItemWriter or + * ItemProcessor to Spring Batch that is hooked directly into a Spring + * Integration MessageChannel. + * + * @author Dave Syer + * @author Mahmoud Ben Hassine + * + */ +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class MessagingGatewayIntegrationTests { + + @Autowired + private ItemProcessor processor; + + @Autowired + private ItemWriter writer; + + /** + * Just for the sake of being able to make assertions. + */ + @Autowired + private EndService service; + + /** + * Just for the sake of being able to make assertions. + */ + @Autowired + private SplitService splitter; + + @Test + public void testProcessor() throws Exception { + String result = processor.process("foo"); + assertEquals("foo: 0: 1", result); + assertNull(processor.process("filter")); + } + + @Test + public void testWriter() throws Exception { + writer.write(Arrays.asList("foo", "bar", "spam")); + assertEquals(3, splitter.count); + assertEquals(3, service.count); + } + + /** + * This service is wrapped into an ItemProcessor and used to transform + * items. This is where the main business processing could take place in a + * real application. To suppress output the service activator can just + * return null (same as an ItemProcessor) but remember to set the reply + * timeout in the gateway. + * + * @author Dave Syer + * + */ + @MessageEndpoint + public static class Activator { + private int count; + + @ServiceActivator + public String transform(String input) { + if (input.equals("filter")) { + return null; + } + return input + ": " + (count++); + } + } + + /** + * The Splitter is wrapped into an ItemWriter and used to relay items to its + * output channel. This one is completely trivial, it just passes the items + * on as they are. More complex splitters might filter or enhance the items + * before passing them on. + * + * @author Dave Syer + * + */ + @MessageEndpoint + public static class SplitService { + // Just for assertions in the test case + private int count; + + @Splitter + public List split(List input) { + count += input.size(); + return input; + } + } + + /** + * This is just used to trap the messages sent by the ItemWriter and make an + * assertion about them in the the test case. In a real application this + * would be the output stage and/or business processing. + * + * @author Dave Syer + * + */ + @MessageEndpoint + public static class EndService { + private int count; + + @ServiceActivator + public void service(String input) { + count++; + return; + } + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobLaunchingGatewayIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobLaunchingGatewayIntegrationTests.java new file mode 100644 index 0000000000..8c50abc9dc --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobLaunchingGatewayIntegrationTests.java @@ -0,0 +1,169 @@ +/* + * Copyright 2002-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.launch; + +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.integration.JobSupport; +import org.springframework.batch.integration.step.TestTasklet; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.integration.support.MessageBuilder; +import org.springframework.messaging.Message; +import org.springframework.messaging.MessageChannel; +import org.springframework.messaging.MessageHandlingException; +import org.springframework.messaging.MessageHeaders; +import org.springframework.messaging.MessagingException; +import org.springframework.messaging.PollableChannel; +import org.springframework.messaging.support.GenericMessage; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +import java.util.HashMap; +import java.util.Map; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * + * @author Gunnar Hillert + * @since 1.3 + * + */ +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class JobLaunchingGatewayIntegrationTests { + + @Autowired + @Qualifier("requests") + private MessageChannel requestChannel; + + @Autowired + @Qualifier("response") + private PollableChannel responseChannel; + + @Autowired + private TestTasklet tasklet; + + @Autowired + private Job testJob; + + private final JobSupport job = new JobSupport("testJob"); + + @Before + public void setUp() { + Object message = ""; + while (message!=null) { + message = responseChannel.receive(10L); + } + } + + @Test + @DirtiesContext + @SuppressWarnings("unchecked") + public void testNoReply() { + GenericMessage trigger = new GenericMessage<>(new JobLaunchRequest(job, + new JobParameters())); + try { + requestChannel.send(trigger); + fail(); + } + catch (MessagingException e) { + String message = e.getMessage(); + assertTrue("Wrong message: " + message, message.contains("replyChannel")); + } + Message executionMessage = (Message) responseChannel.receive(1000); + + assertNull("JobExecution message received when no return address set", executionMessage); + } + + @SuppressWarnings("unchecked") + @Test + @DirtiesContext + public void testReply() { + JobParametersBuilder builder = new JobParametersBuilder(); + builder.addString("dontclash", "12"); + Map map = new HashMap<>(); + map.put(MessageHeaders.REPLY_CHANNEL, "response"); + MessageHeaders headers = new MessageHeaders(map); + GenericMessage trigger = new GenericMessage<>(new JobLaunchRequest(job, + builder.toJobParameters()), headers); + requestChannel.send(trigger); + Message executionMessage = (Message) responseChannel.receive(1000); + + assertNotNull("No response received", executionMessage); + JobExecution execution = executionMessage.getPayload(); + assertNotNull("JobExecution not returned", execution); + } + + @Test + @DirtiesContext + @SuppressWarnings("unchecked") + public void testWrongPayload() { + + final Message stringMessage = MessageBuilder.withPayload("just text").build(); + + try { + requestChannel.send(stringMessage); + fail(); + } + catch (MessageHandlingException e) { + String message = e.getCause().getMessage(); + assertTrue("Wrong message: " + message, message.contains("The payload must be of type JobLaunchRequest.")); + } + Message executionMessage = (Message) responseChannel.receive(1000); + + assertNull("JobExecution message received when no return address set", executionMessage); + } + + @Test + @DirtiesContext + @SuppressWarnings("unchecked") + public void testExceptionRaised() throws InterruptedException { + + this.tasklet.setFail(true); + + JobParametersBuilder builder = new JobParametersBuilder(); + builder.addString("dontclash", "12"); + Map map = new HashMap<>(); + map.put(MessageHeaders.REPLY_CHANNEL, "response"); + MessageHeaders headers = new MessageHeaders(map); + GenericMessage trigger = new GenericMessage<>(new JobLaunchRequest(testJob, + builder.toJobParameters()), headers); + requestChannel.send(trigger); + + Message executionMessage = (Message) responseChannel.receive(1000); + + assertNotNull("No response received", executionMessage); + + JobExecution execution = executionMessage.getPayload(); + assertNotNull("JobExecution not returned", execution); + + assertEquals(ExitStatus.FAILED.getExitCode(), execution.getExitStatus().getExitCode()); + + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobLaunchingGatewayTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobLaunchingGatewayTests.java new file mode 100644 index 0000000000..d33d8c61a5 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobLaunchingGatewayTests.java @@ -0,0 +1,65 @@ +/* + * Copyright 2002-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.launch; + +import org.junit.Test; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersInvalidException; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.integration.JobSupport; +import org.springframework.integration.support.MessageBuilder; +import org.springframework.messaging.Message; +import org.springframework.messaging.MessageHandlingException; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +/** + * + * @author Gunnar Hillert + * @since 1.3 + * + */ +public class JobLaunchingGatewayTests { + + @Test + public void testExceptionRaised() throws Exception { + + final Message message = MessageBuilder.withPayload(new JobLaunchRequest(new JobSupport("testJob"), + new JobParameters())).build(); + + final JobLauncher jobLauncher = mock(JobLauncher.class); + when(jobLauncher.run(any(Job.class), any(JobParameters.class))) + .thenThrow(new JobParametersInvalidException("This is a JobExecutionException.")); + + JobLaunchingGateway jobLaunchingGateway = new JobLaunchingGateway(jobLauncher); + + try { + jobLaunchingGateway.handleMessage(message); + } + catch (MessageHandlingException e) { + assertEquals("This is a JobExecutionException.", e.getCause().getMessage()); + return; + } + + fail("Expecting a MessageHandlingException to be thrown."); + + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobLaunchingMessageHandlerIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobLaunchingMessageHandlerIntegrationTests.java new file mode 100644 index 0000000000..fb8ea90c69 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobLaunchingMessageHandlerIntegrationTests.java @@ -0,0 +1,88 @@ +package org.springframework.batch.integration.launch; + +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; + +import java.util.HashMap; +import java.util.Map; + +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.integration.JobSupport; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.messaging.Message; +import org.springframework.messaging.MessageChannel; +import org.springframework.messaging.MessageHeaders; +import org.springframework.messaging.MessagingException; +import org.springframework.messaging.PollableChannel; +import org.springframework.messaging.support.GenericMessage; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class JobLaunchingMessageHandlerIntegrationTests { + + @Autowired + @Qualifier("requests") + private MessageChannel requestChannel; + + @Autowired + @Qualifier("response") + private PollableChannel responseChannel; + + private final JobSupport job = new JobSupport("testJob"); + + @Before + public void setUp() { + Object message = ""; + while (message!=null) { + message = responseChannel.receive(10L); + } + } + + @Test + @DirtiesContext + @SuppressWarnings("unchecked") + public void testNoReply() { + GenericMessage trigger = new GenericMessage<>(new JobLaunchRequest(job, + new JobParameters())); + try { + requestChannel.send(trigger); + } + catch (MessagingException e) { + String message = e.getMessage(); + assertTrue("Wrong message: " + message, message.contains("replyChannel")); + } + Message executionMessage = (Message) responseChannel.receive(1000); + + assertNull("JobExecution message received when no return address set", executionMessage); + } + + @SuppressWarnings("unchecked") + @Test + @DirtiesContext + public void testReply() { + JobParametersBuilder builder = new JobParametersBuilder(); + builder.addString("dontclash", "12"); + Map map = new HashMap<>(); + map.put(MessageHeaders.REPLY_CHANNEL, "response"); + MessageHeaders headers = new MessageHeaders(map); + GenericMessage trigger = new GenericMessage<>(new JobLaunchRequest(job, + builder.toJobParameters()), headers); + requestChannel.send(trigger); + Message executionMessage = (Message) responseChannel.receive(1000); + + assertNotNull("No response received", executionMessage); + JobExecution execution = executionMessage.getPayload(); + assertNotNull("JobExecution not returned", execution); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobLaunchingMessageHandlerTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobLaunchingMessageHandlerTests.java new file mode 100644 index 0000000000..6500d017e0 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobLaunchingMessageHandlerTests.java @@ -0,0 +1,58 @@ +package org.springframework.batch.integration.launch; + +import static org.junit.Assert.assertEquals; + +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.atomic.AtomicLong; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.integration.JobSupport; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests; + +@ContextConfiguration(locations = { "/job-execution-context.xml" }) +public class JobLaunchingMessageHandlerTests extends AbstractJUnit4SpringContextTests { + + JobLaunchRequestHandler messageHandler; + + StubJobLauncher jobLauncher; + + @Before + public void setUp() { + jobLauncher = new StubJobLauncher(); + messageHandler = new JobLaunchingMessageHandler(jobLauncher); + } + + @Test + public void testSimpleDelivery() throws Exception{ + messageHandler.launch(new JobLaunchRequest(new JobSupport("testjob"), null)); + + assertEquals("Wrong job count", 1, jobLauncher.jobs.size()); + assertEquals("Wrong job name", jobLauncher.jobs.get(0).getName(), "testjob"); + + } + + private static class StubJobLauncher implements JobLauncher { + + List jobs = new ArrayList<>(); + + List parameters = new ArrayList<>(); + + AtomicLong jobId = new AtomicLong(); + + public JobExecution run(Job job, JobParameters jobParameters){ + jobs.add(job); + parameters.add(jobParameters); + return new JobExecution(new JobInstance(jobId.getAndIncrement(), job.getName()), jobParameters); + } + + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobRequestConverter.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobRequestConverter.java new file mode 100644 index 0000000000..42b0253f9a --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/launch/JobRequestConverter.java @@ -0,0 +1,36 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.launch; + +import java.util.Properties; + +import org.springframework.batch.core.converter.DefaultJobParametersConverter; +import org.springframework.batch.integration.JobSupport; +import org.springframework.integration.annotation.ServiceActivator; + +/** + * @author Dave Syer + * + */ +public class JobRequestConverter { + + @ServiceActivator + public JobLaunchRequest convert(String jobName) { + Properties properties = new Properties(); + return new JobLaunchRequest(new JobSupport(jobName), new DefaultJobParametersConverter().getJobParameters(properties)); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/BeanFactoryStepLocatorTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/BeanFactoryStepLocatorTests.java new file mode 100644 index 0000000000..049a2c8973 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/BeanFactoryStepLocatorTests.java @@ -0,0 +1,57 @@ +package org.springframework.batch.integration.partition; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import org.junit.Test; +import org.springframework.batch.core.JobInterruptedException; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.beans.factory.support.DefaultListableBeanFactory; + + +public class BeanFactoryStepLocatorTests { + + private BeanFactoryStepLocator stepLocator = new BeanFactoryStepLocator(); + private DefaultListableBeanFactory beanFactory = new DefaultListableBeanFactory(); + + @Test + public void testGetStep() throws Exception { + beanFactory.registerSingleton("foo", new StubStep("foo")); + stepLocator.setBeanFactory(beanFactory); + assertNotNull(stepLocator.getStep("foo")); + } + + @Test + public void testGetStepNames() throws Exception { + beanFactory.registerSingleton("foo", new StubStep("foo")); + beanFactory.registerSingleton("bar", new StubStep("bar")); + stepLocator.setBeanFactory(beanFactory); + assertEquals(2, stepLocator.getStepNames().size()); + } + + private static final class StubStep implements Step { + + private String name; + + public StubStep(String name) { + this.name = name; + } + + public void execute(StepExecution stepExecution) throws JobInterruptedException { + } + + public String getName() { + return name; + } + + public int getStartLimit() { + return 0; + } + + public boolean isAllowStartIfComplete() { + return false; + } + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/ExampleItemReader.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/ExampleItemReader.java new file mode 100644 index 0000000000..3509c78c82 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/ExampleItemReader.java @@ -0,0 +1,59 @@ +package org.springframework.batch.integration.partition; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemStream; +import org.springframework.batch.item.ItemStreamException; +import org.springframework.lang.Nullable; + +/** + * {@link ItemReader} with hard-coded input data. + */ +public class ExampleItemReader implements ItemReader, ItemStream { + + private Log logger = LogFactory.getLog(getClass()); + + private String[] input = { "Hello", "world!", "Go", "on", "punk", "make", "my", "day!" }; + + private int index = 0; + + public static volatile boolean fail = false; + + /** + * Reads next record from input + */ + @Nullable + public String read() throws Exception { + if (index >= input.length) { + return null; + } + logger.info(String.format("Processing input index=%s, item=%s, in (%s)", index, input[index], this)); + if (fail && index == 4) { + synchronized (ExampleItemReader.class) { + if (fail) { + // Only fail once per flag setting... + fail = false; + logger.info(String.format("Throwing exception index=%s, item=%s, in (%s)", index, input[index], + this)); + index++; + throw new RuntimeException("Planned failure"); + } + } + } + return input[index++]; + } + + public void close() throws ItemStreamException { + } + + public void open(ExecutionContext executionContext) throws ItemStreamException { + index = (int) executionContext.getLong("POSITION", 0); + } + + public void update(ExecutionContext executionContext) throws ItemStreamException { + executionContext.putLong("POSITION", index); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/ExampleItemReaderTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/ExampleItemReaderTests.java new file mode 100644 index 0000000000..3f2a7f96e2 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/ExampleItemReaderTests.java @@ -0,0 +1,70 @@ +package org.springframework.batch.integration.partition; + +import static org.junit.Assert.*; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.item.ExecutionContext; + +public class ExampleItemReaderTests { + + private ExampleItemReader reader = new ExampleItemReader(); + + @Before + @After + public void ensureFailFlagUnset() { + ExampleItemReader.fail = false; + } + + @Test + public void testRead() throws Exception { + int count = 0; + while (reader.read()!=null) { + count++; + } + assertEquals(8, count); + } + + @Test + public void testOpen() throws Exception { + ExecutionContext context = new ExecutionContext(); + for (int i=0; i<4; i++) { + reader.read(); + } + reader.update(context); + reader.open(context); + int count = 0; + while (reader.read()!=null) { + count++; + } + assertEquals(4, count); + } + + @Test + public void testFailAndRestart() throws Exception { + ExecutionContext context = new ExecutionContext(); + ExampleItemReader.fail = true; + for (int i=0; i<4; i++) { + reader.read(); + reader.update(context); + } + try { + reader.read(); + reader.update(context); + fail("Expected Exception"); + } + catch (Exception e) { + // expected + assertEquals("Planned failure", e.getMessage()); + } + assertFalse(ExampleItemReader.fail); + reader.open(context); + int count = 0; + while (reader.read()!=null) { + count++; + } + assertEquals(4, count); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/ExampleItemWriter.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/ExampleItemWriter.java new file mode 100644 index 0000000000..fd3965190c --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/ExampleItemWriter.java @@ -0,0 +1,23 @@ +package org.springframework.batch.integration.partition; + +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.item.ItemWriter; + +/** + * Dummy {@link ItemWriter} which only logs data it receives. + */ +public class ExampleItemWriter implements ItemWriter { + + private static final Log log = LogFactory.getLog(ExampleItemWriter.class); + + /** + * @see ItemWriter#write(List) + */ + public void write(List data) throws Exception { + log.info(data); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/JmsIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/JmsIntegrationTests.java new file mode 100755 index 0000000000..10c90d587b --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/JmsIntegrationTests.java @@ -0,0 +1,81 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + */ +package org.springframework.batch.integration.partition; + +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +/** + * @author Dave Syer + * + */ +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +@DirtiesContext +public class JmsIntegrationTests { + + private Log logger = LogFactory.getLog(getClass()); + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private Job job; + + @Autowired + private JobExplorer jobExplorer; + + @Test + public void testSimpleProperties() throws Exception { + assertNotNull(jobLauncher); + } + + @Test + public void testLaunchJob() throws Exception { + int before = jobExplorer.getJobInstances(job.getName(), 0, 100).size(); + assertNotNull(jobLauncher.run(job, new JobParameters())); + List jobInstances = jobExplorer.getJobInstances(job.getName(), 0, 100); + int after = jobInstances.size(); + assertEquals(1, after - before); + JobExecution jobExecution = jobExplorer.getJobExecutions(jobInstances.get(jobInstances.size() - 1)).get(0); + assertEquals(jobExecution.getExitStatus().getExitDescription(), BatchStatus.COMPLETED, jobExecution.getStatus()); + assertEquals(3, jobExecution.getStepExecutions().size()); + for (StepExecution stepExecution : jobExecution.getStepExecutions()) { + // BATCH-1703: we are using a map dao so the step executions in the job execution are old and we need to + // pull them back out of the repository... + stepExecution = jobExplorer.getStepExecution(jobExecution.getId(), stepExecution.getId()); + logger.debug("" + stepExecution); + assertEquals(BatchStatus.COMPLETED, stepExecution.getStatus()); + } + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/MessageChannelPartitionHandlerTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/MessageChannelPartitionHandlerTests.java new file mode 100644 index 0000000000..fd9170412f --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/MessageChannelPartitionHandlerTests.java @@ -0,0 +1,213 @@ +package org.springframework.batch.integration.partition; + +import java.util.Collection; +import java.util.Collections; +import java.util.HashSet; +import java.util.concurrent.TimeoutException; + +import org.junit.Test; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.partition.StepExecutionSplitter; +import org.springframework.integration.MessageTimeoutException; +import org.springframework.integration.core.MessagingTemplate; +import org.springframework.messaging.Message; +import org.springframework.messaging.PollableChannel; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +/** + * + * @author Will Schipp + * @author Michael Minella + * + */ +@SuppressWarnings("raw") +public class MessageChannelPartitionHandlerTests { + + private MessageChannelPartitionHandler messageChannelPartitionHandler; + + @Test + public void testNoPartitions() throws Exception { + //execute with no default set + messageChannelPartitionHandler = new MessageChannelPartitionHandler(); + //mock + StepExecution masterStepExecution = mock(StepExecution.class); + StepExecutionSplitter stepExecutionSplitter = mock(StepExecutionSplitter.class); + + //execute + Collection executions = messageChannelPartitionHandler.handle(stepExecutionSplitter, masterStepExecution); + //verify + assertTrue(executions.isEmpty()); + } + + @SuppressWarnings({"unchecked", "rawtypes"}) + @Test + public void testHandleNoReply() throws Exception { + //execute with no default set + messageChannelPartitionHandler = new MessageChannelPartitionHandler(); + //mock + StepExecution masterStepExecution = mock(StepExecution.class); + StepExecutionSplitter stepExecutionSplitter = mock(StepExecutionSplitter.class); + MessagingTemplate operations = mock(MessagingTemplate.class); + Message message = mock(Message.class); + //when + HashSet stepExecutions = new HashSet<>(); + stepExecutions.add(new StepExecution("step1", new JobExecution(5L))); + when(stepExecutionSplitter.split(any(StepExecution.class), eq(1))).thenReturn(stepExecutions); + when(message.getPayload()).thenReturn(Collections.emptyList()); + when(operations.receive((PollableChannel) any())).thenReturn(message); + //set + messageChannelPartitionHandler.setMessagingOperations(operations); + + //execute + Collection executions = messageChannelPartitionHandler.handle(stepExecutionSplitter, masterStepExecution); + //verify + assertNotNull(executions); + assertTrue(executions.isEmpty()); + } + + @SuppressWarnings({"unchecked", "rawtypes"}) + @Test + public void testHandleWithReplyChannel() throws Exception { + //execute with no default set + messageChannelPartitionHandler = new MessageChannelPartitionHandler(); + //mock + StepExecution masterStepExecution = mock(StepExecution.class); + StepExecutionSplitter stepExecutionSplitter = mock(StepExecutionSplitter.class); + MessagingTemplate operations = mock(MessagingTemplate.class); + Message message = mock(Message.class); + PollableChannel replyChannel = mock(PollableChannel.class); + //when + HashSet stepExecutions = new HashSet<>(); + stepExecutions.add(new StepExecution("step1", new JobExecution(5L))); + when(stepExecutionSplitter.split(any(StepExecution.class), eq(1))).thenReturn(stepExecutions); + when(message.getPayload()).thenReturn(Collections.emptyList()); + when(operations.receive(replyChannel)).thenReturn(message); + //set + messageChannelPartitionHandler.setMessagingOperations(operations); + messageChannelPartitionHandler.setReplyChannel(replyChannel); + + //execute + Collection executions = messageChannelPartitionHandler.handle(stepExecutionSplitter, masterStepExecution); + //verify + assertNotNull(executions); + assertTrue(executions.isEmpty()); + + } + + @SuppressWarnings("rawtypes") + @Test(expected = MessageTimeoutException.class) + public void messageReceiveTimeout() throws Exception { + //execute with no default set + messageChannelPartitionHandler = new MessageChannelPartitionHandler(); + //mock + StepExecution masterStepExecution = mock(StepExecution.class); + StepExecutionSplitter stepExecutionSplitter = mock(StepExecutionSplitter.class); + MessagingTemplate operations = mock(MessagingTemplate.class); + Message message = mock(Message.class); + //when + HashSet stepExecutions = new HashSet<>(); + stepExecutions.add(new StepExecution("step1", new JobExecution(5L))); + when(stepExecutionSplitter.split(any(StepExecution.class), eq(1))).thenReturn(stepExecutions); + when(message.getPayload()).thenReturn(Collections.emptyList()); + //set + messageChannelPartitionHandler.setMessagingOperations(operations); + + //execute + messageChannelPartitionHandler.handle(stepExecutionSplitter, masterStepExecution); + } + + @Test + public void testHandleWithJobRepositoryPolling() throws Exception { + //execute with no default set + messageChannelPartitionHandler = new MessageChannelPartitionHandler(); + //mock + JobExecution jobExecution = new JobExecution(5L, new JobParameters()); + StepExecution masterStepExecution = new StepExecution("step1", jobExecution, 1L); + StepExecutionSplitter stepExecutionSplitter = mock(StepExecutionSplitter.class); + MessagingTemplate operations = mock(MessagingTemplate.class); + JobExplorer jobExplorer = mock(JobExplorer.class); + //when + HashSet stepExecutions = new HashSet<>(); + StepExecution partition1 = new StepExecution("step1:partition1", jobExecution, 2L); + StepExecution partition2 = new StepExecution("step1:partition2", jobExecution, 3L); + StepExecution partition3 = new StepExecution("step1:partition3", jobExecution, 4L); + StepExecution partition4 = new StepExecution("step1:partition3", jobExecution, 4L); + partition1.setStatus(BatchStatus.COMPLETED); + partition2.setStatus(BatchStatus.COMPLETED); + partition3.setStatus(BatchStatus.STARTED); + partition4.setStatus(BatchStatus.COMPLETED); + stepExecutions.add(partition1); + stepExecutions.add(partition2); + stepExecutions.add(partition3); + when(stepExecutionSplitter.split(any(StepExecution.class), eq(1))).thenReturn(stepExecutions); + when(jobExplorer.getStepExecution(eq(5L), any(Long.class))).thenReturn(partition2, partition1, partition3, partition3, partition3, partition3, partition4); + + //set + messageChannelPartitionHandler.setMessagingOperations(operations); + messageChannelPartitionHandler.setJobExplorer(jobExplorer); + messageChannelPartitionHandler.setStepName("step1"); + messageChannelPartitionHandler.setPollInterval(500L); + messageChannelPartitionHandler.afterPropertiesSet(); + + //execute + Collection executions = messageChannelPartitionHandler.handle(stepExecutionSplitter, masterStepExecution); + //verify + assertNotNull(executions); + assertEquals(3, executions.size()); + assertTrue(executions.contains(partition1)); + assertTrue(executions.contains(partition2)); + assertTrue(executions.contains(partition4)); + + //verify + verify(operations, times(3)).send(any(Message.class)); + } + + @Test(expected = TimeoutException.class) + public void testHandleWithJobRepositoryPollingTimeout() throws Exception { + //execute with no default set + messageChannelPartitionHandler = new MessageChannelPartitionHandler(); + //mock + JobExecution jobExecution = new JobExecution(5L, new JobParameters()); + StepExecution masterStepExecution = new StepExecution("step1", jobExecution, 1L); + StepExecutionSplitter stepExecutionSplitter = mock(StepExecutionSplitter.class); + MessagingTemplate operations = mock(MessagingTemplate.class); + JobExplorer jobExplorer = mock(JobExplorer.class); + //when + HashSet stepExecutions = new HashSet<>(); + StepExecution partition1 = new StepExecution("step1:partition1", jobExecution, 2L); + StepExecution partition2 = new StepExecution("step1:partition2", jobExecution, 3L); + StepExecution partition3 = new StepExecution("step1:partition3", jobExecution, 4L); + partition1.setStatus(BatchStatus.COMPLETED); + partition2.setStatus(BatchStatus.COMPLETED); + partition3.setStatus(BatchStatus.STARTED); + stepExecutions.add(partition1); + stepExecutions.add(partition2); + stepExecutions.add(partition3); + when(stepExecutionSplitter.split(any(StepExecution.class), eq(1))).thenReturn(stepExecutions); + when(jobExplorer.getStepExecution(eq(5L), any(Long.class))).thenReturn(partition2, partition1, partition3); + + //set + messageChannelPartitionHandler.setMessagingOperations(operations); + messageChannelPartitionHandler.setJobExplorer(jobExplorer); + messageChannelPartitionHandler.setStepName("step1"); + messageChannelPartitionHandler.setTimeout(1000L); + messageChannelPartitionHandler.afterPropertiesSet(); + + //execute + messageChannelPartitionHandler.handle(stepExecutionSplitter, masterStepExecution); + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/PollingIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/PollingIntegrationTests.java new file mode 100644 index 0000000000..4b75489f28 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/PollingIntegrationTests.java @@ -0,0 +1,71 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.partition; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import java.util.List; + +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +/** + * @author Dave Syer + * + */ +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class PollingIntegrationTests { + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private Job job; + + @Autowired + private JobExplorer jobExplorer; + + @Test + public void testSimpleProperties() throws Exception { + assertNotNull(jobLauncher); + } + + @Test + public void testLaunchJob() throws Exception { + int before = jobExplorer.getJobInstances(job.getName(), 0, 100).size(); + assertNotNull(jobLauncher.run(job, new JobParameters())); + List jobInstances = jobExplorer.getJobInstances(job.getName(), 0, 100); + int after = jobInstances.size(); + assertEquals(1, after-before); + JobExecution jobExecution = jobExplorer.getJobExecutions(jobInstances.get(jobInstances.size()-1)).get(0); + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + assertEquals(3, jobExecution.getStepExecutions().size()); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/RemotePartitioningMasterStepBuilderTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/RemotePartitioningMasterStepBuilderTests.java new file mode 100644 index 0000000000..b2b5d0d682 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/RemotePartitioningMasterStepBuilderTests.java @@ -0,0 +1,248 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.partition; + +import org.junit.Assert; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; +import org.junit.runner.RunWith; +import org.mockito.Mockito; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.partition.PartitionHandler; +import org.springframework.batch.core.partition.support.Partitioner; +import org.springframework.batch.core.partition.support.StepExecutionAggregator; +import org.springframework.batch.core.repository.JobRepository; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.annotation.Configuration; +import org.springframework.integration.channel.DirectChannel; +import org.springframework.integration.channel.QueueChannel; +import org.springframework.integration.core.MessagingTemplate; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringRunner; + +import static org.springframework.test.util.ReflectionTestUtils.getField; + +/** + * @author Mahmoud Ben Hassine + */ +@RunWith(SpringRunner.class) +@ContextConfiguration(classes = {RemotePartitioningMasterStepBuilderTests.BatchConfiguration.class}) +public class RemotePartitioningMasterStepBuilderTests { + + @Rule + public ExpectedException expectedException = ExpectedException.none(); + + @Autowired + private JobRepository jobRepository; + + @Test + public void inputChannelMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("inputChannel must not be null"); + + // when + new RemotePartitioningMasterStepBuilder("step").inputChannel(null); + + // then + // expected exception + } + + @Test + public void outputChannelMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("outputChannel must not be null"); + + // when + new RemotePartitioningMasterStepBuilder("step").outputChannel(null); + + // then + // expected exception + } + + @Test + public void messagingTemplateMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("messagingTemplate must not be null"); + + // when + new RemotePartitioningMasterStepBuilder("step").messagingTemplate(null); + + // then + // expected exception + } + + @Test + public void jobExplorerMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("jobExplorer must not be null"); + + // when + new RemotePartitioningMasterStepBuilder("step").jobExplorer(null); + + // then + // expected exception + } + + @Test + public void pollIntervalMustBeGreaterThanZero() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("The poll interval must be greater than zero"); + + // when + new RemotePartitioningMasterStepBuilder("step").pollInterval(-1); + + // then + // expected exception + } + + @Test + public void eitherOutputChannelOrMessagingTemplateMustBeProvided() { + // given + RemotePartitioningMasterStepBuilder builder = new RemotePartitioningMasterStepBuilder("step") + .outputChannel(new DirectChannel()) + .messagingTemplate(new MessagingTemplate()); + + this.expectedException.expect(IllegalStateException.class); + this.expectedException.expectMessage("You must specify either an outputChannel or a messagingTemplate but not both."); + + // when + Step step = builder.build(); + + // then + // expected exception + } + + @Test + public void testUnsupportedOperationExceptionWhenSpecifyingPartitionHandler() { + // given + PartitionHandler partitionHandler = Mockito.mock(PartitionHandler.class); + this.expectedException.expect(UnsupportedOperationException.class); + this.expectedException.expectMessage("When configuring a master step " + + "for remote partitioning using the RemotePartitioningMasterStepBuilder, " + + "the partition handler will be automatically set to an instance " + + "of MessageChannelPartitionHandler. The partition handler must " + + "not be provided in this case."); + + // when + new RemotePartitioningMasterStepBuilder("step").partitionHandler(partitionHandler); + + // then + // expected exception + } + + @Test + public void testMasterStepCreationWhenPollingRepository() { + // given + int gridSize = 5; + int startLimit = 3; + long timeout = 1000L; + long pollInterval = 5000L; + DirectChannel outputChannel = new DirectChannel(); + Partitioner partitioner = Mockito.mock(Partitioner.class); + StepExecutionAggregator stepExecutionAggregator = (result, executions) -> { }; + + // when + Step step = new RemotePartitioningMasterStepBuilder("masterStep") + .repository(jobRepository) + .outputChannel(outputChannel) + .partitioner("workerStep", partitioner) + .gridSize(gridSize) + .pollInterval(pollInterval) + .timeout(timeout) + .startLimit(startLimit) + .aggregator(stepExecutionAggregator) + .allowStartIfComplete(true) + .build(); + + // then + Assert.assertNotNull(step); + Assert.assertEquals(getField(step, "startLimit"), startLimit); + Assert.assertEquals(getField(step, "jobRepository"), this.jobRepository); + Assert.assertEquals(getField(step, "stepExecutionAggregator"), stepExecutionAggregator); + Assert.assertTrue((Boolean) getField(step, "allowStartIfComplete")); + + Object partitionHandler = getField(step, "partitionHandler"); + Assert.assertNotNull(partitionHandler); + Assert.assertTrue(partitionHandler instanceof MessageChannelPartitionHandler); + MessageChannelPartitionHandler messageChannelPartitionHandler = (MessageChannelPartitionHandler) partitionHandler; + Assert.assertEquals(getField(messageChannelPartitionHandler, "gridSize"), gridSize); + Assert.assertEquals(getField(messageChannelPartitionHandler, "pollInterval"), pollInterval); + Assert.assertEquals(getField(messageChannelPartitionHandler, "timeout"), timeout); + + Object messagingGateway = getField(messageChannelPartitionHandler, "messagingGateway"); + Assert.assertNotNull(messagingGateway); + MessagingTemplate messagingTemplate = (MessagingTemplate) messagingGateway; + Assert.assertEquals(getField(messagingTemplate, "defaultDestination"), outputChannel); + } + + @Test + public void testMasterStepCreationWhenAggregatingReplies() { + // given + int gridSize = 5; + int startLimit = 3; + DirectChannel outputChannel = new DirectChannel(); + Partitioner partitioner = Mockito.mock(Partitioner.class); + StepExecutionAggregator stepExecutionAggregator = (result, executions) -> { }; + + // when + Step step = new RemotePartitioningMasterStepBuilder("masterStep") + .repository(jobRepository) + .outputChannel(outputChannel) + .partitioner("workerStep", partitioner) + .gridSize(gridSize) + .startLimit(startLimit) + .aggregator(stepExecutionAggregator) + .allowStartIfComplete(true) + .build(); + + // then + Assert.assertNotNull(step); + Assert.assertEquals(getField(step, "startLimit"), startLimit); + Assert.assertEquals(getField(step, "jobRepository"), this.jobRepository); + Assert.assertEquals(getField(step, "stepExecutionAggregator"), stepExecutionAggregator); + Assert.assertTrue((Boolean) getField(step, "allowStartIfComplete")); + + Object partitionHandler = getField(step, "partitionHandler"); + Assert.assertNotNull(partitionHandler); + Assert.assertTrue(partitionHandler instanceof MessageChannelPartitionHandler); + MessageChannelPartitionHandler messageChannelPartitionHandler = (MessageChannelPartitionHandler) partitionHandler; + Assert.assertEquals(getField(messageChannelPartitionHandler, "gridSize"), gridSize); + + Object replyChannel = getField(messageChannelPartitionHandler, "replyChannel"); + Assert.assertNotNull(replyChannel); + Assert.assertTrue(replyChannel instanceof QueueChannel); + + Object messagingGateway = getField(messageChannelPartitionHandler, "messagingGateway"); + Assert.assertNotNull(messagingGateway); + MessagingTemplate messagingTemplate = (MessagingTemplate) messagingGateway; + Assert.assertEquals(getField(messagingTemplate, "defaultDestination"), outputChannel); + } + + @Configuration + @EnableBatchProcessing + public static class BatchConfiguration { + + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/RemotePartitioningWorkerStepBuilderTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/RemotePartitioningWorkerStepBuilderTests.java new file mode 100644 index 0000000000..316f99dbab --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/RemotePartitioningWorkerStepBuilderTests.java @@ -0,0 +1,132 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.partition; + +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; +import org.mockito.Mock; + +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.integration.channel.DirectChannel; + +/** + * @author Mahmoud Ben Hassine + */ +public class RemotePartitioningWorkerStepBuilderTests { + + @Rule + public ExpectedException expectedException = ExpectedException.none(); + + @Mock + private Tasklet tasklet; + + @Test + public void inputChannelMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("inputChannel must not be null"); + + // when + new RemotePartitioningWorkerStepBuilder("step").inputChannel(null); + + // then + // expected exception + } + + @Test + public void outputChannelMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("outputChannel must not be null"); + + // when + new RemotePartitioningWorkerStepBuilder("step").outputChannel(null); + + // then + // expected exception + } + + @Test + public void jobExplorerMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("jobExplorer must not be null"); + + // when + new RemotePartitioningWorkerStepBuilder("step").jobExplorer(null); + + // then + // expected exception + } + + @Test + public void stepLocatorMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("stepLocator must not be null"); + + // when + new RemotePartitioningWorkerStepBuilder("step").stepLocator(null); + + // then + // expected exception + } + + @Test + public void beanFactoryMustNotBeNull() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("beanFactory must not be null"); + + // when + new RemotePartitioningWorkerStepBuilder("step").beanFactory(null); + + // then + // expected exception + } + + @Test + public void testMandatoryInputChannel() { + // given + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("An InputChannel must be provided"); + + // when + new RemotePartitioningWorkerStepBuilder("step").tasklet(this.tasklet); + + // then + // expected exception + } + + @Test + public void testMandatoryJobExplorer() { + // given + DirectChannel inputChannel = new DirectChannel(); + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("A JobExplorer must be provided"); + + // when + new RemotePartitioningWorkerStepBuilder("step") + .inputChannel(inputChannel) + .tasklet(this.tasklet); + + // then + // expected exception + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/StepExecutionRequestTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/StepExecutionRequestTests.java new file mode 100644 index 0000000000..2dc0b98b09 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/StepExecutionRequestTests.java @@ -0,0 +1,57 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.integration.partition; + +import java.io.IOException; + +import com.fasterxml.jackson.databind.ObjectMapper; +import org.junit.Assert; +import org.junit.Test; + +/** + * @author Mahmoud Ben Hassine + */ +public class StepExecutionRequestTests { + + private static final String SERIALIZED_REQUEST = "{\"stepExecutionId\":1,\"stepName\":\"step\",\"jobExecutionId\":1}"; + + private ObjectMapper objectMapper = new ObjectMapper(); + + @Test + public void stepExecutionRequestShouldBeSerializableWithJackson() throws IOException { + // given + StepExecutionRequest request = new StepExecutionRequest("step", 1L, 1L); + + // when + String serializedRequest = this.objectMapper.writeValueAsString(request); + + // then + Assert.assertEquals(SERIALIZED_REQUEST, serializedRequest); + } + + @Test + public void stepExecutionRequestShouldBeDeserializableWithJackson() throws IOException { + // when + StepExecutionRequest deserializedRequest = this.objectMapper.readValue(SERIALIZED_REQUEST, StepExecutionRequest.class); + + // then + Assert.assertNotNull(deserializedRequest); + Assert.assertEquals("step", deserializedRequest.getStepName()); + Assert.assertEquals(1L, deserializedRequest.getJobExecutionId().longValue()); + Assert.assertEquals(1L, deserializedRequest.getStepExecutionId().longValue()); + } +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/VanillaIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/VanillaIntegrationTests.java new file mode 100644 index 0000000000..59f9bbaa39 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/partition/VanillaIntegrationTests.java @@ -0,0 +1,70 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.partition; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import java.util.List; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobInstance; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.explore.JobExplorer; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +/** + * @author Dave Syer + * + */ +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class VanillaIntegrationTests { + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private Job job; + + @Autowired + private JobExplorer jobExplorer; + + @Test + public void testSimpleProperties() throws Exception { + assertNotNull(jobLauncher); + } + + @Test + public void testLaunchJob() throws Exception { + int before = jobExplorer.getJobInstances(job.getName(), 0, 100).size(); + assertNotNull(jobLauncher.run(job, new JobParameters())); + List jobInstances = jobExplorer.getJobInstances(job.getName(), 0, 100); + int after = jobInstances.size(); + assertEquals(1, after-before); + JobExecution jobExecution = jobExplorer.getJobExecutions(jobInstances.get(jobInstances.size()-1)).get(0); + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + assertEquals(3, jobExecution.getStepExecutions().size()); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/RepeatTransactionalPollingIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/RepeatTransactionalPollingIntegrationTests.java new file mode 100644 index 0000000000..b25e698367 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/RepeatTransactionalPollingIntegrationTests.java @@ -0,0 +1,114 @@ +package org.springframework.batch.integration.retry; + +import static org.junit.Assert.assertEquals; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.support.transaction.TransactionAwareProxyFactory; +import org.springframework.beans.BeansException; +import org.springframework.context.ApplicationContext; +import org.springframework.context.ApplicationContextAware; +import org.springframework.context.Lifecycle; +import org.springframework.integration.annotation.MessageEndpoint; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.util.StringUtils; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +@MessageEndpoint +public class RepeatTransactionalPollingIntegrationTests implements ApplicationContextAware { + + private Log logger = LogFactory.getLog(getClass()); + + private static List processed = new ArrayList<>(); + + private static List expected; + + private static List handled = new ArrayList<>(); + + private static List list = new ArrayList<>(); + + private Lifecycle bus; + + private volatile static int count = 0; + + public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { + bus = (Lifecycle) applicationContext; + } + + public String process(String message) { + String result = message + ": " + count; + logger.debug("Handling: " + message); + if (count list = new ArrayList<>(); + + @Autowired + private SimpleRecoverer recoverer; + + @Autowired + private SimpleService service; + + private Lifecycle lifecycle; + + public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { + lifecycle = (Lifecycle) applicationContext; + } + + private static volatile int count = 0; + + @Before + public void clearLists() { + list.clear(); + count = 0; + } + + public String input() { + logger.debug("Polling: " + count); + if (list.isEmpty()) { + return null; + } + return list.remove(0); + } + + public void output(String message) { + count++; + logger.debug("Handled: " + message); + } + + @Test + @DirtiesContext + public void testSunnyDay() throws Exception { + list = TransactionAwareProxyFactory.createTransactionalList(Arrays.asList(StringUtils + .commaDelimitedListToStringArray("a,b,c,d,e,f,g,h,j,k"))); + List expected = Arrays.asList(StringUtils.commaDelimitedListToStringArray("a,b,c,d")); + service.setExpected(expected); + waitForResults(lifecycle, expected.size(), 60); + assertEquals(4, service.getProcessed().size()); // a,b,c,d + assertEquals(expected, service.getProcessed()); + } + + @Test + @DirtiesContext + public void testRollback() throws Exception { + list = TransactionAwareProxyFactory.createTransactionalList(Arrays.asList(StringUtils + .commaDelimitedListToStringArray("a,b,fail,d,e,f,g,h,j,k"))); + List expected = Arrays.asList(StringUtils.commaDelimitedListToStringArray("a,b,fail,fail,d,e,f")); + service.setExpected(expected); + waitForResults(lifecycle, expected.size(), 60); // (a,b), (fail), (fail), ([fail],d), (e,f) + System.err.println(service.getProcessed()); + assertEquals(7, service.getProcessed().size()); // a,b,fail,fail,d,e,f + assertEquals(1, recoverer.getRecovered().size()); // fail + assertEquals(expected, service.getProcessed()); + } + + private void waitForResults(Lifecycle lifecycle, int count, int maxTries) throws InterruptedException { + lifecycle.start(); + int timeout = 0; + while (service.getProcessed().size() < count && timeout++ < maxTries) { + Thread.sleep(5); + } + lifecycle.stop(); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/RetryTransactionalPollingIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/RetryTransactionalPollingIntegrationTests.java new file mode 100644 index 0000000000..4bf38174b2 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/RetryTransactionalPollingIntegrationTests.java @@ -0,0 +1,105 @@ +package org.springframework.batch.integration.retry; + +import static org.junit.Assert.assertEquals; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.concurrent.atomic.AtomicInteger; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.support.transaction.TransactionAwareProxyFactory; +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.ApplicationContext; +import org.springframework.context.ApplicationContextAware; +import org.springframework.context.Lifecycle; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.util.StringUtils; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class RetryTransactionalPollingIntegrationTests implements ApplicationContextAware { + + private Log logger = LogFactory.getLog(getClass()); + + private static List list = new ArrayList<>(); + + @Autowired + private SimpleRecoverer recoverer; + + @Autowired + private SimpleService service; + + private Lifecycle bus; + + public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { + bus = (Lifecycle) applicationContext; + } + + private static AtomicInteger count = new AtomicInteger(0); + + @Before + public void clearLists() { + list.clear(); + count.set(0); + } + + public String input() { + logger.debug("Polling: " + count); + if (list.isEmpty()) { + return null; + } + // This happens in a transaction and the list is transactional so if it rolls back the same item comes back + // again at the head of the list + return list.remove(0); + } + + public void output(String message) { + count.incrementAndGet(); + logger.debug("Handled: " + message); + } + + @Test + @DirtiesContext + public void testSunnyDay() throws Exception { + list = TransactionAwareProxyFactory.createTransactionalList(Arrays.asList(StringUtils + .commaDelimitedListToStringArray("a,b,c,d,e,f,g,h,j,k"))); + List expected = Arrays.asList(StringUtils.commaDelimitedListToStringArray("a,b,c,d")); + service.setExpected(expected); + waitForResults(bus, expected.size(), 60); + assertEquals(4, service.getProcessed().size()); // a,b,c,d + assertEquals(expected, service.getProcessed()); + } + + @Test + @DirtiesContext + public void testRollback() throws Exception { + list = TransactionAwareProxyFactory.createTransactionalList(Arrays.asList(StringUtils + .commaDelimitedListToStringArray("a,b,fail,d,e,f,g,h,j,k"))); + + List expected = Arrays.asList(StringUtils.commaDelimitedListToStringArray("a,b,fail,fail,d,e")); + service.setExpected(expected); + waitForResults(bus, expected.size(), 100); // a, b, (fail, fail, [fail]), d, e + // System.err.println(service.getProcessed()); + assertEquals(6, service.getProcessed().size()); // a,b,fail,fail,d,e + assertEquals(1, recoverer.getRecovered().size()); // fail + assertEquals(expected, service.getProcessed()); + } + + private void waitForResults(Lifecycle lifecycle, int count, int maxTries) throws InterruptedException { + lifecycle.start(); + int timeout = 0; + while (service.getProcessed().size() < count && timeout++ < maxTries) { + Thread.sleep(10); + } + lifecycle.stop(); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/Service.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/Service.java new file mode 100644 index 0000000000..18c11ca382 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/Service.java @@ -0,0 +1,26 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.retry; + +/** + * @author Dave Syer + * + */ +public interface Service { + + String process(String message); + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/SimpleRecoverer.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/SimpleRecoverer.java new file mode 100644 index 0000000000..681899abb9 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/SimpleRecoverer.java @@ -0,0 +1,37 @@ +package org.springframework.batch.integration.retry; + +import java.util.ArrayList; +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.retry.interceptor.MethodInvocationRecoverer; + +/** + * @author Dave Syer + * + */ +public final class SimpleRecoverer implements MethodInvocationRecoverer { + + private Log logger = LogFactory.getLog(getClass()); + + private final List recovered = new ArrayList<>(); + + /** + * Public getter for the recovered. + * @return the recovered + */ + public List getRecovered() { + return recovered; + } + + public String recover(Object[] data, Throwable cause) { + if (data == null) { + return null; + } + String payload = (String) data[0]; + logger.debug("Recovering: " + payload); + recovered.add(payload); + return null; + } +} \ No newline at end of file diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/SimpleService.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/SimpleService.java new file mode 100644 index 0000000000..8ef470a906 --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/SimpleService.java @@ -0,0 +1,49 @@ +package org.springframework.batch.integration.retry; + +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.CopyOnWriteArrayList; +import java.util.concurrent.atomic.AtomicInteger; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.integration.annotation.MessageEndpoint; +import org.springframework.integration.annotation.ServiceActivator; + +@MessageEndpoint +public class SimpleService implements Service { + + private Log logger = LogFactory.getLog(getClass()); + + private List processed = new CopyOnWriteArrayList<>(); + + private List expected = new ArrayList<>(); + + private AtomicInteger count = new AtomicInteger(0); + + public void setExpected(List expected) { + this.expected = expected; + } + + /** + * Public getter for the processed. + * @return the processed + */ + public List getProcessed() { + return processed; + } + + @ServiceActivator(inputChannel = "requests", outputChannel = "replies") + public String process(String message) { + String result = message + ": " + count.incrementAndGet(); + logger.debug("Handling: " + message); + if (count.get() <= expected.size()) { + processed.add(message); + } + if ("fail".equals(message)) { + throw new RuntimeException("Planned failure"); + } + return result; + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/TransactionalPollingIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/TransactionalPollingIntegrationTests.java new file mode 100644 index 0000000000..e57fd7cebc --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/retry/TransactionalPollingIntegrationTests.java @@ -0,0 +1,121 @@ +package org.springframework.batch.integration.retry; + +import static org.junit.Assert.assertEquals; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.support.transaction.TransactionAwareProxyFactory; +import org.springframework.beans.BeansException; +import org.springframework.context.ApplicationContext; +import org.springframework.context.ApplicationContextAware; +import org.springframework.context.Lifecycle; +import org.springframework.integration.annotation.MessageEndpoint; +import org.springframework.integration.annotation.ServiceActivator; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.util.StringUtils; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +@MessageEndpoint +public class TransactionalPollingIntegrationTests implements ApplicationContextAware { + + private Log logger = LogFactory.getLog(getClass()); + + private static List processed = new ArrayList<>(); + + private static List handled = new ArrayList<>(); + + private static List expected = new ArrayList<>(); + + private static List list = new ArrayList<>(); + + private Lifecycle bus; + + public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { + bus = (Lifecycle) applicationContext; + } + + private volatile static int count = 0; + + @ServiceActivator(inputChannel = "requests", outputChannel = "replies") + public String process(String message) { + String result = message + ": " + count; + logger.debug("Handling: " + message); + if (count < expected.size()) { + processed.add(message); + count++; + } + if ("fail".equals(message)) { + throw new RuntimeException("Planned failure"); + } + return result; + } + + public String input() { + logger.debug("Polling: " + count + " of " + list.size()); + if (list.isEmpty()) { + return null; + } + return list.remove(0); + } + + public void output(String message) { + if (count < expected.size()) { + handled.add(message); + } + logger.debug("Handled: " + message); + } + + @Before + public void clearLists() { + list.clear(); + handled.clear(); + processed.clear(); + count = 0; + } + + @Test + @DirtiesContext + public void testSunnyDay() throws Exception { + try { + list = TransactionAwareProxyFactory.createTransactionalList(Arrays.asList(StringUtils + .commaDelimitedListToStringArray("a,b,c,d,e,f,g,h,j,k"))); + expected = Arrays.asList(StringUtils.commaDelimitedListToStringArray("a,b,c,d")); + waitForResults(bus, 4, 60); + assertEquals(expected, processed); + } catch (Throwable t) { + System.out.println(t.getMessage()); + t.printStackTrace(); + } + } + + @Test + @DirtiesContext + public void testRollback() throws Exception { + list = TransactionAwareProxyFactory.createTransactionalList(Arrays.asList(StringUtils + .commaDelimitedListToStringArray("a,b,fail,d,e,f,g,h,j,k"))); + expected = Arrays.asList(StringUtils.commaDelimitedListToStringArray("a,b,fail,fail")); + waitForResults(bus, 4, 30); + assertEquals(expected, processed); + assertEquals(2, handled.size()); // a,b + } + + private void waitForResults(Lifecycle lifecycle, int count, int maxTries) throws InterruptedException { + lifecycle.start(); + int timeout = 0; + while (processed.size() < count && timeout++ < maxTries) { + Thread.sleep(10); + } + lifecycle.stop(); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/step/StepGatewayIntegrationTests.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/step/StepGatewayIntegrationTests.java new file mode 100644 index 0000000000..f2a9217bde --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/step/StepGatewayIntegrationTests.java @@ -0,0 +1,73 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.step; + +import static org.junit.Assert.assertEquals; + +import org.junit.After; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +/** + * @author Dave Syer + * + */ +@ContextConfiguration() +@RunWith(SpringJUnit4ClassRunner.class) +public class StepGatewayIntegrationTests { + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + @Qualifier("job") + private Job job; + + @Autowired + private TestTasklet tasklet; + + @After + public void clear() { + tasklet.setFail(false); + } + + @Test + public void testLaunchJob() throws Exception { + JobExecution jobExecution = jobLauncher.run(job, new JobParameters()); + assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + assertEquals(ExitStatus.COMPLETED, jobExecution.getExitStatus()); + } + + @Test + public void testLaunchFailedJob() throws Exception { + tasklet.setFail(true); + JobExecution jobExecution = jobLauncher.run(job, new JobParametersBuilder().addLong("run.id", 2L).toJobParameters()); + assertEquals(BatchStatus.FAILED, jobExecution.getStatus()); + assertEquals(ExitStatus.FAILED, jobExecution.getExitStatus()); + } + +} diff --git a/spring-batch-integration/src/test/java/org/springframework/batch/integration/step/TestTasklet.java b/spring-batch-integration/src/test/java/org/springframework/batch/integration/step/TestTasklet.java new file mode 100644 index 0000000000..e47a33619b --- /dev/null +++ b/spring-batch-integration/src/test/java/org/springframework/batch/integration/step/TestTasklet.java @@ -0,0 +1,44 @@ +/* + * Copyright 2006-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.integration.step; + +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; + +/** + * @author Dave Syer + * + */ +public class TestTasklet implements Tasklet { + + private boolean fail = false; + + public void setFail(boolean fail) { + this.fail = fail; + } + + @Nullable + public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + if (fail) { + throw new IllegalStateException("Planned Tasklet failure"); + } + return RepeatStatus.FINISHED; + } + +} diff --git a/spring-batch-integration/src/test/resources/config-derby.properties b/spring-batch-integration/src/test/resources/config-derby.properties new file mode 100644 index 0000000000..c41d095056 --- /dev/null +++ b/spring-batch-integration/src/test/resources/config-derby.properties @@ -0,0 +1,14 @@ +# Placeholders batch.* +# for Derby: +batch.jdbc.driver=org.apache.derby.jdbc.EmbeddedDriver +batch.jdbc.url=jdbc:derby:derby-home/test;create=true +batch.jdbc.user=app +batch.jdbc.password= +batch.jdbc.testWhileIdle=false +batch.jdbc.validationQuery= +batch.data.source.init=true +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.DerbyMaxValueIncrementer +batch.schema.script=classpath:/org/springframework/batch/core/schema-derby.sql +batch.drop.script=classpath:/org/springframework/batch/core/schema-drop-derby.sql +integration.schema.script=classpath*:/org/springframework/integration/jdbc/schema-derby.sql +integration.drop.script=classpath*:/org/springframework/integration/jdbc/schema-drop-derby.sql diff --git a/spring-batch-integration/src/test/resources/config-h2.properties b/spring-batch-integration/src/test/resources/config-h2.properties new file mode 100644 index 0000000000..9e5516dbc3 --- /dev/null +++ b/spring-batch-integration/src/test/resources/config-h2.properties @@ -0,0 +1,13 @@ +# Default database platform is HSQLDB: +batch.jdbc.driver=org.h2.Driver +batch.jdbc.url=jdbc:h2:file:target/data/h2;DB_CLOSE_ON_EXIT=FALSE +batch.jdbc.user=sa +batch.jdbc.password= +batch.jdbc.testWhileIdle=false +batch.jdbc.validationQuery= +batch.data.source.init=true +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.H2SequenceMaxValueIncrementer +batch.schema.script=classpath*:/org/springframework/batch/core/schema-h2.sql +batch.drop.script=classpath*:/org/springframework/batch/core/schema-drop-h2.sql +integration.schema.script=classpath*:/org/springframework/integration/jdbc/schema-h2.sql +integration.drop.script=classpath*:/org/springframework/integration/jdbc/schema-drop-h2.sql diff --git a/spring-batch-integration/src/test/resources/config-hsql.properties b/spring-batch-integration/src/test/resources/config-hsql.properties new file mode 100644 index 0000000000..ae92235985 --- /dev/null +++ b/spring-batch-integration/src/test/resources/config-hsql.properties @@ -0,0 +1,16 @@ +# Default database platform is HSQLDB: +batch.jdbc.driver=org.hsqldb.jdbcDriver +batch.jdbc.url=jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true;hsqldb.tx=mvcc +# Override and use this one in for a separate server process so you can inspect +# the results (or add it to system properties with -D to override at run time). +# batch.jdbc.url=jdbc:hsqldb:hsql://localhost:9005/samples +batch.jdbc.user=sa +batch.jdbc.password= +batch.jdbc.testWhileIdle=false +batch.jdbc.validationQuery= +batch.data.source.init=true +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.HsqlMaxValueIncrementer +batch.schema.script=classpath*:/org/springframework/batch/core/schema-hsqldb.sql +batch.drop.script=classpath*:/org/springframework/batch/core/schema-drop-hsqldb.sql +integration.schema.script=classpath*:/org/springframework/integration/jdbc/schema-hsqldb.sql +integration.drop.script=classpath*:/org/springframework/integration/jdbc/schema-drop-hsqldb.sql diff --git a/spring-batch-integration/src/test/resources/config-mysql.properties b/spring-batch-integration/src/test/resources/config-mysql.properties new file mode 100644 index 0000000000..083deef2e1 --- /dev/null +++ b/spring-batch-integration/src/test/resources/config-mysql.properties @@ -0,0 +1,14 @@ +# Placeholders batch.* +# for MySQL: +batch.jdbc.driver=com.mysql.jdbc.Driver +batch.jdbc.url=jdbc:mysql://localhost/test +batch.jdbc.user=root +batch.jdbc.password=root +batch.jdbc.testWhileIdle=true +batch.jdbc.validationQuery=SELECT 1 +batch.data.source.init=true +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.MySQLMaxValueIncrementer +batch.schema.script=classpath:/org/springframework/batch/core/schema-mysql.sql +batch.drop.script=classpath*:/org/springframework/batch/core/schema-drop-mysql.sql +integration.schema.script=classpath*:/org/springframework/integration/jdbc/schema-mysql.sql +integration.drop.script=classpath*:/org/springframework/integration/jdbc/schema-drop-mysql.sql diff --git a/spring-batch-integration/src/test/resources/jms-context.xml b/spring-batch-integration/src/test/resources/jms-context.xml new file mode 100644 index 0000000000..24d362c7c9 --- /dev/null +++ b/spring-batch-integration/src/test/resources/jms-context.xml @@ -0,0 +1,36 @@ + + + + + + + + + + + + + + + + + + + + + vm://localhost + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/job-execution-context.xml b/spring-batch-integration/src/test/resources/job-execution-context.xml new file mode 100644 index 0000000000..3415e61bed --- /dev/null +++ b/spring-batch-integration/src/test/resources/job-execution-context.xml @@ -0,0 +1,14 @@ + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/log4j.properties b/spring-batch-integration/src/test/resources/log4j.properties new file mode 100644 index 0000000000..cbfa277790 --- /dev/null +++ b/spring-batch-integration/src/test/resources/log4j.properties @@ -0,0 +1,13 @@ +log4j.rootCategory=WARN, stdout + +log4j.appender.stdout=org.apache.logging.log4j.core.appender.ConsoleAppender +log4j.appender.stdout.layout=org.apache.logging.log4j.core.layout.PatternLayout +log4j.appender.stdout.layout.ConversionPattern=%d %5p %t [%c] - <%m>%n + +log4j.category.org.springframework.context=INFO +log4j.category.org.springframework.beans=INFO +log4j.category.org.springframework.batch.retry=DEBUG +log4j.category.org.springframework.batch.core=DEBUG +log4j.category.org.springframework.batch.integration=DEBUG +log4j.category.org.springframework.integration=DEBUG +log4j.category.org.springframework.transaction=INFO \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/SmokeTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/SmokeTests-context.xml new file mode 100644 index 0000000000..9513535069 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/SmokeTests-context.xml @@ -0,0 +1,21 @@ + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/async/AsyncItemProcessorMessagingGatewayTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/async/AsyncItemProcessorMessagingGatewayTests-context.xml new file mode 100644 index 0000000000..91c87b923a --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/async/AsyncItemProcessorMessagingGatewayTests-context.xml @@ -0,0 +1,23 @@ + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/async/PollingAsyncItemProcessorMessagingGatewayTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/async/PollingAsyncItemProcessorMessagingGatewayTests-context.xml new file mode 100644 index 0000000000..90239a823b --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/async/PollingAsyncItemProcessorMessagingGatewayTests-context.xml @@ -0,0 +1,28 @@ + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/ChunkMessageItemWriterIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/ChunkMessageItemWriterIntegrationTests-context.xml new file mode 100644 index 0000000000..d0ba00aa58 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/ChunkMessageItemWriterIntegrationTests-context.xml @@ -0,0 +1,33 @@ + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepIntegrationTests-context.xml new file mode 100644 index 0000000000..c97a550074 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepIntegrationTests-context.xml @@ -0,0 +1,111 @@ + + + + + + + + + + + + + + + + + + + 1 + 2 + #{jobParameters['item.three']} + 4 + 5 + 6 + 7 + 8 + 9 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJdbcIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJdbcIntegrationTests-context.xml new file mode 100644 index 0000000000..e04a665bdb --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJdbcIntegrationTests-context.xml @@ -0,0 +1,135 @@ + + + + + + + + + + + + + + + + + + + + + + 1 + 2 + #{jobParameters['item.three']} + 4 + 5 + 6 + 7 + 8 + 9 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJmsIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJmsIntegrationTests-context.xml new file mode 100644 index 0000000000..b27846f9e7 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkFaultTolerantStepJmsIntegrationTests-context.xml @@ -0,0 +1,117 @@ + + + + + + + + + + + + + + + + + + + + + 1 + 2 + #{jobParameters['item.three']} + 4 + 5 + 6 + 7 + 8 + 9 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkStepIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkStepIntegrationTests-context.xml new file mode 100644 index 0000000000..2bfad626cd --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk/RemoteChunkStepIntegrationTests-context.xml @@ -0,0 +1,72 @@ + + + + + + + + + + + + + + + 1 + 2 + #{jobParameters['item.three']} + 4 + 5 + 6 + 7 + 8 + 9 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTests-context.xml new file mode 100644 index 0000000000..64b1146634 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTests-context.xml @@ -0,0 +1,27 @@ + + + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTestsNoJobLauncher-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTestsNoJobLauncher-context.xml new file mode 100644 index 0000000000..5d95e348bd --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTestsNoJobLauncher-context.xml @@ -0,0 +1,22 @@ + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTestsRunning-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTestsRunning-context.xml new file mode 100644 index 0000000000..7dc6b4d536 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTestsRunning-context.xml @@ -0,0 +1,27 @@ + + + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTestsWithEnableBatchProcessing-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTestsWithEnableBatchProcessing-context.xml new file mode 100644 index 0000000000..4c64e6d890 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/JobLaunchingGatewayParserTestsWithEnableBatchProcessing-context.xml @@ -0,0 +1,26 @@ + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingIdAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingIdAttrTests.xml new file mode 100644 index 0000000000..d977fe0a94 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingIdAttrTests.xml @@ -0,0 +1,12 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingMessageTemplateAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingMessageTemplateAttrTests.xml new file mode 100644 index 0000000000..0d349d5c91 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingMessageTemplateAttrTests.xml @@ -0,0 +1,12 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingReplyChannelAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingReplyChannelAttrTests.xml new file mode 100644 index 0000000000..6187eebc22 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingReplyChannelAttrTests.xml @@ -0,0 +1,12 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingStepAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingStepAttrTests.xml new file mode 100644 index 0000000000..be4f748937 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserMissingStepAttrTests.xml @@ -0,0 +1,12 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserTests.xml new file mode 100644 index 0000000000..1c1a58f29f --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingManagerParserTests.xml @@ -0,0 +1,57 @@ + + + + + + + + + + + + + + + + + + 1 + 2 + 3 + 4 + 5 + 6 + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingIdAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingIdAttrTests.xml new file mode 100644 index 0000000000..b7e92bc00f --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingIdAttrTests.xml @@ -0,0 +1,12 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingMessageTemplateAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingMessageTemplateAttrTests.xml new file mode 100644 index 0000000000..ccf1bd943d --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingMessageTemplateAttrTests.xml @@ -0,0 +1,12 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingReplyChannelAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingReplyChannelAttrTests.xml new file mode 100644 index 0000000000..82d102e6eb --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingReplyChannelAttrTests.xml @@ -0,0 +1,12 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingStepAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingStepAttrTests.xml new file mode 100644 index 0000000000..fa17d9c8b8 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserMissingStepAttrTests.xml @@ -0,0 +1,12 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserTests.xml new file mode 100644 index 0000000000..74af85c3e8 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingMasterParserTests.xml @@ -0,0 +1,57 @@ + + + + + + + + + + + + + + + + + + 1 + 2 + 3 + 4 + 5 + 6 + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingIdAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingIdAttrTests.xml new file mode 100644 index 0000000000..5b201bfff9 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingIdAttrTests.xml @@ -0,0 +1,13 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingInputChannelAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingInputChannelAttrTests.xml new file mode 100644 index 0000000000..bcc5becca5 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingInputChannelAttrTests.xml @@ -0,0 +1,13 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingItemWriterAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingItemWriterAttrTests.xml new file mode 100644 index 0000000000..a7af0eef60 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingItemWriterAttrTests.xml @@ -0,0 +1,13 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingOutputChannelAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingOutputChannelAttrTests.xml new file mode 100644 index 0000000000..2284e3c84e --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserMissingOutputChannelAttrTests.xml @@ -0,0 +1,13 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserNoProcessorTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserNoProcessorTests.xml new file mode 100644 index 0000000000..05b96dda5d --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserNoProcessorTests.xml @@ -0,0 +1,21 @@ + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserTests.xml new file mode 100644 index 0000000000..f5cd564680 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingSlaveParserTests.xml @@ -0,0 +1,21 @@ + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingIdAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingIdAttrTests.xml new file mode 100644 index 0000000000..e071d2e5a9 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingIdAttrTests.xml @@ -0,0 +1,13 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingInputChannelAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingInputChannelAttrTests.xml new file mode 100644 index 0000000000..8b5ea77504 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingInputChannelAttrTests.xml @@ -0,0 +1,13 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingItemWriterAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingItemWriterAttrTests.xml new file mode 100644 index 0000000000..b490becd1b --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingItemWriterAttrTests.xml @@ -0,0 +1,13 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingOutputChannelAttrTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingOutputChannelAttrTests.xml new file mode 100644 index 0000000000..75b1d89b08 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserMissingOutputChannelAttrTests.xml @@ -0,0 +1,13 @@ + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserNoProcessorTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserNoProcessorTests.xml new file mode 100644 index 0000000000..3c1842e34c --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserNoProcessorTests.xml @@ -0,0 +1,21 @@ + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserTests.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserTests.xml new file mode 100644 index 0000000000..1254a6ed36 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/RemoteChunkingWorkerParserTests.xml @@ -0,0 +1,21 @@ + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/batch-setup-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/batch-setup-context.xml new file mode 100644 index 0000000000..dc2cef00e0 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/config/xml/batch-setup-context.xml @@ -0,0 +1,24 @@ + + + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/file/FileToMessagesJobIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/file/FileToMessagesJobIntegrationTests-context.xml new file mode 100644 index 0000000000..c5dc85ef20 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/file/FileToMessagesJobIntegrationTests-context.xml @@ -0,0 +1,30 @@ + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/file/ResourceSplitterIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/file/ResourceSplitterIntegrationTests-context.xml new file mode 100644 index 0000000000..ba064dc191 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/file/ResourceSplitterIntegrationTests-context.xml @@ -0,0 +1,19 @@ + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/file/test.txt b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/file/test.txt new file mode 100644 index 0000000000..b2f931a673 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/file/test.txt @@ -0,0 +1,5 @@ +one +two +three +four +five diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/item/MessagingGatewayIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/item/MessagingGatewayIntegrationTests-context.xml new file mode 100644 index 0000000000..241e95c202 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/item/MessagingGatewayIntegrationTests-context.xml @@ -0,0 +1,38 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/launch/JobLaunchingGatewayIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/launch/JobLaunchingGatewayIntegrationTests-context.xml new file mode 100644 index 0000000000..5c33b1b4bf --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/launch/JobLaunchingGatewayIntegrationTests-context.xml @@ -0,0 +1,27 @@ + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/launch/JobLaunchingMessageHandlerIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/launch/JobLaunchingMessageHandlerIntegrationTests-context.xml new file mode 100644 index 0000000000..71ddfda8b4 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/launch/JobLaunchingMessageHandlerIntegrationTests-context.xml @@ -0,0 +1,32 @@ + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/partition/JmsIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/partition/JmsIntegrationTests-context.xml new file mode 100755 index 0000000000..bc7725d8b7 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/partition/JmsIntegrationTests-context.xml @@ -0,0 +1,82 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/partition/PollingIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/partition/PollingIntegrationTests-context.xml new file mode 100644 index 0000000000..9a27c72fe1 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/partition/PollingIntegrationTests-context.xml @@ -0,0 +1,63 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/partition/VanillaIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/partition/VanillaIntegrationTests-context.xml new file mode 100644 index 0000000000..0e6a95c635 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/partition/VanillaIntegrationTests-context.xml @@ -0,0 +1,71 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/retry/RepeatTransactionalPollingIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/retry/RepeatTransactionalPollingIntegrationTests-context.xml new file mode 100644 index 0000000000..05cfb13480 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/retry/RepeatTransactionalPollingIntegrationTests-context.xml @@ -0,0 +1,57 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/retry/RetryRepeatTransactionalPollingIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/retry/RetryRepeatTransactionalPollingIntegrationTests-context.xml new file mode 100644 index 0000000000..666dce1fd6 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/retry/RetryRepeatTransactionalPollingIntegrationTests-context.xml @@ -0,0 +1,92 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/retry/RetryTransactionalPollingIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/retry/RetryTransactionalPollingIntegrationTests-context.xml new file mode 100644 index 0000000000..0b70044c6b --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/retry/RetryTransactionalPollingIntegrationTests-context.xml @@ -0,0 +1,57 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/retry/TransactionalPollingIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/retry/TransactionalPollingIntegrationTests-context.xml new file mode 100644 index 0000000000..e4443d78c4 --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/retry/TransactionalPollingIntegrationTests-context.xml @@ -0,0 +1,39 @@ + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/org/springframework/batch/integration/step/StepGatewayIntegrationTests-context.xml b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/step/StepGatewayIntegrationTests-context.xml new file mode 100644 index 0000000000..c8a15e5c4f --- /dev/null +++ b/spring-batch-integration/src/test/resources/org/springframework/batch/integration/step/StepGatewayIntegrationTests-context.xml @@ -0,0 +1,37 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-integration/src/test/resources/simple-job-launcher-context.xml b/spring-batch-integration/src/test/resources/simple-job-launcher-context.xml new file mode 100644 index 0000000000..06dce322d2 --- /dev/null +++ b/spring-batch-integration/src/test/resources/simple-job-launcher-context.xml @@ -0,0 +1,44 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/spring-batch-parent/pom.xml b/spring-batch-parent/pom.xml deleted file mode 100644 index e5ba7a6615..0000000000 --- a/spring-batch-parent/pom.xml +++ /dev/null @@ -1,763 +0,0 @@ - - - 4.0.0 - org.springframework.batch - spring-batch-parent - 2.2.2.BUILD-SNAPSHOT - Spring Batch Parent - Spring Batch parent project. Defines dependencies and common configuration for the build process. - http://static.springframework.org/spring-batch/${project.artifactId} - pom - - http://github.com/SpringSource/spring-batch - scm:git:git://github.com/SpringSource/spring-batch.git - scm:git:git://github.com/SpringSource/spring-batch.git - HEAD - - - - Apache 2.0 - http://www.apache.org/licenses/LICENSE-2.0.txt - - - - - dsyer - Dave Syer - david.syer@springsource.com - - - mminella - Michael Minella - mminella@vmware.com - - - - 3.2.0.RELEASE - 1.1.2.RELEASE - 4.10 - 1.0.0.RELEASE - false - UTF-8 - - - - fast - - true - true - - - - central - - - sonatype-nexus-snapshots - Sonatype Nexus Snapshots - https://oss.sonatype.org/content/repositories/snapshots/ - - - sonatype-nexus-staging - Nexus Release Repository - https://oss.sonatype.org/service/local/staging/deploy/maven2/ - - - - - - maven-gpg-plugin - - - sign-artifacts - verify - - sign - - - - - - - - - staging - - - staging - file:///${user.dir}/target/staging - - - staging - file:///${user.dir}/target/staging - - - staging - file:///${user.dir}/target/staging - - - - - milestone - - - spring-milestone - Spring Milestone Repository - s3://maven.springframework.org/milestone - - - - - dist - - - - maven-javadoc-plugin - - - javadoc - package - - jar - - - - - - maven-source-plugin - - - attach-sources - - jar - - - - - - - - - bootstrap - - - objectstyle - ObjectStyle.org Repository - http://objectstyle.org/maven2/ - - false - - - - apache-snapshot - Apache Foundation Snapshot Repository - http://people.apache.org/maven-snapshot-repository/ - - - springsource-milestone - SpringSource Milestone Repository - https://repo.springsource.org/libs-milestone/ - - - gemstone - Release bundles for SQLFire and GemFire - http://dist.gemstone.com.s3.amazonaws.com/maven/release - - - neo4j-public-release-repository - http://m2.neo4j.org/releases - - false - - - - - - apache-snapshots - http://people.apache.org/maven-snapshot-repository/ - - - - - - - - org.springframework.build.aws - org.springframework.build.aws.maven - 3.0.0.RELEASE - - - - - - - org.eclipse.m2e - lifecycle-mapping - 1.0.0 - - - - - - com.springsource.bundlor - com.springsource.bundlor.maven - [1.0,) - - bundlor - - - - - - - - - org.apache.maven.plugins - maven-antrun-plugin - [1.0,) - - run - - - - - - - - - - - - maven-jar-plugin - 2.3 - - - ${project.build.outputDirectory}/META-INF/MANIFEST.MF - - - - - org.apache.maven.plugins - maven-antrun-plugin - - - org.apache.ant - ant - 1.7.0 - - - org.apache.ant - ant-trax - 1.7.0 - - - org.apache.ant - ant-apache-regexp - 1.7.0 - - - foundrylogic.vpp - vpp - 2.2.1 - - - - - org.apache.maven.plugins - maven-site-plugin - 3.1 - - - org.apache.maven.wagon - wagon-ssh - 2.2 - - - - - - - - org.apache.maven.plugins - maven-compiler-plugin - 2.5.1 - - 1.6 - 1.6 - - - - com.springsource.bundlor - com.springsource.bundlor.maven - ${bundlor.version} - true - - - bundlor-transform - compile - - bundlor - - - - - - org.apache.maven.plugins - maven-surefire-plugin - 2.12.3 - - - **/*Tests.java - - - **/Abstract*.java - - junit:junit - - - - - - - objectstyle - ObjectStyle.org Repository - http://objectstyle.org/maven2/ - - false - - - - com.springsource.repository.bundles.release - SpringSource Enterprise Bundle Repository - SpringSource Bundle Releases - http://repository.springsource.com/maven/bundles/release - - false - - - - - - - maven-jxr-plugin - 2.3 - - true - - - - maven-surefire-report-plugin - 2.12.3 - - true - - - - maven-project-info-reports-plugin - 2.5.1 - - - maven-javadoc-plugin - 2.8.1 - - true - - - - - javadoc - - - - - - - - - - org.aspectj - aspectjrt - 1.5.4 - - - org.aspectj - aspectjweaver - 1.5.4 - - - junit - junit - ${junit.version} - test - - - org.apache.geronimo.specs - geronimo-jms_1.1_spec - 1.1 - true - - - org.apache.geronimo.specs - geronimo-jta_1.1_spec - 1.1 - true - - - javax.annotation - jsr250-api - 1.0 - test - - - org.codehaus.woodstox - woodstox-core-asl - 4.0.6 - true - - - stax - stax-api - - - - - org.hibernate - hibernate-core - 4.1.9.Final - true - - - cglib - cglib - - - asm - asm - - - asm - asm-attrs - - - javax.transaction - jta - - - - - org.hibernate - hibernate-entitymanager - 4.1.9.Final - true - - - edu.oswego.cs.concurrent - edu.oswego.cs.dl.util.concurrent - - - org.hibernate - hibernate - - - - - org.hibernate - hibernate-validator - 4.3.1.Final - - - org.hibernate - hibernate-annotations - 3.3.1.GA - true - - - org.hibernate - hibernate - - - commons-logging - commons-logging - - - - - cglib - cglib-nodep - 2.1_3 - true - - - org.slf4j - slf4j-api - 1.6.6 - - - org.slf4j - slf4j-log4j12 - 1.6.6 - - - commons-lang - commons-lang - PLEASE DON'T USE COMMONS LANG - true - - - commons-io - commons-io - 1.4 - true - - - commons-collections - commons-collections - 3.2 - true - - - commons-dbcp - commons-dbcp - 1.2.2 - true - - - log4j - log4j - 1.2.14 - true - - - org.hsqldb - hsqldb - 2.2.9 - test - - - com.h2database - h2 - 1.2.125 - test - - - org.apache.derby - derby - 10.8.2.2 - test - - - com.thoughtworks.xstream - xstream - 1.3 - - - org.codehaus.jettison - jettison - 1.1 - - - stax - stax-api - - - - - org.apache.ibatis - ibatis-sqlmap - 2.3.0 - true - - - org.osgi - osgi_R4_core - 1.0 - true - - - asm - asm - 2.2.3 - runtime - true - - - asm - asm-attrs - 2.2.3 - runtime - true - - - asm - asm-commons - 2.2.3 - runtime - true - - - javax.servlet - servlet-api - 2.5 - provided - true - - - stax - stax - 1.2.0 - true - - - org.springframework - spring-aop - ${spring.framework.version} - - - org.springframework - spring-beans - ${spring.framework.version} - - - org.springframework - spring-context - ${spring.framework.version} - - - org.springframework - spring-context-support - ${spring.framework.version} - - - quartz - quartz - - - - - org.springframework - spring-core - ${spring.framework.version} - - - org.springframework - spring-jdbc - ${spring.framework.version} - - - org.springframework - spring-jms - ${spring.framework.version} - - - org.springframework - spring-orm - ${spring.framework.version} - - - org.springframework - spring-test - ${spring.framework.version} - test - - - org.springframework - spring-tx - ${spring.framework.version} - - - org.springframework - spring-web - ${spring.framework.version} - - - org.springframework - spring-oxm - ${spring.framework.version} - true - - - commons-lang - commons-lang - - - - - org.springframework.osgi - spring-osgi-core - 1.1.2 - true - - - org.springframework.retry - spring-retry - 1.0.2.RELEASE - - - org.springframework.amqp - spring-amqp - ${spring.amqp.version} - - - org.springframework.amqp - spring-rabbit - ${spring.amqp.version} - - - org.mockito - mockito-all - 1.9.5 - test - - - org.springframework.data - spring-data-commons - 1.5.0.RELEASE - true - - - org.springframework.data - spring-data-jpa - 1.2.0.RELEASE - true - - - org.springframework.data - spring-data-mongodb - 1.1.0.RELEASE - true - - - org.springframework.data - spring-data-neo4j - 2.2.0.RELEASE - true - - - org.springframework.data - spring-data-gemfire - 1.3.0.RELEASE - true - - - org.springframework.data - spring-data-redis - 1.0.3.RELEASE - true - - - - - - static.springframework.org - scp://static.springframework.org/var/www/domains/springframework.org/static/htdocs/spring-batch/trunk - - - spring-release - Spring Release Repository - s3://maven.springframework.org/release - - - spring-snapshot - Spring Snapshot Repository - s3://maven.springframework.org/snapshot - - - diff --git a/spring-batch-parent/src/site/site.xml b/spring-batch-parent/src/site/site.xml deleted file mode 100644 index 5aad0d342b..0000000000 --- a/spring-batch-parent/src/site/site.xml +++ /dev/null @@ -1,23 +0,0 @@ - - - - ${project.name} - http://www.springframework.org/spring-batch - - - images/shim.gif - - - - - - org.springframework.maven.skins - maven-spring-skin - 1.0.5 - - - - - - - diff --git a/spring-batch-samples/.fbprefs b/spring-batch-samples/.fbprefs deleted file mode 100644 index d498aadbae..0000000000 --- a/spring-batch-samples/.fbprefs +++ /dev/null @@ -1,116 +0,0 @@ -#FindBugs User Preferences -#Wed Jul 23 12:11:15 BST 2008 -detectorAppendingToAnObjectOutputStream=AppendingToAnObjectOutputStream|true -detectorBadAppletConstructor=BadAppletConstructor|false -detectorBadResultSetAccess=BadResultSetAccess|true -detectorBadSyntaxForRegularExpression=BadSyntaxForRegularExpression|true -detectorBadUseOfReturnValue=BadUseOfReturnValue|true -detectorBadlyOverriddenAdapter=BadlyOverriddenAdapter|true -detectorBooleanReturnNull=BooleanReturnNull|true -detectorCheckImmutableAnnotation=CheckImmutableAnnotation|true -detectorCheckTypeQualifiers=CheckTypeQualifiers|true -detectorCloneIdiom=CloneIdiom|true -detectorComparatorIdiom=ComparatorIdiom|true -detectorConfusedInheritance=ConfusedInheritance|true -detectorConfusionBetweenInheritedAndOuterMethod=ConfusionBetweenInheritedAndOuterMethod|true -detectorCrossSiteScripting=CrossSiteScripting|true -detectorDoInsideDoPrivileged=DoInsideDoPrivileged|true -detectorDontCatchIllegalMonitorStateException=DontCatchIllegalMonitorStateException|true -detectorDontUseEnum=DontUseEnum|true -detectorDroppedException=DroppedException|true -detectorDumbMethodInvocations=DumbMethodInvocations|true -detectorDumbMethods=DumbMethods|true -detectorDuplicateBranches=DuplicateBranches|true -detectorEmptyZipFileEntry=EmptyZipFileEntry|true -detectorFinalizerNullsFields=FinalizerNullsFields|true -detectorFindBadCast2=FindBadCast2|true -detectorFindBadForLoop=FindBadForLoop|true -detectorFindCircularDependencies=FindCircularDependencies|false -detectorFindDeadLocalStores=FindDeadLocalStores|true -detectorFindDoubleCheck=FindDoubleCheck|true -detectorFindEmptySynchronizedBlock=FindEmptySynchronizedBlock|true -detectorFindFieldSelfAssignment=FindFieldSelfAssignment|true -detectorFindFinalizeInvocations=FindFinalizeInvocations|true -detectorFindFloatEquality=FindFloatEquality|true -detectorFindHEmismatch=FindHEmismatch|true -detectorFindInconsistentSync2=FindInconsistentSync2|true -detectorFindJSR166LockMonitorenter=FindJSR166LockMonitorenter|true -detectorFindLocalSelfAssignment2=FindLocalSelfAssignment2|true -detectorFindMaskedFields=FindMaskedFields|true -detectorFindMismatchedWaitOrNotify=FindMismatchedWaitOrNotify|true -detectorFindNakedNotify=FindNakedNotify|true -detectorFindNonSerializableStoreIntoSession=FindNonSerializableStoreIntoSession|true -detectorFindNonSerializableValuePassedToWriteObject=FindNonSerializableValuePassedToWriteObject|true -detectorFindNonShortCircuit=FindNonShortCircuit|true -detectorFindNullDeref=FindNullDeref|true -detectorFindOpenStream=FindOpenStream|true -detectorFindPuzzlers=FindPuzzlers|true -detectorFindRefComparison=FindRefComparison|true -detectorFindReturnRef=FindReturnRef|true -detectorFindRunInvocations=FindRunInvocations|true -detectorFindSelfComparison=FindSelfComparison|true -detectorFindSelfComparison2=FindSelfComparison2|true -detectorFindSleepWithLockHeld=FindSleepWithLockHeld|true -detectorFindSpinLoop=FindSpinLoop|true -detectorFindSqlInjection=FindSqlInjection|true -detectorFindTwoLockWait=FindTwoLockWait|true -detectorFindUncalledPrivateMethods=FindUncalledPrivateMethods|true -detectorFindUnconditionalWait=FindUnconditionalWait|true -detectorFindUninitializedGet=FindUninitializedGet|true -detectorFindUnrelatedTypesInGenericContainer=FindUnrelatedTypesInGenericContainer|true -detectorFindUnreleasedLock=FindUnreleasedLock|true -detectorFindUnsyncGet=FindUnsyncGet|true -detectorFindUselessControlFlow=FindUselessControlFlow|true -detectorHugeSharedStringConstants=HugeSharedStringConstants|true -detectorIDivResultCastToDouble=IDivResultCastToDouble|true -detectorIncompatMask=IncompatMask|true -detectorInefficientMemberAccess=InefficientMemberAccess|false -detectorInefficientToArray=InefficientToArray|true -detectorInfiniteLoop=InfiniteLoop|true -detectorInfiniteRecursiveLoop=InfiniteRecursiveLoop|true -detectorInfiniteRecursiveLoop2=InfiniteRecursiveLoop2|false -detectorInheritanceUnsafeGetResource=InheritanceUnsafeGetResource|true -detectorInitializationChain=InitializationChain|true -detectorInstantiateStaticClass=InstantiateStaticClass|true -detectorInvalidJUnitTest=InvalidJUnitTest|true -detectorIteratorIdioms=IteratorIdioms|true -detectorLazyInit=LazyInit|true -detectorLoadOfKnownNullValue=LoadOfKnownNullValue|true -detectorMethodReturnCheck=MethodReturnCheck|true -detectorMultithreadedInstanceAccess=MultithreadedInstanceAccess|true -detectorMutableLock=MutableLock|true -detectorMutableStaticFields=MutableStaticFields|true -detectorNaming=Naming|true -detectorNumberConstructor=NumberConstructor|true -detectorOverridingEqualsNotSymmetrical=OverridingEqualsNotSymmetrical|true -detectorPreferZeroLengthArrays=PreferZeroLengthArrays|true -detectorPublicSemaphores=PublicSemaphores|false -detectorQuestionableBooleanAssignment=QuestionableBooleanAssignment|true -detectorReadReturnShouldBeChecked=ReadReturnShouldBeChecked|true -detectorRedundantInterfaces=RedundantInterfaces|true -detectorRuntimeExceptionCapture=RuntimeExceptionCapture|true -detectorSerializableIdiom=SerializableIdiom|true -detectorStartInConstructor=StartInConstructor|true -detectorStaticCalendarDetector=StaticCalendarDetector|true -detectorStringConcatenation=StringConcatenation|true -detectorSuperfluousInstanceOf=SuperfluousInstanceOf|true -detectorSuspiciousThreadInterrupted=SuspiciousThreadInterrupted|true -detectorSwitchFallthrough=SwitchFallthrough|true -detectorSynchronizeAndNullCheckField=SynchronizeAndNullCheckField|true -detectorSynchronizeOnClassLiteralNotGetClass=SynchronizeOnClassLiteralNotGetClass|true -detectorSynchronizingOnContentsOfFieldToProtectField=SynchronizingOnContentsOfFieldToProtectField|true -detectorURLProblems=URLProblems|true -detectorUncallableMethodOfAnonymousClass=UncallableMethodOfAnonymousClass|true -detectorUnnecessaryMath=UnnecessaryMath|true -detectorUnreadFields=UnreadFields|true -detectorUseObjectEquals=UseObjectEquals|false -detectorUselessSubclassMethod=UselessSubclassMethod|false -detectorVarArgsProblems=VarArgsProblems|true -detectorVolatileUsage=VolatileUsage|true -detectorWaitInLoop=WaitInLoop|true -detectorWrongMapIterator=WrongMapIterator|true -detectorXMLFactoryBypass=XMLFactoryBypass|true -detector_threshold=2 -effort=default -filter_settings=Medium|BAD_PRACTICE,CORRECTNESS,I18N,MALICIOUS_CODE,MT_CORRECTNESS,PERFORMANCE,SECURITY,STYLE|false -filter_settings_neg=| diff --git a/spring-batch-samples/.springBeans b/spring-batch-samples/.springBeans index 291c24367b..477bf9450d 100644 --- a/spring-batch-samples/.springBeans +++ b/spring-batch-samples/.springBeans @@ -1,413 +1,472 @@ - - - 1 - - - - - - - src/main/resources/jobs/multilineJob.xml - src/main/resources/jobs/multilineOrderJob.xml - src/main/resources/jobs/tradeJob.xml - src/main/resources/jobs/restartSample.xml - src/main/resources/data-source-context.xml - src/main/resources/jobs/beanWrapperMapperSampleJob.xml - src/main/resources/jobs/adhocLoopJob.xml - src/main/resources/jobs/infiniteLoopJob.xml - src/main/resources/jobs/footballJob.xml - src/main/resources/jobs/delegatingJob.xml - src/main/resources/jobs/parallelJob.xml - src/main/resources/jobs/retrySample.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/adhoc-job-launcher-context.xml - src/main/resources/quartz-job-launcher-context.xml - src/main/resources/jobs/skipSampleJob.xml - src/main/resources/jobs/multilineOrderInputTokenizers.xml - src/main/resources/jobs/multilineOrderOutputAggregators.xml - src/main/resources/jobs/compositeItemWriterSampleJob.xml - src/main/resources/hibernate-context.xml - src/main/resources/staging-test-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/jobs/headerFooterSample.xml - src/main/resources/jobs/customerFilterJob.xml - src/main/resources/skipSample-job-launcher-context.xml - src/main/resources/jobs/ioSampleJob.xml - src/main/resources/jobs/iosample/delimited.xml - src/main/resources/jobs/iosample/fixedLength.xml - src/main/resources/jobs/iosample/jdbcCursor.xml - src/main/resources/jobs/iosample/multiLine.xml - src/main/resources/jobs/iosample/multiRecordType.xml - src/main/resources/jobs/iosample/multiResource.xml - src/main/resources/jobs/iosample/xml.xml - src/main/resources/jobs/iosample/hibernate.xml - src/main/resources/jobs/hibernateJob.xml - src/main/resources/jobs/iosample/jpa.xml - src/main/resources/jobs/iosample/ibatis.xml - src/test/resources/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDaoTests-context.xml - src/main/resources/jobs/loopFlowSample.xml - src/test/resources/org/springframework/batch/sample/common/StagingItemReaderTests-context.xml - src/test/resources/org/springframework/batch/sample/common/StagingItemWriterTests-context.xml - src/main/resources/jobs/multilineOrderValidator.xml - src/main/resources/jobs/restartFileSampleJob.xml - src/main/resources/jobs/taskletJob.xml - src/main/resources/org/springframework/batch/sample/football-job-context.xml - src/main/resources/jobs/groovyJob.xml - src/main/resources/jobs/iosample/jdbcPaging.xml - src/main/resources/jobs/partitionJdbcJob.xml - src/main/resources/jobs/jobStepSample.xml - src/main/resources/jobs/partitionFileJob.xml - src/test/resources/org/springframework/batch/sample/common/ColumnRangePartitionerTests-context.xml - src/test/resources/job-runner-context.xml - src/test/resources/org/springframework/batch/sample/JobStepFunctionalTests-context.xml - src/main/resources/jobs/mailJob.xml - - - - - true - false - - src/main/resources/data-source-context.xml - - - - - true - false - - src/main/resources/jobs/beanWrapperMapperSampleJob.xml - src/main/resources/data-source-context.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/delegatingJob.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/footballJob.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/jobs/infiniteLoopJob.xml - src/main/resources/data-source-context.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/multilineJob.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/parallelJob.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/restartSample.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/jobs/retrySample.xml - src/main/resources/data-source-context.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/tradeJob.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/quartz-job-launcher-context.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/jobs/compositeItemWriterSampleJob.xml - - - - - true - false - - src/main/resources/jobs/adhocLoopJob.xml - src/main/resources/data-source-context.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/staging-test-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/simple-job-launcher-context.xml - - - - - true - false - - src/main/resources/adhoc-job-launcher-context.xml - src/main/resources/data-source-context.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/headerFooterSample.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/simple-job-launcher-context.xml - - - - - true - false - - src/main/resources/jobs/customerFilterJob.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/data-source-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/hibernate-context.xml - src/main/resources/jobs/hibernateJob.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/skipSampleJob.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/skipSample-job-launcher-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/ioSampleJob.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/hibernate-context.xml - src/main/resources/jobs/iosample/hibernate.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/iosample/jdbcCursor.xml - src/main/resources/jobs/ioSampleJob.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/simple-job-launcher-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/iosample/multiLine.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/simple-job-launcher-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/iosample/multiRecordType.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/simple-job-launcher-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/iosample/jpa.xml - src/main/resources/jobs/ioSampleJob.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/simple-job-launcher-context.xml - - - - - true - false - - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/jobs/iosample/ibatis.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/simple-job-launcher-context.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/multilineOrderJob.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - src/main/resources/jobs/multilineOrderInputTokenizers.xml - src/main/resources/jobs/multilineOrderOutputAggregators.xml - src/main/resources/jobs/multilineOrderValidator.xml - - - - - true - false - - src/main/resources/data-source-context.xml - src/main/resources/jobs/restartFileSampleJob.xml - src/main/resources/simple-job-launcher-context.xml - src/main/resources/org/springframework/batch/sample/config/common-context.xml - - - - + + + 1 + + + + + + + src/main/resources/jobs/multilineJob.xml + src/main/resources/jobs/multilineOrderJob.xml + src/main/resources/jobs/tradeJob.xml + src/main/resources/jobs/restartSample.xml + src/main/resources/data-source-context.xml + src/main/resources/jobs/beanWrapperMapperSampleJob.xml + src/main/resources/jobs/adhocLoopJob.xml + src/main/resources/jobs/infiniteLoopJob.xml + src/main/resources/jobs/footballJob.xml + src/main/resources/jobs/delegatingJob.xml + src/main/resources/jobs/parallelJob.xml + src/main/resources/jobs/retrySample.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/adhoc-job-launcher-context.xml + src/main/resources/quartz-job-launcher-context.xml + src/main/resources/jobs/skipSampleJob.xml + src/main/resources/jobs/multilineOrderInputTokenizers.xml + src/main/resources/jobs/multilineOrderOutputAggregators.xml + src/main/resources/jobs/compositeItemWriterSampleJob.xml + src/main/resources/hibernate-context.xml + src/main/resources/staging-test-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + src/main/resources/jobs/headerFooterSample.xml + src/main/resources/jobs/customerFilterJob.xml + src/main/resources/skipSample-job-launcher-context.xml + src/main/resources/jobs/ioSampleJob.xml + src/main/resources/jobs/iosample/delimited.xml + src/main/resources/jobs/iosample/fixedLength.xml + src/main/resources/jobs/iosample/jdbcCursor.xml + src/main/resources/jobs/iosample/multiLine.xml + src/main/resources/jobs/iosample/multiRecordType.xml + src/main/resources/jobs/iosample/multiResource.xml + src/main/resources/jobs/iosample/xml.xml + src/main/resources/jobs/iosample/hibernate.xml + src/main/resources/jobs/hibernateJob.xml + src/main/resources/jobs/iosample/jpa.xml + src/test/resources/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDaoTests-context.xml + src/main/resources/jobs/loopFlowSample.xml + src/test/resources/org/springframework/batch/sample/common/StagingItemReaderTests-context.xml + src/test/resources/org/springframework/batch/sample/common/StagingItemWriterTests-context.xml + src/main/resources/jobs/multilineOrderValidator.xml + src/main/resources/jobs/restartFileSampleJob.xml + src/main/resources/jobs/taskletJob.xml + src/main/resources/org/springframework/batch/sample/football-job-context.xml + src/main/resources/jobs/groovyJob.xml + src/main/resources/jobs/iosample/jdbcPaging.xml + src/main/resources/jobs/partitionJdbcJob.xml + src/main/resources/jobs/jobStepSample.xml + src/main/resources/jobs/partitionFileJob.xml + src/test/resources/org/springframework/batch/sample/common/ColumnRangePartitionerTests-context.xml + src/test/resources/job-runner-context.xml + src/test/resources/org/springframework/batch/sample/JobStepFunctionalTests-context.xml + src/main/resources/jobs/mailJob.xml + src/main/resources/META-INF/spring/jobs/amqp/amqp-example-job-beans.xml + src/main/resources/jobs/amqp-example-job.xml + src/main/resources/META-INF/spring/jobs/amqp/amqp-example-job.xml + src/main/resources/META-INF/spring/config-beans.xml + src/main/resources/META-INF/spring/jobs/messaging/rabbitmq-beans.xml + src/main/resources/jobs/iosample/repository.xml + java:org.springframework.batch.sample.config.DataSourceConfiguration + java:org.springframework.batch.sample.config.JobRunnerConfiguration + java:org.springframework.batch.sample.config.RetrySampleConfiguration + + + + + true + false + + src/main/resources/data-source-context.xml + + + + + + + true + false + + src/main/resources/jobs/beanWrapperMapperSampleJob.xml + src/main/resources/data-source-context.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/delegatingJob.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/footballJob.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/jobs/infiniteLoopJob.xml + src/main/resources/data-source-context.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/multilineJob.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/parallelJob.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/restartSample.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/jobs/retrySample.xml + src/main/resources/data-source-context.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/tradeJob.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/quartz-job-launcher-context.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + src/main/resources/jobs/compositeItemWriterSampleJob.xml + + + + + + + true + false + + src/main/resources/jobs/adhocLoopJob.xml + src/main/resources/data-source-context.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/staging-test-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + src/main/resources/simple-job-launcher-context.xml + + + + + + + true + false + + src/main/resources/adhoc-job-launcher-context.xml + src/main/resources/data-source-context.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/headerFooterSample.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + src/main/resources/simple-job-launcher-context.xml + + + + + + + true + false + + src/main/resources/jobs/customerFilterJob.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/data-source-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + src/main/resources/hibernate-context.xml + src/main/resources/jobs/hibernateJob.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/skipSampleJob.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + src/main/resources/skipSample-job-launcher-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/ioSampleJob.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/hibernate-context.xml + src/main/resources/jobs/iosample/hibernate.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/iosample/jdbcCursor.xml + src/main/resources/jobs/ioSampleJob.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + src/main/resources/simple-job-launcher-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/iosample/multiLine.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + src/main/resources/simple-job-launcher-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/iosample/multiRecordType.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + src/main/resources/simple-job-launcher-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/iosample/jpa.xml + src/main/resources/jobs/ioSampleJob.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + src/main/resources/simple-job-launcher-context.xml + + + + + + + true + false + + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/simple-job-launcher-context.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/multilineOrderJob.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + src/main/resources/jobs/multilineOrderInputTokenizers.xml + src/main/resources/jobs/multilineOrderOutputAggregators.xml + src/main/resources/jobs/multilineOrderValidator.xml + + + + + + + true + false + + src/main/resources/data-source-context.xml + src/main/resources/jobs/restartFileSampleJob.xml + src/main/resources/simple-job-launcher-context.xml + src/main/resources/org/springframework/batch/sample/config/common-context.xml + + + + + + diff --git a/spring-batch-samples/README.md b/spring-batch-samples/README.md new file mode 100644 index 0000000000..42291a8ad6 --- /dev/null +++ b/spring-batch-samples/README.md @@ -0,0 +1,921 @@ +## Spring Batch Samples + +### Overview + +There is considerable variability in the types of input and output +formats in batch jobs. There are also a number of options to consider +in terms of how the types of strategies that will be used to handle +skips, recovery, and statistics. However, when approaching a new +batch job there are a few standard questions to answer to help +determine how the job will be written and how to use the services +offered by the spring batch framework. Consider the following: + +* How do I configure this batch job? In the samples the pattern is to follow the convention of `[nameOf]Job.xml`. Each sample identifies the XML definition used to configure the job. Job configurations that use a common execution environment have many common items in their respective configurations. +* What is the input source? Each sample batch job identifies its input source. +* What is my output source? Each sample batch job identifies its output source. +* How are records read and validated from the input source? This refers to the input type and its format (e.g. flat file with fixed position, comma separated or XML, etc.) +* What is the policy of the job if a input record fails the validation step? The most important aspect is whether the record can be skipped so that processing can be continued. +* How do I process the data and write to the output source? How and what business logic is being applied to the processing of a record? +* How do I recover from an exception while operating on the output source? There are numerous recovery strategies that can be applied to handling errors on transactional targets. The samples provide a feeling for some of the choices. +* Can I restart the job and if so which strategy can I use to restart the job? The samples show some of the options available to jobs and what the decision criteria is for the respective choices. + +Here is a list of samples with checks to indicate which features each one demonstrates: + +Job/Feature | skip | retry | restart | automatic mapping | asynch launch | validation | delegation | write behind | non-squenetial | asynch process | filtering +:------------------------------------------------ | :--: | :---: | :-----: | :---------------: | :-----------: | :--------: | :--------: | :----------: | :------------: | :------------: | :-------: +[Adhoc Loop and JMX Demo](#adhoc-loop-and-jmx-demo) | | | | | X | | | | | | +[Amqp Job Sample](#amqp-job-sample) | | | | | | | | | | X | +[BeanWrapperMapper Sample](#beanwrappermapper-sample) | | | | X | | | | | | | +[Composite ItemWriter Sample](#composite-itemwriter-sample) | | | | | | | X | | | | +[Customer Filter Sample](#customer-filter-sample) | | | | | | | | | | | X +[Delegating Sample](#delegating-sample) | | | | | | | X | | | | +[Football Job](#football-job) | | | | | | | | | | | +[Header Footer Sample](#header-footer-sample) | | | | | | | | | | | +[Hibernate Sample](#hibernate-sample) | | X | | | | | | X | | | +IO Sample Job | | | | | | X | | X | | | +[Infinite Loop Sample](#infinite-loop-sample) | | | | | | X | | | X | | +[Loop Flow Sample](#loop-flow-sample) | | | | | | | | | | | +[Multiline](#multiline) | | | | | | | X | | | | +[Multiline Order Job](#multiline-order-job) | | | | | | | X | | | | +[Parallel Sample](#parallel-sample) | | | | | | | | | | X | +[Partitioning Sample](#partitioning-sample) | | | | | | | | | | X | +[Remote Chunking Sample](#remote-chunking-sample) | | | | | | | | | | X | +[Quartz Sample](#quartz-sample) | | | | | X | | | | | | +[Restart Sample](#restart-sample) | | | X | | | | | | | | +[Retry Sample](#retry-sample) | | X | | | | | | | | | +[Skip Sample](#skip-sample) | X | | | | | | | | | | +[Trade Job](#trade-job) | | | | | | X | | | | | + +The IO Sample Job has a number of special instances that show different IO features using the same job configuration but with different readers and writers: + +Job/Feature | delimited input | fixed-length input | xml input | db paging input | db cursor input | delimited output | fixed-length output | xml output | db output | multiple files | multi-line | mulit-record +:-------------------------- | :-------------: | :----------------: | :-------: | :-------------: | :-------------: | :--------------: | :-----------------: | :--------: | :-------: | :------------: | :--------: | :----------: +delimited | x | | | | | | | x | | | | +[Fixed Length Import Job](#fixed-length-import-job) | | x | | | | | | | x | | | +[Hibernate Sample](#hibernate-sample) | | | | | x | | | | | | x | +[Jdbc Cursor and Batch Update](#jdbc-cursor-and-batch-update) | | | | | x | | | | | | x | +jpa | | | | x | | | | | | | x | +[Multiline](#multiline) | x | | | | | | | x | | | x | +multiRecordtype | | x | | | | | | | x | | | x +multiResource | x | | | | | | | x | | | | x +[XML Input Output](#xml-input-output) | | | x | | | | | | | x | | + + +### Common Sample Source Structures + +The easiest way to launch a sample job in Spring Batch is to open up +a unit test in your IDE and run it directly. Each sample has a +separate test case in the `org.springframework.batch.samples` +package. The name of the test case is `[JobName]FunctionalTests`. + +**Note:** The test cases do not ship in the samples jar file, but they +are in the .zip distribution and in the source code, which +you can download using subversion (or browse in a web browser if +you need to). See here for a link to the source code repository. + +You can also use the same Spring configuration as the unit test to +launch the job via a main method in `CommmandLineJobRunner`. +The samples source code has an Eclipse launch configuration to do +this, taking the hassle out of setting up a classpath to run the +job. + +### Adhoc Loop and JMX Demo + +This job is simply an infinite loop. It runs forever so it is +useful for testing features to do with stopping and starting jobs. +It is used, for instance, as one of the jobs that can be run from +JMX using the Eclipse launch configuration "jmxLauncher". + +The JMX launcher uses an additional XML configuration file +(adhoc-job-launcher-context.xml) to set up a `JobOperator` for +running jobs asynchronously (i.e. in a background thread). This +follows the same pattern as the [Quartz sample](#quartz-sample), so see that section + for more details of the `JobLauncher` configuration. + +The rest of the configuration for this demo consists of exposing +some components from the application context as JMX managed beans. +The `JobOperator` is exposed so that it can be controlled from a +remote client (such as JConsole from the JDK) which does not have +Spring Batch on the classpath. See the Spring Core Reference Guide +for more details on how to customise the JMX configuration. + +### Jdbc Cursor and Batch Update + +The purpose of this sample is to show to usage of the +`JdbcCursorItemReader` and the `JdbcBatchItemWriter` to make +efficient updates to a database table. + +The `JdbcBatchItemWriter` accepts a special form of +`PreparedStatementSetter` as a (mandatory) dependency. This is +responsible for copying fields from the item to be written to a +`PreparedStatement` matching the SQL query that has been +injected. The implementation of the +`CustomerCreditUpdatePreparedStatementSetter` shows best +practice of keeping all the information needed for the execution in +one place, since it contains a static constant value (`QUERY`) +which is used to configure the query for the writer. + +### Amqp Job Sample + +This sample shows the use of Spring Batch to write to an `AmqpItemWriter`. +The `AmqpItemReader` and Writer were contributed by Chris Schaefer. +It is modeled after the `JmsItemReader` / Writer implementations, which +are popular models for remote chunking. It leverages the `AmqpTemplate`. + +This example requires the env to have a copy of rabbitmq installed +and running. The standard dashboard can be used to see the traffic +from the `MessageProducer` to the `AmqpItemWriter`. Make sure you +launch the `MessageProducer` before launching the test. + +### BeanWrapperMapper Sample + +This sample shows the use of automatic mapping from fields in a file +to a domain object. The `Trade` and `Person` objects needed +by the job are created from the Spring configuration using prototype +beans, and then their properties are set using the +`BeanWrapperFieldSetMapper`, which sets properties of the +prototype according to the field names in the file. + +Nested property paths are resolved in the same way as normal Spring +binding occurs, but with a little extra leeway in terms of spelling +and capitalisation. Thus for instance, the `Trade` object has a +property called `customer` (lower case), but the file has been +configured to have a column name `CUSTOMER` (upper case), and +the mapper will accept the values happily. Underscores instead of +camel-casing (e.g. `CREDIT_CARD` instead of `creditCard`) +also work. + +### Composite ItemWriter Sample + +This shows a common use case using a composite pattern, composing +instances of other framework readers or writers. It is also quite +common for business-specific readers or writers to wrap +off-the-shelf components in a similar way. + +In this job the composite pattern is used just to make duplicate +copies of the output data. The delegates for the +`CompositeItemWriter` have to be separately registered as +streams in the `Step` where they are used, in order for the step +to be restartable. This is a common feature of all delegate +patterns. + +### Customer Filter Sample + +This shows the use of the `ItemProcessor` to filter out items by +returning null. When an item is filtered it leads to an increment +in the `filterCount` in the step execution. + +### Delegating Sample + +This sample shows the delegate pattern again, and also the +`ItemReaderAdapter` which is used to adapt a POJO to the +`ItemReader` interface. + +### Fixed Length Import Job + +The goal is to demonstrate a typical scenario of importing data +from a fixed-length file to database + +This job shows a typical scenario, when reading input data and +processing the data is cleanly separated. The data provider is +responsible for reading input and mapping each record to a domain +object, which is then passed to the module processor. The module +processor handles the processing of the domain objects, in this case +it only writes them to database. + +In this example we are using a simple fixed length record structure +that can be found in the project at +`data/iosample/input`. A considerable amount of +thought can go into designing the folder structures for batch file +management. The fixed length records look like this: + + UK21341EAH4597898.34customer1 + UK21341EAH4611218.12customer2 + UK21341EAH4724512.78customer2 + UK21341EAH48108109.25customer3 + UK21341EAH49854123.39customer4 + +Looking back to the configuration file you will see where this is +documented in the property of the `FixedLengthTokenizer`. You can +infer the following properties: + + +FieldName | Length +--------- | :----: +ISIN | 12 +Quantity | 3 +Price | 5 +Customer | 9 + +*Output target:* database - writes the data to database using a DAO +object + + +### Football Job + +This is a (American) Football statistics loading job. We gave it the +id of `footballJob` in our configuration file. Before diving +into the batch job, we'll examine the two input files that need to +be loaded. First is `player.csv`, which can be found in the +samples project under +src/main/resources/data/footballjob/input/. Each line within this +file represents a player, with a unique id, the player’s name, +position, etc: + + AbduKa00,Abdul-Jabbar,Karim,rb,1974,1996 + AbduRa00,Abdullah,Rabih,rb,1975,1999 + AberWa00,Abercrombie,Walter,rb,1959,1982 + AbraDa00,Abramowicz,Danny,wr,1945,1967 + AdamBo00,Adams,Bob,te,1946,1969 + AdamCh00,Adams,Charlie,wr,1979,2003 + ... + +One of the first noticeable characteristics of the file is that each +data element is separated by a comma, a format most are familiar +with known as 'CSV'. Other separators such as pipes or semicolons +could just as easily be used to delineate between unique +elements. In general, it falls into one of two types of flat file +formats: delimited or fixed length. (The fixed length case was +covered in the `fixedLengthImportJob`. + +The second file, 'games.csv' is formatted the same as the previous +example, and resides in the same directory: + + AbduKa00,1996,mia,10,nwe,0,0,0,0,0,29,104,,16,2 + AbduKa00,1996,mia,11,clt,0,0,0,0,0,18,70,,11,2 + AbduKa00,1996,mia,12,oti,0,0,0,0,0,18,59,,0,0 + AbduKa00,1996,mia,13,pit,0,0,0,0,0,16,57,,0,0 + AbduKa00,1996,mia,14,rai,0,0,0,0,0,18,39,,7,0 + AbduKa00,1996,mia,15,nyg,0,0,0,0,0,17,96,,14,0 + ... + +Each line in the file represents an individual player's performance +in a particular game, containing such statistics as passing yards, +receptions, rushes, and total touchdowns. + +Our example batch job is going to load both files into a database, +and then combine each to summarise how each player performed for a +particular year. Although this example is fairly trivial, it shows +multiple types of input, and the general style is a common batch +scenario. That is, summarising a very large dataset so that it can +be more easily manipulated or viewed by an online web-based +application. In an enterprise solution the third step, the reporting +step, could be implemented through the use of Eclipse BIRT or one of +the many Java Reporting Engines. Given this description, we can then +easily divide our batch job up into 3 'steps': one to load the +player data, one to load the game data, and one to produce a summary +report: + +**Note:** One of the nice features of Spring is a project called +Spring IDE. When you download the project you can install Spring +IDE and add the Spring configurations to the IDE project. This is +not a tutorial on Spring IDE but the visual view into Spring beans +is helpful in understanding the structure of a Job +Configuration. Spring IDE produces the following diagram: + +![Spring Batch Football Object Model](src/site/resources/images/spring-batch-football-graph.jpg "Spring Batch Football Object Model") + +This corresponds exactly with the `footballJob.xml` job +configuration file which can be found in the jobs folder under +`src/main/resources`. When you drill down into the football job +you will see that the configuration has a list of steps: + + + + + + + + + +A step is run until there is no more input to process, which in +this case would mean that each file has been completely +processed. To describe it in a more narrative form: the first step, +playerLoad, begins executing by grabbing one line of input from the +file, and parsing it into a domain object. That domain object is +then passed to a dao, which writes it out to the PLAYERS table. This +action is repeated until there are no more lines in the file, +causing the playerLoad step to finish. Next, the gameLoad step does +the same for the games input file, inserting into the GAMES +table. Once finished, the playerSummarization step can begin. Unlike +the first two steps, playerSummarization input comes from the +database, using a Sql statement to combine the GAMES and PLAYERS +table. Each returned row is packaged into a domain object and +written out to the PLAYER_SUMMARY table. + +Now that we've discussed the entire flow of the batch job, we can +dive deeper into the first step: playerLoad: + + + + + + + + + + + + + + + + +The root bean in this case is a `SimpleStepFactoryBean`, which +can be considered a 'blueprint' of sorts that tells the execution +environment basic details about how the batch job should be +executed. It contains four properties: (others have been removed for +greater clarity) commitInterval, startLimit, itemReader and +itemWriter . After performing all necessary startup, the framework +will periodically delegate to the reader and writer. In this way, +the developer can remain solely concerned with their business +logic. + +* *ItemReader* – the item reader is the source of the information +pipe. At the most basic level input is read in from an input +source, parsed into a domain object and returned. In this way, the +good batch architecture practice of ensuring all data has been +read before beginning processing can be enforced, along with +providing a possible avenue for reuse. + +* *ItemWriter* – this is the business logic. At a high level, +the item writer takes the item returned from the reader +and 'processes' it. In our case it's a data access object that is +simply responsible for inserting a record into the PLAYERS +table. As you can see the developer does very little. + +The application developer simply provides a job configuration with a +configured number of steps, an ItemReader associated to some type +of input source, and ItemWriter associated to some type of +output source and a little mapping of data from flat records to +objects and the pipe is ready wired for processing. + +Another property in the step configuration, the commitInterval, +gives the framework vital information about how to control +transactions during the batch run. Due to the large amount of data +involved in batch processing, it is often advantageous to 'batch' +together multiple logical units of work into one transaction, since +starting and committing a transaction is extremely expensive. For +example, in the playerLoad step, the framework calls read() on the +item reader. The item reader reads one record from the file, and +returns a domain object representation which is passed to the +processor. The writer then writes the one record to the database. It +can then be said that one iteration = one call to +`ItemReader.read()` = one line of the file. Therefore, setting +your commitInterval to 5 would result in the framework committing a +transaction after 5 lines have been read from the file, with 5 +resultant entries in the PLAYERS table. + +Following the general flow of the batch job, the next step is to +describe how each line of the file will be parsed from its string +representation into a domain object. The first thing the provider +will need is an `ItemReader`, which is provided as part of the Spring +Batch infrastructure. Because the input is flat-file based, a +`FlatFileItemReader` is used: + + + + + + + + + + + + + +There are three required dependencies of the item reader; the first +is a resource to read in, which is the file to process. The second +dependency is a `LineTokenizer`. The interface for a +`LineTokenizer` is very simple, given a string; it will return a +`FieldSet` that wraps the results from splitting the provided +string. A `FieldSet` is Spring Batch's abstraction for flat file +data. It allows developers to work with file input in much the same +way as they would work with database input. All the developers need +to provide is a `FieldSetMapper` (similar to a Spring +`RowMapper`) that will map the provided `FieldSet` into an +`Object`. Simply by providing the names of each token to the +`LineTokenizer`, the `ItemReader` can pass the +`FieldSet` into our `PlayerMapper`, which implements the +`FieldSetMapper` interface. There is a single method, +`mapLine()`, which maps `FieldSet`s the same way that +developers are comfortable mapping `ResultSet`s into Java +`Object`s, either by index or field name. This behaviour is by +intention and design similar to the `RowMapper` passed into a +`JdbcTemplate`. You can see this below: + + public class PlayerMapper implements FieldSetMapper { + + public Object mapLine(FieldSet fs) { + + if(fs == null){ + return null; + } + + Player player = new Player(); + player.setID(fs.readString("ID")); + player.setLastName(fs.readString("lastName")); + player.setFirstName(fs.readString("firstName")); + player.setPosition(fs.readString("position")); + player.setDebutYear(fs.readInt("debutYear")); + player.setBirthYear(fs.readInt("birthYear")); + + return player; + } + } + +The flow of the `ItemReader`, in this case, starts with a call +to read the next line from the file. This is passed into the +provided `LineTokenizer`. The `LineTokenizer` splits the +line at every comma, and creates a `FieldSet` using the created +`String` array and the array of names passed in. + +**Note:** it is only necessary to provide the names to create the +`FieldSet` if you wish to access the field by name, rather +than by index. + +Once the domain representation of the data has been returned by the +provider, (i.e. a `Player` object in this case) it is passed to +the `ItemWriter`, which is essentially a Dao that uses a Spring +`JdbcTemplate` to insert a new row in the PLAYERS table. + +The next step, gameLoad, works almost exactly the same as the +playerLoad step, except the games file is used. + +The final step, playerSummarization, is much like the previous two +steps, in that it reads from a reader and returns a domain object to +a writer. However, in this case, the input source is the database, +not a file: + + + + + + + + + SELECT games.player_id, games.year_no, SUM(COMPLETES), + SUM(ATTEMPTS), SUM(PASSING_YARDS), SUM(PASSING_TD), + SUM(INTERCEPTIONS), SUM(RUSHES), SUM(RUSH_YARDS), + SUM(RECEPTIONS), SUM(RECEPTIONS_YARDS), SUM(TOTAL_TD) + from games, players where players.player_id = + games.player_id group by games.player_id, games.year_no + + + + +The `JdbcCursorItemReader` has three dependences: + +* A `DataSource` +* The `RowMapper` to use for each row. +* The Sql statement used to create the cursor. + +When the step is first started, a query will be run against the +database to open a cursor, and each call to `itemReader.read()` +will move the cursor to the next row, using the provided +`RowMapper` to return the correct object. As with the previous +two steps, each record returned by the provider will be written out +to the database in the PLAYER_SUMMARY table. Finally to run this +sample application you can execute the JUnit test +`FootballJobFunctionalTests`, and you'll see an output showing +each of the records as they are processed. Please keep in mind that +AoP is used to wrap the `ItemWriter` and output each record as it +is processed to the logger, which may impact performance. + +### Header Footer Sample + +This sample shows the use of callbacks and listeners to deal with +headers and footers in flat files. It uses two custom callbacks: + +* `HeaderCopyCallback`: copies the header of a file from the +input to the output. +* `SummaryFooterCallback`: creates a summary footer at the end +of the output file. + +### Hibernate Sample + +The purpose of this sample is to show a typical usage of Hibernate +as an ORM tool in the input and output of a job. + +The job uses a `HibernateCursorItemReader` for the input, where +a simple HQL query is used to supply items. It also uses a +non-framework `ItemWriter` wrapping a DAO, which perhaps was +written as part of an online system. + +The output reliability and robustness are improved by the use of +`Session.flush()` inside `ItemWriter.write()`. This +"write-behind" behaviour is provided by Hibernate implicitly, but we +need to take control of it so that the skip and retry features +provided by Spring Batch can work effectively. + +### Infinite Loop Sample + +This sample has a single step that is an infinite loop, reading and +writing fake data. It is used to demonstrate stop signals and +restart capabilities. + +### Loop Flow Sample + +Shows how to implement a job that repeats one of its steps up to a +limit set by a `JobExecutionDecider`. + +### Multiline + +The goal of this sample is to show some common tricks with multiline +records in file input jobs. + +The input file in this case consists of two groups of trades +delimited by special lines in a file (BEGIN and END): + + BEGIN + UK21341EAH4597898.34customer1 + UK21341EAH4611218.12customer2 + END + BEGIN + UK21341EAH4724512.78customer2 + UK21341EAH4810809.25customer3 + UK21341EAH4985423.39customer4 + END + +The goal of the job is to operate on the two groups, so the item +type is naturally `List. To get these items delivered +from an item reader we employ two components from Spring Batch: the +`AggregateItemReader` and the +`PrefixMatchingCompositeLineTokenizer`. The latter is +responsible for recognising the difference between the trade data +and the delimiter records. The former is responsible for +aggregating the trades from each group into a `List` and handing +out the list from its `read()` method. To help these components +perform their responsibilities we also provide some business +knowledge about the data in the form of a `FieldSetMapper` +(`TradeFieldSetMapper`). The `TradeFieldSetMapper` checks +its input for the delimiter fields (BEGIN, END) and if it detects +them, returns the special tokens that `AggregateItemReader` +needs. Otherwise it maps the input into a `Trade` object. + +### Multiline Order Job + +The goal is to demonstrate how to handle a more complex file input +format, where a record meant for processing includes nested records +and spans multiple lines + +The input source is file with multiline records. +`OrderItemReader` is an example of a non-default programmatic +item reader. It reads input until it detects that the multiline +record has finished and encapsulates the record in a single domain +object. + +The output target is a file with multiline records. The concrete +`ItemWriter` passes the object to a an injected 'delegate +writer' which in this case writes the output to a file. The writer +in this case demonstrates how to write multiline output using a +custom aggregator transformer. + +### Parallel Sample + +The purpose of this sample is to show multi-threaded step execution +using the Process Indicator pattern. + +The job reads data from the same file as the [Fixed Length Import sample](#fixed-length-import-job), but instead of +writing it out directly it goes through a staging table, and the +staging table is read in a multi-threaded step. Note that for such +a simple example where the item processing was not expensive, there +is unlikely to be much if any benefit in using a multi-threaded +step. + +Multi-threaded step execution is easy to configure using Spring +Batch, but there are some limitations. Most of the out-of-the-box +`ItemReader` and `ItemWriter` implementations are not +designed to work in this scenario because they need to be +restartable and they are also stateful. There should be no surprise +about this, and reading a file (for instance) is usually fast enough +that multi-threading that part of the process is not likely to +provide much benefit, compared to the cost of managing the state. + +The best strategy to cope with restart state from multiple +concurrent threads depends on the kind of input source involved: + +* For file-based input (and output) restart sate is practically +impossible to manage. Spring Batch does not provide any features +or samples to help with this use case. +* With message middleware input it is trivial to manage restarts, +since there is no state to store (if a transaction rolls back the +messages are returned to the destination they came from). +* With database input state management is still necessary, but it +isn't particularly difficult. The easiest thing to do is rely on +a Process Indicator in the input data, which is a column in the +data indicating for each row if it has been processed or not. The +flag is updated inside the batch transaction, and then in the case +of a failure the updates are lost, and the records will show as +un-processed on a restart. + +This last strategy is implemented in the `StagingItemReader`. +Its companion, the `StagingItemWriter` is responsible for +setting up the data in a staging table which contains the process +indicator. The reader is then driven by a simple SQL query that +includes a where clause for the processed flag, i.e. + + SELECT ID FROM BATCH_STAGING WHERE JOB_ID=? AND PROCESSED=? ORDER BY ID + +It is then responsible for updating the processed flag (which +happens inside the main step transaction). + +### Partitioning Sample + +The purpose of this sample is to show multi-threaded step execution +using the `PartitionHandler` SPI. The example uses a +`TaskExecutorPartitionHandler` to spread the work of reading +some files across multiple threads, with one `Step` execution +per thread. The key components are the `PartitionStep` and the +`MultiResourcePartitioner` which is responsible for dividing up +the work. Notice that the readers and writers in the `Step` +that is being partitioned are step-scoped, so that their state does +not get shared across threads of execution. + +### Remote Partitioning Sample + +This sample shows how to configure a remote partitioning job. The manager step +uses a `MessageChannelPartitionHandler` to send partitions to and receive +replies from workers. Two examples are shown: + +* A manager step that polls the job repository to see if all workers have finished +their work +* A manager step that aggregates replies from workers to notify work completion + +The sample uses an embedded JMS broker and an embedded database for simplicity +but any option supported via Spring Integration for communication is technically +acceptable. + +### Remote Chunking Sample + +This sample shows how to configure a remote chunking job. The manager step will +read numbers from 1 to 6 and send two chunks ({1, 2, 3} and {4, 5, 6}) to workers +for processing and writing. + +This example shows how to use: + +* the `RemoteChunkingManagerStepBuilderFactory` to create a manager step +* the `RemoteChunkingWorkerBuilder` to configure an integration flow on the worker side. + +The sample uses an embedded JMS broker as a communication middleware between the +manager and workers. The usage of an embedded broker is only for simplicity's sake, +the communication between the manager and workers is still done through JMS queues +and Spring Integration channels and messages are sent over the wire through a TCP port. + +### Quartz Sample + +The goal is to demonstrate how to schedule job execution using +Quartz scheduler. In this case there is no unit test to launch the +sample because it just re-uses the football job. There is a main +method in `JobRegistryBackgroundJobRunner` and an Eclipse launch +configuration which runs it with arguments to pick up the football +job. + +The additional XML configuration for this job is in +`quartz-job-launcher.xml`, and it also re-uses +`footballJob.xml` + +The configuration declares a `JobLauncher` bean. The launcher +bean is different from the other samples only in that it uses an +asynchronous task executor, so that the jobs are launched in a +separate thread to the main method: + + + + + + + + +Also, a Quartz `JobDetail` is defined using a Spring +`JobDetailBean` as a convenience. + + + + + + + + + + + + + +Finally, a trigger with a scheduler is defined that will launch the +job detail every 10 seconds: + + + + + + + + + + +The job is thus scheduled to run every 10 seconds. In fact it +should be successful on the first attempt, so the second and +subsequent attempts should through a +`JobInstanceAlreadyCompleteException`. In a production system, +the job detail would probably be modified to account for this +exception (e.g. catch it and re-submit with a new set of job +parameters). The point here is that Spring Batch guarantees that +the job execution is idempotent - you can never inadvertently +process the same data twice. + +### Restart Sample + +The goal of this sample is to show how a job can be restarted after +a failure and continue processing where it left off. + +To simulate a failure we "fake" a failure on the fourth record +though the use of a sample component +`ExceptionThrowingItemReaderProxy`. This is a stateful reader +that counts how many records it has processed and throws a planned +exception in a specified place. Since we re-use the same instance +when we restart the job it will not fail the second time. + +### Retry Sample + +The purpose of this sample is to show how to use the automatic retry +capabilities of Spring Batch. + +The retry is configured in the step through the +`SkipLimitStepFactoryBean`: + + + ... + + + + +Failed items will cause a rollback for all `Exception` types, up +to a limit of 3 attempts. On the 4th attempt, the failed item would +be skipped, and there would be a callback to a +`ItemSkipListener` if one was provided (via the "listeners" +property of the step factory bean). + +An `ItemReader` is provided that will generate unique +`Trade` data by just incrementing a counter. Note that it uses +the counter in its `mark()` and `reset()` methods so that +the same content is returned after a rollback. The same content is +returned, but the instance of `Trade` is different, which means +that the implementation of `equals()` in the `Trade` object +is important. This is because to identify a failed item on retry +(so that the number of attempts can be counted) the framework by +default uses `Object.equals()` to compare the recently failed +item with a cache of previously failed items. Without implementing +a field-based `equals()` method for the domain object, our job +will spin round the retry for potentially quite a long time before +failing because the default implementation of `equals()` is +based on object reference, not on field content. + +### Skip Sample + +The purpose of this sample is to show how to use the skip features +of Spring Batch. Since skip is really just a special case of retry +(with limit 0), the details are quite similar to the [Retry +Sample](#retry-sample), but the use case is less artificial, since it +is based on the [Trade Sample](#trade-job). + +The failure condition is still artificial, since it is triggered by +a special `ItemWriter` wrapper (`ItemTrackingItemWriter`). +The plan is that a certain item (the third) will fail business +validation in the writer, and the system can then respond by +skipping it. We also configure the step so that it will not roll +back on the validation exception, since we know that it didn't +invalidate the transaction, only the item. This is done through the +transaction attribute: + + + + + + .... + + +The format for the transaction attribute specification is given in +the Spring Core documentation (e.g. see the Javadocs for +[TransactionAttributeEditor](https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/transaction/interceptor/TransactionAttributeEditor.html)). + +### Tasklet Job + +The goal is to show the simplest use of the batch framework with a +single job with a single step, which cleans up a directory and runs +a system command. + +*Description:* The +`Job` itself is defined by the bean definition with +`id="taskletJob"`. In this example we have two steps. + +* The first step defines a tasklet that is responsible for +clearing out a directory though a custom `Tasklet`. Each +tasklet has an `execute()` method which is called by the +step. All processing of business data should be handled by this +method. +* The second step uses another tasklet to execute a system (OS) +command line. + +You can visualise the Spring configuration of a job through +Spring-IDE. See [Spring IDE](https://spring.io/tools). The +source view of the configuration is as follows: + + + + + + + + + + + + + + + + + + + + + + + + + + + + +For simplicity we are only displaying the job configuration itself +and leaving out the details of the supporting batch execution +environment configuration. + +### Trade Job + +The goal is to show a reasonably complex scenario, that would +resemble the real-life usage of the framework. + +This job has 3 steps. First, data about trades are imported from a +file to database. Second, the trades are read from the database and +credit on customer accounts is decreased appropriately. Last, a +report about customers is exported to a file. + +### XML Input Output + +The goal here is to show the use of XML input and output through +streaming and Spring OXM marshallers and unmarshallers. + +The job has a single step that copies `Trade` data from one XML +file to another. It uses XStream for the object XML conversion, +because this is simple to configure for basic use cases like this +one. See +[Spring OXM documentation](https://docs.spring.io/spring/docs/current/spring-framework-reference/data-access.html#oxm) for details of other options. + +### Batch metrics with Micrometer + +This sample shows how to use [Micrometer](https://micrometer.io) to collect batch metrics in Spring Batch. +It uses [Prometheus](https://prometheus.io) as the metrics back end and [Grafana](https://grafana.com) as the front end. +The sample consists of two jobs: + +* `job1` : Composed of two tasklets that print `hello` and `world` +* `job2` : Composed of single chunk-oriented step that reads and writes a random number of items + +These two jobs are run repeatedly at regular intervals and might fail randomly for demonstration purposes. + +This sample requires [docker compose](https://docs.docker.com/compose/) to start the monitoring stack. +To run the sample, please follow these steps: + +``` +$>cd spring-batch-samples/src/grafana +$>docker-compose up -d +``` + +This should start the required monitoring stack: + +* Prometheus server on port `9090` +* Prometheus push gateway on port `9091` +* Grafana on port `3000` + +Once started, you need to [configure Prometheus as data source in Grafana](https://grafana.com/docs/features/datasources/prometheus/) +and import the ready-to-use dashboard in `spring-batch-samples/src/grafana/spring-batch-dashboard.json`. + +Finally, run the `org.springframework.batch.sample.metrics.BatchMetricsApplication` +class without any argument to start the sample. diff --git a/spring-batch-samples/hsql-manager.launch b/spring-batch-samples/hsql-manager.launch deleted file mode 100644 index 339e9b47be..0000000000 --- a/spring-batch-samples/hsql-manager.launch +++ /dev/null @@ -1,15 +0,0 @@ - - - - - - - - - - - - - - - diff --git a/spring-batch-samples/hsql-server.launch b/spring-batch-samples/hsql-server.launch deleted file mode 100644 index 3b569d6d3e..0000000000 --- a/spring-batch-samples/hsql-server.launch +++ /dev/null @@ -1,16 +0,0 @@ - - - - - - - - - - - - - - - - diff --git a/spring-batch-samples/jalopy_customized.xml b/spring-batch-samples/jalopy_customized.xml deleted file mode 100644 index 459f95a140..0000000000 --- a/spring-batch-samples/jalopy_customized.xml +++ /dev/null @@ -1,436 +0,0 @@ - - - - - 14 - - - - - true - - - [A-Z][a-zA-Z0-9]+ - [A-Z][a-zA-Z0-9]+ - - - [a-z][\w]+ - [a-z][\w]+ - [a-zA-Z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - [a-zA-Z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - [a-zA-Z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - [a-zA-Z][\w]+ - - [A-Z][a-zA-Z0-9]+ - \w+ - - [a-z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - [a-z][\w]+ - - [a-z]+(?:\.[a-z]+)* - - [a-z][\w]+ - [a-z][\w]+ - - [a-z][\w]* - - - false - false - false - false - false - false - false - false - false - false - false - false - false - false - false - false - false - false - false - - - - 6 - - - - 30000 - 30000 - 30000 - 30000 - 30000 - 30000 - - true - - - 1 - - - - true - false - true - false - false - false - - - bak - 0 - - - - 1 - 0 - 1 - 0 -
        1
        -
        0
        - 1 - 2 - 1 - 1 -
        - - 1 - 0 - 1 - - 1 - 1 - 1 - - 1 - 1 -
        0
        -
        0
        -
        - 1 -
        - - false - - - - false - false - - - false - false - true - false - - - true - false - false - false - false - - - false - false - - - - true - true - - - - true - - - - false - true - true - - true - - 0 - 0 - 0 - 0 - - false - true - - false - - - - @task - - - /**| * DOCUMENT ME!| *| * @author $author$| * @version $Revision: 1.9 $| */ - - */ - * @throws $exceptionType$ DOCUMENT ME! - * @param $paramType$ DOCUMENT ME! - /**| * Creates a new $objectType$ object. - - /**| * DOCUMENT ME!| *| * @author $author$| * @version $Revision: 1.9 $| */ - - */ - * @throws $exceptionType$ DOCUMENT ME! - * @param $paramType$ DOCUMENT ME! - * @return DOCUMENT ME! - /**| * DOCUMENT ME! - - /** DOCUMENT ME! */ - - - - false - false - false - - - - - false - false - - Annotations - Inner Classes - Constructors - Enumerations - Instance fields - Instance initializers - Inner Interfaces - Methods - Static fields/initializers - - - - -
        - false - - 0 - false -
        -
        - true - Geotools2 - OpenSource mapping toolkit - 0 - /*| * GeoTools - OpenSource mapping toolkit| * http://geotools.org| * (C) 2002-2006, GeoTools Project Managment Committee (PMC)| *| * This library is free software; you can redistribute it and/or| * modify it under the terms of the GNU Lesser General Public| * License as published by the Free Software Foundation;| * version 2.1 of the License.| *| * This library is distributed in the hope that it will be useful,| * but WITHOUT ANY WARRANTY; without even the implied warranty of| * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU| * Lesser General Public License for more details.| */ - false -
        - - disabled - - - - 0 - *:0|java:1|javax:1|org.springframework:3|com.accenture:2 - - disabled - true - - - false - - true - false - - true - - - false - - - 1 - 1 - 0 - 1 - 4 - 55 - -1 - 4 - -1 - 0 - 8 - -1 - 1 - - - false - false - - - - false - false - false - true - false - true - false - - false - - - - - false - false - false - true - false - false - - false - - static|field|initializer|constructor|method|interface|class|annotation|enum - false - - - false - public=true|protected=true|private=true|abstract=true|static=true|final=true|synchronized=true|transient=true|volatile=true|native=true|strictfp=true - - - - - true - true - true - - - true - false - false - false - - false - - - false - false - true - - - - true - false - - true - true - true - true - true - true - - false - false - - - - - - 0 - false - false - false - - false - - false - false - - false - - - false - false - false - false - - - false - false - false - - - - - 2147483647 - - - true - - - - - 2147483647 - - - true - - - - - 2147483647 - - - true - - - - true - true - 100 - - - - false - false - false - - false - false - false - - - - false - - - false - - false - - false - - - -
        -
        diff --git a/spring-batch-samples/jmxLauncher.launch b/spring-batch-samples/jmxLauncher.launch deleted file mode 100644 index ef309a72ea..0000000000 --- a/spring-batch-samples/jmxLauncher.launch +++ /dev/null @@ -1,10 +0,0 @@ - - - - - - - - - - diff --git a/spring-batch-samples/jobLauncher.launch b/spring-batch-samples/jobLauncher.launch deleted file mode 100644 index 5c39d9e9a8..0000000000 --- a/spring-batch-samples/jobLauncher.launch +++ /dev/null @@ -1,10 +0,0 @@ - - - - - - - - - - diff --git a/spring-batch-samples/maven_checks_customized.xml b/spring-batch-samples/maven_checks_customized.xml deleted file mode 100644 index 654b243d31..0000000000 --- a/spring-batch-samples/maven_checks_customized.xml +++ /dev/null @@ -1,166 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/spring-batch-samples/pom.xml b/spring-batch-samples/pom.xml deleted file mode 100644 index 2ca49ff062..0000000000 --- a/spring-batch-samples/pom.xml +++ /dev/null @@ -1,439 +0,0 @@ - - - 4.0.0 - spring-batch-samples - jar - Samples - http://static.springframework.org/spring-batch/${artifactId} - Example batch jobs using Spring Batch Core and Execution. - - org.springframework.batch - spring-batch-parent - 2.2.2.BUILD-SNAPSHOT - ../spring-batch-parent - - - - hsql - - - - com.vmware.sqlfire - sqlfireclient - true - 1.0.3 - - - org.springframework.batch - spring-batch-core - ${project.version} - - - org.springframework.batch - spring-batch-test - ${project.version} - - - org.aspectj - aspectjrt - - - org.aspectj - aspectjweaver - - - junit - junit - - - org.springmodules - spring-modules-validation - 0.8 - - - rhino - js - - - commons-validator - commons-validator - - - commons-lang - commons-lang - - - commons-logging - commons-logging - - - commons-beanutils - commons-beanutils - - - commons-digester - commons-digester - - - antlr - antlr - - - org.springframework - spring - - - - - org.opensymphony.quartz - quartz - 1.6.1 - - - mysql - mysql-connector-java - 5.1.6 - runtime - - - net.sourceforge.jtds - jtds - 1.2.4 - runtime - - - org.hsqldb - hsqldb - runtime - - - com.h2database - h2 - runtime - - - commons-io - commons-io - - - commons-dbcp - commons-dbcp - - - com.thoughtworks.xstream - xstream - - - org.codehaus.woodstox - woodstox-core-asl - - - javax.servlet - servlet-api - - - xmlunit - xmlunit - 1.2 - test - - - org.slf4j - slf4j-log4j12 - true - - - log4j - log4j - test - - - org.codehaus.groovy - groovy - 1.6.3 - runtime - - - org.hibernate - hibernate-core - - - org.hibernate - hibernate-entitymanager - true - - - org.hibernate - hibernate-annotations - true - - - org.apache.geronimo.specs - geronimo-jta_1.1_spec - - - cglib - cglib-nodep - - - org.apache.ibatis - ibatis-sqlmap - - - org.apache.derby - derby - true - compile - - - postgresql - postgresql - 9.1-901.jdbc4 - true - - - org.springframework - spring-aop - - - org.springframework - spring-oxm - - - org.springframework - spring-core - - - org.springframework - spring-context-support - - - org.springframework - spring-jdbc - - - org.springframework - spring-orm - - - org.springframework - spring-test - - - org.springframework - spring-tx - - - org.springframework - spring-web - true - runtime - - - org.springframework.data - spring-data-commons - true - - - org.springframework.data - spring-data-jpa - - - javax.mail - mail - 1.4.1 - - - org.springframework.amqp - spring-amqp - true - - - org.springframework.amqp - spring-rabbit - true - - - org.mockito - mockito-all - test - - - - - - maven-surefire-plugin - - - - ENVIRONMENT - ${environment} - - - - - - org.apache.maven.plugins - maven-antrun-plugin - - - generate-sql - generate-sources - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - run - - - - - - - - - org.apache.maven.plugins - maven-surefire-plugin - - junit:junit - - - - - - - - com.springsource.repository.bundles.external - SpringSource Enterprise Bundle Repository - SpringSource Bundle External - http://repository.springsource.com/maven/bundles/external - - false - - - - gemstone - Release bundles for SQLFire and GemFire - http://dist.gemstone.com.s3.amazonaws.com/maven/release - - - diff --git a/spring-batch-samples/server.properties b/spring-batch-samples/server.properties deleted file mode 100644 index f40f3db454..0000000000 --- a/spring-batch-samples/server.properties +++ /dev/null @@ -1,5 +0,0 @@ -server.port=9005 -server.trace=true - -server.database.0=file:target/hsqldb-data/samples -server.dbname.0=samples diff --git a/spring-batch-samples/src/grafana/docker-compose.yml b/spring-batch-samples/src/grafana/docker-compose.yml new file mode 100644 index 0000000000..f82d08a70f --- /dev/null +++ b/spring-batch-samples/src/grafana/docker-compose.yml @@ -0,0 +1,22 @@ +version: '3.3' +services: + + prometheus: + image: prom/prometheus:v2.7.2 + container_name: 'prometheus' + ports: + - '9090:9090' + volumes: + - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml + + pushgateway: + image: prom/pushgateway:v0.6.0 + container_name: 'pushgateway' + ports: + - '9091:9091' + + grafana: + image: grafana/grafana:6.0.2 + container_name: 'grafana' + ports: + - '3000:3000' diff --git a/spring-batch-samples/src/grafana/prometheus/prometheus.yml b/spring-batch-samples/src/grafana/prometheus/prometheus.yml new file mode 100644 index 0000000000..4c469dd3d8 --- /dev/null +++ b/spring-batch-samples/src/grafana/prometheus/prometheus.yml @@ -0,0 +1,9 @@ +global: + scrape_interval: 5s + evaluation_interval: 5s + +scrape_configs: + - job_name: 'springbatch' + honor_labels: true + static_configs: + - targets: ['pushgateway:9091'] diff --git a/spring-batch-samples/src/grafana/spring-batch-dashboard.json b/spring-batch-samples/src/grafana/spring-batch-dashboard.json new file mode 100644 index 0000000000..8660bbb2ec --- /dev/null +++ b/spring-batch-samples/src/grafana/spring-batch-dashboard.json @@ -0,0 +1,568 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": "-- Grafana --", + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "limit": 100, + "name": "Annotations & Alerts", + "showIn": 0, + "type": "dashboard" + } + ] + }, + "description": "Dashboard for Spring Batch applications instrumented with Micrometer", + "editable": true, + "gnetId": 4701, + "graphTooltip": 1, + "id": 1, + "links": [], + "panels": [ + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 132, + "panels": [], + "repeat": null, + "title": "Spring Batch Metrics", + "type": "row" + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": "Prometheus", + "editable": true, + "error": false, + "fill": 1, + "grid": { + "leftLogBase": 1, + "leftMax": null, + "leftMin": null, + "rightLogBase": 1, + "rightMax": null, + "rightMin": null + }, + "gridPos": { + "h": 7, + "w": 8, + "x": 0, + "y": 1 + }, + "id": 37, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "links": [], + "nullPointMode": "null", + "paceLength": 10, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "spring_batch_job_seconds_max", + "format": "time_series", + "interval": "", + "intervalFactor": 1, + "legendFormat": "{{name}} {{status}}", + "metric": "", + "refId": "A", + "step": 1200 + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [], + "timeShift": null, + "title": "Job duration", + "tooltip": { + "msResolution": false, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "type": "graph", + "x-axis": true, + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "y-axis": true, + "y_formats": [ + "short", + "short" + ], + "yaxes": [ + { + "format": "s", + "label": "", + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "format": "short", + "label": "", + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": "Prometheus", + "editable": true, + "error": false, + "fill": 1, + "grid": { + "leftLogBase": 1, + "leftMax": null, + "leftMin": null, + "rightLogBase": 1, + "rightMax": null, + "rightMin": null + }, + "gridPos": { + "h": 7, + "w": 8, + "x": 8, + "y": 1 + }, + "id": 38, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": true, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "links": [], + "nullPointMode": "null", + "paceLength": 10, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "rate(spring_batch_job_seconds_count[1m]) * 60", + "format": "time_series", + "hide": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "{{name}} {{status}}", + "metric": "", + "refId": "A", + "step": 1200 + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [], + "timeShift": null, + "title": "Job execution rate", + "tooltip": { + "msResolution": false, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "type": "graph", + "x-axis": true, + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "y-axis": true, + "y_formats": [ + "ops", + "short" + ], + "yaxes": [ + { + "decimals": null, + "format": "opm", + "label": "", + "logBase": 1, + "max": null, + "min": null, + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": "Prometheus", + "fill": 1, + "gridPos": { + "h": 7, + "w": 8, + "x": 16, + "y": 1 + }, + "id": 138, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": true, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "links": [], + "nullPointMode": "null", + "paceLength": 10, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "(sum(rate(spring_batch_job_seconds_count{status=\"FAILED\"}[5m])) by (name, status)) / (sum(rate(spring_batch_job_seconds_count[5m])) by (name, status))", + "format": "time_series", + "hide": false, + "instant": false, + "intervalFactor": 1, + "legendFormat": "{{name}} {{status}}", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [], + "timeShift": null, + "title": "Job failure rate", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "percentunit", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "fill": 1, + "gridPos": { + "h": 9, + "w": 12, + "x": 0, + "y": 8 + }, + "id": 141, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": true, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "links": [], + "nullPointMode": "null", + "paceLength": 10, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "seriesOverrides": [ + { + "alias": "write success", + "yaxis": 1 + } + ], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "rate(spring_batch_item_read_seconds_count[1m])", + "format": "time_series", + "interval": "", + "intervalFactor": 1, + "legendFormat": "read {{status}}", + "refId": "B" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [], + "timeShift": null, + "title": "Item read throughput", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + }, + { + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "fill": 1, + "gridPos": { + "h": 9, + "w": 12, + "x": 12, + "y": 8 + }, + "id": 140, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": true, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "links": [], + "nullPointMode": "null", + "paceLength": 10, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "seriesOverrides": [ + { + "alias": "write success", + "yaxis": 1 + } + ], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "rate(spring_batch_chunk_write_seconds_count[1m])", + "format": "time_series", + "interval": "", + "intervalFactor": 1, + "legendFormat": "write {{status}}", + "refId": "B" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [], + "timeShift": null, + "title": "Item write throughput", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + }, + { + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + } + ], + "refresh": "5s", + "schemaVersion": 18, + "style": "dark", + "tags": [], + "templating": { + "list": [] + }, + "time": { + "from": "now-5m", + "to": "now" + }, + "timepicker": { + "now": true, + "refresh_intervals": [ + "5s", + "10s", + "30s", + "1m", + "5m", + "15m", + "30m", + "1h", + "2h", + "1d" + ], + "time_options": [ + "5m", + "15m", + "1h", + "6h", + "12h", + "24h", + "2d", + "7d", + "30d" + ] + }, + "timezone": "browser", + "title": "Spring Batch Prometheus", + "uid": "qRLUmOCmk", + "version": 6 +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/ColumnRangePartitioner.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/ColumnRangePartitioner.java index dd9c11e6e3..616b49cec3 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/ColumnRangePartitioner.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/ColumnRangePartitioner.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.common; import java.util.HashMap; @@ -61,19 +76,18 @@ public void setDataSource(DataSource dataSource) { * * @see Partitioner#partition(int) */ + @Override public Map partition(int gridSize) { - - int min = jdbcTemplate.queryForInt("SELECT MIN(" + column + ") from " + table); - int max = jdbcTemplate.queryForInt("SELECT MAX(" + column + ") from " + table); + int min = jdbcTemplate.queryForObject("SELECT MIN(" + column + ") from " + table, Integer.class); + int max = jdbcTemplate.queryForObject("SELECT MAX(" + column + ") from " + table, Integer.class); int targetSize = (max - min) / gridSize + 1; - Map result = new HashMap(); + Map result = new HashMap<>(); int number = 0; int start = min; int end = start + targetSize - 1; while (start <= max) { - ExecutionContext value = new ExecutionContext(); result.put("partition" + number, value); @@ -88,7 +102,5 @@ public Map partition(int gridSize) { } return result; - } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/FieldSetResultSetExtractor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/FieldSetResultSetExtractor.java deleted file mode 100644 index cf20d19bad..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/FieldSetResultSetExtractor.java +++ /dev/null @@ -1,65 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.sample.common; - -import java.sql.ResultSet; -import java.sql.ResultSetMetaData; -import java.sql.SQLException; -import java.util.ArrayList; -import java.util.List; - -import org.springframework.batch.item.file.transform.DefaultFieldSet; -import org.springframework.batch.item.file.transform.FieldSet; - -/** - * ResultSetExtractor implementation that returns list of FieldSets - * for given ResultSet. - * - * @author peter.zozom - * - */ -public final class FieldSetResultSetExtractor { - - // utility class not meant for instantiation - private FieldSetResultSetExtractor(){} - - /** - * Processes single row in ResultSet and returns its FieldSet representation. - * @param rs ResultSet ResultSet to extract data from. - * @return FieldSet representation of current row in ResultSet - * @throws SQLException thrown during processing - */ - public static FieldSet getFieldSet(ResultSet rs) throws SQLException { - ResultSetMetaData metaData = rs.getMetaData(); - int columnCount = metaData.getColumnCount(); - - FieldSet fs; - - List tokens = new ArrayList(); - List names = new ArrayList(); - - for (int i = 1; i <= columnCount; i++) { - tokens.add(rs.getString(i)); - names.add(metaData.getColumnName(i)); - } - - fs = new DefaultFieldSet(tokens.toArray(new String[0]), names.toArray(new String[0])); - - return fs; - } - -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/InfiniteLoopReader.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/InfiniteLoopReader.java index ff0a71ac1d..3b03efb2af 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/InfiniteLoopReader.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/InfiniteLoopReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,8 +17,7 @@ package org.springframework.batch.sample.common; import org.springframework.batch.item.ItemReader; -import org.springframework.batch.item.ParseException; -import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.lang.Nullable; /** * ItemReader implementation that will continually return a new object. It's @@ -29,8 +28,9 @@ */ public class InfiniteLoopReader implements ItemReader { - public Object read() throws Exception, UnexpectedInputException, ParseException { + @Nullable + @Override + public Object read() throws Exception { return new Object(); } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/InfiniteLoopWriter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/InfiniteLoopWriter.java index feae9acc00..a58d0b1bd7 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/InfiniteLoopWriter.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/InfiniteLoopWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -21,7 +21,6 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.core.StepExecution; -import org.springframework.batch.core.StepExecutionListener; import org.springframework.batch.core.listener.StepExecutionListenerSupport; import org.springframework.batch.item.ItemWriter; @@ -34,37 +33,34 @@ * */ public class InfiniteLoopWriter extends StepExecutionListenerSupport implements ItemWriter { + private static final Log LOG = LogFactory.getLog(InfiniteLoopWriter.class); private StepExecution stepExecution; - private int count = 0; - private static final Log logger = LogFactory.getLog(InfiniteLoopWriter.class); - /** - * @see StepExecutionListener#beforeStep(StepExecution) + * @see org.springframework.batch.core.StepExecutionListener#beforeStep(StepExecution) */ @Override public void beforeStep(StepExecution stepExecution) { this.stepExecution = stepExecution; } - /** - * - */ public InfiniteLoopWriter() { super(); } + @Override public void write(List items) throws Exception { try { Thread.sleep(500); } catch (InterruptedException e) { Thread.currentThread().interrupt(); - throw new RuntimeException("Job interrupted."); + throw new IllegalStateException("Job interrupted.", e); } + stepExecution.setWriteCount(++count); - logger.info("Executing infinite loop, at count=" + count); + LOG.info("Executing infinite loop, at count=" + count); } } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/LogAdvice.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/LogAdvice.java index aa29fb2025..bf26cc646d 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/LogAdvice.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/LogAdvice.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,8 +18,6 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; -import org.aspectj.lang.JoinPoint; - /** * Wraps calls for 'Processing' methods which output a single Object to write @@ -28,29 +26,9 @@ * @author Lucas Ward */ public class LogAdvice { - private static Log log = LogFactory.getLog(LogAdvice.class); - /* - * Wraps original method and adds logging both before and after method - */ - public void doBasicLogging(JoinPoint pjp) throws Throwable { - Object[] args = pjp.getArgs(); - StringBuffer output = new StringBuffer(); - - output.append(pjp.getTarget().getClass().getName()).append(": "); - output.append(pjp.toShortString()).append(": "); - - for (Object arg : args) { - output.append(arg).append(" "); - } - - - log.info("Basic: " + output.toString()); - } - public void doStronglyTypedLogging(Object item){ log.info("Processed: " + item); } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/OutputFileListener.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/OutputFileListener.java index ab400c74a6..2d61024f8b 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/OutputFileListener.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/OutputFileListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,7 +30,7 @@ public class OutputFileListener { private String inputKeyName = "fileName"; - private String path = "file:./target/output/"; + private String path = "file:./build/output/"; public void setPath(String path) { this.path = path; diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/OutputFileNameListener.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/OutputFileNameListener.java index 64c0c40542..65614e10e6 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/OutputFileNameListener.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/OutputFileNameListener.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.common; public class OutputFileNameListener { diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/ProcessIndicatorItemWrapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/ProcessIndicatorItemWrapper.java index d1b0c4ea19..db157d4bd2 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/ProcessIndicatorItemWrapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/ProcessIndicatorItemWrapper.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.common; /** diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemListener.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemListener.java index e4012611b7..5785416bec 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemListener.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemListener.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -38,6 +38,7 @@ public void setDataSource(DataSource dataSource) { jdbcTemplate = new JdbcTemplate(dataSource); } + @Override public final void afterPropertiesSet() throws Exception { Assert.notNull(jdbcTemplate, "You must provide a DataSource."); } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemProcessor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemProcessor.java index 3bf84b2345..ffd3d323f5 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemProcessor.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemProcessor.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.common; import javax.sql.DataSource; @@ -7,6 +22,7 @@ import org.springframework.dao.OptimisticLockingFailureException; import org.springframework.jdbc.core.JdbcOperations; import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -33,6 +49,7 @@ public void setDataSource(DataSource dataSource) { this.jdbcTemplate = new JdbcTemplate(dataSource); } + @Override public void afterPropertiesSet() throws Exception { Assert.notNull(jdbcTemplate, "Either jdbcTemplate or dataSource must be set"); } @@ -41,6 +58,8 @@ public void afterPropertiesSet() throws Exception { * Use the technical identifier to mark the input row as processed and * return unwrapped item. */ + @Nullable + @Override public T process(ProcessIndicatorItemWrapper wrapper) throws Exception { int count = jdbcTemplate.update("UPDATE BATCH_STAGING SET PROCESSED=? WHERE ID=? AND PROCESSED=?", diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemReader.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemReader.java index 757033fbc8..140d5964c1 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemReader.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,19 +25,20 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; + import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.StepExecutionListener; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ReaderNotOpenException; -import org.springframework.batch.support.SerializationUtils; import org.springframework.beans.factory.DisposableBean; import org.springframework.beans.factory.InitializingBean; -import org.springframework.dao.DataAccessException; import org.springframework.jdbc.core.JdbcOperations; import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; +import org.springframework.util.SerializationUtils; /** * Thread-safe database {@link ItemReader} implementing the process indicator @@ -46,7 +47,7 @@ * To achieve restartability use together with {@link StagingItemProcessor}. */ public class StagingItemReader implements ItemReader>, StepExecutionListener, - InitializingBean, DisposableBean { +InitializingBean, DisposableBean { private static Log logger = LogFactory.getLog(StagingItemReader.class); @@ -64,11 +65,13 @@ public void setDataSource(DataSource dataSource) { jdbcTemplate = new JdbcTemplate(dataSource); } + @Override public void destroy() throws Exception { initialized = false; keys = null; } + @Override public final void afterPropertiesSet() throws Exception { Assert.notNull(jdbcTemplate, "You must provide a DataSource."); } @@ -79,22 +82,24 @@ private List retrieveKeys() { return jdbcTemplate.query( - "SELECT ID FROM BATCH_STAGING WHERE JOB_ID=? AND PROCESSED=? ORDER BY ID", + "SELECT ID FROM BATCH_STAGING WHERE JOB_ID=? AND PROCESSED=? ORDER BY ID", - new ParameterizedRowMapper() { - public Long mapRow(ResultSet rs, int rowNum) throws SQLException { - return rs.getLong(1); - } - }, + new RowMapper() { + @Override + public Long mapRow(ResultSet rs, int rowNum) throws SQLException { + return rs.getLong(1); + } + }, - stepExecution.getJobExecution().getJobId(), StagingItemWriter.NEW); + stepExecution.getJobExecution().getJobId(), StagingItemWriter.NEW); } } - public ProcessIndicatorItemWrapper read() throws DataAccessException { - + @Nullable + @Override + public ProcessIndicatorItemWrapper read() { if (!initialized) { throw new ReaderNotOpenException("Reader must be open before it can be used."); } @@ -112,21 +117,24 @@ public ProcessIndicatorItemWrapper read() throws DataAccessException { } @SuppressWarnings("unchecked") T result = (T) jdbcTemplate.queryForObject("SELECT VALUE FROM BATCH_STAGING WHERE ID=?", - new ParameterizedRowMapper() { - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { - byte[] blob = rs.getBytes(1); - return SerializationUtils.deserialize(blob); - } - }, id); - - return new ProcessIndicatorItemWrapper(id, result); + new RowMapper() { + @Override + public Object mapRow(ResultSet rs, int rowNum) throws SQLException { + byte[] blob = rs.getBytes(1); + return SerializationUtils.deserialize(blob); + } + }, id); + return new ProcessIndicatorItemWrapper<>(id, result); } + @Nullable + @Override public ExitStatus afterStep(StepExecution stepExecution) { return null; } + @Override public void beforeStep(StepExecution stepExecution) { this.stepExecution = stepExecution; synchronized (lock) { diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemWriter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemWriter.java index a9444f6102..74957c0294 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemWriter.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/StagingItemWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,7 +16,6 @@ package org.springframework.batch.sample.common; -import java.io.Serializable; import java.sql.PreparedStatement; import java.sql.SQLException; import java.util.List; @@ -26,23 +25,22 @@ import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.StepExecutionListener; import org.springframework.batch.item.ItemWriter; -import org.springframework.batch.support.SerializationUtils; import org.springframework.jdbc.core.BatchPreparedStatementSetter; import org.springframework.jdbc.core.support.JdbcDaoSupport; import org.springframework.jdbc.support.incrementer.DataFieldMaxValueIncrementer; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.ClassUtils; +import org.springframework.util.SerializationUtils; /** * Database {@link ItemWriter} implementing the process indicator pattern. */ public class StagingItemWriter extends JdbcDaoSupport implements StepExecutionListener, ItemWriter { - public static final String NEW = "N"; + protected static final String NEW = "N"; - public static final String DONE = "Y"; - - public static final Object WORKING = "W"; + protected static final String DONE = "Y"; private DataFieldMaxValueIncrementer incrementer; @@ -50,9 +48,10 @@ public class StagingItemWriter extends JdbcDaoSupport implements StepExecutio /** * Check mandatory properties. - * + * * @see org.springframework.dao.support.DaoSupport#initDao() */ + @Override protected void initDao() throws Exception { super.initDao(); Assert.notNull(incrementer, "DataFieldMaxValueIncrementer is required - set the incrementer property in the " @@ -61,7 +60,7 @@ protected void initDao() throws Exception { /** * Setter for the key generator for the staging table. - * + * * @param incrementer the {@link DataFieldMaxValueIncrementer} to set */ public void setIncrementer(DataFieldMaxValueIncrementer incrementer) { @@ -70,57 +69,53 @@ public void setIncrementer(DataFieldMaxValueIncrementer incrementer) { /** * Serialize the item to the staging table, and add a NEW processed flag. - * + * * @see ItemWriter#write(java.util.List) */ + @Override public void write(final List items) { - final ListIterator itemIterator = items.listIterator(); + getJdbcTemplate().batchUpdate("INSERT into BATCH_STAGING (ID, JOB_ID, VALUE, PROCESSED) values (?,?,?,?)", new BatchPreparedStatementSetter() { - - public int getBatchSize() { - return items.size(); - } - - public void setValues(PreparedStatement ps, int i) throws SQLException { - - long id = incrementer.nextLongValue(); - long jobId = stepExecution.getJobExecution().getJobId(); - - Assert.state(itemIterator.nextIndex() == i, - "Item ordering must be preserved in batch sql update"); - - byte[] blob = SerializationUtils.serialize((Serializable) itemIterator.next()); - - ps.setLong(1, id); - ps.setLong(2, jobId); - ps.setBytes(3, blob); - ps.setString(4, NEW); - } - }); - + @Override + public int getBatchSize() { + return items.size(); + } + + @Override + public void setValues(PreparedStatement ps, int i) throws SQLException { + Assert.state(itemIterator.nextIndex() == i, "Item ordering must be preserved in batch sql update"); + + ps.setLong(1, incrementer.nextLongValue()); + ps.setLong(2, stepExecution.getJobExecution().getJobId()); + ps.setBytes(3, SerializationUtils.serialize(itemIterator.next())); + ps.setString(4, NEW); + } + }); } /* * (non-Javadoc) - * + * * @see * org.springframework.batch.core.domain.StepListener#afterStep(StepExecution * ) */ + @Nullable + @Override public ExitStatus afterStep(StepExecution stepExecution) { return null; } /* * (non-Javadoc) - * - * @seeorg.springframework.batch.core.domain.StepListener#beforeStep(org. + * + * @see org.springframework.batch.core.domain.StepListener#beforeStep(org. * springframework.batch.core.domain.StepExecution) */ + @Override public void beforeStep(StepExecution stepExecution) { this.stepExecution = stepExecution; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/config/DataSourceConfiguration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/config/DataSourceConfiguration.java index 2983f9957d..e2f7a9dfce 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/config/DataSourceConfiguration.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/config/DataSourceConfiguration.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,7 +18,8 @@ import javax.annotation.PostConstruct; import javax.sql.DataSource; -import org.apache.commons.dbcp.BasicDataSource; +import org.apache.commons.dbcp2.BasicDataSource; + import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/config/RetrySampleConfiguration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/config/RetrySampleConfiguration.java index 4b40f7f333..73dda5eeeb 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/config/RetrySampleConfiguration.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/config/RetrySampleConfiguration.java @@ -1,11 +1,11 @@ /* - * Copyright 2012-2013 the original author or authors. + * Copyright 2012-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -42,12 +42,12 @@ public class RetrySampleConfiguration { private StepBuilderFactory steps; @Bean - public Job retrySample() throws Exception { + public Job retrySample() { return jobs.get("retrySample").start(step()).build(); } @Bean - protected Step step() throws Exception { + protected Step step() { return steps.get("step"). chunk(1).reader(reader()).writer(writer()).faultTolerant() .retry(Exception.class).retryLimit(3).build(); } @@ -61,7 +61,6 @@ protected ItemReader reader() { @Bean protected ItemWriter writer() { - return new RetrySampleItemWriter(); + return new RetrySampleItemWriter<>(); } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/data/CustomerCreditRepository.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/data/CustomerCreditRepository.java index fea3a65e9b..bccf791c9e 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/data/CustomerCreditRepository.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/data/CustomerCreditRepository.java @@ -1,3 +1,18 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.data; import java.math.BigDecimal; diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/Game.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/Game.java index 531a0c5372..a9d003b88c 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/Game.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/Game.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,6 +18,7 @@ import java.io.Serializable; +@SuppressWarnings("serial") public class Game implements Serializable { private String id; @@ -217,6 +218,7 @@ public void setTotalTd(int totalTd) { } + @Override public String toString() { return "Game: ID=" + id + " " + team + " vs. " + opponent + @@ -232,19 +234,25 @@ public int hashCode() { } @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + if (getClass() != obj.getClass()) { return false; + } Game other = (Game) obj; if (id == null) { - if (other.id != null) + if (other.id != null) { return false; + } } - else if (!id.equals(other.id)) + else if (!id.equals(other.id)) { return false; + } + return true; } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/Player.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/Player.java index b97c26cf50..ac13f7c4c4 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/Player.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/Player.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,6 +18,7 @@ import java.io.Serializable; +@SuppressWarnings("serial") public class Player implements Serializable { private String id; @@ -27,6 +28,7 @@ public class Player implements Serializable { private int birthYear; private int debutYear; + @Override public String toString() { return "PLAYER:id=" + id + ",Last Name=" + lastName + diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/PlayerDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/PlayerDao.java index 6d22b55b31..8fd687fc34 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/PlayerDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/PlayerDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/PlayerSummary.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/PlayerSummary.java index a8ef969a83..f92d96d9d3 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/PlayerSummary.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/PlayerSummary.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -13,19 +13,15 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.springframework.batch.sample.domain.football; - /** * Domain object representing the summary of a given Player's * year. * * @author Lucas Ward - * */ public class PlayerSummary { - private String id; private int year; private int completes; @@ -111,13 +107,14 @@ public int getTotalTd() { public void setTotalTd(int totalTd) { this.totalTd = totalTd; } - - + + @Override public String toString() { return "Player Summary: ID=" + id + " Year=" + year + "[" + completes + ";" + attempts + ";" + passingYards + ";" + passingTd + ";" + interceptions + ";" + rushes + ";" + rushYards + ";" + receptions + ";" + receptionYards + ";" + totalTd; } + @Override public int hashCode() { final int prime = 31; @@ -125,22 +122,28 @@ public int hashCode() { result = prime * result + ((id == null) ? 0 : id.hashCode()); return result; } + @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + if (getClass() != obj.getClass()) { return false; + } PlayerSummary other = (PlayerSummary) obj; if (id == null) { - if (other.id != null) + if (other.id != null) { return false; + } } - else if (!id.equals(other.id)) + else if (!id.equals(other.id)) { return false; + } + return true; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/FootballExceptionHandler.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/FootballExceptionHandler.java index a1ff7afcbb..420dd450d5 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/FootballExceptionHandler.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/FootballExceptionHandler.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,6 +26,7 @@ public class FootballExceptionHandler implements ExceptionHandler { private static final Log logger = LogFactory .getLog(FootballExceptionHandler.class); + @Override public void handleException(RepeatContext context, Throwable throwable) throws Throwable { diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/GameFieldSetMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/GameFieldSetMapper.java index 3121b50fe0..0851e650b4 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/GameFieldSetMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/GameFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ public class GameFieldSetMapper implements FieldSetMapper { + @Override public Game mapFieldSet(FieldSet fs) { if(fs == null){ diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcGameDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcGameDao.java index 6074bde1f9..c0215e4ba8 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcGameDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcGameDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ public class JdbcGameDao extends JdbcDaoSupport implements ItemWriter { private SimpleJdbcInsert insertGame; + @Override protected void initDao() throws Exception { super.initDao(); insertGame = new SimpleJdbcInsert(getDataSource()).withTableName("GAMES").usingColumns("player_id", "year_no", @@ -36,6 +37,7 @@ protected void initDao() throws Exception { "rushes", "rush_yards", "receptions", "receptions_yards", "total_td"); } + @Override public void write(List games) { for (Game game : games) { diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerDao.java index b406ed006f..179186e1f6 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,7 +36,8 @@ public class JdbcPlayerDao implements PlayerDao { private NamedParameterJdbcOperations namedParameterJdbcTemplate; - public void savePlayer(Player player) { + @Override + public void savePlayer(Player player) { namedParameterJdbcTemplate.update(INSERT_PLAYER, new BeanPropertySqlParameterSource(player)); } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerSummaryDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerSummaryDao.java index f3ca7b1adc..270713707e 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerSummaryDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerSummaryDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,6 +35,7 @@ public class JdbcPlayerSummaryDao implements ItemWriter { private NamedParameterJdbcOperations namedParameterJdbcTemplate; + @Override public void write(List summaries) { for (PlayerSummary summary : summaries) { diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerFieldSetMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerFieldSetMapper.java index 7fdadf1e2d..5366085172 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerFieldSetMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ public class PlayerFieldSetMapper implements FieldSetMapper { + @Override public Player mapFieldSet(FieldSet fs) { if(fs == null){ diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerItemWriter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerItemWriter.java index 57945a2177..7ed7575ac6 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerItemWriter.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,6 +26,7 @@ public class PlayerItemWriter implements ItemWriter { private PlayerDao playerDao; + @Override public void write(List players) throws Exception { for (Player player : players) { playerDao.savePlayer(player); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryMapper.java index fea712fd03..27c48f8200 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryMapper.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,19 +19,21 @@ import java.sql.SQLException; import org.springframework.batch.sample.domain.football.PlayerSummary; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; /** - * RowMapper used to map a ResultSet to a {@link PlayerSummary} + * RowMapper used to map a ResultSet to a {@link org.springframework.batch.sample.domain.football.PlayerSummary} * * @author Lucas Ward + * @author Mahmoud Ben Hassine * */ -public class PlayerSummaryMapper implements ParameterizedRowMapper { +public class PlayerSummaryMapper implements RowMapper { /* (non-Javadoc) * @see org.springframework.jdbc.core.RowMapper#mapRow(java.sql.ResultSet, int) */ + @Override public PlayerSummary mapRow(ResultSet rs, int rowNum) throws SQLException { PlayerSummary summary = new PlayerSummary(); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryRowMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryRowMapper.java index e821833d43..96cde00f74 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryRowMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/football/internal/PlayerSummaryRowMapper.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,17 +22,19 @@ import org.springframework.jdbc.core.RowMapper; /** - * RowMapper used to map a ResultSet to a {@link PlayerSummary} + * RowMapper used to map a ResultSet to a {@link org.springframework.batch.sample.domain.football.PlayerSummary} * * @author Lucas Ward + * @author Mahmoud Ben Hassine * */ -public class PlayerSummaryRowMapper implements RowMapper { +public class PlayerSummaryRowMapper implements RowMapper { /* (non-Javadoc) * @see org.springframework.jdbc.core.RowMapper#mapRow(java.sql.ResultSet, int) */ - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { + @Override + public PlayerSummary mapRow(ResultSet rs, int rowNum) throws SQLException { PlayerSummary summary = new PlayerSummary(); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/User.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/User.java index 72956657f0..b9817a28a3 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/User.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/User.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/internal/TestMailErrorHandler.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/internal/TestMailErrorHandler.java index 4b18f54c87..405d0beeed 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/internal/TestMailErrorHandler.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/internal/TestMailErrorHandler.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2010 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,6 +19,8 @@ import java.util.List; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; import org.springframework.batch.item.mail.MailErrorHandler; import org.springframework.mail.MailMessage; @@ -33,13 +35,14 @@ * @since 2.1 */ public class TestMailErrorHandler implements MailErrorHandler { + private static final Log LOGGER = LogFactory.getLog(TestMailErrorHandler.class); - private List failedMessages = new ArrayList(); + private List failedMessages = new ArrayList<>(); + @Override public void handle(MailMessage failedMessage, Exception ex) { this.failedMessages.add(failedMessage); - System.out.println("Mail message failed: " + failedMessage); - System.out.println(ex); + LOGGER.error("Mail message failed: " + failedMessage, ex); } public List getFailedMessages() { diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/internal/TestMailSender.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/internal/TestMailSender.java index 27cdcf4a70..662c28f0c3 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/internal/TestMailSender.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/internal/TestMailSender.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,24 +35,26 @@ */ public class TestMailSender implements MailSender { - private List subjectsToFail = new ArrayList(); + private List subjectsToFail = new ArrayList<>(); - private List received = new ArrayList(); + private List received = new ArrayList<>(); public void clear() { received.clear(); } + @Override public void send(SimpleMailMessage simpleMessage) throws MailException { - throw new UnsupportedOperationException("Not implememted. Use send(SimpleMailMessage[])."); + throw new UnsupportedOperationException("Not implemented. Use send(SimpleMailMessage[])."); } public void setSubjectsToFail(List subjectsToFail) { this.subjectsToFail = subjectsToFail; } - public void send(SimpleMailMessage[] simpleMessages) throws MailException { - Map failedMessages = new LinkedHashMap(); + @Override + public void send(SimpleMailMessage... simpleMessages) throws MailException { + Map failedMessages = new LinkedHashMap<>(); for (SimpleMailMessage simpleMessage : simpleMessages) { if (subjectsToFail.contains(simpleMessage.getSubject())) { failedMessages.put(simpleMessage, new MessagingException()); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/internal/UserMailItemProcessor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/internal/UserMailItemProcessor.java index 38c6b4215c..347c40059b 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/internal/UserMailItemProcessor.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/mail/internal/UserMailItemProcessor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2010 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,6 +19,7 @@ import org.springframework.batch.item.ItemProcessor; import org.springframework.batch.sample.domain.mail.User; +import org.springframework.lang.Nullable; import org.springframework.mail.SimpleMailMessage; /** @@ -33,7 +34,9 @@ public class UserMailItemProcessor implements /** * @see org.springframework.batch.item.ItemProcessor#process(java.lang.Object) */ - public SimpleMailMessage process( User user ) throws Exception { + @Nullable + @Override + public SimpleMailMessage process( User user ) throws Exception { SimpleMailMessage message = new SimpleMailMessage(); message.setTo( user.getEmail() ); message.setFrom( "communications@thecompany.com" ); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItem.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItem.java index 708de125ba..2820d415f4 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItem.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItem.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -15,8 +15,6 @@ */ package org.springframework.batch.sample.domain.multiline; -import org.springframework.batch.item.ItemReaderException; - /** * A wrapper type for an item that is used by {@link AggregateItemReader} to * identify the start and end of an aggregate record. @@ -27,11 +25,10 @@ * */ public class AggregateItem { - @SuppressWarnings("rawtypes") private static final AggregateItem FOOTER = new AggregateItem(false, true) { @Override - public Object getItem() throws ItemReaderException { + public Object getItem() { throw new IllegalStateException("Footer record has no item."); } }; @@ -41,14 +38,14 @@ public Object getItem() throws ItemReaderException { * @return a static {@link AggregateItem} that is a footer. */ @SuppressWarnings("unchecked") - public static final AggregateItem getFooter() { + public static AggregateItem getFooter() { return FOOTER; } @SuppressWarnings("rawtypes") private static final AggregateItem HEADER = new AggregateItem(true, false) { @Override - public Object getItem() throws ItemReaderException { + public Object getItem() { throw new IllegalStateException("Header record has no item."); } }; @@ -58,7 +55,7 @@ public Object getItem() throws ItemReaderException { * @return a static {@link AggregateItem} that is a header. */ @SuppressWarnings("unchecked") - public static final AggregateItem getHeader() { + public static AggregateItem getHeader() { return HEADER; } @@ -69,7 +66,7 @@ public static final AggregateItem getHeader() { private boolean header = false; /** - * @param item + * @param item the item to wrap */ public AggregateItem(T item) { super(); @@ -89,7 +86,7 @@ public AggregateItem(boolean header, boolean footer) { * @throws IllegalStateException if called on a record for which either * {@link #isHeader()} or {@link #isFooter()} answers true. */ - public T getItem() throws IllegalStateException { + public T getItem() { return item; } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItemFieldSetMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItemFieldSetMapper.java index 24997b8efa..51cc52ed31 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItemFieldSetMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItemFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -70,6 +70,7 @@ public void setBegin(String begin) { * Check mandatory properties (delegate). * @see org.springframework.beans.factory.InitializingBean#afterPropertiesSet() */ + @Override public void afterPropertiesSet() throws Exception { Assert.notNull(delegate, "A FieldSetMapper delegate must be provided."); } @@ -85,6 +86,7 @@ public void afterPropertiesSet() throws Exception { * delegate * @throws BindException if one of the delegates does */ + @Override public AggregateItem mapFieldSet(FieldSet fieldSet) throws BindException { if (fieldSet.readString(0).equals(begin)) { @@ -94,7 +96,7 @@ public AggregateItem mapFieldSet(FieldSet fieldSet) throws BindException { return AggregateItem.getFooter(); } - return new AggregateItem(delegate.mapFieldSet(fieldSet)); + return new AggregateItem<>(delegate.mapFieldSet(fieldSet)); } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItemReader.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItemReader.java index 2b12ced752..a81d6ce24f 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItemReader.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.batch.item.ItemReader; +import org.springframework.lang.Nullable; /** * An {@link ItemReader} that delivers a list as its item, storing up objects @@ -30,10 +31,10 @@ * {@link ItemReader} that can identify the record boundaries. The custom reader * should mark the beginning and end of records by returning an * {@link AggregateItem} which responds true to its query methods - * is*().

        + * is*().

        * - * This class is thread safe (it can be used concurrently by multiple threads) - * as long as the {@link ItemReader} is also thread safe. + * This class is thread-safe (it can be used concurrently by multiple threads) + * as long as the {@link ItemReader} is also thread-safe. * * @see AggregateItem#isHeader() * @see AggregateItem#isFooter() @@ -42,17 +43,17 @@ * */ public class AggregateItemReader implements ItemReader> { - - private static final Log log = LogFactory.getLog(AggregateItemReader.class); + private static final Log LOG = LogFactory.getLog(AggregateItemReader.class); private ItemReader> itemReader; /** * Get the next list of records. - * @throws Exception - * + * * @see org.springframework.batch.item.ItemReader#read() */ + @Nullable + @Override public List read() throws Exception { ResultHolder holder = new ResultHolder(); @@ -60,8 +61,8 @@ public List read() throws Exception { continue; } - if (!holder.exhausted) { - return holder.records; + if (!holder.isExhausted()) { + return holder.getRecords(); } else { return null; @@ -71,26 +72,26 @@ public List read() throws Exception { private boolean process(AggregateItem value, ResultHolder holder) { // finish processing if we hit the end of file if (value == null) { - log.debug("Exhausted ItemReader"); - holder.exhausted = true; + LOG.debug("Exhausted ItemReader"); + holder.setExhausted(true); return false; } // start a new collection if (value.isHeader()) { - log.debug("Start of new record detected"); + LOG.debug("Start of new record detected"); return true; } // mark we are finished with current collection if (value.isFooter()) { - log.debug("End of record detected"); + LOG.debug("End of record detected"); return false; } // add a simple record to the current collection - log.debug("Mapping: " + value); - holder.records.add(value.getItem()); + LOG.debug("Mapping: " + value); + holder.addRecord(value.getItem()); return true; } @@ -106,9 +107,23 @@ public void setItemReader(ItemReader> itemReader) { * */ private class ResultHolder { - List records = new ArrayList(); + private List records = new ArrayList<>(); + private boolean exhausted = false; - boolean exhausted = false; - } + public List getRecords() { + return records; + } + + public boolean isExhausted() { + return exhausted; + } + + public void addRecord(T record) { + records.add(record); + } + public void setExhausted(boolean exhausted) { + this.exhausted = exhausted; + } + } } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/Address.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/Address.java index 1d58a2c638..f9ad86a955 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/Address.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/Address.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -109,32 +109,41 @@ public int hashCode() { @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + if (getClass() != obj.getClass()) { return false; + } Address other = (Address) obj; if (addressee == null) { - if (other.addressee != null) + if (other.addressee != null) { return false; + } } - else if (!addressee.equals(other.addressee)) + else if (!addressee.equals(other.addressee)) { return false; + } if (country == null) { - if (other.country != null) + if (other.country != null) { return false; + } } - else if (!country.equals(other.country)) + else if (!country.equals(other.country)) { return false; + } if (zipCode == null) { - if (other.zipCode != null) + if (other.zipCode != null) { return false; + } } - else if (!zipCode.equals(other.zipCode)) + else if (!zipCode.equals(other.zipCode)) { return false; + } + return true; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/BillingInfo.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/BillingInfo.java index e54f309eeb..971fa4bcc4 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/BillingInfo.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/BillingInfo.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -56,26 +56,33 @@ public int hashCode() { @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + if (getClass() != obj.getClass()) { return false; + } BillingInfo other = (BillingInfo) obj; if (paymentDesc == null) { - if (other.paymentDesc != null) + if (other.paymentDesc != null) { return false; + } } - else if (!paymentDesc.equals(other.paymentDesc)) + else if (!paymentDesc.equals(other.paymentDesc)) { return false; + } if (paymentId == null) { - if (other.paymentId != null) + if (other.paymentId != null) { return false; + } } - else if (!paymentId.equals(other.paymentId)) + else if (!paymentId.equals(other.paymentId)) { return false; + } + return true; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/Customer.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/Customer.java index f5528fc7f2..5231650708 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/Customer.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/Customer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -126,45 +126,60 @@ public int hashCode() { @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + if (getClass() != obj.getClass()) { return false; + } Customer other = (Customer) obj; - if (businessCustomer != other.businessCustomer) + if (businessCustomer != other.businessCustomer) { return false; + } if (companyName == null) { - if (other.companyName != null) + if (other.companyName != null) { return false; + } } - else if (!companyName.equals(other.companyName)) + else if (!companyName.equals(other.companyName)) { return false; + } if (firstName == null) { - if (other.firstName != null) + if (other.firstName != null) { return false; + } } - else if (!firstName.equals(other.firstName)) + else if (!firstName.equals(other.firstName)) { return false; + } if (lastName == null) { - if (other.lastName != null) + if (other.lastName != null) { return false; + } } - else if (!lastName.equals(other.lastName)) + else if (!lastName.equals(other.lastName)) { return false; + } if (middleName == null) { - if (other.middleName != null) + if (other.middleName != null) { return false; + } } - else if (!middleName.equals(other.middleName)) + else if (!middleName.equals(other.middleName)) { return false; - if (registered != other.registered) + } + if (registered != other.registered) { return false; - if (registrationId != other.registrationId) + } + if (registrationId != other.registrationId) { return false; - if (vip != other.vip) + } + if (vip != other.vip) { return false; + } return true; } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/LineItem.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/LineItem.java index 3ec0798a63..3e90916cfc 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/LineItem.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/LineItem.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,9 +18,9 @@ import java.math.BigDecimal; - public class LineItem { public static final String LINE_ID_ITEM = "LIT"; + private long itemId; private BigDecimal price; private BigDecimal discountPerc; @@ -110,18 +110,28 @@ public int hashCode() { @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + + if (getClass() != obj.getClass()) { return false; + } + LineItem other = (LineItem) obj; - if (itemId != other.itemId) + + if (itemId != other.itemId) { return false; - if (quantity != other.quantity) + } + + if (quantity != other.quantity) { return false; + } + return true; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/Order.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/Order.java index 4a1896deb4..e5fa9ee972 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/Order.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/Order.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -164,68 +164,90 @@ public int hashCode() { @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + if (getClass() != obj.getClass()) { return false; + } Order other = (Order) obj; if (billing == null) { - if (other.billing != null) + if (other.billing != null) { return false; + } } - else if (!billing.equals(other.billing)) + else if (!billing.equals(other.billing)) { return false; + } if (billingAddress == null) { - if (other.billingAddress != null) + if (other.billingAddress != null) { return false; + } } - else if (!billingAddress.equals(other.billingAddress)) + else if (!billingAddress.equals(other.billingAddress)) { return false; + } if (customer == null) { - if (other.customer != null) + if (other.customer != null) { return false; + } } - else if (!customer.equals(other.customer)) + else if (!customer.equals(other.customer)) { return false; + } if (lineItems == null) { - if (other.lineItems != null) + if (other.lineItems != null) { return false; + } } - else if (!lineItems.equals(other.lineItems)) + else if (!lineItems.equals(other.lineItems)) { return false; + } if (orderDate == null) { - if (other.orderDate != null) + if (other.orderDate != null) { return false; + } } - else if (!orderDate.equals(other.orderDate)) + else if (!orderDate.equals(other.orderDate)) { return false; - if (orderId != other.orderId) + } + if (orderId != other.orderId) { return false; + } if (shipping == null) { - if (other.shipping != null) + if (other.shipping != null) { return false; + } } - else if (!shipping.equals(other.shipping)) + else if (!shipping.equals(other.shipping)) { return false; + } if (shippingAddress == null) { - if (other.shippingAddress != null) + if (other.shippingAddress != null) { return false; + } } - else if (!shippingAddress.equals(other.shippingAddress)) + else if (!shippingAddress.equals(other.shippingAddress)) { return false; - if (totalItems != other.totalItems) + } + if (totalItems != other.totalItems) { return false; - if (totalLines != other.totalLines) + } + if (totalLines != other.totalLines) { return false; + } if (totalPrice == null) { - if (other.totalPrice != null) + if (other.totalPrice != null) { return false; + } } - else if (!totalPrice.equals(other.totalPrice)) + else if (!totalPrice.equals(other.totalPrice)) { return false; + } + return true; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/OrderDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/OrderDao.java deleted file mode 100644 index 2d7a279a55..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/OrderDao.java +++ /dev/null @@ -1,27 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.sample.domain.order; - - -/** - * Interface for writing Order objects. - */ -public interface OrderDao { - - public void write(Order order); - -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/ShippingInfo.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/ShippingInfo.java index efdf8befb7..51aeb8c925 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/ShippingInfo.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/ShippingInfo.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -62,32 +62,41 @@ public int hashCode() { @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + if (getClass() != obj.getClass()) { return false; + } ShippingInfo other = (ShippingInfo) obj; if (shipperId == null) { - if (other.shipperId != null) + if (other.shipperId != null) { return false; + } } - else if (!shipperId.equals(other.shipperId)) + else if (!shipperId.equals(other.shipperId)) { return false; + } if (shippingInfo == null) { - if (other.shippingInfo != null) + if (other.shippingInfo != null) { return false; + } } - else if (!shippingInfo.equals(other.shippingInfo)) + else if (!shippingInfo.equals(other.shippingInfo)) { return false; + } if (shippingTypeId == null) { - if (other.shippingTypeId != null) + if (other.shippingTypeId != null) { return false; + } } - else if (!shippingTypeId.equals(other.shippingTypeId)) + else if (!shippingTypeId.equals(other.shippingTypeId)) { return false; + } + return true; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/OrderItemReader.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/OrderItemReader.java index d846fa6a41..3a4c110bc5 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/OrderItemReader.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/OrderItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ import org.springframework.batch.sample.domain.order.LineItem; import org.springframework.batch.sample.domain.order.Order; import org.springframework.batch.sample.domain.order.ShippingInfo; +import org.springframework.lang.Nullable; /** * @author peter.zozom @@ -56,9 +57,10 @@ public class OrderItemReader implements ItemReader { private ItemReader
        fieldSetReader; /** - * @throws Exception * @see org.springframework.batch.item.ItemReader#read() */ + @Nullable + @Override public Order read() throws Exception { recordFinished = false; @@ -85,7 +87,6 @@ private void process(FieldSet fieldSet) throws Exception { String lineId = fieldSet.readString(0); - // start a new Order if (Order.LINE_ID_HEADER.equals(lineId)) { log.debug("STARTING NEW RECORD"); order = headerMapper.mapFieldSet(fieldSet); @@ -138,14 +139,13 @@ else if (ShippingInfo.LINE_ID_SHIPPING_INFO.equals(lineId)) { else if (LineItem.LINE_ID_ITEM.equals(lineId)) { log.debug("MAPPING LINE ITEM"); if (order.getLineItems() == null) { - order.setLineItems(new ArrayList()); + order.setLineItems(new ArrayList<>()); } order.getLineItems().add(itemMapper.mapFieldSet(fieldSet)); } else { log.debug("Could not map LINE_ID=" + lineId); } - } /** diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/OrderLineAggregator.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/OrderLineAggregator.java index 262d92498b..9991b54c53 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/OrderLineAggregator.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/OrderLineAggregator.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,6 +34,7 @@ public class OrderLineAggregator implements LineAggregator { private Map> aggregators; + @Override public String aggregate(Order order) { StringBuilder result = new StringBuilder(); @@ -54,7 +55,8 @@ public String aggregate(Order order) { /** * Set aggregators for all types of lines in the output file * - * @param aggregators + * @param aggregators Map of LineAggregators used to map the various record types for + * each order */ public void setAggregators(Map> aggregators) { this.aggregators = aggregators; diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/AddressFieldExtractor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/AddressFieldExtractor.java index e100431f8b..0b61d84fd1 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/AddressFieldExtractor.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/AddressFieldExtractor.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order.internal.extractor; import org.springframework.batch.item.file.transform.FieldExtractor; @@ -10,6 +25,7 @@ */ public class AddressFieldExtractor implements FieldExtractor { + @Override public Object[] extract(Order order) { Address address = order.getBillingAddress(); return new Object[] { "ADDRESS:", address.getAddrLine1(), address.getCity(), address.getZipCode() }; diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/BillingInfoFieldExtractor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/BillingInfoFieldExtractor.java index 4114425323..d2f94e7f69 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/BillingInfoFieldExtractor.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/BillingInfoFieldExtractor.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order.internal.extractor; import org.springframework.batch.item.file.transform.FieldExtractor; @@ -10,6 +25,7 @@ */ public class BillingInfoFieldExtractor implements FieldExtractor { + @Override public Object[] extract(Order order) { BillingInfo billingInfo = order.getBilling(); return new Object[] { "BILLING:", billingInfo.getPaymentId(), billingInfo.getPaymentDesc() }; diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/CustomerFieldExtractor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/CustomerFieldExtractor.java index 7c06489c49..3411cd88bc 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/CustomerFieldExtractor.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/CustomerFieldExtractor.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order.internal.extractor; import org.springframework.batch.item.file.transform.FieldExtractor; @@ -10,6 +25,7 @@ */ public class CustomerFieldExtractor implements FieldExtractor { + @Override public Object[] extract(Order order) { Customer customer = order.getCustomer(); return new Object[] { "CUSTOMER:", customer.getRegistrationId(), emptyIfNull(customer.getFirstName()), diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/FooterFieldExtractor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/FooterFieldExtractor.java index f8295e29d2..6937c490d7 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/FooterFieldExtractor.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/FooterFieldExtractor.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order.internal.extractor; import org.springframework.batch.item.file.transform.FieldExtractor; @@ -9,6 +24,7 @@ */ public class FooterFieldExtractor implements FieldExtractor { + @Override public Object[] extract(Order order) { return new Object[] { "END_ORDER:", order.getTotalPrice() }; } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/HeaderFieldExtractor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/HeaderFieldExtractor.java index 61923c2153..4497e3af1a 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/HeaderFieldExtractor.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/HeaderFieldExtractor.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order.internal.extractor; import java.text.SimpleDateFormat; @@ -10,11 +25,10 @@ * @since 2.0.1 */ public class HeaderFieldExtractor implements FieldExtractor { + private SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd"); - private static final SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd"); - + @Override public Object[] extract(Order order) { return new Object[] { "BEGIN_ORDER:", order.getOrderId(), dateFormat.format(order.getOrderDate()) }; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/LineItemFieldExtractor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/LineItemFieldExtractor.java index 3fa8cf67de..abbf93d57d 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/LineItemFieldExtractor.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/extractor/LineItemFieldExtractor.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order.internal.extractor; import org.springframework.batch.item.file.transform.FieldExtractor; @@ -9,6 +24,7 @@ */ public class LineItemFieldExtractor implements FieldExtractor { + @Override public Object[] extract(LineItem item) { return new Object[] { "ITEM:", item.getItemId(), item.getPrice() }; } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/AddressFieldSetMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/AddressFieldSetMapper.java index 4ff9079d9f..9c82cd4d12 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/AddressFieldSetMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/AddressFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,6 +30,7 @@ public class AddressFieldSetMapper implements FieldSetMapper
        { public static final String STATE_COLUMN = "STATE"; public static final String COUNTRY_COLUMN = "COUNTRY"; + @Override public Address mapFieldSet(FieldSet fieldSet) { Address address = new Address(); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/BillingFieldSetMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/BillingFieldSetMapper.java index 3c28a35d8c..e86e11d172 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/BillingFieldSetMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/BillingFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,6 +25,7 @@ public class BillingFieldSetMapper implements FieldSetMapper { public static final String PAYMENT_TYPE_ID_COLUMN = "PAYMENT_TYPE_ID"; public static final String PAYMENT_DESC_COLUMN = "PAYMENT_DESC"; + @Override public BillingInfo mapFieldSet(FieldSet fieldSet) { BillingInfo info = new BillingInfo(); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/CustomerFieldSetMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/CustomerFieldSetMapper.java index 1d7f3c077f..d4149d769b 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/CustomerFieldSetMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/CustomerFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,6 +32,7 @@ public class CustomerFieldSetMapper implements FieldSetMapper { public static final String REG_ID_COLUMN = "REG_ID"; public static final String VIP_COLUMN = "VIP"; + @Override public Customer mapFieldSet(FieldSet fieldSet) { Customer customer = new Customer(); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/HeaderFieldSetMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/HeaderFieldSetMapper.java index 168b68d658..1410ce7d26 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/HeaderFieldSetMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/HeaderFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,6 +25,7 @@ public class HeaderFieldSetMapper implements FieldSetMapper { public static final String ORDER_ID_COLUMN = "ORDER_ID"; public static final String ORDER_DATE_COLUMN = "ORDER_DATE"; + @Override public Order mapFieldSet(FieldSet fieldSet) { Order order = new Order(); order.setOrderId(fieldSet.readLong(ORDER_ID_COLUMN)); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/OrderItemFieldSetMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/OrderItemFieldSetMapper.java index 86a3d3d8f8..f738ec3390 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/OrderItemFieldSetMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/OrderItemFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,6 +31,7 @@ public class OrderItemFieldSetMapper implements FieldSetMapper { public static final String PRICE_COLUMN = "PRICE"; public static final String ITEM_ID_COLUMN = "ITEM_ID"; + @Override public LineItem mapFieldSet(FieldSet fieldSet) { LineItem item = new LineItem(); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/ShippingFieldSetMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/ShippingFieldSetMapper.java index 6bc319bfee..9b45268073 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/ShippingFieldSetMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/mapper/ShippingFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,6 +26,7 @@ public class ShippingFieldSetMapper implements FieldSetMapper { public static final String SHIPPING_TYPE_ID_COLUMN = "SHIPPING_TYPE_ID"; public static final String SHIPPER_ID_COLUMN = "SHIPPER_ID"; + @Override public ShippingInfo mapFieldSet(FieldSet fieldSet) { ShippingInfo info = new ShippingInfo(); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/FutureDateFunction.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/FutureDateFunction.java deleted file mode 100644 index 47cb1a2c08..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/FutureDateFunction.java +++ /dev/null @@ -1,55 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.sample.domain.order.internal.valang; - -import java.util.Date; - -import org.springmodules.validation.valang.functions.AbstractFunction; -import org.springmodules.validation.valang.functions.Function; - - -/** - * Returns Boolean.TRUE if given value is future date, else it returns Boolean.FALSE - * @author peter.zozom - */ -public class FutureDateFunction extends AbstractFunction { - - public FutureDateFunction(Function[] arguments, int line, int column) { - super(arguments, line, column); - definedExactNumberOfArguments(1); - } - - /** - * @see org.springmodules.validation.valang.functions.AbstractFunction#doGetResult(java.lang.Object) - */ - protected Object doGetResult(final Object target) throws Exception { - //get argument - final Object value = getArguments()[0].getResult(target); - - Boolean result; - - if (value instanceof Date) { - final Date now = new Date(System.currentTimeMillis()); - final Date date = (Date) value; - result = (now.compareTo(date) < 0) ? Boolean.TRUE : Boolean.FALSE; - } else { - throw new Exception("No Date value for validation"); - } - - return result; - } -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/TotalOrderItemsFunction.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/TotalOrderItemsFunction.java deleted file mode 100644 index b541c37ad1..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/TotalOrderItemsFunction.java +++ /dev/null @@ -1,63 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.sample.domain.order.internal.valang; - -import java.util.List; - -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.AbstractFunction; -import org.springmodules.validation.valang.functions.Function; - - -/** - * Validates total items count in Order. - * - * @author peter.zozom - */ -public class TotalOrderItemsFunction extends AbstractFunction { - public TotalOrderItemsFunction(Function[] arguments, int line, int column) { - super(arguments, line, column); - definedExactNumberOfArguments(2); - } - - /** - * @see org.springmodules.validation.valang.functions.AbstractFunction#doGetResult(java.lang.Object) - */ - @SuppressWarnings("unchecked") - protected Object doGetResult(Object target) throws Exception { - //get arguments - int count = (Integer) getArguments()[0].getResult(target); - Object value = getArguments()[1].getResult(target); - - Boolean result; - - //count items in list of order lines - if (value instanceof List) { - int totalItems = 0; - - for (LineItem lineItem : ((List) value)) { - totalItems += lineItem.getQuantity(); - } - - result = (totalItems == count) ? Boolean.TRUE : Boolean.FALSE; - } else { - throw new Exception("No list for validation"); - } - - return result; - } -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateDiscountsFunction.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateDiscountsFunction.java deleted file mode 100644 index 38fcc23fb5..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateDiscountsFunction.java +++ /dev/null @@ -1,68 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.sample.domain.order.internal.valang; - -import java.math.BigDecimal; -import java.util.List; - -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.AbstractFunction; -import org.springmodules.validation.valang.functions.Function; - - -/** - * @author peter.zozom - * - */ -public class ValidateDiscountsFunction extends AbstractFunction { - private static final BigDecimal BD_0 = new BigDecimal(0.0); - private static final BigDecimal BD_PERC_MAX = new BigDecimal(100.0); - - public ValidateDiscountsFunction(Function[] arguments, int line, int column) { - super(arguments, line, column); - definedExactNumberOfArguments(1); - } - - /** - * @see org.springmodules.validation.valang.functions.AbstractFunction#doGetResult(java.lang.Object) - */ - @SuppressWarnings("unchecked") - protected Object doGetResult(Object target) throws Exception { - List lineItems = (List) getArguments()[0].getResult(target); - - for (LineItem item : lineItems) { - - if (BD_0.compareTo(item.getDiscountPerc()) != 0) { - //DiscountPerc must be between 0.0 and 100.0 - if ((BD_0.compareTo(item.getDiscountPerc()) > 0) - || (BD_PERC_MAX.compareTo(item.getDiscountPerc()) < 0) - || (BD_0.compareTo(item.getDiscountAmount()) != 0)) { //only one of DiscountAmount and DiscountPerc should be non-zero - - return Boolean.FALSE; - } - } else { - //DiscountAmount must be between 0.0 and item.price - if ((BD_0.compareTo(item.getDiscountAmount()) > 0) - || (item.getPrice().compareTo(item.getDiscountAmount()) < 0)) { - return Boolean.FALSE; - } - } - } - - return Boolean.TRUE; - } -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateHandlingPricesFunction.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateHandlingPricesFunction.java deleted file mode 100644 index 29558b768c..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateHandlingPricesFunction.java +++ /dev/null @@ -1,57 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.sample.domain.order.internal.valang; - -import java.math.BigDecimal; -import java.util.List; - -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.AbstractFunction; -import org.springmodules.validation.valang.functions.Function; - - -/** - * @author peter.zozom - * - */ -public class ValidateHandlingPricesFunction extends AbstractFunction { - private static final BigDecimal BD_MIN = new BigDecimal(0.0); - private static final BigDecimal BD_MAX = new BigDecimal(99999999.99); - - public ValidateHandlingPricesFunction(Function[] arguments, int line, int column) { - super(arguments, line, column); - definedExactNumberOfArguments(1); - } - - /** - * @see org.springmodules.validation.valang.functions.AbstractFunction#doGetResult(java.lang.Object) - */ - @SuppressWarnings("unchecked") - protected Object doGetResult(Object target) throws Exception { - List lineItems = (List) getArguments()[0].getResult(target); - - for (LineItem item : lineItems) { - - if ((BD_MIN.compareTo(item.getHandlingPrice()) > 0) - || (BD_MAX.compareTo(item.getHandlingPrice()) < 0)) { - return Boolean.FALSE; - } - } - - return Boolean.TRUE; - } -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateIdsFunction.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateIdsFunction.java deleted file mode 100644 index 85f08feb05..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateIdsFunction.java +++ /dev/null @@ -1,54 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.sample.domain.order.internal.valang; - -import java.util.List; - -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.AbstractFunction; -import org.springmodules.validation.valang.functions.Function; - - -/** - * @author peter.zozom - * - */ -public class ValidateIdsFunction extends AbstractFunction { - private static final long MAX_ID = 9999999999L; - - public ValidateIdsFunction(Function[] arguments, int line, int column) { - super(arguments, line, column); - definedExactNumberOfArguments(1); - } - - /** - * @see org.springmodules.validation.valang.functions.AbstractFunction#doGetResult(java.lang.Object) - */ - @SuppressWarnings("unchecked") - protected Object doGetResult(Object target) throws Exception { - List lineItems = (List) getArguments()[0].getResult(target); - - for (LineItem item : lineItems) { - - if ((item.getItemId() <= 0) || (item.getItemId() > MAX_ID)) { - return Boolean.FALSE; - } - } - - return Boolean.TRUE; - } -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidatePricesFunction.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidatePricesFunction.java deleted file mode 100644 index 7157abc86a..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidatePricesFunction.java +++ /dev/null @@ -1,56 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.sample.domain.order.internal.valang; - -import java.math.BigDecimal; -import java.util.List; - -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.AbstractFunction; -import org.springmodules.validation.valang.functions.Function; - - -/** - * @author peter.zozom - * - */ -public class ValidatePricesFunction extends AbstractFunction { - private static final BigDecimal BD_MIN = new BigDecimal(0.0); - private static final BigDecimal BD_MAX = new BigDecimal(99999999.99); - - public ValidatePricesFunction(Function[] arguments, int line, int column) { - super(arguments, line, column); - definedExactNumberOfArguments(1); - } - - /** - * @see org.springmodules.validation.valang.functions.AbstractFunction#doGetResult(java.lang.Object) - */ - @SuppressWarnings("unchecked") - protected Object doGetResult(Object target) throws Exception { - List lineItems = (List) getArguments()[0].getResult(target); - - for (LineItem item : lineItems) { - - if ((BD_MIN.compareTo(item.getPrice()) > 0) || (BD_MAX.compareTo(item.getPrice()) < 0)) { - return Boolean.FALSE; - } - } - - return Boolean.TRUE; - } -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateQuantitiesFunction.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateQuantitiesFunction.java deleted file mode 100644 index fbb17e77dd..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateQuantitiesFunction.java +++ /dev/null @@ -1,54 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.sample.domain.order.internal.valang; - -import java.util.List; - -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.AbstractFunction; -import org.springmodules.validation.valang.functions.Function; - - -/** - * @author peter.zozom - * - */ -public class ValidateQuantitiesFunction extends AbstractFunction { - private static final int MAX_QUANTITY = 9999; - - public ValidateQuantitiesFunction(Function[] arguments, int line, int column) { - super(arguments, line, column); - definedExactNumberOfArguments(1); - } - - /** - * @see org.springmodules.validation.valang.functions.AbstractFunction#doGetResult(java.lang.Object) - */ - @SuppressWarnings("unchecked") - protected Object doGetResult(Object target) throws Exception { - List lineItems = (List) getArguments()[0].getResult(target); - - for (LineItem item : lineItems) { - - if ((item.getQuantity() <= 0) || (item.getQuantity() > MAX_QUANTITY)) { - return Boolean.FALSE; - } - } - - return Boolean.TRUE; - } -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateShippingPricesFunction.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateShippingPricesFunction.java deleted file mode 100644 index e393a92733..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateShippingPricesFunction.java +++ /dev/null @@ -1,57 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.sample.domain.order.internal.valang; - -import java.math.BigDecimal; -import java.util.List; - -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.AbstractFunction; -import org.springmodules.validation.valang.functions.Function; - - -/** - * @author peter.zozom - * - */ -public class ValidateShippingPricesFunction extends AbstractFunction { - private static final BigDecimal BD_MIN = new BigDecimal(0.0); - private static final BigDecimal BD_MAX = new BigDecimal(99999999.99); - - public ValidateShippingPricesFunction(Function[] arguments, int line, int column) { - super(arguments, line, column); - definedExactNumberOfArguments(1); - } - - /** - * @see org.springmodules.validation.valang.functions.AbstractFunction#doGetResult(java.lang.Object) - */ - @SuppressWarnings("unchecked") - protected Object doGetResult(Object target) throws Exception { - List lineItems = (List) getArguments()[0].getResult(target); - - for (LineItem item : lineItems) { - - if ((BD_MIN.compareTo(item.getShippingPrice()) > 0) - || (BD_MAX.compareTo(item.getShippingPrice()) < 0)) { - return Boolean.FALSE; - } - } - - return Boolean.TRUE; - } -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateTotalPricesFunction.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateTotalPricesFunction.java deleted file mode 100644 index 3bf54744dc..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateTotalPricesFunction.java +++ /dev/null @@ -1,83 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.sample.domain.order.internal.valang; - -import java.math.BigDecimal; -import java.util.List; - -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.AbstractFunction; -import org.springmodules.validation.valang.functions.Function; - - -/** - * @author peter.zozom - * - */ -public class ValidateTotalPricesFunction extends AbstractFunction { - private static final BigDecimal BD_MIN = new BigDecimal(0.0); - private static final BigDecimal BD_MAX = new BigDecimal(99999999.99); - private static final BigDecimal BD_100 = new BigDecimal(100.00); - - public ValidateTotalPricesFunction(Function[] arguments, int line, int column) { - super(arguments, line, column); - definedExactNumberOfArguments(1); - } - - /** - * @see org.springmodules.validation.valang.functions.AbstractFunction#doGetResult(java.lang.Object) - */ - @SuppressWarnings("unchecked") - protected Object doGetResult(Object target) throws Exception { - List lineItems = (List) getArguments()[0].getResult(target); - - for (LineItem item : lineItems) { - - if ((BD_MIN.compareTo(item.getTotalPrice()) > 0) - || (BD_MAX.compareTo(item.getTotalPrice()) < 0)) { - return Boolean.FALSE; - } - - //calculate total price - - //discount coeficient = (100.00 - discountPerc) / 100.00 - BigDecimal coef = BD_100.subtract(item.getDiscountPerc()) - .divide(BD_100, 4, BigDecimal.ROUND_HALF_UP); - - //discountedPrice = (price * coef) - discountAmount - //at least one of discountPerc and discountAmount is 0 - this is validated by ValidateDiscountsFunction - BigDecimal discountedPrice = item.getPrice().multiply(coef) - .subtract(item.getDiscountAmount()); - - //price for single item = discountedPrice + shipping + handling - BigDecimal singleItemPrice = discountedPrice.add(item.getShippingPrice()) - .add(item.getHandlingPrice()); - - //total price = singleItemPrice * quantity - BigDecimal quantity = new BigDecimal(item.getQuantity()); - BigDecimal totalPrice = singleItemPrice.multiply(quantity) - .setScale(2, BigDecimal.ROUND_HALF_UP); - - //calculatedPrice should equal to item.totalPrice - if (totalPrice.compareTo(item.getTotalPrice()) != 0) { - return Boolean.FALSE; - } - } - - return Boolean.TRUE; - } -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/validator/OrderValidator.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/validator/OrderValidator.java new file mode 100644 index 0000000000..d24452a66d --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/validator/OrderValidator.java @@ -0,0 +1,289 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.domain.order.internal.validator; + +import java.math.BigDecimal; +import java.util.ArrayList; +import java.util.Date; +import java.util.List; + +import org.springframework.batch.sample.domain.order.Address; +import org.springframework.batch.sample.domain.order.BillingInfo; +import org.springframework.batch.sample.domain.order.Customer; +import org.springframework.batch.sample.domain.order.LineItem; +import org.springframework.batch.sample.domain.order.Order; +import org.springframework.batch.sample.domain.order.ShippingInfo; +import org.springframework.util.StringUtils; +import org.springframework.validation.Errors; +import org.springframework.validation.Validator; + +public class OrderValidator implements Validator { + + private static final List CARD_TYPES = new ArrayList<>(); + private static final List SHIPPER_IDS = new ArrayList<>(); + private static final List SHIPPER_TYPES = new ArrayList<>(); + private static final long MAX_ID = 9999999999L; + private static final BigDecimal BD_MIN = new BigDecimal("0.0"); + private static final BigDecimal BD_MAX = new BigDecimal("99999999.99"); + private static final BigDecimal BD_PERC_MAX = new BigDecimal("100.0"); + private static final int MAX_QUANTITY = 9999; + private static final BigDecimal BD_100 = new BigDecimal("100.00"); + + static { + CARD_TYPES.add("VISA"); + CARD_TYPES.add("AMEX"); + CARD_TYPES.add("ECMC"); + CARD_TYPES.add("DCIN"); + CARD_TYPES.add("PAYP"); + + SHIPPER_IDS.add("FEDX"); + SHIPPER_IDS.add("UPS"); + SHIPPER_IDS.add("DHL"); + SHIPPER_IDS.add("DPD"); + + SHIPPER_TYPES.add("STD"); + SHIPPER_TYPES.add("EXP"); + SHIPPER_TYPES.add("AMS"); + SHIPPER_TYPES.add("AME"); + } + + @Override + public boolean supports(Class arg0) { + return arg0.isAssignableFrom(Order.class); + } + + @Override + public void validate(Object arg0, Errors errors) { + Order item = null; + try { + item = (Order) arg0; + } catch (ClassCastException cce) { + errors.reject("Incorrect type"); + } + + if(item != null) { + validateOrder(item, errors); + validateCustomer(item.getCustomer(), errors); + validateAddress(item.getBillingAddress(), errors, "billingAddress"); + validateAddress(item.getShippingAddress(), errors, "shippingAddress"); + validatePayment(item.getBilling(), errors); + validateShipping(item.getShipping(), errors); + validateLineItems(item.getLineItems(), errors); + } + } + + protected void validateLineItems(List lineItems, Errors errors) { + boolean ids = true; + boolean prices = true; + boolean discounts = true; + boolean shippingPrices = true; + boolean handlingPrices = true; + boolean quantities = true; + boolean totalPrices = true; + + for (LineItem lineItem : lineItems) { + if(lineItem.getItemId() <= 0 || lineItem.getItemId() > MAX_ID) { + ids = false; + } + + if((BD_MIN.compareTo(lineItem.getPrice()) > 0) || (BD_MAX.compareTo(lineItem.getPrice()) < 0)) { + prices = false; + } + + if (BD_MIN.compareTo(lineItem.getDiscountPerc()) != 0) { + //DiscountPerc must be between 0.0 and 100.0 + if ((BD_MIN.compareTo(lineItem.getDiscountPerc()) > 0) + || (BD_PERC_MAX.compareTo(lineItem.getDiscountPerc()) < 0) + || (BD_MIN.compareTo(lineItem.getDiscountAmount()) != 0)) { //only one of DiscountAmount and DiscountPerc should be non-zero + discounts = false; + } + } else { + //DiscountAmount must be between 0.0 and item.price + if ((BD_MIN.compareTo(lineItem.getDiscountAmount()) > 0) + || (lineItem.getPrice().compareTo(lineItem.getDiscountAmount()) < 0)) { + discounts = false; + } + } + + if ((BD_MIN.compareTo(lineItem.getShippingPrice()) > 0) || (BD_MAX.compareTo(lineItem.getShippingPrice()) < 0)) { + shippingPrices = false; + } + + if ((BD_MIN.compareTo(lineItem.getHandlingPrice()) > 0) || (BD_MAX.compareTo(lineItem.getHandlingPrice()) < 0)) { + handlingPrices = false; + } + + if ((lineItem.getQuantity() <= 0) || (lineItem.getQuantity() > MAX_QUANTITY)) { + quantities = false; + } + + + if ((BD_MIN.compareTo(lineItem.getTotalPrice()) > 0) + || (BD_MAX.compareTo(lineItem.getTotalPrice()) < 0)) { + totalPrices = false; + } + + //calculate total price + + //discount coefficient = (100.00 - discountPerc) / 100.00 + BigDecimal coef = BD_100.subtract(lineItem.getDiscountPerc()) + .divide(BD_100, 4, BigDecimal.ROUND_HALF_UP); + + //discountedPrice = (price * coefficient) - discountAmount + //at least one of discountPerc and discountAmount is 0 - this is validated by ValidateDiscountsFunction + BigDecimal discountedPrice = lineItem.getPrice().multiply(coef) + .subtract(lineItem.getDiscountAmount()); + + //price for single item = discountedPrice + shipping + handling + BigDecimal singleItemPrice = discountedPrice.add(lineItem.getShippingPrice()) + .add(lineItem.getHandlingPrice()); + + //total price = singleItemPrice * quantity + BigDecimal quantity = new BigDecimal(lineItem.getQuantity()); + BigDecimal totalPrice = singleItemPrice.multiply(quantity) + .setScale(2, BigDecimal.ROUND_HALF_UP); + + //calculatedPrice should equal to item.totalPrice + if (totalPrice.compareTo(lineItem.getTotalPrice()) != 0) { + totalPrices = false; + } + } + + String lineItemsFieldName = "lineItems"; + + if(!ids) { + errors.rejectValue(lineItemsFieldName, "error.lineitems.id"); + } + + if(!prices) { + errors.rejectValue(lineItemsFieldName, "error.lineitems.price"); + } + + if(!discounts) { + errors.rejectValue(lineItemsFieldName, "error.lineitems.discount"); + } + + if(!shippingPrices) { + errors.rejectValue(lineItemsFieldName, "error.lineitems.shipping"); + } + + if(!handlingPrices) { + errors.rejectValue(lineItemsFieldName, "error.lineitems.handling"); + } + + if(!quantities) { + errors.rejectValue(lineItemsFieldName, "error.lineitems.quantity"); + } + + if(!totalPrices) { + errors.rejectValue(lineItemsFieldName, "error.lineitems.totalprice"); + } + } + + protected void validateShipping(ShippingInfo shipping, Errors errors) { + if(!SHIPPER_IDS.contains(shipping.getShipperId())) { + errors.rejectValue("shipping.shipperId", "error.shipping.shipper"); + } + + if(!SHIPPER_TYPES.contains(shipping.getShippingTypeId())) { + errors.rejectValue("shipping.shippingTypeId", "error.shipping.type"); + } + + if(StringUtils.hasText(shipping.getShippingInfo())) { + validateStringLength(shipping.getShippingInfo(), errors, "shipping.shippingInfo", "error.shipping.shippinginfo.length", 100); + } + } + + protected void validatePayment(BillingInfo billing, Errors errors) { + if(!CARD_TYPES.contains(billing.getPaymentId())) { + errors.rejectValue("billing.paymentId", "error.billing.type"); + } + + if(!billing.getPaymentDesc().matches("[A-Z]{4}-[0-9]{10,11}")) { + errors.rejectValue("billing.paymentDesc", "error.billing.desc"); + } + } + + protected void validateAddress(Address address, Errors errors, + String prefix) { + if(address != null) { + if(StringUtils.hasText(address.getAddressee())) { + validateStringLength(address.getAddressee(), errors, prefix + ".addressee", "error.baddress.addresse.length", 60); + } + + validateStringLength(address.getAddrLine1(), errors, prefix + ".addrLine1", "error.baddress.addrline1.length", 50); + + if(StringUtils.hasText(address.getAddrLine2())) { + validateStringLength(address.getAddrLine2(), errors, prefix + ".addrLine2", "error.baddress.addrline2.length", 50); + } + validateStringLength(address.getCity(), errors, prefix + ".city", "error.baddress.city.length", 30); + validateStringLength(address.getZipCode(), errors, prefix + ".zipCode", "error.baddress.zipcode.length", 5); + + if(StringUtils.hasText(address.getZipCode()) && !address.getZipCode().matches("[0-9]{5}")) { + errors.rejectValue(prefix + ".zipCode", "error.baddress.zipcode.format"); + } + + if((!StringUtils.hasText(address.getState()) && ("United States".equals(address.getCountry())) || StringUtils.hasText(address.getState()) && address.getState().length() != 2)) { + errors.rejectValue(prefix + ".state", "error.baddress.state.length"); + } + + validateStringLength(address.getCountry(), errors, prefix + ".country", "error.baddress.country.length", 50); + } + } + + protected void validateStringLength(String string, Errors errors, + String field, String message, int length) { + if(!StringUtils.hasText(string) || string.length() > length) { + errors.rejectValue(field, message); + } + } + + protected void validateCustomer(Customer customer, Errors errors) { + if(!customer.isRegistered() && customer.isBusinessCustomer()) { + errors.rejectValue("customer.registered", "error.customer.registration"); + } + + if(!StringUtils.hasText(customer.getCompanyName()) && customer.isBusinessCustomer()) { + errors.rejectValue("customer.companyName", "error.customer.companyname"); + } + + if(!StringUtils.hasText(customer.getFirstName()) && !customer.isBusinessCustomer()) { + errors.rejectValue("customer.firstName", "error.customer.firstname"); + } + + if(!StringUtils.hasText(customer.getLastName()) && !customer.isBusinessCustomer()) { + errors.rejectValue("customer.lastName", "error.customer.lastname"); + } + + if(customer.isRegistered() && (customer.getRegistrationId() < 0 || customer.getRegistrationId() >= 99999999L)) { + errors.rejectValue("customer.registrationId", "error.customer.registrationid"); + } + } + + protected void validateOrder(Order item, Errors errors) { + if(item.getOrderId() < 0 || item.getOrderId() > 9999999999L) { + errors.rejectValue("orderId", "error.order.id"); + } + + if(new Date().compareTo(item.getOrderDate()) < 0) { + errors.rejectValue("orderDate", "error.order.date.future"); + } + + if(item.getLineItems() != null && item.getTotalLines() != item.getLineItems().size()) { + errors.rejectValue("totalLines", "error.order.lines.badcount"); + } + } +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/Customer.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/Customer.java index 39dcb6ab3b..2d711fd4de 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/Customer.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/Customer.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/LineItem.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/LineItem.java index e614ce4490..5a497431b0 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/LineItem.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/LineItem.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/Order.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/Order.java index 238d0099f9..0e076b09a3 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/Order.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/Order.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -42,11 +42,11 @@ public void setCustomer(Customer customer) { } public Date getDate() { - return date; + return date != null ? new Date(date.getTime()) : null; } public void setDate(Date date) { - this.date = date; + this.date = date != null ? new Date(date.getTime()) : null; } public List getLineItems() { diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/Shipper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/Shipper.java index 696e28cd1b..5761515eeb 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/Shipper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/order/internal/xml/Shipper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/Child.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/Child.java index 0571fdaae3..8abea70dba 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/Child.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/Child.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -13,12 +13,9 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.springframework.batch.sample.domain.person; - public class Child { - private String name; public void setName(String name){ @@ -44,20 +41,29 @@ public int hashCode() { @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + + if (getClass() != obj.getClass()) { return false; + } + Child other = (Child) obj; + if (name == null) { - if (other.name != null) + if (other.name != null) { return false; + } } - else if (!name.equals(other.name)) + else if (!name.equals(other.name)) { return false; + } + return true; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/Person.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/Person.java index 4ee7af83fa..3430a33bdd 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/Person.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/Person.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,18 +22,12 @@ import org.springframework.batch.sample.domain.order.Address; public class Person { - private String title = ""; - private String firstName = ""; - private String last_name = ""; - private int age = 0; - private Address address = new Address(); - - private List children = new ArrayList(); + private List children = new ArrayList<>(); public Person() { children.add(new Child()); @@ -147,46 +141,69 @@ public int hashCode() { @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + + if (getClass() != obj.getClass()) { return false; + } + Person other = (Person) obj; + if (address == null) { - if (other.address != null) + if (other.address != null) { return false; + } } - else if (!address.equals(other.address)) + else if (!address.equals(other.address)) { return false; - if (age != other.age) + } + + if (age != other.age) { return false; + } + if (children == null) { - if (other.children != null) + if (other.children != null) { return false; + } } - else if (!children.equals(other.children)) + else if (!children.equals(other.children)) { return false; + } + if (firstName == null) { - if (other.firstName != null) + if (other.firstName != null) { return false; + } } - else if (!firstName.equals(other.firstName)) + else if (!firstName.equals(other.firstName)) { return false; + } + if (last_name == null) { - if (other.last_name != null) + if (other.last_name != null) { return false; + } } - else if (!last_name.equals(other.last_name)) + else if (!last_name.equals(other.last_name)) { return false; + } + if (title == null) { - if (other.title != null) + if (other.title != null) { return false; + } } - else if (!title.equals(other.title)) + else if (!title.equals(other.title)) { return false; + } + return true; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/PersonService.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/PersonService.java index 364640faa8..4dbdc8058d 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/PersonService.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/PersonService.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,33 +19,31 @@ import java.util.ArrayList; import java.util.List; -import org.springframework.batch.item.ItemReader; -import org.springframework.batch.item.ItemWriter; import org.springframework.batch.sample.domain.order.Address; /** * Custom class that contains logic that would normally be be contained in - * {@link ItemReader} and {@link ItemWriter}. + * {@link org.springframework.batch.item.ItemReader} and + * {@link javax.batch.api.chunk.ItemWriter}. * * @author tomas.slanina * @author Robert Kasanicky */ public class PersonService { - private static final int GENERATION_LIMIT = 10; private int generatedCounter = 0; - private int processedCounter = 0; public Person getData() { - if (generatedCounter >= GENERATION_LIMIT) + if (generatedCounter >= GENERATION_LIMIT) { return null; + } Person person = new Person(); Address address = new Address(); Child child = new Child(); - List children = new ArrayList(1); + List children = new ArrayList<>(1); children.add(child); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/internal/PersonWriter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/internal/PersonWriter.java index c83838a876..bd10d66137 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/internal/PersonWriter.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/person/internal/PersonWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,13 +23,11 @@ import org.springframework.batch.item.ItemWriter; import org.springframework.batch.sample.domain.person.Person; - - public class PersonWriter implements ItemWriter { private static Log log = LogFactory.getLog(PersonWriter.class); - public void write(List data) { + @Override + public void write(List data) { log.debug("Processing: " + data); } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CompositeCustomerUpdateLineTokenizer.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CompositeCustomerUpdateLineTokenizer.java index 3f10b72810..d124d7f443 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CompositeCustomerUpdateLineTokenizer.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CompositeCustomerUpdateLineTokenizer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ import org.springframework.batch.core.listener.StepExecutionListenerSupport; import org.springframework.batch.item.file.transform.FieldSet; import org.springframework.batch.item.file.transform.LineTokenizer; +import org.springframework.lang.Nullable; /** * Composite {@link LineTokenizer} that delegates the tokenization of a line to one of two potential @@ -39,7 +40,8 @@ public class CompositeCustomerUpdateLineTokenizer extends StepExecutionListenerS /* (non-Javadoc) * @see org.springframework.batch.item.file.transform.LineTokenizer#tokenize(java.lang.String) */ - public FieldSet tokenize(String line) { + @Override + public FieldSet tokenize(@Nullable String line) { if(line.charAt(0) == 'F'){ //line starts with F, so the footer tokenizer should tokenize it. @@ -74,8 +76,8 @@ public void beforeStep(StepExecution stepExecution) { /** * Set the {@link LineTokenizer} that will be used to tokenize any lines that begin with * A, U, or D, and are thus a customer operation. - * - * @param customerTokenizer + * + * @param customerTokenizer tokenizer to delegate to for customer operation records */ public void setCustomerTokenizer(LineTokenizer customerTokenizer) { this.customerTokenizer = customerTokenizer; @@ -85,7 +87,7 @@ public void setCustomerTokenizer(LineTokenizer customerTokenizer) { * Set the {@link LineTokenizer} that will be used to tokenize any lines that being with * F and is thus a footer record. * - * @param footerTokenizer + * @param footerTokenizer tokenizer to delegate to for footer records */ public void setFooterTokenizer(LineTokenizer footerTokenizer) { this.footerTokenizer = footerTokenizer; diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerCredit.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerCredit.java index 589ad581ba..e0bcdd90b9 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerCredit.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerCredit.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -42,6 +42,7 @@ public CustomerCredit(int id, String name, BigDecimal credit) { this.credit = credit; } + @Override public String toString() { return "CustomerCredit [id=" + id + ",name=" + name + ", credit=" + credit + "]"; } @@ -78,10 +79,12 @@ public CustomerCredit increaseCreditBy(BigDecimal sum) { return newCredit; } + @Override public boolean equals(Object o) { return (o instanceof CustomerCredit) && ((CustomerCredit) o).id == id; } + @Override public int hashCode() { return id; } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerCreditDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerCreditDao.java index 83f598be8f..4147d096af 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerCreditDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerCreditDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerDao.java index fcfd688364..3612c46d03 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerDebit.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerDebit.java index 95566979d4..48979adbce 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerDebit.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerDebit.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -47,7 +47,8 @@ public void setName(String name) { this.name = name; } - public String toString() { + @Override + public String toString() { return "CustomerDebit [name=" + name + ", debit=" + debit + "]"; } @@ -62,26 +63,33 @@ public int hashCode() { @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + if (getClass() != obj.getClass()) { return false; + } CustomerDebit other = (CustomerDebit) obj; if (debit == null) { - if (other.debit != null) + if (other.debit != null) { return false; + } } - else if (!debit.equals(other.debit)) + else if (!debit.equals(other.debit)) { return false; + } if (name == null) { - if (other.name != null) + if (other.name != null) { return false; + } } - else if (!name.equals(other.name)) + else if (!name.equals(other.name)) { return false; + } + return true; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerDebitDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerDebitDao.java index df52bd652b..2ff1533023 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerDebitDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerDebitDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerOperation.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerOperation.java index de5dcc5f92..40d229fc01 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerOperation.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerOperation.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,22 +30,22 @@ public enum CustomerOperation { ADD('A'), UPDATE('U'), DELETE('D'); private final char code; - private static final Map codeMap; + private static final Map CODE_MAP; private CustomerOperation(char code) { this.code = code; } static{ - codeMap = new HashMap(); + CODE_MAP = new HashMap<>(); for(CustomerOperation operation:values()){ - codeMap.put(operation.getCode(), operation); + CODE_MAP.put(operation.getCode(), operation); } } public static CustomerOperation fromCode(char code){ - if(codeMap.containsKey(code)){ - return codeMap.get(code); + if(CODE_MAP.containsKey(code)){ + return CODE_MAP.get(code); } else{ throw new IllegalArgumentException("Invalid code: [" + code + "]"); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerUpdate.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerUpdate.java index 22e5106671..6559e090be 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerUpdate.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerUpdate.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,7 +20,7 @@ /** * Immutable Value Object representing an update to the customer as stored in the database. - * This object has the customer name, credit ammount, and the operation to be performed + * This object has the customer name, credit amount, and the operation to be performed * on them. In the case of an add, a new customer will be entered with the appropriate * credit. In the case of an update, the customer's credit is considered an absolute update. * Deletes are currently not supported, but can still be read in from a file. diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerUpdateFieldSetMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerUpdateFieldSetMapper.java index e8f9b85c75..44cdc501db 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerUpdateFieldSetMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerUpdateFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,6 +29,7 @@ */ public class CustomerUpdateFieldSetMapper implements FieldSetMapper { + @Override public CustomerUpdate mapFieldSet(FieldSet fs) { if (fs == null) { diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerUpdateProcessor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerUpdateProcessor.java index 74f5737ce3..7ffe0df4e4 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerUpdateProcessor.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/CustomerUpdateProcessor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,6 +19,7 @@ import static org.springframework.batch.sample.domain.trade.CustomerOperation.*; import org.springframework.batch.item.ItemProcessor; +import org.springframework.lang.Nullable; /** * @author Lucas Ward @@ -29,6 +30,8 @@ public class CustomerUpdateProcessor implements ItemProcessor { private CustomerDao customerDao; + @Override public void write(List items) throws Exception { for(CustomerUpdate customerUpdate : items){ if(customerUpdate.getOperation() == CustomerOperation.ADD){ diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/InvalidCustomerLogger.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/InvalidCustomerLogger.java index a3c7079f02..111df25381 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/InvalidCustomerLogger.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/InvalidCustomerLogger.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/Trade.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/Trade.java index f16de15162..e31d16d33b 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/Trade.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/Trade.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,10 +24,11 @@ * @author Rob Harrop * @author Dave Syer */ +@SuppressWarnings("serial") public class Trade implements Serializable { private String isin = ""; private long quantity = 0; - private BigDecimal price = new BigDecimal(0); + private BigDecimal price = BigDecimal.ZERO; private String customer = ""; private Long id; private long version = 0; @@ -43,7 +44,7 @@ public Trade(String isin, long quantity, BigDecimal price, String customer){ } /** - * @param id + * @param id id of the trade */ public Trade(long id) { this.id = id; @@ -97,7 +98,8 @@ public String getCustomer() { return customer; } - public String toString() { + @Override + public String toString() { return "Trade: [isin=" + this.isin + ",quantity=" + this.quantity + ",price=" + this.price + ",customer=" + this.customer + "]"; } @@ -116,36 +118,46 @@ public int hashCode() { @Override public boolean equals(Object obj) { - if (this == obj) + if (this == obj) { return true; - if (obj == null) + } + if (obj == null) { return false; - if (getClass() != obj.getClass()) + } + if (getClass() != obj.getClass()) { return false; + } Trade other = (Trade) obj; if (customer == null) { - if (other.customer != null) + if (other.customer != null) { return false; + } } - else if (!customer.equals(other.customer)) + else if (!customer.equals(other.customer)) { return false; + } if (isin == null) { - if (other.isin != null) + if (other.isin != null) { return false; + } } - else if (!isin.equals(other.isin)) + else if (!isin.equals(other.isin)) { return false; + } if (price == null) { - if (other.price != null) + if (other.price != null) { return false; + } } - else if (!price.equals(other.price)) + else if (!price.equals(other.price)) { return false; - if (quantity != other.quantity) + } + if (quantity != other.quantity) { return false; - if (version != other.version) + } + if (version != other.version) { return false; + } return true; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/TradeDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/TradeDao.java index 2eddfb363a..9bc5d31351 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/TradeDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/TradeDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CommonsLoggingInvalidCustomerLogger.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CommonsLoggingInvalidCustomerLogger.java index 71c3b8a2c9..fc402cc123 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CommonsLoggingInvalidCustomerLogger.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CommonsLoggingInvalidCustomerLogger.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,16 +26,14 @@ * @author Lucas Ward * */ -public class CommonsLoggingInvalidCustomerLogger implements - InvalidCustomerLogger { - - protected static final Log logger = LogFactory.getLog(CommandLineJobRunner.class); +public class CommonsLoggingInvalidCustomerLogger implements InvalidCustomerLogger { + protected static final Log LOG = LogFactory.getLog(CommandLineJobRunner.class); /* (non-Javadoc) * @see org.springframework.batch.sample.domain.trade.InvalidCustomerLogger#log(org.springframework.batch.sample.domain.trade.CustomerUpdate) */ + @Override public void log(CustomerUpdate customerUpdate) { - logger.error("invalid customer encountered: [ " + customerUpdate + "]"); + LOG.error("invalid customer encountered: [ " + customerUpdate + "]"); } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditFieldSetMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditFieldSetMapper.java index ac26678785..3e625e0a7a 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditFieldSetMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditFieldSetMapper.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,13 +25,12 @@ * @since 2.0 */ public class CustomerCreditFieldSetMapper implements FieldSetMapper { - public static final int ID_COLUMN = 0; public static final int NAME_COLUMN = 1; public static final int CREDIT_COLUMN = 2; + @Override public CustomerCredit mapFieldSet(FieldSet fieldSet) { - CustomerCredit trade = new CustomerCredit(); trade.setId(fieldSet.readInt(ID_COLUMN)); trade.setName(fieldSet.readString(NAME_COLUMN)); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditIncreaseProcessor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditIncreaseProcessor.java index 0987183f3d..03f9139811 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditIncreaseProcessor.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditIncreaseProcessor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ import org.springframework.batch.item.ItemProcessor; import org.springframework.batch.sample.domain.trade.CustomerCredit; +import org.springframework.lang.Nullable; /** * Increases customer's credit by a fixed amount. @@ -27,9 +28,10 @@ * @author Robert Kasanicky */ public class CustomerCreditIncreaseProcessor implements ItemProcessor { - public static final BigDecimal FIXED_AMOUNT = new BigDecimal("5"); + @Nullable + @Override public CustomerCredit process(CustomerCredit item) throws Exception { return item.increaseCreditBy(FIXED_AMOUNT); } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditItemWriter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditItemWriter.java index 47a1116344..7322b8a169 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditItemWriter.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -39,6 +39,7 @@ public void setCustomerCreditDao(CustomerCreditDao customerCreditDao) { this.customerCreditDao = customerCreditDao; } + @Override public void write(List customerCredits) throws Exception { for (CustomerCredit customerCredit : customerCredits) { customerCreditDao.writeCredit(customerCredit); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditRowMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditRowMapper.java index 17cc4a44fb..0434ee58aa 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditRowMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditRowMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,13 +22,14 @@ import org.springframework.batch.sample.domain.trade.CustomerCredit; import org.springframework.jdbc.core.RowMapper; -public class CustomerCreditRowMapper implements RowMapper { +public class CustomerCreditRowMapper implements RowMapper { public static final String ID_COLUMN = "id"; public static final String NAME_COLUMN = "name"; public static final String CREDIT_COLUMN = "credit"; - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { + @Override + public CustomerCredit mapRow(ResultSet rs, int rowNum) throws SQLException { CustomerCredit customerCredit = new CustomerCredit(); customerCredit.setId(rs.getInt(ID_COLUMN)); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdatePreparedStatementSetter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdatePreparedStatementSetter.java index 1f2ef6bd28..f93d833a16 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdatePreparedStatementSetter.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdatePreparedStatementSetter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,17 +27,15 @@ * */ public class CustomerCreditUpdatePreparedStatementSetter implements ItemPreparedStatementSetter { - public static final BigDecimal FIXED_AMOUNT = new BigDecimal(1000); - public static final String QUERY = "UPDATE CUSTOMER SET CREDIT=? WHERE ID=?"; /* (non-Javadoc) * @see org.springframework.batch.io.support.ItemPreparedStatementSetter#setValues(java.lang.Object, java.sql.PreparedStatement) */ + @Override public void setValues(CustomerCredit customerCredit, PreparedStatement ps) throws SQLException { ps.setBigDecimal(1, customerCredit.getCredit().add(FIXED_AMOUNT)); ps.setLong(2, customerCredit.getId()); } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdateWriter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdateWriter.java index 46586d438a..3fdf66ec18 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdateWriter.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdateWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,6 +27,7 @@ public class CustomerCreditUpdateWriter implements ItemWriter { private CustomerCreditDao dao; + @Override public void write(List customerCredits) throws Exception { for (CustomerCredit customerCredit : customerCredits) { if (customerCredit.getCredit().doubleValue() > creditFilter) { @@ -42,5 +43,4 @@ public void setCreditFilter(double creditFilter) { public void setDao(CustomerCreditDao dao) { this.dao = dao; } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerDebitRowMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerDebitRowMapper.java index 09a02190d6..b4e84df5b7 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerDebitRowMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerDebitRowMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,12 +23,13 @@ import org.springframework.jdbc.core.RowMapper; -public class CustomerDebitRowMapper implements RowMapper { +public class CustomerDebitRowMapper implements RowMapper { public static final String CUSTOMER_COLUMN = "customer"; public static final String PRICE_COLUMN = "price"; - public Object mapRow(ResultSet rs, int ignoredRowNumber) + @Override + public CustomerDebit mapRow(ResultSet rs, int ignoredRowNumber) throws SQLException { CustomerDebit customerDebit = new CustomerDebit(); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerUpdateWriter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerUpdateWriter.java index 7b6ebac723..3fd5a68066 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerUpdateWriter.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/CustomerUpdateWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -33,6 +33,7 @@ public class CustomerUpdateWriter implements ItemWriter { private CustomerDebitDao dao; + @Override public void write(List trades) { for (Trade trade : trades) { CustomerDebit customerDebit = new CustomerDebit(); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/FlatFileCustomerCreditDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/FlatFileCustomerCreditDao.java index a94303f068..91fd9464bb 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/FlatFileCustomerCreditDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/FlatFileCustomerCreditDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,6 +40,7 @@ public class FlatFileCustomerCreditDao implements CustomerCreditDao, private volatile boolean opened = false; + @Override public void writeCredit(CustomerCredit customerCredit) throws Exception { if (!opened) { @@ -78,6 +79,7 @@ public void close() throws Exception { * * @see org.springframework.beans.factory.DisposableBean#destroy() */ + @Override public void destroy() throws Exception { close(); } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/GeneratingTradeItemReader.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/GeneratingTradeItemReader.java index 6a52c826a2..aa738b2f73 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/GeneratingTradeItemReader.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/GeneratingTradeItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ import org.springframework.batch.item.ItemReader; import org.springframework.batch.sample.domain.trade.Trade; +import org.springframework.lang.Nullable; /** * Generates configurable number of {@link Trade} items. @@ -32,6 +33,8 @@ public class GeneratingTradeItemReader implements ItemReader { private int counter = 0; + @Nullable + @Override public Trade read() throws Exception { if (counter < limit) { counter++; diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/HibernateAwareCustomerCreditItemWriter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/HibernateAwareCustomerCreditItemWriter.java index 4e11588e35..fc8bd6abbc 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/HibernateAwareCustomerCreditItemWriter.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/HibernateAwareCustomerCreditItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -38,6 +38,7 @@ public class HibernateAwareCustomerCreditItemWriter implements ItemWriter items) throws Exception { for (CustomerCredit credit : items) { dao.writeCredit(credit); @@ -61,6 +62,7 @@ public void setSessionFactory(SessionFactory sessionFactory) { this.sessionFactory = sessionFactory; } + @Override public void afterPropertiesSet() throws Exception { Assert.state(sessionFactory != null, "Hibernate SessionFactory is required"); Assert.notNull(dao, "Delegate DAO must be set"); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/HibernateCreditDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/HibernateCreditDao.java index 2cf1a53e81..cf6af264b0 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/HibernateCreditDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/HibernateCreditDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,7 +34,7 @@ public class HibernateCreditDao implements CustomerCreditDao, RepeatListener { private int failOnFlush = -1; - private List errors = new ArrayList(); + private List errors = new ArrayList<>(); private SessionFactory sessionFactory; public void setSessionFactory(SessionFactory sessionFactory) { @@ -55,6 +55,7 @@ public List getErrors() { * * @see org.springframework.batch.sample.domain.trade.internal.CustomerCreditWriter#write(org.springframework.batch.sample.domain.CustomerCredit) */ + @Override public void writeCredit(CustomerCredit customerCredit) { if (customerCredit.getId() == failOnFlush) { // try to insert one with a duplicate ID @@ -87,6 +88,7 @@ public void setFailOnFlush(int failOnFlush) { this.failOnFlush = failOnFlush; } + @Override public void onError(RepeatContext context, Throwable e) { errors.add(e); } @@ -94,24 +96,28 @@ public void onError(RepeatContext context, Throwable e) { /* (non-Javadoc) * @see org.springframework.batch.repeat.RepeatInterceptor#after(org.springframework.batch.repeat.RepeatContext, org.springframework.batch.repeat.ExitStatus) */ + @Override public void after(RepeatContext context, RepeatStatus result) { } /* (non-Javadoc) * @see org.springframework.batch.repeat.RepeatInterceptor#before(org.springframework.batch.repeat.RepeatContext) */ + @Override public void before(RepeatContext context) { } /* (non-Javadoc) * @see org.springframework.batch.repeat.RepeatInterceptor#close(org.springframework.batch.repeat.RepeatContext) */ + @Override public void close(RepeatContext context) { } /* (non-Javadoc) * @see org.springframework.batch.repeat.RepeatInterceptor#open(org.springframework.batch.repeat.RepeatContext) */ + @Override public void open(RepeatContext context) { } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/IbatisCustomerCreditDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/IbatisCustomerCreditDao.java deleted file mode 100644 index 23342431bd..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/IbatisCustomerCreditDao.java +++ /dev/null @@ -1,56 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.sample.domain.trade.internal; - -import org.springframework.batch.sample.domain.trade.CustomerCredit; -import org.springframework.batch.sample.domain.trade.CustomerCreditDao; -import org.springframework.orm.ibatis.support.SqlMapClientDaoSupport; - -/** - * @author Lucas Ward - * - */ -public class IbatisCustomerCreditDao extends SqlMapClientDaoSupport - implements CustomerCreditDao { - - String statementId; - - /* (non-Javadoc) - * @see org.springframework.batch.sample.domain.trade.internal.CustomerCreditWriter#write(org.springframework.batch.sample.domain.CustomerCredit) - */ - public void writeCredit(CustomerCredit customerCredit) { - - getSqlMapClientTemplate().update(statementId, customerCredit); - } - - /* (non-Javadoc) - * @see org.springframework.batch.item.ResourceLifecycle#close() - */ - public void close() { - } - - /* (non-Javadoc) - * @see org.springframework.batch.item.ResourceLifecycle#open() - */ - public void open() { - } - - - public void setStatementId(String statementId) { - this.statementId = statementId; - } - -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDao.java index 76d522f67b..bea06a88cc 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -43,14 +43,15 @@ public void setIncrementer(DataFieldMaxValueIncrementer incrementer) { this.incrementer = incrementer; } + @Override public CustomerCredit getCustomerByName(String name) { - @SuppressWarnings("unchecked") - List customers = (List) getJdbcTemplate().query(GET_CUSTOMER_BY_NAME, new Object[]{name}, + List customers = getJdbcTemplate().query(GET_CUSTOMER_BY_NAME, new Object[]{name}, - new RowMapper(){ + new RowMapper(){ - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { + @Override + public CustomerCredit mapRow(ResultSet rs, int rowNum) throws SQLException { CustomerCredit customer = new CustomerCredit(); customer.setName(rs.getString("NAME")); customer.setId(rs.getInt("ID")); @@ -69,11 +70,13 @@ public Object mapRow(ResultSet rs, int rowNum) throws SQLException { } + @Override public void insertCustomer(String name, BigDecimal credit) { getJdbcTemplate().update(INSERT_CUSTOMER, new Object[]{incrementer.nextIntValue(), name, credit}); } + @Override public void updateCustomer(String name, BigDecimal credit) { getJdbcTemplate().update(UPDATE_CUSTOMER, new Object[]{credit, name}); } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDao.java index efda0ab1b9..704470e106 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,7 +36,8 @@ public class JdbcCustomerDebitDao implements CustomerDebitDao { private JdbcOperations jdbcTemplate; - public void write(CustomerDebit customerDebit) { + @Override + public void write(CustomerDebit customerDebit) { jdbcTemplate.update(UPDATE_CREDIT, customerDebit.getDebit(), customerDebit.getName()); } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/JdbcTradeDao.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/JdbcTradeDao.java index 3b68f97b50..5f947c7124 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/JdbcTradeDao.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/JdbcTradeDao.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,19 +40,20 @@ public class JdbcTradeDao implements TradeDao { private static final String INSERT_TRADE_RECORD = "INSERT INTO TRADE (id, version, isin, quantity, price, customer) VALUES (?, 0, ?, ? ,?, ?)"; /** - * handles the processing of sql query + * handles the processing of SQL query */ private JdbcOperations jdbcTemplate; /** - * database is not expected to be setup for autoincrement + * database is not expected to be setup for auto increment */ private DataFieldMaxValueIncrementer incrementer; /** * @see TradeDao */ - public void writeTrade(Trade trade) { + @Override + public void writeTrade(Trade trade) { Long id = incrementer.nextLongValue(); log.debug("Processing: " + trade); jdbcTemplate.update(INSERT_TRADE_RECORD, diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeFieldSetMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeFieldSetMapper.java index a506f43f45..a698bd2ad8 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeFieldSetMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeFieldSetMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,7 +29,8 @@ public class TradeFieldSetMapper implements FieldSetMapper { public static final int PRICE_COLUMN = 2; public static final int CUSTOMER_COLUMN = 3; - public Trade mapFieldSet(FieldSet fieldSet) { + @Override + public Trade mapFieldSet(FieldSet fieldSet) { Trade trade = new Trade(); trade.setIsin(fieldSet.readString(ISIN_COLUMN)); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeProcessor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeProcessor.java index 23fcad4c2f..a5269f0d1c 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeProcessor.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeProcessor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,6 +19,7 @@ import org.springframework.batch.item.ItemProcessor; import org.springframework.batch.item.validator.ValidationException; import org.springframework.batch.sample.domain.trade.Trade; +import org.springframework.lang.Nullable; /** * Processes the Trade - throwing validation errors if necessary. @@ -40,6 +41,8 @@ public void setValidationFailure(int failure) { this.failure = failure; } + @Nullable + @Override public Trade process(Trade item) throws Exception { if ((failedItem == null && index++ == failure) || (failedItem != null && failedItem.equals(item))) { failedItem = item; diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeRowMapper.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeRowMapper.java index b2451ada6a..d4f970bf41 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeRowMapper.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeRowMapper.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,7 +22,7 @@ import org.springframework.batch.sample.domain.trade.Trade; import org.springframework.jdbc.core.RowMapper; -public class TradeRowMapper implements RowMapper { +public class TradeRowMapper implements RowMapper { public static final int ISIN_COLUMN = 1; public static final int QUANTITY_COLUMN = 2; @@ -31,7 +31,8 @@ public class TradeRowMapper implements RowMapper { public static final int ID_COLUMN = 5; public static final int VERSION_COLUMN = 6; - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { + @Override + public Trade mapRow(ResultSet rs, int rowNum) throws SQLException { Trade trade = new Trade(rs.getLong(ID_COLUMN)); trade.setIsin(rs.getString(ISIN_COLUMN)); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeWriter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeWriter.java index d85a39bd6f..69f0c24d59 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeWriter.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/TradeWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,7 +24,6 @@ import org.apache.commons.logging.LogFactory; import org.springframework.batch.core.annotation.AfterWrite; import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.item.ItemStreamException; import org.springframework.batch.item.ItemStreamSupport; import org.springframework.batch.item.ItemWriter; import org.springframework.batch.item.WriteFailedException; @@ -44,10 +43,11 @@ public class TradeWriter extends ItemStreamSupport implements ItemWriter private TradeDao dao; - private List failingCustomers = new ArrayList(); + private List failingCustomers = new ArrayList<>(); private BigDecimal totalPrice = BigDecimal.ZERO; + @Override public void write(List trades) { for (Trade trade : trades) { @@ -56,7 +56,7 @@ public void write(List trades) { dao.writeTrade(trade); - Assert.notNull(trade.getPrice()); // There must be a price to total + Assert.notNull(trade.getPrice(), "price must not be null"); // There must be a price to total if (this.failingCustomers.contains(trade.getCustomer())) { throw new WriteFailedException("Something unexpected happened!"); @@ -73,7 +73,7 @@ public void updateTotalPrice(List trades) { } @Override - public void open(ExecutionContext executionContext) throws ItemStreamException { + public void open(ExecutionContext executionContext) { if (executionContext.containsKey(TOTAL_AMOUNT_KEY)) { this.totalPrice = (BigDecimal) executionContext.get(TOTAL_AMOUNT_KEY); } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/validator/TradeValidator.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/validator/TradeValidator.java new file mode 100644 index 0000000000..2e0fe71f64 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/trade/internal/validator/TradeValidator.java @@ -0,0 +1,39 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.domain.trade.internal.validator; + +import org.springframework.validation.Errors; +import org.springframework.validation.Validator; +import org.springframework.batch.sample.domain.trade.Trade; + +/** + * @author Michael Minella + */ +public class TradeValidator implements Validator { + @Override + public boolean supports(Class clazz) { + return clazz.equals(Trade.class); + } + + @Override + public void validate(Object target, Errors errors) { + Trade trade = (Trade) target; + + if(trade.getIsin().length() >= 13) { + errors.rejectValue("isin", "isin_length"); + } + } +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/jmx/JobExecutionNotificationPublisher.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jmx/JobExecutionNotificationPublisher.java index 98a55e09a2..b00f8138de 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/jmx/JobExecutionNotificationPublisher.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jmx/JobExecutionNotificationPublisher.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,9 +31,8 @@ * @author Dave Syer * @since 1.0 */ -public class JobExecutionNotificationPublisher implements ApplicationListener, NotificationPublisherAware { - - protected static final Log logger = LogFactory.getLog(JobExecutionNotificationPublisher.class); +public class JobExecutionNotificationPublisher implements ApplicationListener, NotificationPublisherAware { + private static final Log LOG = LogFactory.getLog(JobExecutionNotificationPublisher.class); private NotificationPublisher notificationPublisher; @@ -44,6 +43,7 @@ public class JobExecutionNotificationPublisher implements ApplicationListener, N * * @see org.springframework.jmx.export.notification.NotificationPublisherAware#setNotificationPublisher(org.springframework.jmx.export.notification.NotificationPublisher) */ + @Override public void setNotificationPublisher(NotificationPublisher notificationPublisher) { this.notificationPublisher = notificationPublisher; } @@ -55,12 +55,11 @@ public void setNotificationPublisher(NotificationPublisher notificationPublisher * * @see ApplicationListener#onApplicationEvent(ApplicationEvent) */ - public void onApplicationEvent(ApplicationEvent applicationEvent) { - if (applicationEvent instanceof SimpleMessageApplicationEvent) { - String message = applicationEvent.toString(); - logger.info(message); - publish(message); - } + @Override + public void onApplicationEvent(SimpleMessageApplicationEvent applicationEvent) { + String message = applicationEvent.toString(); + LOG.info(message); + publish(message); } /** @@ -82,5 +81,4 @@ private void publish(String message) { notificationPublisher.sendNotification(notification); } } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/jmx/SimpleMessageApplicationEvent.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jmx/SimpleMessageApplicationEvent.java index 45f81e20e1..54a2ab48e0 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/jmx/SimpleMessageApplicationEvent.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jmx/SimpleMessageApplicationEvent.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,6 +22,7 @@ * @author Dave Syer * */ +@SuppressWarnings("serial") public class SimpleMessageApplicationEvent extends ApplicationEvent { private String message; @@ -34,6 +35,7 @@ public SimpleMessageApplicationEvent(Object source, String message) { /* (non-Javadoc) * @see java.util.EventObject#toString() */ + @Override public String toString() { return "message=["+message+"], " + super.toString(); } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/jmx/StepExecutionApplicationEventAdvice.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jmx/StepExecutionApplicationEventAdvice.java index 1037286aa6..9cd91b6be4 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/jmx/StepExecutionApplicationEventAdvice.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jmx/StepExecutionApplicationEventAdvice.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,13 +18,12 @@ import org.aspectj.lang.JoinPoint; import org.springframework.batch.core.StepExecution; -import org.springframework.context.ApplicationEvent; import org.springframework.context.ApplicationEventPublisher; import org.springframework.context.ApplicationEventPublisherAware; /** * Wraps calls for methods taking {@link StepExecution} as an argument and - * publishes notifications in the form of {@link ApplicationEvent}. + * publishes notifications in the form of {@link org.springframework.context.ApplicationEvent}. * * @author Dave Syer */ @@ -36,6 +35,7 @@ public class StepExecutionApplicationEventAdvice implements ApplicationEventPubl * (non-Javadoc) * @see org.springframework.context.ApplicationEventPublisherAware#setApplicationEventPublisher(org.springframework.context.ApplicationEventPublisher) */ + @Override public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) { this.applicationEventPublisher = applicationEventPublisher; } @@ -62,5 +62,4 @@ public void onError(JoinPoint jp, StepExecution stepExecution, Throwable t) { private void publish(Object source, String message) { applicationEventPublisher.publishEvent(new SimpleMessageApplicationEvent(source, message)); } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleBatchlet.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleBatchlet.java new file mode 100644 index 0000000000..150fc272bd --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleBatchlet.java @@ -0,0 +1,50 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.jsr352; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import javax.batch.api.AbstractBatchlet; +import javax.batch.api.BatchProperty; +import javax.inject.Inject; + +/** + *

        + * Sample {@link javax.batch.api.Batchlet} implementation. + *

        + * + * @since 3.0 + * @author Chris Schaefer + */ +public class JsrSampleBatchlet extends AbstractBatchlet { + private static final Log LOG = LogFactory.getLog(JsrSampleBatchlet.class); + + @Inject + @BatchProperty + private String remoteServiceURL; + + @Override + public String process() throws Exception { + LOG.info("Calling remote service at: " + remoteServiceURL); + + Thread.sleep(2000); + + LOG.info("Remote service call complete"); + + return null; + } +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleItemProcessor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleItemProcessor.java new file mode 100644 index 0000000000..ba46665d24 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleItemProcessor.java @@ -0,0 +1,42 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.jsr352; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import javax.batch.api.chunk.ItemProcessor; + +/** + *

        + * Sample {@link javax.batch.api.chunk.ItemProcessor} implementation. + *

        + * + * @since 3.0 + * @author Chris Schaefer + */ +public class JsrSampleItemProcessor implements ItemProcessor { + private static final Log LOG = LogFactory.getLog(JsrSampleItemProcessor.class); + + @Override + public Object processItem(Object o) throws Exception { + String person = (String) o; + + LOG.info("Transforming person: " + person + " to uppercase"); + + return person.toUpperCase(); + } +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleItemReader.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleItemReader.java new file mode 100644 index 0000000000..f72ce2c8ff --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleItemReader.java @@ -0,0 +1,57 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.jsr352; + +import java.util.ArrayList; +import java.util.List; +import javax.batch.api.chunk.AbstractItemReader; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +/** + *

        + * Sample {@link javax.batch.api.chunk.ItemReader} implementation. + *

        + * + * @since 3.0 + * @author Chris Schaefer + */ +@SuppressWarnings("serial") +public class JsrSampleItemReader extends AbstractItemReader { + private static final Log LOG = LogFactory.getLog(JsrSampleItemReader.class); + + private List people = new ArrayList() {{ + add("John"); + add("Joe"); + add("Mark"); + add("Jane"); + }}; + + @Override + public Object readItem() throws Exception { + String person = null; + + if(people.iterator().hasNext()) { + person = people.iterator().next(); + people.remove(person); + + LOG.info("Read person: " + person); + } + + return person; + } +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleItemWriter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleItemWriter.java new file mode 100644 index 0000000000..3623689075 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleItemWriter.java @@ -0,0 +1,41 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.jsr352; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import javax.batch.api.chunk.AbstractItemWriter; +import java.util.List; + +/** + *

        + * Sample {@link javax.batch.api.chunk.ItemWriter} implementation. + *

        + * + * @since 3.0 + * @author Chris Schaefer + */ +public class JsrSampleItemWriter extends AbstractItemWriter { + private static final Log LOG = LogFactory.getLog(JsrSampleItemWriter.class); + + @Override + public void writeItems(List people) throws Exception { + for(Object person : people) { + LOG.info("Writing person: " + person); + } + } +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleTasklet.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleTasklet.java new file mode 100644 index 0000000000..79d17cf8e1 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/jsr352/JsrSampleTasklet.java @@ -0,0 +1,55 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.jsr352; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; + +import javax.batch.api.BatchProperty; +import javax.inject.Inject; + +/** + *

        + * Sample {@link org.springframework.batch.core.step.tasklet.Tasklet} implementation. + *

        + * + * @since 3.0 + * @author Chris Schaefer + */ +public class JsrSampleTasklet implements Tasklet { + private static final Log LOG = LogFactory.getLog(JsrSampleTasklet.class); + + @Inject + @BatchProperty + private String remoteServiceURL; + + @Nullable + @Override + public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception { + LOG.info("Calling remote service at: " + remoteServiceURL); + + Thread.sleep(2000); + + LOG.info("Remote service call complete"); + + return RepeatStatus.FINISHED; + } +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/launch/DefaultJobLoader.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/launch/DefaultJobLoader.java index 14054ad9d6..8726196a55 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/launch/DefaultJobLoader.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/launch/DefaultJobLoader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,8 +36,9 @@ public class DefaultJobLoader implements JobLoader, ApplicationContextAware { private ApplicationContext applicationContext; - private Map configurations = new HashMap(); + private Map configurations = new HashMap<>(); + @Override public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { this.applicationContext = applicationContext; } @@ -46,8 +47,9 @@ public void setRegistry(ListableJobLocator registry) { this.registry = registry; } + @Override public Map getConfigurations() { - Map result = new HashMap(configurations); + Map result = new HashMap<>(configurations); for (String jobName : registry.getJobNames()) { try { Job configuration = registry.getJob(jobName); @@ -57,12 +59,14 @@ public Map getConfigurations() { } } catch (NoSuchJobException e) { - throw new IllegalStateException("Registry could not locate its own job (NoSuchJobException)."); + throw new IllegalStateException("Registry could not locate its own job (NoSuchJobException).", e); } } return result; } + @Override + @SuppressWarnings("resource") public void loadResource(String path) { ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext(new String[] { path }, applicationContext); @@ -72,6 +76,7 @@ public void loadResource(String path) { } } + @Override public Object getJobConfiguration(String name) { try { return registry.getJob(name); @@ -81,6 +86,7 @@ public Object getJobConfiguration(String name) { } } + @Override public Object getProperty(String path) { int index = PropertyAccessorUtils.getFirstNestedPropertySeparatorIndex(path); BeanWrapperImpl wrapper = createBeanWrapper(path, index); @@ -88,6 +94,7 @@ public Object getProperty(String path) { return wrapper.getPropertyValue(key); } + @Override public void setProperty(String path, String value) { int index = PropertyAccessorUtils.getFirstNestedPropertySeparatorIndex(path); BeanWrapperImpl wrapper = createBeanWrapper(path, index); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/launch/ExportedJobLoader.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/launch/ExportedJobLoader.java deleted file mode 100644 index 8e851a091c..0000000000 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/launch/ExportedJobLoader.java +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Copyright 2006-2007 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.sample.launch; - -import java.util.Map; - - -/** - * @author Dave Syer - * - */ -public interface ExportedJobLoader { - - void loadResource(String path); - - Map getConfigurations(); - - String getJob(String path); - - String getProperty(String path); - - void setProperty(String path, String value); - -} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/launch/JobLoader.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/launch/JobLoader.java index 066a25c720..ee5adae9e5 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/launch/JobLoader.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/launch/JobLoader.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/loop/GeneratingTradeResettingListener.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/loop/GeneratingTradeResettingListener.java index 741b8a2575..cd8d827b28 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/loop/GeneratingTradeResettingListener.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/loop/GeneratingTradeResettingListener.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,6 +20,7 @@ import org.springframework.batch.core.listener.StepExecutionListenerSupport; import org.springframework.batch.sample.domain.trade.internal.GeneratingTradeItemReader; import org.springframework.beans.factory.InitializingBean; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -33,6 +34,8 @@ public class GeneratingTradeResettingListener extends StepExecutionListenerSuppo private GeneratingTradeItemReader reader; + @Nullable + @Override public ExitStatus afterStep(StepExecution stepExecution) { this.reader.resetCounter(); return null; @@ -42,6 +45,7 @@ public void setReader(GeneratingTradeItemReader reader) { this.reader = reader; } + @Override public void afterPropertiesSet() throws Exception { Assert.notNull(this.reader, "The 'reader' must be set."); } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/loop/LimitDecider.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/loop/LimitDecider.java index 7a4e7c29aa..d98e68d332 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/loop/LimitDecider.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/loop/LimitDecider.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -19,6 +19,7 @@ import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.job.flow.FlowExecutionStatus; import org.springframework.batch.core.job.flow.JobExecutionDecider; +import org.springframework.lang.Nullable; /** * This decider will return "CONTINUE" until the limit it reached, at which @@ -33,7 +34,8 @@ public class LimitDecider implements JobExecutionDecider { private int limit = 1; - public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) { + @Override + public FlowExecutionStatus decide(JobExecution jobExecution, @Nullable StepExecution stepExecution) { if (++count >= limit) { return new FlowExecutionStatus("COMPLETED"); } @@ -43,7 +45,7 @@ public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepE } /** - * @param limit + * @param limit number of times to return "CONTINUE" */ public void setLimit(int limit) { this.limit = limit; diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/BatchMetricsApplication.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/BatchMetricsApplication.java new file mode 100644 index 0000000000..eb6e2aa23b --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/BatchMetricsApplication.java @@ -0,0 +1,31 @@ +package org.springframework.batch.sample.metrics; + +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Import; +import org.springframework.context.annotation.PropertySource; +import org.springframework.scheduling.annotation.EnableScheduling; +import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler; + +@EnableScheduling +@EnableBatchProcessing +@Import({Job1Configuration.class, Job2Configuration.class, JobScheduler.class, PrometheusConfiguration.class}) +@PropertySource("metrics-sample.properties") +public class BatchMetricsApplication { + + public static void main(String[] args) { + AnnotationConfigApplicationContext applicationContext = new AnnotationConfigApplicationContext(BatchMetricsApplication.class); + applicationContext.start(); + } + + @Bean(destroyMethod = "shutdown") + public ThreadPoolTaskScheduler taskScheduler(@Value("${thread.pool.size}") int threadPoolSize) { + ThreadPoolTaskScheduler threadPoolTaskScheduler = new ThreadPoolTaskScheduler(); + threadPoolTaskScheduler.setPoolSize(threadPoolSize); + return threadPoolTaskScheduler; + } + +} + diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/Job1Configuration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/Job1Configuration.java new file mode 100644 index 0000000000..406c4dbc2c --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/Job1Configuration.java @@ -0,0 +1,62 @@ +package org.springframework.batch.sample.metrics; + +import java.util.Random; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +@Configuration +public class Job1Configuration { + + private Random random; + private JobBuilderFactory jobs; + private StepBuilderFactory steps; + + public Job1Configuration(JobBuilderFactory jobs, StepBuilderFactory steps) { + this.jobs = jobs; + this.steps = steps; + this.random = new Random(); + } + + @Bean + public Job job1() { + return jobs.get("job1") + .start(step1()) + .next(step2()) + .build(); + } + + @Bean + public Step step1() { + return steps.get("step1") + .tasklet((contribution, chunkContext) -> { + System.out.println("hello"); + // simulate processing time + Thread.sleep(random.nextInt(3000)); + return RepeatStatus.FINISHED; + }) + .build(); + } + + @Bean + public Step step2() { + return steps.get("step2") + .tasklet((contribution, chunkContext) -> { + System.out.println("world"); + // simulate step failure + int nextInt = random.nextInt(3000); + Thread.sleep(nextInt); + if (nextInt % 5 == 0) { + throw new Exception("Boom!"); + } + return RepeatStatus.FINISHED; + }) + .build(); + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/Job2Configuration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/Job2Configuration.java new file mode 100644 index 0000000000..fe12891095 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/Job2Configuration.java @@ -0,0 +1,72 @@ +package org.springframework.batch.sample.metrics; + +import java.util.LinkedList; +import java.util.List; +import java.util.Random; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepScope; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.ListItemReader; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +@Configuration +public class Job2Configuration { + + private Random random; + private JobBuilderFactory jobs; + private StepBuilderFactory steps; + + public Job2Configuration(JobBuilderFactory jobs, StepBuilderFactory steps) { + this.jobs = jobs; + this.steps = steps; + this.random = new Random(); + } + + @Bean + public Job job2() { + return jobs.get("job2") + .start(step()) + .build(); + } + + @Bean + public Step step() { + return steps.get("step1") + .chunk(3) + .reader(itemReader()) + .writer(itemWriter()) + .build(); + } + + @Bean + @StepScope + public ListItemReader itemReader() { + List items = new LinkedList<>(); + // read a random number of items in each run + for (int i = 0; i < random.nextInt(100); i++) { + items.add(i); + } + return new ListItemReader<>(items); + } + + @Bean + public ItemWriter itemWriter() { + return items -> { + for (Integer item : items) { + int nextInt = random.nextInt(1000); + Thread.sleep(nextInt); + // simulate write failure + if (nextInt % 57 == 0) { + throw new Exception("Boom!"); + } + System.out.println("item = " + item); + } + }; + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/JobScheduler.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/JobScheduler.java new file mode 100644 index 0000000000..43382b1f78 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/JobScheduler.java @@ -0,0 +1,43 @@ +package org.springframework.batch.sample.metrics; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.scheduling.annotation.Scheduled; +import org.springframework.stereotype.Component; + +@Component +public class JobScheduler { + + private final Job job1; + private final Job job2; + private final JobLauncher jobLauncher; + + @Autowired + public JobScheduler(Job job1, Job job2, JobLauncher jobLauncher) { + this.job1 = job1; + this.job2 = job2; + this.jobLauncher = jobLauncher; + } + + @Scheduled(cron="*/10 * * * * *") + public void launchJob1() throws Exception { + JobParameters jobParameters = new JobParametersBuilder() + .addLong("time", System.currentTimeMillis()) + .toJobParameters(); + + jobLauncher.run(job1, jobParameters); + } + + @Scheduled(cron="*/15 * * * * *") + public void launchJob2() throws Exception { + JobParameters jobParameters = new JobParametersBuilder() + .addLong("time", System.currentTimeMillis()) + .toJobParameters(); + + jobLauncher.run(job2, jobParameters); + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/PrometheusConfiguration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/PrometheusConfiguration.java new file mode 100644 index 0000000000..b073a1cbf2 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/metrics/PrometheusConfiguration.java @@ -0,0 +1,55 @@ +package org.springframework.batch.sample.metrics; + +import java.util.HashMap; +import java.util.Map; +import javax.annotation.PostConstruct; + +import io.micrometer.core.instrument.Metrics; +import io.micrometer.prometheus.PrometheusConfig; +import io.micrometer.prometheus.PrometheusMeterRegistry; +import io.prometheus.client.CollectorRegistry; +import io.prometheus.client.exporter.PushGateway; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Configuration; +import org.springframework.scheduling.annotation.Scheduled; + +@Configuration +public class PrometheusConfiguration { + + private static final Logger LOGGER = LoggerFactory.getLogger(PrometheusConfiguration.class); + + @Value("${prometheus.job.name}") + private String prometheusJobName; + + @Value("${prometheus.grouping.key}") + private String prometheusGroupingKey; + + @Value("${prometheus.pushgateway.url}") + private String prometheusPushGatewayUrl; + + private Map groupingKey = new HashMap<>(); + private PushGateway pushGateway; + private CollectorRegistry collectorRegistry; + + @PostConstruct + public void init() { + pushGateway = new PushGateway(prometheusPushGatewayUrl); + groupingKey.put(prometheusGroupingKey, prometheusJobName); + PrometheusMeterRegistry prometheusMeterRegistry = new PrometheusMeterRegistry(PrometheusConfig.DEFAULT); + collectorRegistry = prometheusMeterRegistry.getPrometheusRegistry(); + Metrics.globalRegistry.add(prometheusMeterRegistry); + } + + @Scheduled(fixedRateString = "${prometheus.push.rate}") + public void pushMetrics() { + try { + pushGateway.pushAdd(collectorRegistry, prometheusJobName, groupingKey); + } catch (Throwable ex) { + LOGGER.error("Unable to push metrics to Prometheus Push Gateway", ex); + } + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/quartz/JobLauncherDetails.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/quartz/JobLauncherDetails.java index 0e3b00cc41..a38a313ab9 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/quartz/JobLauncherDetails.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/quartz/JobLauncherDetails.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -62,7 +62,7 @@ public void setJobLauncher(JobLauncher jobLauncher) { this.jobLauncher = jobLauncher; } - @SuppressWarnings("unchecked") + @Override protected void executeInternal(JobExecutionContext context) { Map jobDataMap = context.getMergedJobDataMap(); String jobName = (String) jobDataMap.get(JOB_NAME); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/rabbitmq/amqp/AmqpMessageProducer.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/rabbitmq/amqp/AmqpMessageProducer.java index 72e053f493..8da6ca391b 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/rabbitmq/amqp/AmqpMessageProducer.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/rabbitmq/amqp/AmqpMessageProducer.java @@ -1,3 +1,18 @@ +/* + * Copyright 2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.rabbitmq.amqp; import org.springframework.amqp.core.AmqpTemplate; @@ -11,9 +26,11 @@ * Simple producer class that sends {@link String} messages to the configured queue to be processed. *

        */ -public class AmqpMessageProducer { - public static final int SEND_MESSAGE_COUNT = 10; - public static final String[] BEAN_CONFIG = { "classpath:/META-INF/spring/jobs/messaging/rabbitmq-beans.xml", +public final class AmqpMessageProducer { + private AmqpMessageProducer() {} + + private static final int SEND_MESSAGE_COUNT = 10; + private static final String[] BEAN_CONFIG = { "classpath:/META-INF/spring/jobs/messaging/rabbitmq-beans.xml", "classpath:/META-INF/spring/config-beans.xml" }; public static void main(String[] args) { diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/rabbitmq/processor/MessageProcessor.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/rabbitmq/processor/MessageProcessor.java index e527a2973a..9b1fd9e856 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/rabbitmq/processor/MessageProcessor.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/rabbitmq/processor/MessageProcessor.java @@ -1,6 +1,22 @@ +/* + * Copyright 2012-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.rabbitmq.processor; import org.springframework.batch.item.ItemProcessor; +import org.springframework.lang.Nullable; import java.util.Date; @@ -11,7 +27,9 @@ */ public class MessageProcessor implements ItemProcessor { - public String process(String message) throws Exception { + @Nullable + @Override + public String process(String message) throws Exception { return "Message: \"" + message + "\" processed on: " + new Date(); } } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotechunking/ManagerConfiguration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotechunking/ManagerConfiguration.java new file mode 100644 index 0000000000..861557de5c --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotechunking/ManagerConfiguration.java @@ -0,0 +1,129 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.remotechunking; + +import java.util.Arrays; + +import org.apache.activemq.ActiveMQConnectionFactory; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.core.step.tasklet.TaskletStep; +import org.springframework.batch.integration.chunk.RemoteChunkingManagerStepBuilderFactory; +import org.springframework.batch.integration.config.annotation.EnableBatchIntegration; +import org.springframework.batch.item.support.ListItemReader; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.PropertySource; +import org.springframework.integration.channel.DirectChannel; +import org.springframework.integration.channel.QueueChannel; +import org.springframework.integration.config.EnableIntegration; +import org.springframework.integration.dsl.IntegrationFlow; +import org.springframework.integration.dsl.IntegrationFlows; +import org.springframework.integration.jms.dsl.Jms; + +/** + * This configuration class is for the manager side of the remote chunking sample. + * The manager step reads numbers from 1 to 6 and sends 2 chunks {1, 2, 3} and + * {4, 5, 6} to workers for processing and writing. + * + * @author Mahmoud Ben Hassine + */ +@Configuration +@EnableBatchProcessing +@EnableBatchIntegration +@EnableIntegration +@PropertySource("classpath:remote-chunking.properties") +public class ManagerConfiguration { + + @Value("${broker.url}") + private String brokerUrl; + + @Autowired + private JobBuilderFactory jobBuilderFactory; + + @Autowired + private RemoteChunkingManagerStepBuilderFactory managerStepBuilderFactory; + + @Bean + public ActiveMQConnectionFactory connectionFactory() { + ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(); + connectionFactory.setBrokerURL(this.brokerUrl); + connectionFactory.setTrustAllPackages(true); + return connectionFactory; + } + + /* + * Configure outbound flow (requests going to workers) + */ + @Bean + public DirectChannel requests() { + return new DirectChannel(); + } + + @Bean + public IntegrationFlow outboundFlow(ActiveMQConnectionFactory connectionFactory) { + return IntegrationFlows + .from(requests()) + .handle(Jms.outboundAdapter(connectionFactory).destination("requests")) + .get(); + } + + /* + * Configure inbound flow (replies coming from workers) + */ + @Bean + public QueueChannel replies() { + return new QueueChannel(); + } + + @Bean + public IntegrationFlow inboundFlow(ActiveMQConnectionFactory connectionFactory) { + return IntegrationFlows + .from(Jms.messageDrivenChannelAdapter(connectionFactory).destination("replies")) + .channel(replies()) + .get(); + } + + /* + * Configure master step components + */ + @Bean + public ListItemReader itemReader() { + return new ListItemReader<>(Arrays.asList(1, 2, 3, 4, 5, 6)); + } + + @Bean + public TaskletStep managerStep() { + return this.managerStepBuilderFactory.get("managerStep") + .chunk(3) + .reader(itemReader()) + .outputChannel(requests()) + .inputChannel(replies()) + .build(); + } + + @Bean + public Job remoteChunkingJob() { + return this.jobBuilderFactory.get("remoteChunkingJob") + .start(managerStep()) + .build(); + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotechunking/WorkerConfiguration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotechunking/WorkerConfiguration.java new file mode 100644 index 0000000000..f190cbcca8 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotechunking/WorkerConfiguration.java @@ -0,0 +1,131 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.remotechunking; + +import org.apache.activemq.ActiveMQConnectionFactory; + +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.integration.chunk.RemoteChunkingWorkerBuilder; +import org.springframework.batch.integration.config.annotation.EnableBatchIntegration; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemWriter; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.PropertySource; +import org.springframework.integration.channel.DirectChannel; +import org.springframework.integration.config.EnableIntegration; +import org.springframework.integration.dsl.IntegrationFlow; +import org.springframework.integration.dsl.IntegrationFlows; +import org.springframework.integration.jms.dsl.Jms; + +/** + * This configuration class is for the worker side of the remote chunking sample. + * It uses the {@link RemoteChunkingWorkerBuilder} to configure an + * {@link IntegrationFlow} in order to: + *
          + *
        • receive requests from the master
        • + *
        • process chunks with the configured item processor and writer
        • + *
        • send replies to the master
        • + *
        + * + * @author Mahmoud Ben Hassine + */ +@Configuration +@EnableBatchProcessing +@EnableBatchIntegration +@EnableIntegration +@PropertySource("classpath:remote-chunking.properties") +public class WorkerConfiguration { + + @Value("${broker.url}") + private String brokerUrl; + + @Autowired + private RemoteChunkingWorkerBuilder remoteChunkingWorkerBuilder; + + @Bean + public ActiveMQConnectionFactory connectionFactory() { + ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(); + connectionFactory.setBrokerURL(this.brokerUrl); + connectionFactory.setTrustAllPackages(true); + return connectionFactory; + } + + /* + * Configure inbound flow (requests coming from the master) + */ + @Bean + public DirectChannel requests() { + return new DirectChannel(); + } + + @Bean + public IntegrationFlow inboundFlow(ActiveMQConnectionFactory connectionFactory) { + return IntegrationFlows + .from(Jms.messageDrivenChannelAdapter(connectionFactory).destination("requests")) + .channel(requests()) + .get(); + } + + /* + * Configure outbound flow (replies going to the master) + */ + @Bean + public DirectChannel replies() { + return new DirectChannel(); + } + + @Bean + public IntegrationFlow outboundFlow(ActiveMQConnectionFactory connectionFactory) { + return IntegrationFlows + .from(replies()) + .handle(Jms.outboundAdapter(connectionFactory).destination("replies")) + .get(); + } + + /* + * Configure worker components + */ + @Bean + public ItemProcessor itemProcessor() { + return item -> { + System.out.println("processing item " + item); + return item; + }; + } + + @Bean + public ItemWriter itemWriter() { + return items -> { + for (Integer item : items) { + System.out.println("writing item " + item); + } + }; + } + + @Bean + public IntegrationFlow workerIntegrationFlow() { + return this.remoteChunkingWorkerBuilder + .itemProcessor(itemProcessor()) + .itemWriter(itemWriter()) + .inputChannel(requests()) + .outputChannel(replies()) + .build(); + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/BasicPartitioner.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/BasicPartitioner.java new file mode 100644 index 0000000000..e9a396a2f4 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/BasicPartitioner.java @@ -0,0 +1,43 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.sample.remotepartitioning; + +import java.util.Map; + +import org.springframework.batch.core.partition.support.SimplePartitioner; +import org.springframework.batch.item.ExecutionContext; + +/** + * Simple partitioner for demonstration purpose. + * + * @author Mahmoud Ben Hassine + */ +public class BasicPartitioner extends SimplePartitioner { + + private static final String PARTITION_KEY = "partition"; + + @Override + public Map partition(int gridSize) { + Map partitions = super.partition(gridSize); + int i = 0; + for (ExecutionContext context : partitions.values()) { + context.put(PARTITION_KEY, PARTITION_KEY + (i++)); + } + return partitions; + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/BrokerConfiguration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/BrokerConfiguration.java new file mode 100644 index 0000000000..60d627c09c --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/BrokerConfiguration.java @@ -0,0 +1,44 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.sample.remotepartitioning; + +import org.apache.activemq.ActiveMQConnectionFactory; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.PropertySource; + +/** + * @author Mahmoud Ben Hassine + */ +@Configuration +@PropertySource("classpath:remote-partitioning.properties") +public class BrokerConfiguration { + + @Value("${broker.url}") + private String brokerUrl; + + @Bean + public ActiveMQConnectionFactory connectionFactory() { + ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(); + connectionFactory.setBrokerURL(this.brokerUrl); + connectionFactory.setTrustAllPackages(true); + return connectionFactory; + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/DataSourceConfiguration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/DataSourceConfiguration.java new file mode 100644 index 0000000000..028c72c7c4 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/DataSourceConfiguration.java @@ -0,0 +1,53 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.sample.remotepartitioning; + +import javax.sql.DataSource; + +import org.apache.commons.dbcp2.BasicDataSource; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.PropertySource; + +/** + * @author Mahmoud Ben Hassine + */ +@Configuration +@PropertySource("classpath:remote-partitioning.properties") +public class DataSourceConfiguration { + + @Value("${datasource.url}") + private String url; + + @Value("${datasource.username}") + private String username; + + @Value("${datasource.password}") + private String password; + + @Bean + public DataSource dataSource() { + BasicDataSource dataSource = new BasicDataSource(); + dataSource.setUrl(url); + dataSource.setUsername(username); + dataSource.setPassword(password); + return dataSource; + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/aggregating/ManagerConfiguration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/aggregating/ManagerConfiguration.java new file mode 100644 index 0000000000..07078272b4 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/aggregating/ManagerConfiguration.java @@ -0,0 +1,115 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.remotepartitioning.aggregating; + +import org.apache.activemq.ActiveMQConnectionFactory; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.integration.config.annotation.EnableBatchIntegration; +import org.springframework.batch.integration.partition.RemotePartitioningManagerStepBuilderFactory; +import org.springframework.batch.sample.remotepartitioning.BasicPartitioner; +import org.springframework.batch.sample.remotepartitioning.BrokerConfiguration; +import org.springframework.batch.sample.remotepartitioning.DataSourceConfiguration; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; +import org.springframework.integration.channel.DirectChannel; +import org.springframework.integration.dsl.IntegrationFlow; +import org.springframework.integration.dsl.IntegrationFlows; +import org.springframework.integration.jms.dsl.Jms; + +/** + * This configuration class is for the manager side of the remote partitioning sample. + * The manager step will create 3 partitions for workers to process. + * + * @author Mahmoud Ben Hassine + */ +@Configuration +@EnableBatchProcessing +@EnableBatchIntegration +@Import(value = {DataSourceConfiguration.class, BrokerConfiguration.class}) +public class ManagerConfiguration { + + private static final int GRID_SIZE = 3; + + private final JobBuilderFactory jobBuilderFactory; + + private final RemotePartitioningManagerStepBuilderFactory managerStepBuilderFactory; + + + public ManagerConfiguration(JobBuilderFactory jobBuilderFactory, + RemotePartitioningManagerStepBuilderFactory managerStepBuilderFactory) { + + this.jobBuilderFactory = jobBuilderFactory; + this.managerStepBuilderFactory = managerStepBuilderFactory; + } + + /* + * Configure outbound flow (requests going to workers) + */ + @Bean + public DirectChannel requests() { + return new DirectChannel(); + } + + @Bean + public IntegrationFlow outboundFlow(ActiveMQConnectionFactory connectionFactory) { + return IntegrationFlows + .from(requests()) + .handle(Jms.outboundAdapter(connectionFactory).destination("requests")) + .get(); + } + + /* + * Configure inbound flow (replies coming from workers) + */ + @Bean + public DirectChannel replies() { + return new DirectChannel(); + } + + @Bean + public IntegrationFlow inboundFlow(ActiveMQConnectionFactory connectionFactory) { + return IntegrationFlows + .from(Jms.messageDrivenChannelAdapter(connectionFactory).destination("replies")) + .channel(replies()) + .get(); + } + + /* + * Configure the manager step + */ + @Bean + public Step managerStep() { + return this.managerStepBuilderFactory.get("managerStep") + .partitioner("workerStep", new BasicPartitioner()) + .gridSize(GRID_SIZE) + .outputChannel(requests()) + .inputChannel(replies()) + .build(); + } + + @Bean + public Job remotePartitioningJob() { + return this.jobBuilderFactory.get("remotePartitioningJob") + .start(managerStep()) + .build(); + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/aggregating/WorkerConfiguration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/aggregating/WorkerConfiguration.java new file mode 100644 index 0000000000..ad756f3c05 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/aggregating/WorkerConfiguration.java @@ -0,0 +1,110 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.remotepartitioning.aggregating; + +import org.apache.activemq.ActiveMQConnectionFactory; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.StepScope; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.integration.config.annotation.EnableBatchIntegration; +import org.springframework.batch.integration.partition.RemotePartitioningWorkerStepBuilderFactory; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.batch.sample.remotepartitioning.BrokerConfiguration; +import org.springframework.batch.sample.remotepartitioning.DataSourceConfiguration; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; +import org.springframework.integration.channel.DirectChannel; +import org.springframework.integration.dsl.IntegrationFlow; +import org.springframework.integration.dsl.IntegrationFlows; +import org.springframework.integration.jms.dsl.Jms; + +/** + * This configuration class is for the worker side of the remote partitioning sample. + * Each worker will process a partition sent by the manager step. + * + * @author Mahmoud Ben Hassine + */ +@Configuration +@EnableBatchProcessing +@EnableBatchIntegration +@Import(value = {DataSourceConfiguration.class, BrokerConfiguration.class}) +public class WorkerConfiguration { + + private final RemotePartitioningWorkerStepBuilderFactory workerStepBuilderFactory; + + + public WorkerConfiguration(RemotePartitioningWorkerStepBuilderFactory workerStepBuilderFactory) { + this.workerStepBuilderFactory = workerStepBuilderFactory; + } + + /* + * Configure inbound flow (requests coming from the manager) + */ + @Bean + public DirectChannel requests() { + return new DirectChannel(); + } + + @Bean + public IntegrationFlow inboundFlow(ActiveMQConnectionFactory connectionFactory) { + return IntegrationFlows + .from(Jms.messageDrivenChannelAdapter(connectionFactory).destination("requests")) + .channel(requests()) + .get(); + } + + /* + * Configure outbound flow (replies going to the manager) + */ + @Bean + public DirectChannel replies() { + return new DirectChannel(); + } + + @Bean + public IntegrationFlow outboundFlow(ActiveMQConnectionFactory connectionFactory) { + return IntegrationFlows + .from(replies()) + .handle(Jms.outboundAdapter(connectionFactory).destination("replies")) + .get(); + } + + /* + * Configure the worker step + */ + @Bean + public Step workerStep() { + return this.workerStepBuilderFactory.get("workerStep") + .inputChannel(requests()) + .outputChannel(replies()) + .tasklet(tasklet(null)) + .build(); + } + + @Bean + @StepScope + public Tasklet tasklet(@Value("#{stepExecutionContext['partition']}") String partition) { + return (contribution, chunkContext) -> { + System.out.println("processing " + partition); + return RepeatStatus.FINISHED; + }; + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/polling/ManagerConfiguration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/polling/ManagerConfiguration.java new file mode 100644 index 0000000000..b5e3beff98 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/polling/ManagerConfiguration.java @@ -0,0 +1,98 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.remotepartitioning.polling; + +import org.apache.activemq.ActiveMQConnectionFactory; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.integration.config.annotation.EnableBatchIntegration; +import org.springframework.batch.integration.partition.RemotePartitioningManagerStepBuilderFactory; +import org.springframework.batch.sample.remotepartitioning.BasicPartitioner; +import org.springframework.batch.sample.remotepartitioning.BrokerConfiguration; +import org.springframework.batch.sample.remotepartitioning.DataSourceConfiguration; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; +import org.springframework.integration.channel.DirectChannel; +import org.springframework.integration.dsl.IntegrationFlow; +import org.springframework.integration.dsl.IntegrationFlows; +import org.springframework.integration.jms.dsl.Jms; + +/** + * This configuration class is for the manager side of the remote partitioning sample. + * The manager step will create 3 partitions for workers to process. + * + * @author Mahmoud Ben Hassine + */ +@Configuration +@EnableBatchProcessing +@EnableBatchIntegration +@Import(value = {DataSourceConfiguration.class, BrokerConfiguration.class}) +public class ManagerConfiguration { + + private static final int GRID_SIZE = 3; + + private final JobBuilderFactory jobBuilderFactory; + + private final RemotePartitioningManagerStepBuilderFactory managerStepBuilderFactory; + + + public ManagerConfiguration(JobBuilderFactory jobBuilderFactory, + RemotePartitioningManagerStepBuilderFactory managerStepBuilderFactory) { + + this.jobBuilderFactory = jobBuilderFactory; + this.managerStepBuilderFactory = managerStepBuilderFactory; + } + + /* + * Configure outbound flow (requests going to workers) + */ + @Bean + public DirectChannel requests() { + return new DirectChannel(); + } + + @Bean + public IntegrationFlow outboundFlow(ActiveMQConnectionFactory connectionFactory) { + return IntegrationFlows + .from(requests()) + .handle(Jms.outboundAdapter(connectionFactory).destination("requests")) + .get(); + } + + /* + * Configure the manager step + */ + @Bean + public Step managerStep() { + return this.managerStepBuilderFactory.get("managerStep") + .partitioner("workerStep", new BasicPartitioner()) + .gridSize(GRID_SIZE) + .outputChannel(requests()) + .build(); + } + + @Bean + public Job remotePartitioningJob() { + return this.jobBuilderFactory.get("remotePartitioningJob") + .start(managerStep()) + .build(); + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/polling/WorkerConfiguration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/polling/WorkerConfiguration.java new file mode 100644 index 0000000000..d7b5adf63e --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/polling/WorkerConfiguration.java @@ -0,0 +1,93 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.remotepartitioning.polling; + +import org.apache.activemq.ActiveMQConnectionFactory; + +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.StepScope; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.integration.config.annotation.EnableBatchIntegration; +import org.springframework.batch.integration.partition.RemotePartitioningWorkerStepBuilderFactory; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.batch.sample.remotepartitioning.BrokerConfiguration; +import org.springframework.batch.sample.remotepartitioning.DataSourceConfiguration; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; +import org.springframework.integration.channel.DirectChannel; +import org.springframework.integration.dsl.IntegrationFlow; +import org.springframework.integration.dsl.IntegrationFlows; +import org.springframework.integration.jms.dsl.Jms; + +/** + * This configuration class is for the worker side of the remote partitioning sample. + * Each worker will process a partition sent by the master step. + * + * @author Mahmoud Ben Hassine + */ +@Configuration +@EnableBatchProcessing +@EnableBatchIntegration +@Import(value = {DataSourceConfiguration.class, BrokerConfiguration.class}) +public class WorkerConfiguration { + + private final RemotePartitioningWorkerStepBuilderFactory workerStepBuilderFactory; + + + public WorkerConfiguration(RemotePartitioningWorkerStepBuilderFactory workerStepBuilderFactory) { + this.workerStepBuilderFactory = workerStepBuilderFactory; + } + + /* + * Configure inbound flow (requests coming from the master) + */ + @Bean + public DirectChannel requests() { + return new DirectChannel(); + } + + @Bean + public IntegrationFlow inboundFlow(ActiveMQConnectionFactory connectionFactory) { + return IntegrationFlows + .from(Jms.messageDrivenChannelAdapter(connectionFactory).destination("requests")) + .channel(requests()) + .get(); + } + + /* + * Configure the worker step + */ + @Bean + public Step workerStep() { + return this.workerStepBuilderFactory.get("workerStep") + .inputChannel(requests()) + .tasklet(tasklet(null)) + .build(); + } + + @Bean + @StepScope + public Tasklet tasklet(@Value("#{stepExecutionContext['partition']}") String partition) { + return (contribution, chunkContext) -> { + System.out.println("processing " + partition); + return RepeatStatus.FINISHED; + }; + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/DummyItemWriter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/DummyItemWriter.java index 0dfa29bbd4..de2f5588e6 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/DummyItemWriter.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/DummyItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -25,6 +25,7 @@ */ public class DummyItemWriter implements ItemWriter { + @Override public void write(List item) throws Exception { // NO-OP Thread.sleep(500); diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/ExceptionThrowingItemReaderProxy.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/ExceptionThrowingItemReaderProxy.java index 1384091c33..c0e4ae135d 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/ExceptionThrowingItemReaderProxy.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/ExceptionThrowingItemReaderProxy.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -18,6 +18,7 @@ import org.springframework.batch.core.UnexpectedJobExecutionException; import org.springframework.batch.item.ItemReader; +import org.springframework.lang.Nullable; /** * Hacked {@link ItemReader} that throws exception on a given record number @@ -43,6 +44,8 @@ public void setThrowExceptionOnRecordNumber(int throwExceptionOnRecordNumber) { this.throwExceptionOnRecordNumber = throwExceptionOnRecordNumber; } + @Nullable + @Override public T read() throws Exception { counter++; diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/HeaderCopyCallback.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/HeaderCopyCallback.java index f3025235ea..b2a9859a45 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/HeaderCopyCallback.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/HeaderCopyCallback.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,28 +20,25 @@ import java.io.Writer; import org.springframework.batch.item.file.FlatFileHeaderCallback; -import org.springframework.batch.item.file.FlatFileItemReader; -import org.springframework.batch.item.file.FlatFileItemWriter; import org.springframework.batch.item.file.LineCallbackHandler; import org.springframework.util.Assert; /** - * Designed to be registered with both {@link FlatFileItemReader} and - * {@link FlatFileItemWriter} and copy header line from input file to output - * file. + * Designed to be registered with both {@link org.springframework.batch.item.file.FlatFileItemReader} + * and {@link org.springframework.batch.item.file.FlatFileItemWriter} and copy header line from input + * file to output file. */ public class HeaderCopyCallback implements LineCallbackHandler, FlatFileHeaderCallback { - private String header = ""; + @Override public void handleLine(String line) { - Assert.notNull(line); + Assert.notNull(line, "line must not be null"); this.header = line; } + @Override public void writeHeader(Writer writer) throws IOException { writer.write("header from input: " + header); - } - } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/RetrySampleItemWriter.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/RetrySampleItemWriter.java index 4920caf00c..169881605e 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/RetrySampleItemWriter.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/RetrySampleItemWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,11 +30,12 @@ public class RetrySampleItemWriter implements ItemWriter { private int counter = 0; + @Override public void write(List items) throws Exception { int current = counter; counter += items.size(); if (current < 3 && (counter >= 2 || counter >= 3)) { - throw new RuntimeException("Temporary error"); + throw new IllegalStateException("Temporary error"); } } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/SummaryFooterCallback.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/SummaryFooterCallback.java index 9cfc152a22..046ac7fc80 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/SummaryFooterCallback.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/support/SummaryFooterCallback.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -30,6 +30,7 @@ public class SummaryFooterCallback extends StepExecutionListenerSupport implemen private StepExecution stepExecution; + @Override public void writeFooter(Writer writer) throws IOException { writer.write("footer - number of items written: " + stepExecution.getWriteCount()); } diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/validation/ValidationSampleConfiguration.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/validation/ValidationSampleConfiguration.java new file mode 100644 index 0000000000..2291e71dc6 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/validation/ValidationSampleConfiguration.java @@ -0,0 +1,85 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.sample.validation; + +import java.util.Arrays; + +import org.springframework.batch.core.Job; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.item.support.ListItemReader; +import org.springframework.batch.item.support.ListItemWriter; +import org.springframework.batch.item.validator.BeanValidatingItemProcessor; +import org.springframework.batch.sample.validation.domain.Person; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * @author Mahmoud Ben Hassine + */ +@Configuration +@EnableBatchProcessing +public class ValidationSampleConfiguration { + + @Autowired + private JobBuilderFactory jobs; + + @Autowired + private StepBuilderFactory steps; + + @Bean + public ListItemReader itemReader() { + Person person1 = new Person(1, "foo"); + Person person2 = new Person(2, ""); + return new ListItemReader<>(Arrays.asList(person1, person2)); + } + + @Bean + public ListItemWriter itemWriter() { + return new ListItemWriter<>(); + } + + @Bean + public BeanValidatingItemProcessor itemValidator() throws Exception { + BeanValidatingItemProcessor validator = new BeanValidatingItemProcessor<>(); + validator.setFilter(true); + validator.afterPropertiesSet(); + + return validator; + } + + @Bean + public Step step() throws Exception { + return this.steps.get("step") + .chunk(1) + .reader(itemReader()) + .processor(itemValidator()) + .writer(itemWriter()) + .build(); + } + + @Bean + public Job job() throws Exception { + return this.jobs.get("job") + .start(step()) + .build(); + } + +} diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/validation/domain/Person.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/validation/domain/Person.java new file mode 100644 index 0000000000..ce28a2c0e6 --- /dev/null +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/validation/domain/Person.java @@ -0,0 +1,62 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.sample.validation.domain; + +import javax.validation.constraints.NotEmpty; + +/** + * @author Mahmoud Ben Hassine + */ +public class Person { + + private int id; + + @NotEmpty + private String name; + + public Person() { + } + + public Person(int id, String name) { + this.id = id; + this.name = name; + } + + public int getId() { + return id; + } + + public void setId(int id) { + this.id = id; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + @Override + public String toString() { + return "Person{" + + "id=" + id + + ", name='" + name + '\'' + + '}'; + } +} diff --git a/spring-batch-samples/src/main/resources/META-INF/batch-jobs/batchXmlConfigSample.xml b/spring-batch-samples/src/main/resources/META-INF/batch-jobs/batchXmlConfigSample.xml new file mode 100644 index 0000000000..b62343d6cb --- /dev/null +++ b/spring-batch-samples/src/main/resources/META-INF/batch-jobs/batchXmlConfigSample.xml @@ -0,0 +1,17 @@ + + + + + + + + + + + + + + + + + diff --git a/spring-batch-samples/src/main/resources/META-INF/batch-jobs/inlineConfigSample.xml b/spring-batch-samples/src/main/resources/META-INF/batch-jobs/inlineConfigSample.xml new file mode 100644 index 0000000000..d99b61db9d --- /dev/null +++ b/spring-batch-samples/src/main/resources/META-INF/batch-jobs/inlineConfigSample.xml @@ -0,0 +1,17 @@ + + + + + + + + + + + + + + + + + diff --git a/spring-batch-samples/src/main/resources/META-INF/batch-jobs/springConfigSample.xml b/spring-batch-samples/src/main/resources/META-INF/batch-jobs/springConfigSample.xml new file mode 100644 index 0000000000..0f7dcb3004 --- /dev/null +++ b/spring-batch-samples/src/main/resources/META-INF/batch-jobs/springConfigSample.xml @@ -0,0 +1,17 @@ + + + + + + + + + + + + + + + + + diff --git a/spring-batch-samples/src/main/resources/META-INF/batch-jobs/springConfigSampleContext.xml b/spring-batch-samples/src/main/resources/META-INF/batch-jobs/springConfigSampleContext.xml new file mode 100644 index 0000000000..3e3221c53c --- /dev/null +++ b/spring-batch-samples/src/main/resources/META-INF/batch-jobs/springConfigSampleContext.xml @@ -0,0 +1,29 @@ + + + + + + + + + + + + John + Joe + Mark + Jane + + + + + + + + + + + diff --git a/spring-batch-samples/src/main/resources/META-INF/batch.xml b/spring-batch-samples/src/main/resources/META-INF/batch.xml new file mode 100644 index 0000000000..47caaf4ff7 --- /dev/null +++ b/spring-batch-samples/src/main/resources/META-INF/batch.xml @@ -0,0 +1,6 @@ + + + + + + diff --git a/spring-batch-samples/src/main/resources/META-INF/persistence.xml b/spring-batch-samples/src/main/resources/META-INF/persistence.xml index bc3353f679..dc3dc605d5 100644 --- a/spring-batch-samples/src/main/resources/META-INF/persistence.xml +++ b/spring-batch-samples/src/main/resources/META-INF/persistence.xml @@ -1,13 +1,10 @@ - org.springframework.batch.sample.domain.trade.CustomerCredit true - - - \ No newline at end of file + diff --git a/spring-batch-samples/src/main/resources/META-INF/spring/config-beans.xml b/spring-batch-samples/src/main/resources/META-INF/spring/config-beans.xml index f5dc575def..e83c28ddb2 100644 --- a/spring-batch-samples/src/main/resources/META-INF/spring/config-beans.xml +++ b/spring-batch-samples/src/main/resources/META-INF/spring/config-beans.xml @@ -5,9 +5,8 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/context https://www.springframework.org/schema/context/spring-context.xsd"> - diff --git a/spring-batch-samples/src/main/resources/META-INF/spring/jobs/amqp/amqp-example-job-beans.xml b/spring-batch-samples/src/main/resources/META-INF/spring/jobs/amqp/amqp-example-job-beans.xml index 8d77e3866a..2979a57f0e 100644 --- a/spring-batch-samples/src/main/resources/META-INF/spring/jobs/amqp/amqp-example-job-beans.xml +++ b/spring-batch-samples/src/main/resources/META-INF/spring/jobs/amqp/amqp-example-job-beans.xml @@ -4,7 +4,7 @@ xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> diff --git a/spring-batch-samples/src/main/resources/META-INF/spring/jobs/amqp/amqp-example-job.xml b/spring-batch-samples/src/main/resources/META-INF/spring/jobs/amqp/amqp-example-job.xml index 0f92248cb0..e251ef6b8c 100644 --- a/spring-batch-samples/src/main/resources/META-INF/spring/jobs/amqp/amqp-example-job.xml +++ b/spring-batch-samples/src/main/resources/META-INF/spring/jobs/amqp/amqp-example-job.xml @@ -5,8 +5,8 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:batch="http://www.springframework.org/schema/batch" xsi:schemaLocation=" - http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + http://www.springframework.org/schema/batch https://www.springframework.org/schema/batch/spring-batch.xsd + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> diff --git a/spring-batch-samples/src/main/resources/META-INF/spring/jobs/messaging/rabbitmq-beans.xml b/spring-batch-samples/src/main/resources/META-INF/spring/jobs/messaging/rabbitmq-beans.xml index 4bca5f1437..f5fff1a0a0 100644 --- a/spring-batch-samples/src/main/resources/META-INF/spring/jobs/messaging/rabbitmq-beans.xml +++ b/spring-batch-samples/src/main/resources/META-INF/spring/jobs/messaging/rabbitmq-beans.xml @@ -6,8 +6,8 @@ xmlns:rabbit="http://www.springframework.org/schema/rabbit" xmlns:p="http://www.springframework.org/schema/p" xsi:schemaLocation=" - http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.0.xsd - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + http://www.springframework.org/schema/rabbit https://www.springframework.org/schema/rabbit/spring-rabbit.xsd + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -17,7 +17,7 @@ + p:defaultReceiveQueue="${rabbitmq.inbound.queue}"/> diff --git a/spring-batch-samples/src/main/resources/adhoc-job-launcher-context.xml b/spring-batch-samples/src/main/resources/adhoc-job-launcher-context.xml index f7a6803093..ed8d3e4547 100644 --- a/spring-batch-samples/src/main/resources/adhoc-job-launcher-context.xml +++ b/spring-batch-samples/src/main/resources/adhoc-job-launcher-context.xml @@ -1,10 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans+%20%20%20%20%20%20%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -57,5 +54,4 @@ - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/batch-derby.properties b/spring-batch-samples/src/main/resources/batch-derby.properties index 385ec9ca5b..2eea26309e 100644 --- a/spring-batch-samples/src/main/resources/batch-derby.properties +++ b/spring-batch-samples/src/main/resources/batch-derby.properties @@ -16,3 +16,4 @@ batch.grid.size=2 batch.jdbc.pool.size=6 batch.verify.cursor.position=false batch.isolationlevel=ISOLATION_SERIALIZABLE +batch.table.prefix=BATCH_ diff --git a/spring-batch-samples/src/main/resources/batch-h2.properties b/spring-batch-samples/src/main/resources/batch-h2.properties index 925428ceda..38c325f4ba 100644 --- a/spring-batch-samples/src/main/resources/batch-h2.properties +++ b/spring-batch-samples/src/main/resources/batch-h2.properties @@ -1,7 +1,7 @@ # Placeholders batch.* # for H2: batch.jdbc.driver=org.h2.Driver -batch.jdbc.url=jdbc:h2:file:target/data/h2 +batch.jdbc.url=jdbc:h2:file:build/data/h2 batch.jdbc.user=sa batch.jdbc.password= batch.jdbc.testWhileIdle=false @@ -16,3 +16,4 @@ batch.grid.size=2 batch.jdbc.pool.size=6 batch.verify.cursor.position=true batch.isolationlevel=ISOLATION_SERIALIZABLE +batch.table.prefix=BATCH_ diff --git a/spring-batch-samples/src/main/resources/batch-hsql.properties b/spring-batch-samples/src/main/resources/batch-hsql.properties index 41423fa1d9..7b6e6846b4 100644 --- a/spring-batch-samples/src/main/resources/batch-hsql.properties +++ b/spring-batch-samples/src/main/resources/batch-hsql.properties @@ -1,7 +1,7 @@ # Placeholders batch.* # for HSQLDB: batch.jdbc.driver=org.hsqldb.jdbcDriver -batch.jdbc.url=jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true +batch.jdbc.url=jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true;hsqldb.tx=mvcc # use this one for a separate server process so you can inspect the results # (or add it to system properties with -D to override at run time). # batch.jdbc.url=jdbc:hsqldb:hsql://localhost:9005/samples @@ -19,4 +19,5 @@ batch.jdbc.pool.size=6 batch.grid.size=6 batch.verify.cursor.position=true batch.isolationlevel=ISOLATION_SERIALIZABLE - +batch.data.source.init=true +batch.table.prefix=BATCH_ diff --git a/spring-batch-samples/src/main/resources/batch-mysql.properties b/spring-batch-samples/src/main/resources/batch-mysql.properties index 947c24b7b4..e1a8c17bcd 100644 --- a/spring-batch-samples/src/main/resources/batch-mysql.properties +++ b/spring-batch-samples/src/main/resources/batch-mysql.properties @@ -16,4 +16,5 @@ batch.jdbc.pool.size=6 batch.grid.size=50 batch.verify.cursor.position=true batch.isolationlevel=ISOLATION_SERIALIZABLE +batch.table.prefix=BATCH_ diff --git a/spring-batch-samples/src/main/resources/batch-oracle.properties b/spring-batch-samples/src/main/resources/batch-oracle.properties index 22c75d5058..f039e2aba2 100644 --- a/spring-batch-samples/src/main/resources/batch-oracle.properties +++ b/spring-batch-samples/src/main/resources/batch-oracle.properties @@ -16,4 +16,5 @@ batch.grid.size=2 batch.jdbc.pool.size=6 batch.verify.cursor.position=true batch.isolationlevel=ISOLATION_SERIALIZABLE +batch.table.prefix=BATCH_ diff --git a/spring-batch-samples/src/main/resources/batch-postgresql.properties b/spring-batch-samples/src/main/resources/batch-postgresql.properties index bf554dde7f..58b97f0803 100644 --- a/spring-batch-samples/src/main/resources/batch-postgresql.properties +++ b/spring-batch-samples/src/main/resources/batch-postgresql.properties @@ -11,10 +11,11 @@ batch.drop.script=classpath:/org/springframework/batch/core/schema-drop-postgres batch.schema.script=classpath:/org/springframework/batch/core/schema-postgresql.sql batch.business.schema.script=business-schema-postgresql.sql batch.data.source.init=true -batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.PostgreSQLSequenceMaxValueIncrementer +batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.PostgresSequenceMaxValueIncrementer batch.database.incrementer.parent=sequenceIncrementerParent batch.lob.handler.class=org.springframework.jdbc.support.lob.DefaultLobHandler batch.grid.size=2 batch.jdbc.pool.size=6 batch.verify.cursor.position=false -batch.isolationlevel=ISOLATION_SERIALIZABLE \ No newline at end of file +batch.isolationlevel=ISOLATION_SERIALIZABLE +batch.table.prefix=BATCH_ diff --git a/spring-batch-samples/src/main/resources/batch-sqlf.properties b/spring-batch-samples/src/main/resources/batch-sqlf.properties index 532ab10fb8..d2a57b94b6 100644 --- a/spring-batch-samples/src/main/resources/batch-sqlf.properties +++ b/spring-batch-samples/src/main/resources/batch-sqlf.properties @@ -19,4 +19,5 @@ batch.jdbc.pool.size=6 batch.verify.cursor.position=false #add this for your jobRepository bean p:isolationLevelForCreate = "${batch.isolationlevel}" #see simple-job-launcher-context.xml for example -batch.isolationlevel=ISOLATION_READ_COMMITTED \ No newline at end of file +batch.isolationlevel=ISOLATION_READ_COMMITTED +batch.table.prefix=BATCH_ diff --git a/spring-batch-samples/src/main/resources/batch-sqlserver.properties b/spring-batch-samples/src/main/resources/batch-sqlserver.properties index f6307adf99..b6d018d84d 100644 --- a/spring-batch-samples/src/main/resources/batch-sqlserver.properties +++ b/spring-batch-samples/src/main/resources/batch-sqlserver.properties @@ -16,4 +16,5 @@ batch.grid.size=2 batch.jdbc.pool.size=6 batch.verify.cursor.position=true batch.isolationlevel=ISOLATION_SERIALIZABLE +batch.table.prefix=BATCH_ diff --git a/spring-batch-samples/src/main/resources/batch-sybase.properties b/spring-batch-samples/src/main/resources/batch-sybase.properties index 706bc4c622..041d1b2cb1 100644 --- a/spring-batch-samples/src/main/resources/batch-sybase.properties +++ b/spring-batch-samples/src/main/resources/batch-sybase.properties @@ -16,5 +16,6 @@ batch.jdbc.pool.size=6 batch.grid.size=6 batch.verify.cursor.position=true batch.isolationlevel=ISOLATION_SERIALIZABLE +batch.table.prefix=BATCH_ diff --git a/spring-batch-samples/src/main/resources/business-schema-mysql.sql b/spring-batch-samples/src/main/resources/business-schema-mysql.sql index 99fd85c657..7f098b7876 100644 --- a/spring-batch-samples/src/main/resources/business-schema-mysql.sql +++ b/spring-batch-samples/src/main/resources/business-schema-mysql.sql @@ -1,4 +1,5 @@ -- Autogenerated: do not edit this file +-- You might need to remove this section the first time you run against a clean database DROP TABLE IF EXISTS BATCH_STAGING_SEQ ; DROP TABLE IF EXISTS TRADE_SEQ ; DROP TABLE IF EXISTS CUSTOMER_SEQ ; @@ -12,11 +13,11 @@ DROP TABLE IF EXISTS ERROR_LOG ; -- Autogenerated: do not edit this file -CREATE TABLE CUSTOMER_SEQ (ID BIGINT NOT NULL) type=MYISAM; +CREATE TABLE CUSTOMER_SEQ (ID BIGINT NOT NULL) engine=InnoDB; INSERT INTO CUSTOMER_SEQ values(5); -CREATE TABLE BATCH_STAGING_SEQ (ID BIGINT NOT NULL) type=MYISAM; +CREATE TABLE BATCH_STAGING_SEQ (ID BIGINT NOT NULL) engine=InnoDB; INSERT INTO BATCH_STAGING_SEQ values(0); -CREATE TABLE TRADE_SEQ (ID BIGINT NOT NULL) type=MYISAM; +CREATE TABLE TRADE_SEQ (ID BIGINT NOT NULL) engine=InnoDB; INSERT INTO TRADE_SEQ values(0); CREATE TABLE BATCH_STAGING ( @@ -24,7 +25,7 @@ CREATE TABLE BATCH_STAGING ( JOB_ID BIGINT NOT NULL, VALUE BLOB NOT NULL, PROCESSED CHAR(1) NOT NULL -) type=InnoDB; +) engine=InnoDB; CREATE TABLE TRADE ( ID BIGINT NOT NULL PRIMARY KEY , @@ -33,14 +34,14 @@ CREATE TABLE TRADE ( QUANTITY BIGINT , PRICE DECIMAL(8,2) , CUSTOMER VARCHAR(45) -) type=InnoDB; +) engine=InnoDB; CREATE TABLE CUSTOMER ( ID BIGINT NOT NULL PRIMARY KEY , VERSION BIGINT , NAME VARCHAR(45) , CREDIT DECIMAL(10,2) -) type=InnoDB; +) engine=InnoDB; INSERT INTO CUSTOMER (ID, VERSION, NAME, CREDIT) VALUES (1, 0, 'customer1', 100000); INSERT INTO CUSTOMER (ID, VERSION, NAME, CREDIT) VALUES (2, 0, 'customer2', 100000); @@ -54,7 +55,7 @@ CREATE TABLE PLAYERS ( POS VARCHAR(10) , YEAR_OF_BIRTH BIGINT NOT NULL, YEAR_DRAFTED BIGINT NOT NULL -) type=InnoDB; +) engine=InnoDB; CREATE TABLE GAMES ( PLAYER_ID CHAR(8) NOT NULL, @@ -72,7 +73,7 @@ CREATE TABLE GAMES ( RECEPTIONS BIGINT , RECEPTIONS_YARDS BIGINT , TOTAL_TD BIGINT -) type=InnoDB; +) engine=InnoDB; CREATE TABLE PLAYER_SUMMARY ( ID CHAR(8) NOT NULL, @@ -87,10 +88,10 @@ CREATE TABLE PLAYER_SUMMARY ( RECEPTIONS BIGINT NOT NULL , RECEPTIONS_YARDS BIGINT NOT NULL , TOTAL_TD BIGINT NOT NULL -) type=InnoDB; +) engine=InnoDB; CREATE TABLE ERROR_LOG ( JOB_NAME CHAR(20) , STEP_NAME CHAR(20) , MESSAGE VARCHAR(300) NOT NULL -) type=InnoDB; +) engine=InnoDB; diff --git a/spring-batch-samples/src/main/resources/data-source-context.xml b/spring-batch-samples/src/main/resources/data-source-context.xml index 9c34cc82c9..c3559f4244 100644 --- a/spring-batch-samples/src/main/resources/data-source-context.xml +++ b/spring-batch-samples/src/main/resources/data-source-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd+http://www.springframework.org/schema/jdbc%20https://www.springframework.org/schema/jdbc/spring-jdbc.xsd"> @@ -11,12 +11,12 @@ - + - + diff --git a/spring-batch-samples/src/main/resources/hibernate-context.xml b/spring-batch-samples/src/main/resources/hibernate-context.xml index 845c8be5c2..4af30c3104 100644 --- a/spring-batch-samples/src/main/resources/hibernate-context.xml +++ b/spring-batch-samples/src/main/resources/hibernate-context.xml @@ -1,9 +1,9 @@ - - - - + + + + @@ -16,8 +16,7 @@ - + - - + diff --git a/spring-batch-samples/src/main/resources/ibatis-config.xml b/spring-batch-samples/src/main/resources/ibatis-config.xml deleted file mode 100644 index 9d2f4e8c32..0000000000 --- a/spring-batch-samples/src/main/resources/ibatis-config.xml +++ /dev/null @@ -1,9 +0,0 @@ - - - - - - - - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/ibatis-customer-credit.xml b/spring-batch-samples/src/main/resources/ibatis-customer-credit.xml deleted file mode 100644 index cd23cf0a7d..0000000000 --- a/spring-batch-samples/src/main/resources/ibatis-customer-credit.xml +++ /dev/null @@ -1,28 +0,0 @@ - - - - - - - - - - - - - - - - - - update CUSTOMER set CREDIT = #credit# where NAME = #name# - - - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/adhocLoopJob.xml b/spring-batch-samples/src/main/resources/jobs/adhocLoopJob.xml index 8c341129ae..0865fe9287 100644 --- a/spring-batch-samples/src/main/resources/jobs/adhocLoopJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/adhocLoopJob.xml @@ -4,11 +4,11 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop" xsi:schemaLocation="http://www.springframework.org/schema/beans-%20%20%20%20%20%20%20http://www.springframework.org/schema/beans/spring-beans.xsd+%20%20%20%20%20%20%20https://www.springframework.org/schema/beans/spring-beans.xsd%20%20%20%20%20%20%20http://www.springframework.org/schema/batch-%20%20%20%20%20%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd+%20%20%20%20%20%20https://www.springframework.org/schema/batch/spring-batch.xsd%20%20%20%20%20%20%20%20http://www.springframework.org/schema/aop-%20http://www.springframework.org/schema/aop/spring-aop.xsd"> + https://www.springframework.org/schema/aop/spring-aop.xsd"> @@ -40,5 +40,4 @@ method="onError" /> - diff --git a/spring-batch-samples/src/main/resources/jobs/amqp-example-job.xml b/spring-batch-samples/src/main/resources/jobs/amqp-example-job.xml index 0f92248cb0..e251ef6b8c 100644 --- a/spring-batch-samples/src/main/resources/jobs/amqp-example-job.xml +++ b/spring-batch-samples/src/main/resources/jobs/amqp-example-job.xml @@ -5,8 +5,8 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:batch="http://www.springframework.org/schema/batch" xsi:schemaLocation=" - http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> + http://www.springframework.org/schema/batch https://www.springframework.org/schema/batch/spring-batch.xsd + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> diff --git a/spring-batch-samples/src/main/resources/jobs/beanWrapperMapperSampleJob.xml b/spring-batch-samples/src/main/resources/jobs/beanWrapperMapperSampleJob.xml index 3b910e5e1f..2b00aa6928 100644 --- a/spring-batch-samples/src/main/resources/jobs/beanWrapperMapperSampleJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/beanWrapperMapperSampleJob.xml @@ -1,12 +1,9 @@ - + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/batch https://www.springframework.org/schema/batch/spring-batch.xsd"> @@ -60,15 +57,7 @@ - - - - - - - + @@ -102,5 +91,4 @@ - diff --git a/spring-batch-samples/src/main/resources/jobs/compositeItemWriterSampleJob.xml b/spring-batch-samples/src/main/resources/jobs/compositeItemWriterSampleJob.xml index 75c2deba74..388e08bc8d 100644 --- a/spring-batch-samples/src/main/resources/jobs/compositeItemWriterSampleJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/compositeItemWriterSampleJob.xml @@ -1,12 +1,9 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/batch https://www.springframework.org/schema/batch/spring-batch.xsd"> - - - - - @@ -66,15 +58,7 @@ - - - - - - - + @@ -90,7 +74,7 @@ - + @@ -98,12 +82,11 @@ - + - - \ No newline at end of file + diff --git a/spring-batch-samples/src/main/resources/jobs/customerFilterJob.xml b/spring-batch-samples/src/main/resources/jobs/customerFilterJob.xml index ac57161f4d..ad727d9e19 100644 --- a/spring-batch-samples/src/main/resources/jobs/customerFilterJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/customerFilterJob.xml @@ -1,9 +1,8 @@ + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd+http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd"> @@ -72,5 +71,4 @@ - diff --git a/spring-batch-samples/src/main/resources/jobs/delegatingJob.xml b/spring-batch-samples/src/main/resources/jobs/delegatingJob.xml index 3236ebf9f1..9465c0f25e 100644 --- a/spring-batch-samples/src/main/resources/jobs/delegatingJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/delegatingJob.xml @@ -1,14 +1,9 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/batch https://www.springframework.org/schema/batch/spring-batch.xsd"> The intent is to to give an example of how existing bean @@ -41,5 +36,4 @@ - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/footballJob.xml b/spring-batch-samples/src/main/resources/jobs/footballJob.xml index 0e85cdb203..f73c42808d 100644 --- a/spring-batch-samples/src/main/resources/jobs/footballJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/footballJob.xml @@ -1,11 +1,10 @@ + xsi:schemaLocation="http://www.springframework.org/schema/aop%20https://www.springframework.org/schema/aop/spring-aop.xsd+http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -127,5 +126,4 @@ - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/groovyJob.xml b/spring-batch-samples/src/main/resources/jobs/groovyJob.xml index 922e25fdf4..d964ced128 100644 --- a/spring-batch-samples/src/main/resources/jobs/groovyJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/groovyJob.xml @@ -1,10 +1,10 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd+http://www.springframework.org/schema/lang%20https://www.springframework.org/schema/lang/spring-lang.xsd"> @@ -35,7 +35,7 @@ void execute() { def ant = new AntBuilder() ant.unzip(src:"src/main/resources/data/groovyJob/input/files.zip", - dest:"target/groovyJob/staging") + dest:"build/groovyJob/staging") } } @@ -46,12 +46,11 @@ class ZipTasklet { void execute() { def ant = new AntBuilder() - ant.mkdir(dir:"target/groovyJob/output") - ant.zip(destfile:"target/groovyJob/output/files.zip", - basedir:"target/groovyJob/staging", includes:"**") + ant.mkdir(dir:"build/groovyJob/output") + ant.zip(destfile:"build/groovyJob/output/files.zip", + basedir:"build/groovyJob/staging", includes:"**") } } - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/headerFooterSample.xml b/spring-batch-samples/src/main/resources/jobs/headerFooterSample.xml index 917f5ed3de..bf2c845557 100644 --- a/spring-batch-samples/src/main/resources/jobs/headerFooterSample.xml +++ b/spring-batch-samples/src/main/resources/jobs/headerFooterSample.xml @@ -1,12 +1,9 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/batch https://www.springframework.org/schema/batch/spring-batch.xsd"> Showcases reading and writing of headers and footers. Copies header from input to output and adds a footer. @@ -64,6 +61,6 @@ + value="build/test-outputs/headerFooterOutput.txt" /> \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/hibernateJob.xml b/spring-batch-samples/src/main/resources/jobs/hibernateJob.xml index 47292f82a6..f016980389 100644 --- a/spring-batch-samples/src/main/resources/jobs/hibernateJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/hibernateJob.xml @@ -1,9 +1,9 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/batch https://www.springframework.org/schema/batch/spring-batch.xsd"> Example for Hibernate integration. @@ -42,5 +42,4 @@ - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/infiniteLoopJob.xml b/spring-batch-samples/src/main/resources/jobs/infiniteLoopJob.xml index 560935156b..53ad9ab963 100644 --- a/spring-batch-samples/src/main/resources/jobs/infiniteLoopJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/infiniteLoopJob.xml @@ -1,10 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans+%20%20%20%20%20%20%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -37,11 +34,6 @@ - @@ -54,5 +46,4 @@ - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/ioSampleJob.xml b/spring-batch-samples/src/main/resources/jobs/ioSampleJob.xml index 31050e0e2f..7bf5b6e527 100644 --- a/spring-batch-samples/src/main/resources/jobs/ioSampleJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/ioSampleJob.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -13,7 +13,5 @@ - - + diff --git a/spring-batch-samples/src/main/resources/jobs/iosample/delimited.xml b/spring-batch-samples/src/main/resources/jobs/iosample/delimited.xml index 3027fec6ba..084991ee78 100644 --- a/spring-batch-samples/src/main/resources/jobs/iosample/delimited.xml +++ b/spring-batch-samples/src/main/resources/jobs/iosample/delimited.xml @@ -1,15 +1,8 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> diff --git a/spring-batch-samples/src/main/resources/jobs/iosample/fixedLength.xml b/spring-batch-samples/src/main/resources/jobs/iosample/fixedLength.xml index 9253f7e703..5cb917bc38 100644 --- a/spring-batch-samples/src/main/resources/jobs/iosample/fixedLength.xml +++ b/spring-batch-samples/src/main/resources/jobs/iosample/fixedLength.xml @@ -1,13 +1,8 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> diff --git a/spring-batch-samples/src/main/resources/jobs/iosample/hibernate.xml b/spring-batch-samples/src/main/resources/jobs/iosample/hibernate.xml index cee7064878..9f2602418a 100644 --- a/spring-batch-samples/src/main/resources/jobs/iosample/hibernate.xml +++ b/spring-batch-samples/src/main/resources/jobs/iosample/hibernate.xml @@ -1,7 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> diff --git a/spring-batch-samples/src/main/resources/jobs/iosample/ibatis.xml b/spring-batch-samples/src/main/resources/jobs/iosample/ibatis.xml deleted file mode 100644 index 6261739d7c..0000000000 --- a/spring-batch-samples/src/main/resources/jobs/iosample/ibatis.xml +++ /dev/null @@ -1,24 +0,0 @@ - - - - - - - - - - - - - - - - - - - - diff --git a/spring-batch-samples/src/main/resources/jobs/iosample/jdbcCursor.xml b/spring-batch-samples/src/main/resources/jobs/iosample/jdbcCursor.xml index d0e0b69a5d..c9286db477 100644 --- a/spring-batch-samples/src/main/resources/jobs/iosample/jdbcCursor.xml +++ b/spring-batch-samples/src/main/resources/jobs/iosample/jdbcCursor.xml @@ -1,15 +1,8 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -28,5 +21,4 @@ - diff --git a/spring-batch-samples/src/main/resources/jobs/iosample/jdbcPaging.xml b/spring-batch-samples/src/main/resources/jobs/iosample/jdbcPaging.xml index 74f080fb9b..1720b24129 100644 --- a/spring-batch-samples/src/main/resources/jobs/iosample/jdbcPaging.xml +++ b/spring-batch-samples/src/main/resources/jobs/iosample/jdbcPaging.xml @@ -1,12 +1,8 @@ - + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -45,5 +41,4 @@ - diff --git a/spring-batch-samples/src/main/resources/jobs/iosample/jpa.xml b/spring-batch-samples/src/main/resources/jobs/iosample/jpa.xml index 726e537661..8921196a61 100644 --- a/spring-batch-samples/src/main/resources/jobs/iosample/jpa.xml +++ b/spring-batch-samples/src/main/resources/jobs/iosample/jpa.xml @@ -1,7 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> diff --git a/spring-batch-samples/src/main/resources/jobs/iosample/multiLine.xml b/spring-batch-samples/src/main/resources/jobs/iosample/multiLine.xml index 3e22bed2fb..b4070db3d7 100644 --- a/spring-batch-samples/src/main/resources/jobs/iosample/multiLine.xml +++ b/spring-batch-samples/src/main/resources/jobs/iosample/multiLine.xml @@ -1,17 +1,10 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/batch https://www.springframework.org/schema/batch/spring-batch.xsd"> @@ -42,12 +35,11 @@ - + - diff --git a/spring-batch-samples/src/main/resources/jobs/iosample/multiRecordType.xml b/spring-batch-samples/src/main/resources/jobs/iosample/multiRecordType.xml index 25906509fd..fed5fd0983 100644 --- a/spring-batch-samples/src/main/resources/jobs/iosample/multiRecordType.xml +++ b/spring-batch-samples/src/main/resources/jobs/iosample/multiRecordType.xml @@ -1,17 +1,10 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/batch https://www.springframework.org/schema/batch/spring-batch.xsd"> @@ -34,12 +27,12 @@ - - - - - - + + + + + + + class="org.springframework.batch.sample.domain.trade.internal.TradeFieldSetMapper" /> - + diff --git a/spring-batch-samples/src/main/resources/jobs/iosample/multiResource.xml b/spring-batch-samples/src/main/resources/jobs/iosample/multiResource.xml index 050c3265fd..727c5eb412 100644 --- a/spring-batch-samples/src/main/resources/jobs/iosample/multiResource.xml +++ b/spring-batch-samples/src/main/resources/jobs/iosample/multiResource.xml @@ -1,13 +1,8 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -59,5 +54,4 @@ - diff --git a/spring-batch-samples/src/main/resources/jobs/iosample/repository.xml b/spring-batch-samples/src/main/resources/jobs/iosample/repository.xml index 73f4f462d4..e82d7fef76 100644 --- a/spring-batch-samples/src/main/resources/jobs/iosample/repository.xml +++ b/spring-batch-samples/src/main/resources/jobs/iosample/repository.xml @@ -2,8 +2,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd+%20http://www.springframework.org/schema/data/jpa%20https://www.springframework.org/schema/data/jpa/spring-jpa.xsd"> diff --git a/spring-batch-samples/src/main/resources/jobs/iosample/xml.xml b/spring-batch-samples/src/main/resources/jobs/iosample/xml.xml index caac90308f..c866b93a51 100644 --- a/spring-batch-samples/src/main/resources/jobs/iosample/xml.xml +++ b/spring-batch-samples/src/main/resources/jobs/iosample/xml.xml @@ -1,13 +1,10 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/util https://www.springframework.org/schema/util/spring-util.xsd"> @@ -34,7 +31,6 @@ - + - diff --git a/spring-batch-samples/src/main/resources/jobs/jobStepSample.xml b/spring-batch-samples/src/main/resources/jobs/jobStepSample.xml index 94d6507d8b..3b2ca325ad 100644 --- a/spring-batch-samples/src/main/resources/jobs/jobStepSample.xml +++ b/spring-batch-samples/src/main/resources/jobs/jobStepSample.xml @@ -1,7 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -49,15 +49,7 @@ - - - - - - - + diff --git a/spring-batch-samples/src/main/resources/jobs/loopFlowSample.xml b/spring-batch-samples/src/main/resources/jobs/loopFlowSample.xml index e77079f516..c69942fce3 100644 --- a/spring-batch-samples/src/main/resources/jobs/loopFlowSample.xml +++ b/spring-batch-samples/src/main/resources/jobs/loopFlowSample.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -40,5 +40,4 @@ - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/mailJob.xml b/spring-batch-samples/src/main/resources/jobs/mailJob.xml index bf813e328d..5ce9cedcb6 100644 --- a/spring-batch-samples/src/main/resources/jobs/mailJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/mailJob.xml @@ -1,9 +1,8 @@ - + @@ -12,9 +11,6 @@ - @@ -48,5 +44,4 @@ - diff --git a/spring-batch-samples/src/main/resources/jobs/multilineJob.xml b/spring-batch-samples/src/main/resources/jobs/multilineJob.xml index fb80f375f9..6e6425621a 100644 --- a/spring-batch-samples/src/main/resources/jobs/multilineJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/multilineJob.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -23,7 +23,7 @@ + value="file:build/test-outputs/20070122.testStream.multilineStep.txt" /> @@ -75,5 +75,4 @@ - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/multilineOrderInputTokenizers.xml b/spring-batch-samples/src/main/resources/jobs/multilineOrderInputTokenizers.xml index 02b7736556..f4a8582cb7 100644 --- a/spring-batch-samples/src/main/resources/jobs/multilineOrderInputTokenizers.xml +++ b/spring-batch-samples/src/main/resources/jobs/multilineOrderInputTokenizers.xml @@ -1,7 +1,7 @@ + https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -61,5 +61,4 @@ - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/multilineOrderJob.xml b/spring-batch-samples/src/main/resources/jobs/multilineOrderJob.xml index 56c7f41664..7aafd7bc48 100644 --- a/spring-batch-samples/src/main/resources/jobs/multilineOrderJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/multilineOrderJob.xml @@ -1,12 +1,10 @@ + https://www.springframework.org/schema/batch/spring-batch.xsd"> @@ -56,15 +54,15 @@ + - + - - \ No newline at end of file + diff --git a/spring-batch-samples/src/main/resources/jobs/multilineOrderOutputAggregators.xml b/spring-batch-samples/src/main/resources/jobs/multilineOrderOutputAggregators.xml index 93c5ebe717..007b8d2df0 100644 --- a/spring-batch-samples/src/main/resources/jobs/multilineOrderOutputAggregators.xml +++ b/spring-batch-samples/src/main/resources/jobs/multilineOrderOutputAggregators.xml @@ -1,9 +1,9 @@ + https://www.springframework.org/schema/beans/spring-beans.xsd + http://www.springframework.org/schema/util https://www.springframework.org/schema/util/spring-util.xsd"> @@ -57,5 +57,4 @@ - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/multilineOrderValidator.xml b/spring-batch-samples/src/main/resources/jobs/multilineOrderValidator.xml index 1ef36c919c..68cc765153 100644 --- a/spring-batch-samples/src/main/resources/jobs/multilineOrderValidator.xml +++ b/spring-batch-samples/src/main/resources/jobs/multilineOrderValidator.xml @@ -1,76 +1,12 @@ - + https://www.springframework.org/schema/beans/spring-beans.xsd"> - - - - - 0 AND ? <= 9999999999 : 'Incorrect order ID' : 'error.order.id' } - { orderDate : isFutureDate(?) = FALSE : 'Future date is not allowed' : 'error.order.date.future' } - { totalLines : ? = size(lineItems) : 'Bad count of order lines' : 'error.order.lines.badcount'} - - { customer.registered : customer.businessCustomer = FALSE OR ? = TRUE : 'Business customer must be registered' : 'error.customer.registration'} - { customer.companyName : customer.businessCustomer = FALSE OR ? HAS TEXT : 'Company name for business customer is mandatory' : 'error.customer.companyname'} - { customer.firstName : customer.businessCustomer = TRUE OR ? HAS TEXT : 'Firstname for non-business customer is mandatory' : 'error.customer.firstname'} - { customer.lastName : customer.businessCustomer = TRUE OR ? HAS TEXT : 'Lastname name for non-business customer is mandatory' : 'error.customer.lastname'} - { customer.registrationId : customer.registered = FALSE OR (? > 0 AND ? < 99999999) : 'Incorrect registration ID' : 'error.customer.registrationid'} - - { billingAddress.addressee : ? HAS NO TEXT OR length(?) <= 60 : 'Maximum length for Addressee is 60 characters' : 'error.baddress.addresse.length'} - { billingAddress.addrLine1 : ? HAS TEXT AND length(?) <= 50 : 'Address line1 is mandatory and maximum length for address line1 is 50 characters' : 'error.baddress.addrline1.length'} - { billingAddress.addrLine2 : ? HAS NO TEXT OR length(?) <= 50 : 'Maximum length for address line2 is 50 characters' : 'error.baddress.addrline2.length'} - { billingAddress.city : ? HAS TEXT AND length(?) <= 30 : 'City is mandatory and maximum length for city is 30 characters' : 'error.baddress.city.length'} - { billingAddress.zipCode : ? HAS TEXT AND length(?) <= 50 : 'Zipcode is mandatory and maximum length for zipcode is 5 characters' : 'error.baddress.zipcode.length'} - { billingAddress.zipCode : match('[0-9]{5}',?) = TRUE : 'ZipCode must contain exactly 5 digits' : 'error.baddress.zipcode.format'} - { billingAddress.state : (? HAS NO TEXT AND billingAddress.country != 'United States') OR (? HAS TEXT AND length(?) <= 2) : 'Maximum length for state is 2 characters' : 'error.baddress.state.length'} - { billingAddress.country : ? HAS TEXT AND length(?) <= 50 : 'Country is mandatory and maximum length for country is 50 characters' : 'error.baddress.country.length'} - - { shippingAddress.addressee : shippingAddress IS NULL OR (? HAS TEXT AND length(?) <= 60) : 'Addressee is mandatory and maximum length for addressee is 60 characters' : 'error.saddress.addresse.length'} - { shippingAddress.addrLine1 : shippingAddress IS NULL OR (? HAS TEXT AND length(?) <= 50) : 'Address line1 is mandatory and maximum length for address line1 is 50 characters' : 'error.baddress.addrline1.length'} - { shippingAddress.addrLine2 : shippingAddress IS NULL OR (? HAS NO TEXT OR length(?) <= 50) : 'Maximum length for address line2 is 50 characters' : 'error.baddress.addrline2.length'} - { shippingAddress.city : shippingAddress IS NULL OR (? HAS TEXT AND length(?) <= 30) : 'City is mandatory and maximum length for city is 30 characters' : 'error.baddress.city.length'} - { shippingAddress.zipCode : shippingAddress IS NULL OR (? HAS TEXT AND length(?) <= 50) : 'Zipcode is mandatory and maximum length for zipcode is 5 characters' : 'error.baddress.zipcode.length'} - { shippingAddress.zipCode : shippingAddress IS NULL OR (match('[0-9]{5}',?) = TRUE) : 'Zipcode must contain exactly 5 digits' : 'error.baddress.zipcode.format'} - { shippingAddress.state : shippingAddress IS NULL OR ((? HAS NO TEXT AND billingAddress.country != 'United States') OR (? HAS TEXT AND length(?) <= 2)) : 'Maximum length for state is 2 characters' : 'error.baddress.state.length'} - { shippingAddress.country : shippingAddress IS NULL OR (? HAS TEXT AND length(?) <= 50) : 'Country is mandatory and maximum length for country is 50 characters' : 'error.baddress.country.length'} - - { billing.paymentId : ? IN 'VISA','AMEX','ECMC','DCIN','PAYP' : 'Invalid payment type' : 'error.billing.type' } - { billing.paymentDesc : match('[A-Z]{4}-[0-9]{10,11}',?) = TRUE : 'Invalid format of payment description' : 'error.billing.desc' } - - { shipping.shipperId : ? IN 'FEDX', 'UPS', 'DHL', 'DPD' : 'Invalid shipper ID' : 'error.shipping.shipper'} - { shipping.shippingTypeId : ? IN 'STD', 'EXP', 'AMS', 'AME' : 'Invalid shipping type' : 'error.shipping.type' } - { shipping.shippingInfo : ? HAS NO TEXT OR length(?) <= 100 : 'Maximum length for additional shipping info is 100 characters' } - - { lineItems : validateTotalItemsCount(totalItems,?) = TRUE : 'Bad count of total line items' : 'error.lineitems.badcount' } - { lineItems : validateIds(?) = TRUE : 'One or more invalid item IDs' : 'error.lineitems.id' } - { lineItems : validatePrices(?) = TRUE : 'One or more invalid item prices' : 'error.lineitems.price' } - { lineItems : validateDiscounts(?) = TRUE : 'One or more invalid item discounts' : 'error.lineitems.discount' } - { lineItems : validateShippingPrices(?) = TRUE : 'One or more invalid item shipping prices' : 'error.lineitems.shipping' } - { lineItems : validateHandlingPrices(?) = TRUE : 'One or more invalid item handling prices' : 'error.lineitems.handling' } - { lineItems : validateQuantities(?) = TRUE : 'One or more invalid item quantities' : 'error.lineitems.quantity' } - { lineItems : validateTotalPrices(?) = TRUE : 'One or more invalid item total prices' : 'error.lineitems.totalprice' } - ]]> - - - - - - - - - - - - - - - - + - \ No newline at end of file + diff --git a/spring-batch-samples/src/main/resources/jobs/parallelJob.xml b/spring-batch-samples/src/main/resources/jobs/parallelJob.xml index 272f438441..21c5822f26 100644 --- a/spring-batch-samples/src/main/resources/jobs/parallelJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/parallelJob.xml @@ -2,9 +2,9 @@ + xsi:schemaLocation="http://www.springframework.org/schema/aop%20https://www.springframework.org/schema/aop/spring-aop.xsd+http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -82,15 +82,7 @@ - - - - - - - + @@ -115,5 +107,4 @@ method="doStronglyTypedLogging" /> - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/partitionFileJob.xml b/spring-batch-samples/src/main/resources/jobs/partitionFileJob.xml index 3dba318047..f00e3a08ed 100644 --- a/spring-batch-samples/src/main/resources/jobs/partitionFileJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/partitionFileJob.xml @@ -1,9 +1,8 @@ - + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -17,7 +16,7 @@ - + @@ -29,7 +28,7 @@ - + @@ -43,7 +42,7 @@ - + @@ -82,5 +81,4 @@ - diff --git a/spring-batch-samples/src/main/resources/jobs/partitionJdbcJob.xml b/spring-batch-samples/src/main/resources/jobs/partitionJdbcJob.xml index 09949e95be..fc35d4ebfd 100644 --- a/spring-batch-samples/src/main/resources/jobs/partitionJdbcJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/partitionJdbcJob.xml @@ -1,9 +1,8 @@ - + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -31,7 +30,7 @@ - + @@ -61,7 +60,7 @@ - + @@ -113,5 +112,4 @@ - diff --git a/spring-batch-samples/src/main/resources/jobs/restartFileSampleJob.xml b/spring-batch-samples/src/main/resources/jobs/restartFileSampleJob.xml index 9a50f37dbb..db0a61ca81 100644 --- a/spring-batch-samples/src/main/resources/jobs/restartFileSampleJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/restartFileSampleJob.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -61,7 +61,6 @@ - + - diff --git a/spring-batch-samples/src/main/resources/jobs/restartSample.xml b/spring-batch-samples/src/main/resources/jobs/restartSample.xml index ab790740ad..75d50974ec 100644 --- a/spring-batch-samples/src/main/resources/jobs/restartSample.xml +++ b/spring-batch-samples/src/main/resources/jobs/restartSample.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -16,8 +16,6 @@ - - @@ -48,15 +46,7 @@ - - - - - - - + @@ -71,5 +61,4 @@ - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/retrySample.xml b/spring-batch-samples/src/main/resources/jobs/retrySample.xml index 6d4fa47c7b..c891efe19e 100644 --- a/spring-batch-samples/src/main/resources/jobs/retrySample.xml +++ b/spring-batch-samples/src/main/resources/jobs/retrySample.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -24,5 +24,4 @@ - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/jobs/skipSampleJob.xml b/spring-batch-samples/src/main/resources/jobs/skipSampleJob.xml index 14e4dee716..8116ae2e81 100644 --- a/spring-batch-samples/src/main/resources/jobs/skipSampleJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/skipSampleJob.xml @@ -1,9 +1,9 @@ + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd+http://www.springframework.org/schema/util%20https://www.springframework.org/schema/util/spring-util.xsd"> @@ -133,5 +133,4 @@ - diff --git a/spring-batch-samples/src/main/resources/jobs/taskletJob.xml b/spring-batch-samples/src/main/resources/jobs/taskletJob.xml index 68cb7aa5b2..5824f8aa79 100644 --- a/spring-batch-samples/src/main/resources/jobs/taskletJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/taskletJob.xml @@ -1,8 +1,8 @@ + + + + + + + + + foo2 + 3 + 3.14 + + + + diff --git a/spring-batch-samples/src/main/resources/jobs/tradeJob.xml b/spring-batch-samples/src/main/resources/jobs/tradeJob.xml index 6d13958d72..69dc816475 100644 --- a/spring-batch-samples/src/main/resources/jobs/tradeJob.xml +++ b/spring-batch-samples/src/main/resources/jobs/tradeJob.xml @@ -3,31 +3,31 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p" xmlns:batch="http://www.springframework.org/schema/batch" - xsi:schemaLocation="http://www.springframework.org/schema/batch%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd-http://www.springframework.org/schema/beans%20http://www.springframework.org/schema/beans/spring-beans.xsd"> + xsi:schemaLocation="http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch.xsd+http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> - - + - - + + - + - - + + - - + + @@ -80,7 +80,7 @@ - + @@ -106,14 +106,7 @@ - - - - - - - + - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/log4j.properties b/spring-batch-samples/src/main/resources/log4j.properties index b708a7b370..c5d0358955 100644 --- a/spring-batch-samples/src/main/resources/log4j.properties +++ b/spring-batch-samples/src/main/resources/log4j.properties @@ -2,9 +2,9 @@ log4j.rootLogger=info, stdout # log4j.rootLogger=info, stdout, chainsaw ### direct log messages to stdout ### -log4j.appender.stdout=org.apache.log4j.ConsoleAppender +log4j.appender.stdout=org.apache.logging.log4j.core.appender.ConsoleAppender log4j.appender.stdout.Target=System.out -log4j.appender.stdout.layout=org.apache.log4j.PatternLayout +log4j.appender.stdout.layout=org.apache.logging.log4j.core.layout.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %t %c{1}:%L - %m%n log4j.appender.chainsaw=org.apache.log4j.RollingFileAppender diff --git a/spring-batch-samples/src/main/resources/metrics-sample.properties b/spring-batch-samples/src/main/resources/metrics-sample.properties new file mode 100644 index 0000000000..2c30f3a8b6 --- /dev/null +++ b/spring-batch-samples/src/main/resources/metrics-sample.properties @@ -0,0 +1,5 @@ +thread.pool.size=3 +prometheus.push.rate=5000 +prometheus.job.name=springbatch +prometheus.grouping.key=appname +prometheus.pushgateway.url=0.0.0.0:9091 diff --git a/spring-batch-samples/src/main/resources/org/springframework/batch/sample/config/common-context.xml b/spring-batch-samples/src/main/resources/org/springframework/batch/sample/config/common-context.xml index b6d093d450..7896ff2d5c 100644 --- a/spring-batch-samples/src/main/resources/org/springframework/batch/sample/config/common-context.xml +++ b/spring-batch-samples/src/main/resources/org/springframework/batch/sample/config/common-context.xml @@ -1,11 +1,7 @@ - + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -14,5 +10,4 @@ - diff --git a/spring-batch-samples/src/main/resources/org/springframework/batch/sample/domain/trade/CustomerCredit.hbm.xml b/spring-batch-samples/src/main/resources/org/springframework/batch/sample/domain/trade/CustomerCredit.hbm.xml index 69b8507193..9aa6e514cf 100644 --- a/spring-batch-samples/src/main/resources/org/springframework/batch/sample/domain/trade/CustomerCredit.hbm.xml +++ b/spring-batch-samples/src/main/resources/org/springframework/batch/sample/domain/trade/CustomerCredit.hbm.xml @@ -1,7 +1,7 @@ + "https://hibernate.org/dtd/hibernate-mapping-3.0.dtd"> + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> - diff --git a/spring-batch-samples/src/main/resources/quartz-job-launcher-context.xml b/spring-batch-samples/src/main/resources/quartz-job-launcher-context.xml deleted file mode 100644 index be921bc83d..0000000000 --- a/spring-batch-samples/src/main/resources/quartz-job-launcher-context.xml +++ /dev/null @@ -1,42 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/spring-batch-samples/src/main/resources/remote-chunking.properties b/spring-batch-samples/src/main/resources/remote-chunking.properties new file mode 100644 index 0000000000..0a85c22526 --- /dev/null +++ b/spring-batch-samples/src/main/resources/remote-chunking.properties @@ -0,0 +1 @@ +broker.url=tcp://localhost:61616 diff --git a/spring-batch-samples/src/main/resources/remote-partitioning.properties b/spring-batch-samples/src/main/resources/remote-partitioning.properties new file mode 100644 index 0000000000..ed040df333 --- /dev/null +++ b/spring-batch-samples/src/main/resources/remote-partitioning.properties @@ -0,0 +1,4 @@ +broker.url=tcp://localhost:61617 +datasource.url=jdbc:hsqldb:mem:testdb +datasource.username=sa +datasource.password= diff --git a/spring-batch-samples/src/main/resources/simple-job-launcher-context.xml b/spring-batch-samples/src/main/resources/simple-job-launcher-context.xml index 147ae9dd2e..38bfb7311b 100644 --- a/spring-batch-samples/src/main/resources/simple-job-launcher-context.xml +++ b/spring-batch-samples/src/main/resources/simple-job-launcher-context.xml @@ -1,11 +1,8 @@ + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -19,7 +16,7 @@ @@ -47,5 +44,4 @@ - - \ No newline at end of file + diff --git a/spring-batch-samples/src/main/resources/skipSample-job-launcher-context.xml b/spring-batch-samples/src/main/resources/skipSample-job-launcher-context.xml index bcf62cae5f..39307ffcb8 100644 --- a/spring-batch-samples/src/main/resources/skipSample-job-launcher-context.xml +++ b/spring-batch-samples/src/main/resources/skipSample-job-launcher-context.xml @@ -1,9 +1,8 @@ - + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -45,6 +44,4 @@ - - diff --git a/spring-batch-samples/src/main/resources/staging-test-context.xml b/spring-batch-samples/src/main/resources/staging-test-context.xml index 0c0dc8714f..345eb73141 100644 --- a/spring-batch-samples/src/main/resources/staging-test-context.xml +++ b/spring-batch-samples/src/main/resources/staging-test-context.xml @@ -1,8 +1,6 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -18,5 +16,4 @@ - \ No newline at end of file diff --git a/spring-batch-samples/src/main/resources/xstream-config.xml b/spring-batch-samples/src/main/resources/xstream-config.xml deleted file mode 100644 index a61f8a2a83..0000000000 --- a/spring-batch-samples/src/main/resources/xstream-config.xml +++ /dev/null @@ -1,75 +0,0 @@ - - - 1003 - purchaseOrders - - - xmlns - http://adsj.accenture.com/purchaseorders - - - xmlns:xsi - http://www.w3.org/2001/XMLSchema-instance - - - xsi:schemaLocation - http://adsj.accenture.com/purchaseorders purchaseorders.xsd - - - - - - - - - org.springframework.batch.sample.domain.order.internal.xml.Customer - org.springframework.batch.sample.domain.order.internal.xml.Order - customer - - - org.springframework.batch.sample.domain.order.internal.xml.Shipper - org.springframework.batch.sample.domain.order.internal.xml.Order - shipper - - - - - - - - - - - - - - - - - - - http://adsj.accenture.com/purchaseorders - order - - org.springframework.batch.sample.domain.order.internal.xml.Order - - - http://adsj.accenture.com/purchaseorders - customer - - org.springframework.batch.sample.domain.order.internal.xml.Customer - - - http://adsj.accenture.com/purchaseorders - shipper - - org.springframework.batch.sample.domain.order.internal.xml.Shipper - - - http://adsj.accenture.com/purchaseorders - lineItem - - org.springframework.batch.sample.domain.order.internal.xml.LineItem - - - diff --git a/spring-batch-samples/src/main/sql/mysql.properties b/spring-batch-samples/src/main/sql/mysql.properties index 763b6442fc..e498ef01b7 100644 --- a/spring-batch-samples/src/main/sql/mysql.properties +++ b/spring-batch-samples/src/main/sql/mysql.properties @@ -1,9 +1,9 @@ -platform=oracle10g +platform=mysql # SQL language oddities BIGINT = BIGINT IDENTITY = GENERATED = -VOODOO = type=InnoDB +VOODOO = engine=InnoDB IFEXISTSBEFORE = IF EXISTS DOUBLE = DOUBLE PRECISION DECIMAL = DECIMAL diff --git a/spring-batch-samples/src/main/sql/mysql.vpp b/spring-batch-samples/src/main/sql/mysql.vpp index 9897be54ec..cd2e6103dc 100644 --- a/spring-batch-samples/src/main/sql/mysql.vpp +++ b/spring-batch-samples/src/main/sql/mysql.vpp @@ -1,3 +1,3 @@ -#macro (sequence $name $value)CREATE TABLE ${name} (ID BIGINT NOT NULL) type=MYISAM; +#macro (sequence $name $value)CREATE TABLE ${name} (ID BIGINT NOT NULL) engine=InnoDB; INSERT INTO ${name} values(${value}); #end diff --git a/spring-batch-samples/src/site/apt/changelog.apt b/spring-batch-samples/src/site/apt/changelog.apt deleted file mode 100644 index e67691400d..0000000000 --- a/spring-batch-samples/src/site/apt/changelog.apt +++ /dev/null @@ -1,8 +0,0 @@ -Changelog: Spring Batch Samples - -* 1.0-M2 - -** 2007/06/25 - - * Dave Syer: fixed xml sample by using scope="step" to control bean - lifecycle and stimulte call to close(). diff --git a/spring-batch-samples/src/site/apt/index.apt b/spring-batch-samples/src/site/apt/index.apt deleted file mode 100644 index 24ac4845e5..0000000000 --- a/spring-batch-samples/src/site/apt/index.apt +++ /dev/null @@ -1,988 +0,0 @@ -Spring Batch Samples - -* Zip Downloads - - There is a ZIP artifact containing the release JARs called <<>>. This file contains the JAR files for the release, including source code and the samples. - - * Full releases: {{{http://static.springframework.org/downloads/nightly/release-download.php?project=BATCH}here}}. - - * Milestones: {{{http://static.springframework.org/downloads/nightly/milestone-download.php?project=BATCH}here}}. - -Source code can also be browsed and downloaded at {{{http://github.com/SpringSource/spring-batch}Github}}. - -* Overview - - There is considerable variability in the types of input and output - formats in batch jobs. There are also a number of options to consider - in terms of how the types of strategies that will be used to handle - skips, recovery, and statistics. However, when approaching a new - batch job there are a few standard questions to answer to help - determine how the job will be written and how to use the services - offered by the spring batch framework. Consider the following: - - * How do I configure this batch job? In the samples the pattern is - to follow the convention of <<<[nameOf]Job.xml>>>. Each sample - identifies the XML definition used to configure the job. Job - configurations that use a common execution environment have - many common items in their respective configurations. - - * What is the input source? Each sample batch job identifies - its input source. - - * What is my output source? Each sample batch job identifies - its output source. - - * How are records read and validated from the input source? This - refers to the input type and its format (e.g. flat file with fixed - position, comma separated or XML, etc.) - - * What is the policy of the job if a input record fails the - validation step? The most important aspect is whether the record - can be skipped so that processing can be continued. - - * How do I process the data and write to the output source? How - and what business logic is being applied to the processing of a - record? -{{adhocLoop}} | | | | |x | | | | | | | | | - - * How do I recover from an exception while operating on the output - source? There are numerous recovery strategies that can be applied - to handling errors on transactional targets. The samples provide a - feeling for some of the choices. - - * Can I restart the job and if so which strategy can I use to - restart the job? The samples show some of the options available to - jobs and what the decision criteria is for the respective choices. - - Here is a list of samples with checks to indicate which features each one demonstrates: - -*----+----+----+----+----+----+----+----+----+----+----+----+ -|<> | <> | <> | <> | <> | <> | <> | <> | <> | <> | <> | <> | <> | <> | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{adhocLoop}} | | | | |x | | | | | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{AmqpJobFunctionalTests}} | | | | | | | | | |x | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{beanWrapperMapperSample}} | | | |x | | | | | | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{compositeItemWriterSample}}| | | | | | |x | | | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{customerFilter}} | | | | | | | | | | |x | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{delegating}} | | | | | | |x | | | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{football}} | | | | | | | | | | | |x |x | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{headerFooter}} | | | | | | | | | | | |x | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{hibernate}} | |x | | | | | |x | | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{ioSample}} | | | | | |x | |x | | | |x | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{infiniteLoop}} | | | | | |x | | | | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{loopflow}} | | | | | | | | |x | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{multiline}} | | | | | | |x | | | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{multilineOrder}} | | | | | | |x | | | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{parallel}} | | | | | | | | | |x | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{partition}} | | | | | | | | | |x | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{quartz}} | | | | |x | | | | | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{restart}} | | |x | | | | | | | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{retry}} | |x | | | | | | | | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{skip}} |x | | | | | | | | | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----+ -{{trade}} | | | | | |x | | | | | |x | | -*----+----+----+----+----+----+----+----+----+----+----+----+ - - The <<>> ({ioSample}) has a number of special instances that show different IO features using the same job configuration but with different readers and writers: - -*----+----+----+----+----+----+----+----+----+----+----+----++ -|<> | <> | <> | <> | <> | <> | <> | <> | <> | <> | <> | <> | <> | -*----+----+----+----+----+----+----+----+----+----+----+----++ -delimited |x | | | | | | |x | | | | | -*----+----+----+----+----+----+----+----+----+----+----+----++ -{{fixedLength}} | |x | | | | | | |x | | | | -*----+----+----+----+----+----+----+----+----+----+----+----++ -{{ibatis}} | | | | |x | | | | | |x | | -*----+----+----+----+----+----+----+----+----+----+----+----++ -hibernate | | | | |x | | | | | |x | | -*----+----+----+----+----+----+----+----+----+----+----+----++ -{{jdbcCursor}} | | | | |x | | | | | |x | | -*----+----+----+----+----+----+----+----+----+----+----+----++ -jpa | | | |x | | | | | | |x | | -*----+----+----+----+----+----+----+----+----+----+----+----++ -multiLine |x | | | | | | |x | | |x | | -*----+----+----+----+----+----+----+----+----+----+----+----++ -multiRecordtype | |x | | | | | | |x | | |x | -*----+----+----+----+----+----+----+----+----+----+----+----++ -multiResource |x | | | | | | |x | | | |x | -*----+----+----+----+----+----+----+----+----+----+----+----++ -{{xml}} | | |x | | | | | | |x | | | -*----+----+----+----+----+----+----+----+----+----+----+----++ - - -* Common Sample Source Structures - - The easiest way to launch a sample job in Spring Batch is to open up - a unit test in your IDE and run it directly. Each sample has a - separate test case in the <<>> - package. The name of the test case is - <<<[JobName]FunctionalTests>>>. - - [Note:] The test cases do not ship in the samples jar file, but - they are in the .zip distribution and in the source code, which - you can download using subversion (or browse in a web browser if - you need to). See {{{source-repository.html}here}} for a link to - the source code repository. - - You can also use the same Spring configuration as the unit test to - launch the job via a main method in <<>>. - The samples source code has an Eclipse launch configuration to do - this, taking the hassle out of setting up a classpath to run the - job. - -* Adhoc Loop and JMX Demo ({adhocLoop}) - - This job is simply an infinite loop. It runs forever so it is - useful for testing features to do with stopping and starting jobs. - It is used, for instance, as one of the jobs that can be run from - JMX using the Eclipse launch configuration "jmxLauncher". - - The JMX launcher uses an additional XML configuration file - (adhoc-job-launcher-context.xml) to set up a <<>> for - running jobs asynchronously (i.e. in a background thread). This - follows the same pattern as the {{{quartzSample}Quartz sample}}, so - see that section for more details of the <<>> - configuration. - - The rest of the configuration for this demo consists of exposing - some components from the application context as JMX managed beans. - The <<>> is exposed so that it can be controlled from a - remote client (such as JConsole from the JDK) which does not have - Spring Batch on the classpath. See the Spring Core Reference Guide - for more details on how to customise the JMX configuration. - -* Jdbc Cursor and Batch Update ({jdbcCursor}) - - The purpose of this sample is to show to usage of the - <<>> and the <<>> to make - efficient updates to a database table. - - The <<>> accepts a special form of - <<>> as a (mandatory) dependency. This is - responsible for copying fields from the item to be written to a - <<>> matching the SQL query that has been - injected. The implementation of the - <<>> shows best - practice of keeping all the information needed for the execution in - one place, since it contains a static constant value (<<>>) - which is used to configure the query for the writer. - -* Amqp Job Sample ({AmqpJobFunctionalTests}) - - This sample shows the use of Spring Batch to write to an <<>>. - The AmqpItemReader and Writer were contributed by Chris Schaefer and - a blog entry can be found at {{http://blog.dtzq.com/2012/08/spring-batch-amqp-itemreader-itemwriter.html}}. - It is modeled after the JmsItemReader / Writer implementations, which - are popular models for remote chunking. It leverages the AmqpTemplate. - - This example requires the env to have a copy of rabbitmq installed - and running. The standard dashboard can be used to see the traffic - from the <<>> to the AmqpItemWriter. Make sure you - launch the <<>> before launching the test. - -* BeanWrapperMapper Sample ({beanWrapperMapperSample}) - - This sample shows the use of automatic mapping from fields in a file - to a domain object. The <<>> and <<>> objects needed - by the job are created from the Spring configuration using prototype - beans, and then their properties are set using the - <<>>, which sets properties of the - prototype according to the field names in the file. - - Nested property paths are resolved in the same way as normal Spring - binding occurs, but with a little extra leeway in terms of spelling - and capitalisation. Thus for instance, the <<>> object has a - property called <<>> (lower case), but the file has been - configured to have a column name <<>> (upper case), and - the mapper will accept the values happily. Underscores instead of - camel-casing (e.g. <<>> instead of <<>>) - also work. - - -* Composite ItemWriter Sample ({compositeItemWriterSample}) - - This shows a common use case using a composite pattern, composing - instances of other framework readers or writers. It is also quite - common for business-specific readers or writers to wrap - off-the-shelf components in a similar way. - - In this job the composite pattern is used just to make duplicate - copies of the output data. The delegates for the - <<>> have to be separately registered as - streams in the <<>> where they are used, in order for the step - to be restartable. This is a common feature of all delegate - patterns. - -* Customer Filter Sample ({customerFilter}) - - This shows the use of the <<>> to filter out items by - returning null. When an item is filtered it leads to an increment - in the <<>> in the step execution. - -* Delegating Sample ({delegating}) - - This sample shows the delegate pattern again, and also the - <<>> which is used to adapt a POJO to the - <<>> interface. - -* Fixed Length Import Job ({fixedLength}) - - The goal is to demonstrate a typical scenario of importing data - from a fixed-length file to database - - This job shows a typical scenario, when reading input data and - processing the data is cleanly separated. The data provider is - responsible for reading input and mapping each record to a domain - object, which is then passed to the module processor. The module - processor handles the processing of the domain objects, in this case - it only writes them to database. - - In this example we are using a simple fixed length record structure - that can be found in the project at - <<>>. A considerable amount of - thought can go into designing the folder structures for batch file - management. The fixed length records look like this: - -+--- - UK21341EAH4597898.34customer1 - UK21341EAH4611218.12customer2 - UK21341EAH4724512.78customer2 - UK21341EAH48108109.25customer3 - UK21341EAH49854123.39customer4 -+--- - - Looking back to the configuration file you will see where this is - documented in the property of the <<>>. You can - infer the following properties: - -*---+---+ -|<>|<>| -*---+---+ -|ISIN|12| -*---+---+ -|Quantity|3| -*---+---+ -|Price|5| -*---+---+ -|Customer|9| -*---+---+ - - database - writes the data to database using a DAO - object - - -* Football Job ({football}) - - This is a (American) Football statistics loading job. We gave it the - id of <<>> in our configuration file. Before diving - into the batch job, we'll examine the two input files that need to - be loaded. First is <<>>, which can be found in the - samples project under - src/main/resources/data/footballjob/input/. Each line within this - file represents a player, with a unique id, the player’s name, - position, etc: - -+--- -AbduKa00,Abdul-Jabbar,Karim,rb,1974,1996 -AbduRa00,Abdullah,Rabih,rb,1975,1999 -AberWa00,Abercrombie,Walter,rb,1959,1982 -AbraDa00,Abramowicz,Danny,wr,1945,1967 -AdamBo00,Adams,Bob,te,1946,1969 -AdamCh00,Adams,Charlie,wr,1979,2003 -... -+--- - - One of the first noticeable characteristics of the file is that each - data element is separated by a comma, a format most are familiar - with known as 'CSV'. Other separators such as pipes or semicolons - could just as easily be used to delineate between unique - elements. In general, it falls into one of two types of flat file - formats: delimited or fixed length. (The fixed length case was - covered in the <<>>. - - The second file, 'games.csv' is formatted the same as the previous - example, and resides in the same directory: - -+--- -AbduKa00,1996,mia,10,nwe,0,0,0,0,0,29,104,,16,2 -AbduKa00,1996,mia,11,clt,0,0,0,0,0,18,70,,11,2 -AbduKa00,1996,mia,12,oti,0,0,0,0,0,18,59,,0,0 -AbduKa00,1996,mia,13,pit,0,0,0,0,0,16,57,,0,0 -AbduKa00,1996,mia,14,rai,0,0,0,0,0,18,39,,7,0 -AbduKa00,1996,mia,15,nyg,0,0,0,0,0,17,96,,14,0 -... -+--- - - Each line in the file represents an individual player's performance - in a particular game, containing such statistics as passing yards, - receptions, rushes, and total touchdowns. - - Our example batch job is going to load both files into a database, - and then combine each to summarise how each player performed for a - particular year. Although this example is fairly trivial, it shows - multiple types of input, and the general style is a common batch - scenario. That is, summarising a very large dataset so that it can - be more easily manipulated or viewed by an online web-based - application. In an enterprise solution the third step, the reporting - step, could be implemented through the use of Eclipse BIRT or one of - the many Java Reporting Engines. Given this description, we can then - easily divide our batch job up into 3 'steps': one to load the - player data, one to load the game data, and one to produce a summary - report: - - [Note:] One of the nice features of Spring is a project called - Spring IDE. When you download the project you can install Spring - IDE and add the Spring configurations to the IDE project. This is - not a tutorial on Spring IDE but the visual view into Spring beans - is helpful in understanding the structure of a Job - Configuration. Spring IDE produces the following diagram: - -[images/spring-batch-football-graph.jpg] - - This corresponds exactly with the <<>> job - configuration file which can be found in the jobs folder under - <<>>. When you drill down into the football job - you will see that the configuration has a list of steps: - -+--- - - - - - - - -+--- - - A step is run until there is no more input to process, which in - this case would mean that each file has been completely - processed. To describe it in a more narrative form: the first step, - playerLoad, begins executing by grabbing one line of input from the - file, and parsing it into a domain object. That domain object is - then passed to a dao, which writes it out to the PLAYERS table. This - action is repeated until there are no more lines in the file, - causing the playerLoad step to finish. Next, the gameLoad step does - the same for the games input file, inserting into the GAMES - table. Once finished, the playerSummarization step can begin. Unlike - the first two steps, playerSummarization input comes from the - database, using a Sql statement to combine the GAMES and PLAYERS - table. Each returned row is packaged into a domain object and - written out to the PLAYER_SUMMARY table. - - Now that we've discussed the entire flow of the batch job, we can - dive deeper into the first step: playerLoad: - -+--- - - - - - - - - - - - - - - -+--- - - The root bean in this case is a <<>>, which - can be considered a 'blueprint' of sorts that tells the execution - environment basic details about how the batch job should be - executed. It contains four properties: (others have been removed for - greater clarity) commitInterval, startLimit, itemReader and - itemWriter . After performing all necessary startup, the framework - will periodically delegate to the reader and writer. In this way, - the developer can remain solely concerned with their business - logic. - - * – the item reader is the source of the information - pipe. At the most basic level input is read in from an input - source, parsed into a domain object and returned. In this way, the - good batch architecture practice of ensuring all data has been - read before beginning processing can be enforced, along with - providing a possible avenue for reuse. - - * – this is the business logic. At a high level, - the item writer takes the item returned from the reader - and 'processes' it. In our case it's a data access object that is - simply responsible for inserting a record into the PLAYERS - table. As you can see the developer does very little. - - The application developer simply provides a job configuration with a - configured number of steps, an ItemReader associated to some type - of input source, and ItemWriter associated to some type of - output source and a little mapping of data from flat records to - objects and the pipe is ready wired for processing. - - Another property in the step configuration, the commitInterval, - gives the framework vital information about how to control - transactions during the batch run. Due to the large amount of data - involved in batch processing, it is often advantageous to 'batch' - together multiple logical units of work into one transaction, since - starting and committing a transaction is extremely expensive. For - example, in the playerLoad step, the framework calls read() on the - item reader. The item reader reads one record from the file, and - returns a domain object representation which is passed to the - processor. The writer then writes the one record to the database. It - can then be said that one iteration = one call to - <<>> = one line of the file. Therefore, setting - your commitInterval to 5 would result in the framework committing a - transaction after 5 lines have been read from the file, with 5 - resultant entries in the PLAYERS table. - - Following the general flow of the batch job, the next step is to - describe how each line of the file will be parsed from its string - representation into a domain object. The first thing the provider - will need is an <<>>, which is provided as part of the Spring - Batch infrastructure. Because the input is flat-file based, a - <<>> is used: - -+--- - - - - - - - - - - - -+--- - - There are three required dependencies of the item reader; the first - is a resource to read in, which is the file to process. The second - dependency is a <<>>. The interface for a - <<>> is very simple, given a string; it will return a - <<
        >> that wraps the results from splitting the provided - string. A <<
        >> is Spring Batch's abstraction for flat file - data. It allows developers to work with file input in much the same - way as they would work with database input. All the developers need - to provide is a <<>> (similar to a Spring - <<>>) that will map the provided <<
        >> into an - <<>>. Simply by providing the names of each token to the - <<>>, the <<>> can pass the - <<
        >> into our <<>>, which implements the - <<>> interface. There is a single method, - <<>>, which maps <<
        >>s the same way that - developers are comfortable mapping <<>>s into Java - <<>>s, either by index or field name. This behaviour is by - intention and design similar to the <<>> passed into a - <<>>. You can see this below: - -+--- -public class - PlayerMapper implements FieldSetMapper { - - public Object mapLine(FieldSet fs) { - - if(fs == null){ - return null; - } - - Player player = new Player(); - player.setID(fs.readString("ID")); - player.setLastName(fs.readString("lastName")); - player.setFirstName(fs.readString("firstName")); - player.setPosition(fs.readString("position")); - player.setDebutYear(fs.readInt("debutYear")); - player.setBirthYear(fs.readInt("birthYear")); - - return player; - } -} -+--- - - The flow of the <<>>, in this case, starts with a call - to read the next line from the file. This is passed into the - provided <<>>. The <<>> splits the - line at every comma, and creates a <<
        >> using the created - <<>> array and the array of names passed in. - - [Note:] it is only necessary to provide the names to create the - <<
        >> if you wish to access the field by name, rather - than by index. - - Once the domain representation of the data has been returned by the - provider, (i.e. a <<>> object in this case) it is passed to - the <<>>, which is essentially a Dao that uses a Spring - <<>> to insert a new row in the PLAYERS table. - - The next step, gameLoad, works almost exactly the same as the - playerLoad step, except the games file is used. - - The final step, playerSummarization, is much like the previous two - steps, in that it reads from a reader and returns a domain object to - a writer. However, in this case, the input source is the database, - not a file: - -+---- - - - - - - - -SELECT games.player_id, games.year_no, SUM(COMPLETES), -SUM(ATTEMPTS), SUM(PASSING_YARDS), SUM(PASSING_TD), -SUM(INTERCEPTIONS), SUM(RUSHES), SUM(RUSH_YARDS), -SUM(RECEPTIONS), SUM(RECEPTIONS_YARDS), SUM(TOTAL_TD) -from games, players where players.player_id = -games.player_id group by games.player_id, games.year_no - - - -+---- - - The <<>> has three dependences: - - * A <<>> - - * The <<>> to use for each row. - - * The Sql statement used to create the cursor. - - When the step is first started, a query will be run against the - database to open a cursor, and each call to <<>> - will move the cursor to the next row, using the provided - <<>> to return the correct object. As with the previous - two steps, each record returned by the provider will be written out - to the database in the PLAYER_SUMMARY table. Finally to run this - sample application you can execute the JUnit test - <<>>, and you'll see an output showing - each of the records as they are processed. Please keep in mind that - AoP is used to wrap the <<>> and output each record as it - is processed to the logger, which may impact performance. - -* Header Footer Sample ({headerFooter}) - - This sample shows the use of callbacks and listeners to deal with - headers and footers in flat files. It uses two custom callbacks: - - * <<>>: copies the header of a file from the - input to the output. - - * <<>>: creates a summary footer at the end - of the output file. - -* Hibernate Sample {hibernate} - - The purpose of this sample is to show a typical usage of Hibernate - as an ORM tool in the input and output of a job. - - The job uses a <<>> for the input, where - a simple HQL query is used to supply items. It also uses a - non-framework <<>> wrapping a DAO, which perhaps was - written as part of an online system. - - - The output reliability and robustness are improved by the use of - <<>> inside <<>>. This - "write-behind" behaviour is provided by Hibernate implicitly, but we - need to take control of it so that the skip and retry features - provided by Spring Batch can work effectively. - -* Ibatis Sample ({ibatis}) - - The goal of this sample is to show the use of Ibatis as a query - mapping tool. Its features are similar to the Hibernate sample, but - it uses Ibatis to drive its input and output. - -* Infinite Loop Sample ({infiniteLoop}) - - This sample has a single step that is an infinite loop, reading and - writing fake data. It is used to demonstrate stop signals and - restart capabilities. - -* Loop Flow Sample ({loopflow}) - - Shows how to implement a job that repeats one of its steps up to a - limit set by a <<>>. - -* Multiline ({multiline}) - - The goal of this sample is to show some common tricks with multiline - records in file input jobs. - - The input file in this case consists of two groups of trades - delimited by special lines in a file (BEGIN and END): - -+--- -BEGIN -UK21341EAH4597898.34customer1 -UK21341EAH4611218.12customer2 -END -BEGIN -UK21341EAH4724512.78customer2 -UK21341EAH4810809.25customer3 -UK21341EAH4985423.39customer4 -END -+--- - - The goal of the job is to operate on the two groups, so the item - type is naturally <<>>>. To get these items delivered - from an item reader we employ two components from Spring Batch: the - <<>> and the - <<>>. The latter is - responsible for recognising the difference between the trade data - and the delimiter records. The former is responsible for - aggregating the trades from each group into a <<>> and handing - out the list from its <<>> method. To help these components - perform their responsibilities we also provide some business - knowledge about the data in the form of a <<>> - (<<>>). The <<>> checks - its input for the delimiter fields (BEGIN, END) and if it detects - them, returns the special tokens that <<>> - needs. Otherwise it maps the input into a <<>> object. - -* Multiline Order Job ({multilineOrder}) - - The goal is to demonstrate how to handle a more complex file input - format, where a record meant for processing includes nested records - and spans multiple lines - - The input source is file with multiline records. - <<>> is an example of a non-default programmatic - item reader. It reads input until it detects that the multiline - record has finished and encapsulates the record in a single domain - object. - - The output target is a file with multiline records. The concrete - <<>> passes the object to a an injected 'delegate - writer' which in this case writes the output to a file. The writer - in this case demonstrates how to write multiline output using a - custom aggregator transformer. - -* Parallel Sample ({parallel}) - - The purpose of this sample is to show multi-threaded step execution - using the Process Indicator pattern. - - The job reads data from the same file as the - {{{fixedLengthImport}Fixed Length Import}} sample, but instead of - writing it out directly it goes through a staging table, and the - staging table is read in a multi-threaded step. Note that for such - a simple example where the item processing was not expensive, there - is unlikely to be much if any benefit in using a multi-threaded - step. - - Multi-threaded step execution is easy to configure using Spring - Batch, but there are some limitations. Most of the out-of-the-box - <<>> and <<>> implementations are not - designed to work in this scenario because they need to be - restartable and they are also stateful. There should be no surprise - about this, and reading a file (for instance) is usually fast enough - that multi-threading that part of the process is not likely to - provide much benefit, compared to the cost of managing the state. - - The best strategy to cope with restart state from multiple - concurrent threads depends on the kind of input source involved: - - * For file-based input (and output) restart sate is practically - impossible to manage. Spring Batch does not provide any features - or samples to help with this use case. - - * With message middleware input it is trivial to manage restarts, - since there is no state to store (if a transaction rolls back the - messages are returned to the destination they came from). - - * With database input state management is still necessary, but it - isn't particularly difficult. The easiest thing to do is rely on - a Process Indicator in the input data, which is a column in the - data indicating for each row if it has been processed or not. The - flag is updated inside the batch transaction, and then in the case - of a failure the updates are lost, and the records will show as - un-processed on a restart. - - This last strategy is implemented in the <<>>. - Its companion, the <<>> is responsible for - setting up the data in a staging table which contains the process - indicator. The reader is then driven by a simple SQL query that - includes a where clause for the processed flag, i.e. - -+--- -SELECT ID FROM BATCH_STAGING WHERE JOB_ID=? AND PROCESSED=? ORDER BY ID -+--- - - It is then responsible for updating the processed flag (which - happens inside the main step transaction). - -* Partitioning Sample ({partition}) - - The purpose of this sample is to show multi-threaded step execution - using the <<>> SPI. The example uses a - <<>> to spread the work of reading - some files acrosss multiple threads, with one <<>> execution - per thread. The key components are the <<>> and the - <<>> which is responsible for dividing up - the work. Notice that the readers and writers in the <<>> - that is being partitioned are step-scoped, so that their state does - not get shared across threads of execution. - -* Quartz Sample ({quartz}) - - The goal is to demonstrate how to schedule job execution using - Quartz scheduler. In this case there is no unit test to launch the - sample because it just re-uses the football job. There is a main - method in <<>> and an Eclipse launch - configuration which runs it with arguments to pick up the football - job. - - The additional XML configuration for this job is in - <<>>, and it also re-uses - <<>> - - The configuration declares a <<>> bean. The launcher - bean is different from the other samples only in that it uses an - asynchronous task executor, so that the jobs are launched in a - separate thread to the main method: - -+--- - - - - - - -+--- - - Also, a Quartz <<>> is defined using a Spring - <<>> as a convenience. - -+-- - - - - - - - - - - - -+-- - - Finally, a trigger with a scheduler is defined that will launch the - job detail every 10 seconds: - -+--- - - - - - - - - -+--- - - The job is thus scheduled to run every 10 seconds. In fact it - should be successful on the first attempt, so the second and - subsequent attempts should through a - <<>>. In a production system, - the job detail would probably be modified to account for this - exception (e.g. catch it and re-submit with a new set of job - parameters). The point here is that Spring Batch guarantees that - the job execution is idempotent - you can never inadvertently - process the same data twice. - -* Restart Sample ({restart}) - - The goal of this sample is to show how a job can be restarted after - a failure and continue processing where it left off. - - To simulate a failure we "fake" a failure on the fourth record - though the use of a sample component - <<>>. This is a stateful reader - that counts how many records it has processed and throws a planned - exception in a specified place. Since we re-use the same instance - when we restart the job it will not fail the second time. - -* Retry Sample ({retry}) - - The purpose of this sample is to show how to use the automatic retry - capabilities of Spring Batch. - - The retry is configured in the step through the - <<>>: - -+--- - - ... - - - -+--- - - Failed items will cause a rollback for all <<>> types, up - to a limit of 3 attempts. On the 4th attempt, the failed item would - be skipped, and there would be a callback to a - <<>> if one was provided (via the "listeners" - property of the step factory bean). - - An <<>> is provided that will generate unique - <<>> data by just incrementing a counter. Note that it uses - the counter in its <<>> and <<>> methods so that - the same content is returned after a rollback. The same content is - returned, but the instance of <<>> is different, which means - that the implementation of <<>> in the <<>> object - is important. This is because to identify a failed item on retry - (so that the number of attempts can be counted) the framework by - default uses <<>> to compare the recently failed - item with a cache of previously failed items. Without implementing - a field-based <<>> method for the domain object, our job - will spin round the retry for potentially quite a long time before - failing because the default implementation of <<>> is - based on object reference, not on field content. - -* Skip Sample ({skip}) - - The purpose of this sample is to show how to use the skip features - of Spring Batch. Since skip is really just a special case of retry - (with limit 0), the details are quite similar to the {{{retrySample}Retry - Sample}}, but the use case is less artificial, since it - is based on the {{{trade}Trade Sample}}. - - The failure condition is still artificial, since it is triggered by - a special <<>> wrapper (<<>>). - The plan is that a certain item (the third) will fail business - validation in the writer, and the system can then respond by - skipping it. We also configure the step so that it will not roll - back on the validation exception, since we know that it didn't - invalidate the transaction, only the item. This is done through the - transaction attribute: - -+--- - - - - - .... - -+--- - - The format for the transaction attribute specification is given in - the Spring Core documentation (e.g. see the Javadocs for - {{{http://static.springframework.org/spring/docs/2.5.x/api/org/springframework/transaction/interceptor/TransactionAttributeEditor.html}TransactionAttributeEditor}}). - -* Tasklet Job ({tasklet}) - - The goal is to show the simplest use of the batch framework with a - single job with a single step, which cleans up a directory and runs - a system command. - - The - <<>> itself is defined by the bean definition with - <<>>. In this example we have two steps. - - * The first step defines a tasklet that is responsible for - clearing out a directory though a custom <<>>. Each - tasklet has an <<>> method which is called by the - step. All processing of business data should be handled by this - method. - - * The second step uses another tasklet to execute a system (OS) - command line. - - You can visualise the Spring configuration of a job through - Spring-IDE. See {{{http://springide.org/blog/}Spring IDE}}. The - source view of the configuration is as follows: - -+--- - - - - - - - - - - - - - - - - - - - - - - - - - - - -+--- - - For simplicity we are only displaying the job configuration itself - and leaving out the details of the supporting batch execution - environment configuration. - -* Trade Job ({trade}) - - The goal is to show a reasonably complex scenario, that would - resemble the real-life usage of the framework. - - This job has 3 steps. First, data about trades are imported from a - file to database. Second, the trades are read from the database and - credit on customer accounts is decreased appropriately. Last, a - report about customers is exported to a file. - -* XML Input Output ({xml}) - - The goal here is to show the use of XML input and output through - streaming and Spring OXM marshallers and unmarshallers. - - The job has a single step that copies <<>> data from one XML - file to another. It uses XStream for the object XML conversion, - because this is simple to configure for basic use cases like this - one. See - {{{http://static.springframework.org/spring-ws/sites/1.5/reference/html/oxm.html}Spring - OXM documentation}} for details of other options. diff --git a/spring-batch-samples/src/site/site.xml b/spring-batch-samples/src/site/site.xml deleted file mode 100644 index 56a12dcc84..0000000000 --- a/spring-batch-samples/src/site/site.xml +++ /dev/null @@ -1,18 +0,0 @@ - - - - - - - - - - - - - - - - - - diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/AMQPJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/AMQPJobFunctionalTests.java index 335871e549..e610d6f2e3 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/AMQPJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/AMQPJobFunctionalTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample; import static org.junit.Assert.assertTrue; diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/BeanWrapperMapperSampleJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/BeanWrapperMapperSampleJobFunctionalTests.java index dcae66d72b..cf9352d652 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/BeanWrapperMapperSampleJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/BeanWrapperMapperSampleJobFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/CompositeItemWriterSampleFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/CompositeItemWriterSampleFunctionalTests.java index 37f42a5c75..e53d22c2a0 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/CompositeItemWriterSampleFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/CompositeItemWriterSampleFunctionalTests.java @@ -1,7 +1,20 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample; -import static org.junit.Assert.assertEquals; - import java.io.FileInputStream; import java.io.IOException; import java.math.BigDecimal; @@ -9,12 +22,12 @@ import java.sql.SQLException; import java.util.ArrayList; import java.util.List; - import javax.sql.DataSource; import org.apache.commons.io.IOUtils; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.sample.domain.trade.Trade; import org.springframework.batch.test.JobLauncherTestUtils; import org.springframework.beans.factory.annotation.Autowired; @@ -24,12 +37,12 @@ import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import static org.junit.Assert.assertEquals; + @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/compositeItemWriterSampleJob.xml", "/job-runner-context.xml" }) public class CompositeItemWriterSampleFunctionalTests { - private static final String GET_TRADES = "SELECT isin, quantity, price, customer FROM TRADE order by isin"; - private static final String EXPECTED_OUTPUT_FILE = "Trade: [isin=UK21341EAH41,quantity=211,price=31.11,customer=customer1]" + "Trade: [isin=UK21341EAH42,quantity=212,price=32.11,customer=customer2]" + "Trade: [isin=UK21341EAH43,quantity=213,price=33.11,customer=customer3]" @@ -48,19 +61,18 @@ public void setDataSource(DataSource dataSource) { @Test public void testJobLaunch() throws Exception { - jdbcTemplate.update("DELETE from TRADE"); - int before = jdbcTemplate.queryForInt("SELECT COUNT(*) from TRADE"); + int before = jdbcTemplate.queryForObject("SELECT COUNT(*) from TRADE", Integer.class); jobLauncherTestUtils.launchJob(); - checkOutputFile("target/test-outputs/CustomerReport1.txt"); - checkOutputFile("target/test-outputs/CustomerReport2.txt"); + checkOutputFile("build/test-outputs/CustomerReport1.txt"); + checkOutputFile("build/test-outputs/CustomerReport2.txt"); checkOutputTable(before); - } private void checkOutputTable(int before) { + @SuppressWarnings("serial") final List trades = new ArrayList() { { add(new Trade("UK21341EAH41", 211, new BigDecimal("31.11"), "customer1")); @@ -71,13 +83,14 @@ private void checkOutputTable(int before) { } }; - int after = jdbcTemplate.queryForInt("SELECT COUNT(*) from TRADE"); + int after = jdbcTemplate.queryForObject("SELECT COUNT(*) from TRADE", Integer.class); assertEquals(before + 5, after); - jdbcTemplate.query(GET_TRADES, new RowCallbackHandler() { private int activeRow = 0; + + @Override public void processRow(ResultSet rs) throws SQLException { Trade trade = trades.get(activeRow++); @@ -91,8 +104,8 @@ public void processRow(ResultSet rs) throws SQLException { } private void checkOutputFile(String fileName) throws IOException { - @SuppressWarnings("unchecked") - List outputLines = IOUtils.readLines(new FileInputStream(fileName)); + @SuppressWarnings("resource") + List outputLines = IOUtils.readLines(new FileInputStream(fileName), "UTF-8"); String output = ""; for (String line : outputLines) { @@ -101,5 +114,4 @@ private void checkOutputFile(String fileName) throws IOException { assertEquals(EXPECTED_OUTPUT_FILE, output); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/CustomerFilterJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/CustomerFilterJobFunctionalTests.java index afea2890f3..fc657be9de 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/CustomerFilterJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/CustomerFilterJobFunctionalTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -43,14 +43,11 @@ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/customerFilterJob.xml", "/job-runner-context.xml" }) public class CustomerFilterJobFunctionalTests { - private static final String GET_CUSTOMERS = "select NAME, CREDIT from CUSTOMER order by NAME"; - private List customers; private int activeRow = 0; - private JdbcOperations jdbcTemplate; - private Map credits = new HashMap(); + private Map credits = new HashMap<>(); @Autowired private JobLauncherTestUtils jobLauncherTestUtils; @@ -65,7 +62,9 @@ public void onSetUp() throws Exception { jdbcTemplate.update("delete from TRADE"); jdbcTemplate.update("delete from CUSTOMER where ID > 4"); jdbcTemplate.update("update CUSTOMER set credit=100000"); + List> list = jdbcTemplate.queryForList("select name, CREDIT from CUSTOMER"); + for (Map map : list) { credits.put((String) map.get("NAME"), ((Number) map.get("CREDIT")).doubleValue()); } @@ -79,17 +78,15 @@ public void tearDown() throws Exception { @Test public void testFilterJob() throws Exception { - JobExecution jobExecution = jobLauncherTestUtils.launchJob(); customers = Arrays.asList(new Customer("customer1", (credits.get("customer1"))), new Customer("customer2", (credits.get("customer2"))), new Customer("customer3", 100500), new Customer("customer4", credits .get("customer4")), new Customer("customer5", 32345), new Customer("customer6", 123456)); - // check content of the customer table activeRow = 0; jdbcTemplate.query(GET_CUSTOMERS, new RowCallbackHandler() { - + @Override public void processRow(ResultSet rs) throws SQLException { Customer customer = customers.get(activeRow++); assertEquals(customer.getName(), rs.getString(1)); @@ -101,7 +98,6 @@ public void processRow(ResultSet rs) throws SQLException { assertEquals("4", step1Execution.get("READ_COUNT").toString()); assertEquals("1", step1Execution.get("FILTER_COUNT").toString()); assertEquals("3", step1Execution.get("WRITE_COUNT").toString()); - } private Map getStepExecution(JobExecution jobExecution, String stepName) { @@ -174,7 +170,5 @@ else if (!name.equals(other.name)) return false; return true; } - } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/DatabaseShutdownFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/DatabaseShutdownFunctionalTests.java index 3ee2b8568f..efba4a834d 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/DatabaseShutdownFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/DatabaseShutdownFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/DelegatingJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/DelegatingJobFunctionalTests.java index d03ca853c6..c4aef3f4bc 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/DelegatingJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/DelegatingJobFunctionalTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2007-2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/FootballJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/FootballJobFunctionalTests.java index 1bdd75935a..fdbbeaef91 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/FootballJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/FootballJobFunctionalTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2007-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample; import static org.junit.Assert.assertTrue; @@ -16,10 +31,8 @@ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/footballJob.xml", "/job-runner-context.xml" }) public class FootballJobFunctionalTests { - @Autowired private JobLauncherTestUtils jobLauncherTestUtils; - private JdbcOperations jdbcTemplate; @Autowired @@ -29,16 +42,13 @@ public void setDataSource(DataSource dataSource) { @Test public void testLaunchJob() throws Exception { - jdbcTemplate.update("DELETE FROM PLAYERS"); jdbcTemplate.update("DELETE FROM GAMES"); jdbcTemplate.update("DELETE FROM PLAYER_SUMMARY"); jobLauncherTestUtils.launchJob(); - int count = jdbcTemplate.queryForInt("SELECT COUNT(*) from PLAYER_SUMMARY"); + int count = jdbcTemplate.queryForObject("SELECT COUNT(*) from PLAYER_SUMMARY", Integer.class); assertTrue(count > 0); - } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/GracefulShutdownFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/GracefulShutdownFunctionalTests.java index 62f5afd54f..d9a25ebc0f 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/GracefulShutdownFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/GracefulShutdownFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/GroovyJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/GroovyJobFunctionalTests.java index 38dbfc8016..4ce778e2ba 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/GroovyJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/GroovyJobFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -41,14 +41,14 @@ public class GroovyJobFunctionalTests { @Before public void removeOldData() throws IOException { - FileUtils.deleteDirectory(new File("target/groovyJob")); + FileUtils.deleteDirectory(new File("build/groovyJob")); } @Test public void testLaunchJob() throws Exception { - assertFalse(new File("target/groovyJob/output/files.zip").exists()); + assertFalse(new File("build/groovyJob/output/files.zip").exists()); jobLauncherTestUtils.launchJob(); - assertTrue(new File("target/groovyJob/output/files.zip").exists()); + assertTrue(new File("build/groovyJob/output/files.zip").exists()); } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/HeaderFooterSampleFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/HeaderFooterSampleFunctionalTests.java index 35c3652c00..511a9086a7 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/HeaderFooterSampleFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/HeaderFooterSampleFunctionalTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample; import static org.junit.Assert.assertTrue; @@ -50,6 +65,9 @@ public void testJob() throws Exception { // footer contains the item count int itemCount = lineCount - 1; // minus 1 due to header line assertTrue(outputReader.readLine().contains(String.valueOf(itemCount))); + + inputReader.close(); + outputReader.close(); } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/HibernateFailureJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/HibernateFailureJobFunctionalTests.java index 2f12229357..2955d8d256 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/HibernateFailureJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/HibernateFailureJobFunctionalTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2007-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample; import static org.junit.Assert.assertEquals; @@ -13,6 +28,7 @@ import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.JobParametersBuilder; import org.springframework.batch.sample.domain.trade.internal.CustomerCreditIncreaseProcessor; @@ -23,8 +39,8 @@ import org.springframework.jdbc.core.JdbcOperations; import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jdbc.core.RowCallbackHandler; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; -import org.springframework.orm.hibernate3.HibernateJdbcException; +import org.springframework.jdbc.core.RowMapper; +import org.springframework.orm.hibernate5.HibernateJdbcException; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.transaction.PlatformTransactionManager; @@ -42,29 +58,21 @@ @ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/hibernate-context.xml", "/jobs/hibernateJob.xml", "/job-runner-context.xml" }) public class HibernateFailureJobFunctionalTests { - - @Autowired - private HibernateCreditDao writer; - - private JdbcOperations jdbcTemplate; - - private PlatformTransactionManager transactionManager; - private static final BigDecimal CREDIT_INCREASE = CustomerCreditIncreaseProcessor.FIXED_AMOUNT; - - private static String[] customers = { "INSERT INTO CUSTOMER (id, version, name, credit) VALUES (1, 0, 'customer1', 100000)", - "INSERT INTO CUSTOMER (id, version, name, credit) VALUES (2, 0, 'customer2', 100000)", - "INSERT INTO CUSTOMER (id, version, name, credit) VALUES (3, 0, 'customer3', 100000)", - "INSERT INTO CUSTOMER (id, version, name, credit) VALUES (4, 0, 'customer4', 100000)"}; - - private static String DELETE_CUSTOMERS = "DELETE FROM CUSTOMER"; - + private static final String DELETE_CUSTOMERS = "DELETE FROM CUSTOMER"; private static final String ALL_CUSTOMERS = "select * from CUSTOMER order by ID"; - private static final String CREDIT_COLUMN = "CREDIT"; + private static String[] customers = { "INSERT INTO CUSTOMER (id, version, name, credit) VALUES (1, 0, 'customer1', 100000)", + "INSERT INTO CUSTOMER (id, version, name, credit) VALUES (2, 0, 'customer2', 100000)", + "INSERT INTO CUSTOMER (id, version, name, credit) VALUES (3, 0, 'customer3', 100000)", + "INSERT INTO CUSTOMER (id, version, name, credit) VALUES (4, 0, 'customer4', 100000)"}; protected static final String ID_COLUMN = "ID"; + @Autowired + private HibernateCreditDao writer; + private JdbcOperations jdbcTemplate; + private PlatformTransactionManager transactionManager; private List creditsBeforeUpdate; @Autowired @@ -82,7 +90,6 @@ public void setTransactionManager(PlatformTransactionManager transactionManager) @Test public void testLaunchJob() throws Exception { - validatePreConditions(); JobParameters params = new JobParametersBuilder().addString("key", "failureJob").toJobParameters(); @@ -100,22 +107,23 @@ public void testLaunchJob() throws Exception { // assertEquals(1, writer.getErrors().size()); throw e; } - int after = jdbcTemplate.queryForInt("SELECT COUNT(*) from CUSTOMER"); + + int after = jdbcTemplate.queryForObject("SELECT COUNT(*) from CUSTOMER", Integer.class); assertEquals(4, after); validatePostConditions(); - } /** * All customers have the same credit */ - @SuppressWarnings("unchecked") protected void validatePreConditions() throws Exception { ensureState(); - creditsBeforeUpdate = (List) new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { - return jdbcTemplate.query(ALL_CUSTOMERS, new ParameterizedRowMapper() { + creditsBeforeUpdate = new TransactionTemplate(transactionManager).execute(new TransactionCallback>() { + @Override + public List doInTransaction(TransactionStatus status) { + return jdbcTemplate.query(ALL_CUSTOMERS, new RowMapper() { + @Override public BigDecimal mapRow(ResultSet rs, int rowNum) throws SQLException { return rs.getBigDecimal(CREDIT_COLUMN); } @@ -129,32 +137,32 @@ public BigDecimal mapRow(ResultSet rs, int rowNum) throws SQLException { * customer table and reading the expected defaults. */ private void ensureState(){ - new TransactionTemplate(transactionManager).execute(new TransactionCallback(){ + new TransactionTemplate(transactionManager).execute(new TransactionCallback(){ - public Object doInTransaction(TransactionStatus status) { + @Override + public Void doInTransaction(TransactionStatus status) { jdbcTemplate.update(DELETE_CUSTOMERS); for (String customer : customers) { - jdbcTemplate.update(customer); + jdbcTemplate.update(customer); } return null; } - }); - + }); } /** * Credit was increased by CREDIT_INCREASE */ protected void validatePostConditions() throws Exception { + final List matches = new ArrayList<>(); - final List matches = new ArrayList(); - - new TransactionTemplate(transactionManager).execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus status) { + new TransactionTemplate(transactionManager).execute(new TransactionCallback() { + @Override + public Void doInTransaction(TransactionStatus status) { jdbcTemplate.query(ALL_CUSTOMERS, new RowCallbackHandler() { - private int i = 0; + @Override public void processRow(ResultSet rs) throws SQLException { final BigDecimal creditBeforeUpdate = creditsBeforeUpdate.get(i++); final BigDecimal expectedCredit = creditBeforeUpdate.add(CREDIT_INCREASE); @@ -164,6 +172,7 @@ public void processRow(ResultSet rs) throws SQLException { } }); + return null; } }); diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/JobOperatorFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/JobOperatorFunctionalTests.java index 9ab073b2f4..c6fb2c3d73 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/JobOperatorFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/JobOperatorFunctionalTests.java @@ -1,140 +1,155 @@ -package org.springframework.batch.sample; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - -import java.util.List; -import java.util.Map; -import java.util.Set; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.springframework.batch.core.BatchStatus; -import org.springframework.batch.core.Job; -import org.springframework.batch.core.JobParametersBuilder; -import org.springframework.batch.core.configuration.JobRegistry; -import org.springframework.batch.core.configuration.support.ReferenceJobFactory; -import org.springframework.batch.core.launch.JobOperator; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.test.context.ContextConfiguration; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; - -@RunWith(SpringJUnit4ClassRunner.class) -@ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/infiniteLoopJob.xml" }) -public class JobOperatorFunctionalTests { - - private static final Log logger = LogFactory.getLog(JobOperatorFunctionalTests.class); - - @Autowired - private JobOperator operator; - - @Autowired - private Job job; - - @Autowired - private JobRegistry jobRegistry; - - @Before - public void setUp() throws Exception { - if (!jobRegistry.getJobNames().contains(job.getName())) { - jobRegistry.register(new ReferenceJobFactory(job)); - } - } - - @Test - public void testStartStopResumeJob() throws Exception { - - String params = new JobParametersBuilder().addLong("jobOperatorTestParam", 7L).toJobParameters().toString(); - - long executionId = operator.start(job.getName(), params); - assertEquals(params, operator.getParameters(executionId)); - stopAndCheckStatus(executionId); - - long resumedExecutionId = operator.restart(executionId); - assertEquals(params, operator.getParameters(resumedExecutionId)); - stopAndCheckStatus(resumedExecutionId); - - List instances = operator.getJobInstances(job.getName(), 0, 1); - assertEquals(1, instances.size()); - long instanceId = instances.get(0); - - List executions = operator.getExecutions(instanceId); - assertEquals(2, executions.size()); - // latest execution is the first in the returned list - assertEquals(resumedExecutionId, executions.get(0).longValue()); - assertEquals(executionId, executions.get(1).longValue()); - - } - - /** - * @param executionId id of running job execution - */ - private void stopAndCheckStatus(long executionId) throws Exception { - - // wait to the job to get up and running - Thread.sleep(1000); - - Set runningExecutions = operator.getRunningExecutions(job.getName()); - assertTrue("Wrong executions: " + runningExecutions + " expected: " + executionId, runningExecutions - .contains(executionId)); - assertTrue("Wrong summary: " + operator.getSummary(executionId), operator.getSummary(executionId).contains( - BatchStatus.STARTED.toString())); - - operator.stop(executionId); - - int count = 0; - while (operator.getRunningExecutions(job.getName()).contains(executionId) && count <= 10) { - logger.info("Checking for running JobExecution: count=" + count); - Thread.sleep(100); - count++; - } - - runningExecutions = operator.getRunningExecutions(job.getName()); - assertFalse("Wrong executions: " + runningExecutions + " expected: " + executionId, runningExecutions - .contains(executionId)); - assertTrue("Wrong summary: " + operator.getSummary(executionId), operator.getSummary(executionId).contains( - BatchStatus.STOPPED.toString())); - - // there is just a single step in the test job - Map summaries = operator.getStepExecutionSummaries(executionId); - System.err.println(summaries); - assertTrue(summaries.values().toString().contains(BatchStatus.STOPPED.toString())); - } - - @Test - public void testMultipleSimultaneousInstances() throws Exception { - String jobName = job.getName(); - - Set names = operator.getJobNames(); - assertEquals(1, names.size()); - assertTrue(names.contains(jobName)); - - long exec1 = operator.startNextInstance(jobName); - long exec2 = operator.startNextInstance(jobName); - - assertTrue(exec1 != exec2); - assertTrue(operator.getParameters(exec1) != operator.getParameters(exec2)); - - Set executions = operator.getRunningExecutions(jobName); - assertTrue(executions.contains(exec1)); - assertTrue(executions.contains(exec2)); - int count = 0; - boolean running = operator.getSummary(exec1).contains("STARTED") - && operator.getSummary(exec2).contains("STARTED"); - while (count++ < 10 && !running) { - Thread.sleep(100L); - running = operator.getSummary(exec1).contains("STARTED") && operator.getSummary(exec2).contains("STARTED"); - } - assertTrue(String.format("Jobs not started: [%s] and [%s]", operator.getSummary(exec1), operator - .getSummary(exec1)), running); - - operator.stop(exec1); - operator.stop(exec2); - - } - -} +/* + * Copyright 2008-2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +import java.util.List; +import java.util.Map; +import java.util.Set; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.BatchStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.configuration.JobRegistry; +import org.springframework.batch.core.configuration.support.ReferenceJobFactory; +import org.springframework.batch.core.launch.JobOperator; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/infiniteLoopJob.xml" }) +public class JobOperatorFunctionalTests { + private static final Log LOG = LogFactory.getLog(JobOperatorFunctionalTests.class); + + @Autowired + private JobOperator operator; + + @Autowired + private Job job; + + @Autowired + private JobRegistry jobRegistry; + + @Before + public void setUp() throws Exception { + if (!jobRegistry.getJobNames().contains(job.getName())) { + jobRegistry.register(new ReferenceJobFactory(job)); + } + } + + @Test + public void testStartStopResumeJob() throws Exception { + String params = new JobParametersBuilder().addLong("jobOperatorTestParam", 7L).toJobParameters().toString(); + + long executionId = operator.start(job.getName(), params); + assertEquals(params, operator.getParameters(executionId)); + stopAndCheckStatus(executionId); + + long resumedExecutionId = operator.restart(executionId); + assertEquals(params, operator.getParameters(resumedExecutionId)); + stopAndCheckStatus(resumedExecutionId); + + List instances = operator.getJobInstances(job.getName(), 0, 1); + assertEquals(1, instances.size()); + long instanceId = instances.get(0); + + List executions = operator.getExecutions(instanceId); + assertEquals(2, executions.size()); + // latest execution is the first in the returned list + assertEquals(resumedExecutionId, executions.get(0).longValue()); + assertEquals(executionId, executions.get(1).longValue()); + } + + /** + * @param executionId id of running job execution + */ + private void stopAndCheckStatus(long executionId) throws Exception { + // wait to the job to get up and running + Thread.sleep(1000); + + Set runningExecutions = operator.getRunningExecutions(job.getName()); + assertTrue("Wrong executions: " + runningExecutions + " expected: " + executionId, runningExecutions + .contains(executionId)); + assertTrue("Wrong summary: " + operator.getSummary(executionId), operator.getSummary(executionId).contains( + BatchStatus.STARTED.toString())); + + operator.stop(executionId); + + int count = 0; + while (operator.getRunningExecutions(job.getName()).contains(executionId) && count <= 10) { + LOG.info("Checking for running JobExecution: count=" + count); + Thread.sleep(100); + count++; + } + + runningExecutions = operator.getRunningExecutions(job.getName()); + assertFalse("Wrong executions: " + runningExecutions + " expected: " + executionId, runningExecutions + .contains(executionId)); + assertTrue("Wrong summary: " + operator.getSummary(executionId), operator.getSummary(executionId).contains( + BatchStatus.STOPPED.toString())); + + // there is just a single step in the test job + Map summaries = operator.getStepExecutionSummaries(executionId); + LOG.info(summaries); + assertTrue(summaries.values().toString().contains(BatchStatus.STOPPED.toString())); + } + + @Test + public void testMultipleSimultaneousInstances() throws Exception { + String jobName = job.getName(); + + Set names = operator.getJobNames(); + assertEquals(1, names.size()); + assertTrue(names.contains(jobName)); + + long exec1 = operator.startNextInstance(jobName); + long exec2 = operator.startNextInstance(jobName); + + assertTrue(exec1 != exec2); + assertTrue(!operator.getParameters(exec1).equals(operator.getParameters(exec2))); + + // Give the asynchronous task executor a chance to start executions + Thread.sleep(1000); + + Set executions = operator.getRunningExecutions(jobName); + assertTrue(executions.contains(exec1)); + assertTrue(executions.contains(exec2)); + + int count = 0; + boolean running = operator.getSummary(exec1).contains("STARTED") + && operator.getSummary(exec2).contains("STARTED"); + + while (count++ < 10 && !running) { + Thread.sleep(100L); + running = operator.getSummary(exec1).contains("STARTED") && operator.getSummary(exec2).contains("STARTED"); + } + + assertTrue(String.format("Jobs not started: [%s] and [%s]", operator.getSummary(exec1), operator + .getSummary(exec1)), running); + + operator.stop(exec1); + operator.stop(exec2); + } +} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/JobStepFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/JobStepFunctionalTests.java index d787b94913..8a279619e2 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/JobStepFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/JobStepFunctionalTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -39,11 +39,8 @@ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration public class JobStepFunctionalTests { - @Autowired private JobLauncherTestUtils jobLauncherTestUtils; - - // auto-injected attributes private JdbcOperations jdbcTemplate; @Autowired @@ -53,16 +50,13 @@ public void setDataSource(DataSource dataSource) { @Test public void testJobLaunch() throws Exception { - jdbcTemplate.update("DELETE FROM TRADE"); jobLauncherTestUtils.launchJob(new DefaultJobParametersConverter() .getJobParameters(PropertiesConverter .stringToProperties("run.id(long)=1,parameter=true,run.date=20070122,input.file=classpath:data/fixedLengthImportJob/input/20070122.teststream.ImportTradeDataStep.txt"))); - int after = jdbcTemplate.queryForInt("SELECT COUNT(*) FROM TRADE"); + int after = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM TRADE", Integer.class); assertEquals(5, after); - } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/JsonSupportIntegrationTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/JsonSupportIntegrationTests.java new file mode 100644 index 0000000000..71c096f260 --- /dev/null +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/JsonSupportIntegrationTests.java @@ -0,0 +1,129 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.sample; + +import java.io.File; +import java.io.FileInputStream; +import java.nio.file.Files; +import java.nio.file.Paths; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.item.json.GsonJsonObjectReader; +import org.springframework.batch.item.json.JacksonJsonObjectMarshaller; +import org.springframework.batch.item.json.JsonItemReader; +import org.springframework.batch.item.json.JsonFileItemWriter; +import org.springframework.batch.item.json.builder.JsonItemReaderBuilder; +import org.springframework.batch.item.json.builder.JsonFileItemWriterBuilder; +import org.springframework.batch.sample.domain.trade.Trade; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.ApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.core.io.FileSystemResource; +import org.springframework.util.DigestUtils; + +/** + * @author Mahmoud Ben Hassine + */ +public class JsonSupportIntegrationTests { + + private static final String INPUT_FILE_DIRECTORY = "src/test/resources/org/springframework/batch/item/json/"; + private static final String OUTPUT_FILE_DIRECTORY = "build/"; + + @Before + public void setUp() throws Exception { + Files.deleteIfExists(Paths.get("build", "trades.json")); + } + + @Configuration + @EnableBatchProcessing + public static class JobConfiguration { + + @Autowired + private JobBuilderFactory jobs; + + @Autowired + private StepBuilderFactory steps; + + @Bean + public JsonItemReader itemReader() { + return new JsonItemReaderBuilder() + .name("tradesJsonItemReader") + .resource(new FileSystemResource(INPUT_FILE_DIRECTORY + "trades.json")) + .jsonObjectReader(new GsonJsonObjectReader<>(Trade.class)) + .build(); + } + + @Bean + public JsonFileItemWriter itemWriter() { + return new JsonFileItemWriterBuilder() + .resource(new FileSystemResource(OUTPUT_FILE_DIRECTORY + "trades.json")) + .jsonObjectMarshaller(new JacksonJsonObjectMarshaller<>()) + .name("tradesJsonFileItemWriter") + .build(); + } + + @Bean + public Step step() { + return steps.get("step") + .chunk(2) + .reader(itemReader()) + .writer(itemWriter()) + .build(); + } + + @Bean + public Job job() { + return jobs.get("job") + .start(step()) + .build(); + } + } + + @Test + public void testJsonReadingAndWriting() throws Exception { + ApplicationContext context = new AnnotationConfigApplicationContext(JobConfiguration.class); + JobLauncher jobLauncher = context.getBean(JobLauncher.class); + Job job = context.getBean(Job.class); + JobExecution jobExecution = jobLauncher.run(job, new JobParameters()); + + Assert.assertEquals(ExitStatus.COMPLETED.getExitCode(), jobExecution.getExitStatus().getExitCode()); + assertFileEquals( + new File(INPUT_FILE_DIRECTORY + "trades.json"), + new File(OUTPUT_FILE_DIRECTORY + "trades.json")); + } + + private void assertFileEquals(File expected, File actual) throws Exception { + String expectedHash = DigestUtils.md5DigestAsHex(new FileInputStream(expected)); + String actualHash = DigestUtils.md5DigestAsHex(new FileInputStream(actual)); + Assert.assertEquals(expectedHash, actualHash); + } + +} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/LoopFlowSampleFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/LoopFlowSampleFunctionalTests.java index 5c34f61d38..b68640fc5f 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/LoopFlowSampleFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/LoopFlowSampleFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/MailJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/MailJobFunctionalTests.java index 1d0cdfdfd1..8dfc4ff392 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/MailJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/MailJobFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/MultilineJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/MultilineJobFunctionalTests.java index 4bf8c4a55c..2d1930cb84 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/MultilineJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/MultilineJobFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,11 +16,10 @@ package org.springframework.batch.sample; -import static org.junit.Assert.assertEquals; - import org.apache.commons.io.IOUtils; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.test.JobLauncherTestUtils; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.core.io.FileSystemResource; @@ -29,6 +28,8 @@ import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.util.StringUtils; +import static org.junit.Assert.assertEquals; + @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/multilineJob.xml", "/job-runner-context.xml" }) @@ -42,12 +43,12 @@ public class MultilineJobFunctionalTests { private static final String EXPECTED_RESULT = "[Trade: [isin=UK21341EAH45,quantity=978,price=98.34,customer=customer1], Trade: [isin=UK21341EAH46,quantity=112,price=18.12,customer=customer2]]" + "[Trade: [isin=UK21341EAH47,quantity=245,price=12.78,customer=customer2], Trade: [isin=UK21341EAH48,quantity=108,price=9.25,customer=customer3], Trade: [isin=UK21341EAH49,quantity=854,price=23.39,customer=customer4]]"; - private Resource output = new FileSystemResource("target/test-outputs/20070122.testStream.multilineStep.txt"); + private Resource output = new FileSystemResource("build/test-outputs/20070122.testStream.multilineStep.txt"); @Test public void testJobLaunch() throws Exception { jobLauncherTestUtils.launchJob(); - assertEquals(EXPECTED_RESULT, StringUtils.replace(IOUtils.toString(output.getInputStream()), System + assertEquals(EXPECTED_RESULT, StringUtils.replace(IOUtils.toString(output.getInputStream(), "UTF-8"), System .getProperty("line.separator"), "")); } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/MultilineOrderJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/MultilineOrderJobFunctionalTests.java index 8ec71f3907..db705182d6 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/MultilineOrderJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/MultilineOrderJobFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,7 +32,7 @@ "/job-runner-context.xml" }) public class MultilineOrderJobFunctionalTests { - private static final String ACTUAL = "target/test-outputs/multilineOrderOutput.txt"; + private static final String ACTUAL = "build/test-outputs/multilineOrderOutput.txt"; private static final String EXPECTED = "data/multilineOrderJob/result/multilineOrderOutput.txt"; @Autowired diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/ParallelJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/ParallelJobFunctionalTests.java index 0078800b08..c2b485a0d3 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/ParallelJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/ParallelJobFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/PartitionFileJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/PartitionFileJobFunctionalTests.java index de7fe3241d..4f4de98352 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/PartitionFileJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/PartitionFileJobFunctionalTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -46,16 +46,15 @@ @ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/partitionFileJob.xml", "/job-runner-context.xml" }) public class PartitionFileJobFunctionalTests implements ApplicationContextAware { - @Autowired @Qualifier("inputTestReader") private ItemReader inputReader; @Autowired private JobLauncherTestUtils jobLauncherTestUtils; - private ApplicationContext applicationContext; + @Override public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { this.applicationContext = applicationContext; } @@ -66,12 +65,11 @@ public void setApplicationContext(ApplicationContext applicationContext) throws */ @Test public void testUpdateCredit() throws Exception { - assertTrue("Define a prototype bean called 'outputTestReader' to check the output", applicationContext .containsBeanDefinition("outputTestReader")); open(inputReader); - List inputs = new ArrayList(getCredits(inputReader)); + List inputs = new ArrayList<>(getCredits(inputReader)); close(inputReader); JobExecution jobExecution = jobLauncherTestUtils.launchJob(); @@ -81,7 +79,7 @@ public void testUpdateCredit() throws Exception { ItemReader outputReader = (ItemReader) applicationContext .getBean("outputTestReader"); open(outputReader); - List outputs = new ArrayList(getCredits(outputReader)); + List outputs = new ArrayList<>(getCredits(outputReader)); close(outputReader); assertEquals(inputs.size(), outputs.size()); @@ -93,7 +91,6 @@ public void testUpdateCredit() throws Exception { assertEquals(inputs.get(i).getCredit().add(CustomerCreditIncreaseProcessor.FIXED_AMOUNT).intValue(), outputs.get(i).getCredit().intValue()); } - } /** @@ -101,12 +98,13 @@ public void testUpdateCredit() throws Exception { */ private Set getCredits(ItemReader reader) throws Exception { CustomerCredit credit; - Set result = new LinkedHashSet(); + Set result = new LinkedHashSet<>(); + while ((credit = reader.read()) != null) { result.add(credit); } - return result; + return result; } /** @@ -126,5 +124,4 @@ private void close(ItemReader reader) { ((ItemStream) reader).close(); } } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/PartitionJdbcJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/PartitionJdbcJobFunctionalTests.java index 0bfb8a7e85..9574a10e92 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/PartitionJdbcJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/PartitionJdbcJobFunctionalTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -46,16 +46,15 @@ @ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/partitionJdbcJob.xml", "/job-runner-context.xml" }) public class PartitionJdbcJobFunctionalTests implements ApplicationContextAware { - @Autowired @Qualifier("inputTestReader") private ItemReader inputReader; @Autowired private JobLauncherTestUtils jobLauncherTestUtils; - private ApplicationContext applicationContext; + @Override public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { this.applicationContext = applicationContext; } @@ -66,12 +65,11 @@ public void setApplicationContext(ApplicationContext applicationContext) throws */ @Test public void testUpdateCredit() throws Exception { - assertTrue("Define a prototype bean called 'outputTestReader' to check the output", applicationContext .containsBeanDefinition("outputTestReader")); open(inputReader); - List inputs = new ArrayList(getCredits(inputReader)); + List inputs = new ArrayList<>(getCredits(inputReader)); close(inputReader); JobExecution jobExecution = jobLauncherTestUtils.launchJob(); @@ -81,7 +79,7 @@ public void testUpdateCredit() throws Exception { ItemReader outputReader = (ItemReader) applicationContext .getBean("outputTestReader"); open(outputReader); - List outputs = new ArrayList(getCredits(outputReader)); + List outputs = new ArrayList<>(getCredits(outputReader)); close(outputReader); assertEquals(inputs.size(), outputs.size()); @@ -93,7 +91,6 @@ public void testUpdateCredit() throws Exception { assertEquals(inputs.get(i).getCredit().add(CustomerCreditIncreaseProcessor.FIXED_AMOUNT).intValue(), outputs.get(i).getCredit().intValue()); } - } /** @@ -101,7 +98,7 @@ public void testUpdateCredit() throws Exception { */ private Set getCredits(ItemReader reader) throws Exception { CustomerCredit credit; - Set result = new LinkedHashSet(); + Set result = new LinkedHashSet<>(); while ((credit = reader.read()) != null) { result.add(credit); } @@ -126,5 +123,4 @@ private void close(ItemReader reader) { ((ItemStream) reader).close(); } } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/RemoteChunkingJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RemoteChunkingJobFunctionalTests.java new file mode 100644 index 0000000000..5a9256e2f7 --- /dev/null +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RemoteChunkingJobFunctionalTests.java @@ -0,0 +1,88 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample; + +import org.apache.activemq.broker.BrokerService; +import org.junit.After; +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.sample.config.JobRunnerConfiguration; +import org.springframework.batch.sample.remotechunking.ManagerConfiguration; +import org.springframework.batch.sample.remotechunking.WorkerConfiguration; +import org.springframework.batch.test.JobLauncherTestUtils; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.PropertySource; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringRunner; + +/** + * The manager step of the job under test will read data and send chunks to the worker + * (started in {@link RemoteChunkingJobFunctionalTests#setUp()}) for processing and writing. + * + * @author Mahmoud Ben Hassine + */ +@RunWith(SpringRunner.class) +@ContextConfiguration(classes = {JobRunnerConfiguration.class, ManagerConfiguration.class}) +@PropertySource("classpath:remote-chunking.properties") +public class RemoteChunkingJobFunctionalTests { + + private static final String BROKER_DATA_DIRECTORY = "build/activemq-data"; + + @Value("${broker.url}") + private String brokerUrl; + + @Autowired + private JobLauncherTestUtils jobLauncherTestUtils; + + private BrokerService brokerService; + + private AnnotationConfigApplicationContext workerApplicationContext; + + @Before + public void setUp() throws Exception { + this.brokerService = new BrokerService(); + this.brokerService.addConnector(this.brokerUrl); + this.brokerService.setDataDirectory(BROKER_DATA_DIRECTORY); + this.brokerService.start(); + this.workerApplicationContext = new AnnotationConfigApplicationContext(WorkerConfiguration.class); + } + + @After + public void tearDown() throws Exception { + this.workerApplicationContext.close(); + this.brokerService.stop(); + } + + @Test + public void testRemoteChunkingJob() throws Exception { + // when + JobExecution jobExecution = this.jobLauncherTestUtils.launchJob(); + + // then + Assert.assertEquals(ExitStatus.COMPLETED.getExitCode(), jobExecution.getExitStatus().getExitCode()); + Assert.assertEquals( + "Waited for 2 results.", // the manager sent 2 chunks ({1, 2, 3} and {4, 5, 6}) to workers + jobExecution.getExitStatus().getExitDescription()); + } + +} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/RemotePartitioningJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RemotePartitioningJobFunctionalTests.java new file mode 100644 index 0000000000..059979598d --- /dev/null +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RemotePartitioningJobFunctionalTests.java @@ -0,0 +1,91 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample; + +import org.apache.activemq.broker.BrokerService; +import org.junit.After; +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.test.JobLauncherTestUtils; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.PropertySource; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabase; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder; +import org.springframework.test.context.junit4.SpringRunner; + +/** + * Base class for remote partitioning tests. + * + * @author Mahmoud Ben Hassine + */ +@RunWith(SpringRunner.class) +@PropertySource("classpath:remote-partitioning.properties") +public abstract class RemotePartitioningJobFunctionalTests { + + private static final String BROKER_DATA_DIRECTORY = "build/activemq-data"; + + @Value("${broker.url}") + private String brokerUrl; + + @Autowired + protected JobLauncherTestUtils jobLauncherTestUtils; + + private BrokerService brokerService; + + private EmbeddedDatabase embeddedDatabase; + + private AnnotationConfigApplicationContext workerApplicationContext; + + protected abstract Class getWorkerConfigurationClass(); + + @Before + public void setUp() throws Exception { + this.brokerService = new BrokerService(); + this.brokerService.addConnector(this.brokerUrl); + this.brokerService.setDataDirectory(BROKER_DATA_DIRECTORY); + this.brokerService.start(); + this.embeddedDatabase = new EmbeddedDatabaseBuilder() + .addScript("/org/springframework/batch/core/schema-drop-hsqldb.sql") + .addScript("/org/springframework/batch/core/schema-hsqldb.sql") + .build(); + this.workerApplicationContext = new AnnotationConfigApplicationContext(getWorkerConfigurationClass()); + } + + @Test + public void testRemotePartitioningJob() throws Exception { + // when + JobExecution jobExecution = this.jobLauncherTestUtils.launchJob(); + + // then + Assert.assertEquals(ExitStatus.COMPLETED.getExitCode(), jobExecution.getExitStatus().getExitCode()); + Assert.assertEquals(4, jobExecution.getStepExecutions().size()); // master + 3 workers + } + + @After + public void tearDown() throws Exception { + this.workerApplicationContext.close(); + this.brokerService.stop(); + this.embeddedDatabase.shutdown(); + } + +} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/RemotePartitioningJobWithMessageAggregationFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RemotePartitioningJobWithMessageAggregationFunctionalTests.java new file mode 100644 index 0000000000..c15507c79d --- /dev/null +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RemotePartitioningJobWithMessageAggregationFunctionalTests.java @@ -0,0 +1,37 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample; + +import org.springframework.batch.sample.config.JobRunnerConfiguration; +import org.springframework.batch.sample.remotepartitioning.aggregating.ManagerConfiguration; +import org.springframework.batch.sample.remotepartitioning.aggregating.WorkerConfiguration; +import org.springframework.test.context.ContextConfiguration; + +/** + * The manager step of the job under test will create 3 partitions for workers + * to process. + * + * @author Mahmoud Ben Hassine + */ +@ContextConfiguration(classes = {JobRunnerConfiguration.class, ManagerConfiguration.class}) +public class RemotePartitioningJobWithMessageAggregationFunctionalTests extends RemotePartitioningJobFunctionalTests { + + @Override + protected Class getWorkerConfigurationClass() { + return WorkerConfiguration.class; + } + +} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/RemotePartitioningJobWithRepositoryPollingFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RemotePartitioningJobWithRepositoryPollingFunctionalTests.java new file mode 100644 index 0000000000..cca8cfb4b8 --- /dev/null +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RemotePartitioningJobWithRepositoryPollingFunctionalTests.java @@ -0,0 +1,37 @@ +/* + * Copyright 2018-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample; + +import org.springframework.batch.sample.config.JobRunnerConfiguration; +import org.springframework.batch.sample.remotepartitioning.polling.ManagerConfiguration; +import org.springframework.batch.sample.remotepartitioning.polling.WorkerConfiguration; +import org.springframework.test.context.ContextConfiguration; + +/** + * The manager step of the job under test will create 3 partitions for workers + * to process. + * + * @author Mahmoud Ben Hassine + */ +@ContextConfiguration(classes = {JobRunnerConfiguration.class, ManagerConfiguration.class}) +public class RemotePartitioningJobWithRepositoryPollingFunctionalTests extends RemotePartitioningJobFunctionalTests { + + @Override + protected Class getWorkerConfigurationClass() { + return WorkerConfiguration.class; + } + +} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/RestartFileSampleFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RestartFileSampleFunctionalTests.java index 78ecaca314..b42ec64296 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/RestartFileSampleFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RestartFileSampleFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/RestartFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RestartFunctionalTests.java index 47b4557b23..63f384ad7b 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/RestartFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RestartFunctionalTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -44,13 +44,11 @@ @ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/restartSample.xml", "/job-runner-context.xml" }) public class RestartFunctionalTests { + private JdbcOperations jdbcTemplate; @Autowired private JobLauncherTestUtils jobLauncherTestUtils; - // auto-injected attributes - private JdbcOperations jdbcTemplate; - @Autowired public void setDataSource(DataSource dataSource) { this.jdbcTemplate = new JdbcTemplate(dataSource); @@ -72,8 +70,7 @@ public void onTearDown() throws Exception { */ @Test public void testLaunchJob() throws Exception { - - int before = jdbcTemplate.queryForInt("SELECT COUNT(*) FROM TRADE"); + int before = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM TRADE", Integer.class); JobExecution jobExecution = runJobForRestartTest(); assertEquals(BatchStatus.FAILED, jobExecution.getStatus()); @@ -86,14 +83,14 @@ public void testLaunchJob() throws Exception { throw new RuntimeException(ex); } - int medium = jdbcTemplate.queryForInt("SELECT COUNT(*) FROM TRADE"); + int medium = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM TRADE", Integer.class); // assert based on commit interval = 2 assertEquals(before + 2, medium); jobExecution = runJobForRestartTest(); assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); - int after = jdbcTemplate.queryForInt("SELECT COUNT(*) FROM TRADE"); + int after = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM TRADE", Integer.class); assertEquals(before + 5, after); } @@ -105,5 +102,4 @@ private JobExecution runJobForRestartTest() throws Exception { .getJobParameters(PropertiesConverter .stringToProperties("run.id(long)=1,parameter=true,run.date=20070122,input.file=classpath:data/fixedLengthImportJob/input/20070122.teststream.ImportTradeDataStep.txt"))); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/RetrySampleConfigurationTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RetrySampleConfigurationTests.java index 58a26ff375..3284e50e5f 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/RetrySampleConfigurationTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RetrySampleConfigurationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/RetrySampleFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RetrySampleFunctionalTests.java index bc0da463e1..8cb53b02dc 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/RetrySampleFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/RetrySampleFunctionalTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/SkipSampleFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/SkipSampleFunctionalTests.java index 5ba3c2f6ae..fd6d006087 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/SkipSampleFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/SkipSampleFunctionalTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample; import static org.junit.Assert.assertEquals; @@ -172,7 +187,7 @@ private void validateLaunchWithSkips(JobExecution jobExecution) { // Step2: 7 input records, 1 skipped on process, 1 on write => 5 written // to output // System.err.println(jdbcTemplate.queryForList("SELECT * FROM TRADE")); - assertEquals(5, jdbcTemplate.queryForInt("SELECT COUNT(*) from TRADE where VERSION=?", 1)); + assertEquals(5, jdbcTemplate.queryForObject("SELECT COUNT(*) from TRADE where VERSION=?", Integer.class, 1).intValue()); // 1 record skipped in processing second step assertEquals(1, SkipCheckingListener.getProcessSkips()); @@ -200,7 +215,7 @@ private void validateLaunchWithoutSkips(JobExecution jobExecution) { assertEquals(5, JdbcTestUtils.countRowsInTable((JdbcTemplate) jdbcTemplate, "TRADE")); // Step2: 5 input records => 5 written to output - assertEquals(5, jdbcTemplate.queryForInt("SELECT COUNT(*) from TRADE where VERSION=?", 1)); + assertEquals(5, jdbcTemplate.queryForObject("SELECT COUNT(*) from TRADE where VERSION=?", Integer.class, 1).intValue()); // Neither step contained skips assertEquals(0, JdbcTestUtils.countRowsInTable((JdbcTemplate) jdbcTemplate, "ERROR_LOG")); diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/TaskletJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/TaskletJobFunctionalTests.java index 28b236a384..be512f9a3e 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/TaskletJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/TaskletJobFunctionalTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,6 +23,7 @@ import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.test.JobLauncherTestUtils; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.test.context.ContextConfiguration; @@ -41,6 +42,7 @@ public void testLaunchJob() throws Exception { JobExecution jobExecution = jobLauncherTestUtils.launchJob(new JobParametersBuilder().addString("value", "foo") .toJobParameters()); assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); + assertEquals("yes", jobExecution.getExecutionContext().getString("done")); } public static class TestBean { @@ -50,9 +52,25 @@ public void setValue(String value) { this.value = value; } - public void execute() { + public void execute(String strValue, Integer integerValue, double doubleValue) { assertEquals("foo", value); + assertEquals("foo2", strValue); + assertEquals(3, integerValue.intValue()); + assertEquals(3.14, doubleValue, 0.01); } } + + public static class Task { + + public boolean doWork(ChunkContext chunkContext) { + chunkContext. + getStepContext(). + getStepExecution(). + getJobExecution(). + getExecutionContext().put("done", "yes"); + return true; + } + + } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/TestSuite.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/TestSuite.java index 660da40839..a89a1b6d32 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/TestSuite.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/TestSuite.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,7 +22,7 @@ import org.junit.runners.Suite.SuiteClasses; /** - * Temporary test suite to find bug in build.... + * Temporary test suite to find bug in build. * */ @Ignore diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/TradeJobFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/TradeJobFunctionalTests.java index 354d924c81..0e426bd5f7 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/TradeJobFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/TradeJobFunctionalTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -46,16 +46,14 @@ @ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/tradeJob.xml", "/job-runner-context.xml" }) public class TradeJobFunctionalTests { - private static final String GET_TRADES = "select ISIN, QUANTITY, PRICE, CUSTOMER, ID, VERSION from TRADE order by ISIN"; private static final String GET_CUSTOMERS = "select NAME, CREDIT from CUSTOMER order by NAME"; private List customers; private List trades; private int activeRow = 0; - private JdbcOperations jdbcTemplate; - private Map credits = new HashMap(); + private Map credits = new HashMap<>(); @Autowired private JobLauncherTestUtils jobLauncherTestUtils; @@ -69,6 +67,7 @@ public void setDataSource(DataSource dataSource) { public void onSetUp() throws Exception { jdbcTemplate.update("delete from TRADE"); List> list = jdbcTemplate.queryForList("select NAME, CREDIT from CUSTOMER"); + for (Map map : list) { credits.put((String) map.get("NAME"), ((Number) map.get("CREDIT")).doubleValue()); } @@ -81,7 +80,6 @@ public void tearDown() throws Exception { @Test public void testLaunchJob() throws Exception { - jobLauncherTestUtils.launchJob(); customers = Arrays.asList(new Customer("customer1", (credits.get("customer1") - 98.34)), @@ -95,9 +93,8 @@ public void testLaunchJob() throws Exception { new Trade("UK21341EAH48", 108, new BigDecimal("109.25"), "customer3"), new Trade("UK21341EAH49", 854, new BigDecimal("123.39"), "customer4")); - // check content of the trade table jdbcTemplate.query(GET_TRADES, new RowCallbackHandler() { - + @Override public void processRow(ResultSet rs) throws SQLException { Trade trade = trades.get(activeRow++); @@ -110,10 +107,9 @@ public void processRow(ResultSet rs) throws SQLException { assertEquals(activeRow, trades.size()); - // check content of the customer table activeRow = 0; jdbcTemplate.query(GET_CUSTOMERS, new RowCallbackHandler() { - + @Override public void processRow(ResultSet rs) throws SQLException { Customer customer = customers.get(activeRow++); @@ -123,8 +119,6 @@ public void processRow(ResultSet rs) throws SQLException { }); assertEquals(customers.size(), activeRow); - - // check content of the output file } private static class Customer { @@ -179,9 +173,5 @@ public boolean equals(Object obj) { return false; return true; } - - } - - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/ColumnRangePartitionerTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/ColumnRangePartitionerTests.java index 7cb7ceb4c6..bea7615ec1 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/ColumnRangePartitionerTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/ColumnRangePartitionerTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.common; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/CustomItemReaderTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/CustomItemReaderTests.java index 612c55bcb7..3ab0ada6a2 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/CustomItemReaderTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/CustomItemReaderTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -27,8 +27,7 @@ import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ItemStream; import org.springframework.batch.item.ItemStreamException; -import org.springframework.batch.item.ParseException; -import org.springframework.batch.item.UnexpectedInputException; +import org.springframework.lang.Nullable; /** * Unit test class that was used as part of the Reference Documentation. I'm only including it in the @@ -38,26 +37,23 @@ * */ public class CustomItemReaderTests { - - ItemReader itemReader; + private ItemReader itemReader; /* (non-Javadoc) * @see junit.framework.TestCase#setUp() */ @Before public void setUp() throws Exception { - - List items = new ArrayList(); + List items = new ArrayList<>(); items.add("1"); items.add("2"); items.add("3"); - itemReader = new CustomItemReader(items); + itemReader = new CustomItemReader<>(items); } @Test public void testRead() throws Exception{ - assertEquals("1", itemReader.read()); assertEquals("2", itemReader.read()); assertEquals("3", itemReader.read()); @@ -66,40 +62,40 @@ public void testRead() throws Exception{ @Test public void testRestart() throws Exception{ - ExecutionContext executionContext = new ExecutionContext(); ((ItemStream)itemReader).open(executionContext); assertEquals("1", itemReader.read()); ((ItemStream)itemReader).update(executionContext); - List items = new ArrayList(); + List items = new ArrayList<>(); items.add("1"); items.add("2"); items.add("3"); - itemReader = new CustomItemReader(items); + itemReader = new CustomItemReader<>(items); ((ItemStream)itemReader).open(executionContext); assertEquals("2", itemReader.read()); } public static class CustomItemReader implements ItemReader, ItemStream { - - List items; - int currentIndex = 0; private static final String CURRENT_INDEX = "current.index"; - + + private List items; + private int currentIndex = 0; + public CustomItemReader(List items) { this.items = items; } - public T read() throws Exception, UnexpectedInputException, - ParseException { - + @Nullable + @Override + public T read() throws Exception { if (currentIndex < items.size()) { return items.get(currentIndex++); } return null; } - + + @Override public void open(ExecutionContext executionContext) throws ItemStreamException { if(executionContext.containsKey(CURRENT_INDEX)){ currentIndex = executionContext.getInt(CURRENT_INDEX); @@ -109,11 +105,12 @@ public void open(ExecutionContext executionContext) throws ItemStreamException { } } + @Override public void close() throws ItemStreamException {} + @Override public void update(ExecutionContext executionContext) throws ItemStreamException { executionContext.putInt(CURRENT_INDEX, currentIndex); - }; - + } } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/CustomItemWriterTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/CustomItemWriterTests.java index 66dff1a537..5d3d24b06e 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/CustomItemWriterTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/CustomItemWriterTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,21 +34,19 @@ * */ public class CustomItemWriterTests { - @Test public void testFlush() throws Exception { - - CustomItemWriter itemWriter = new CustomItemWriter(); + CustomItemWriter itemWriter = new CustomItemWriter<>(); itemWriter.write(Collections.singletonList("1")); assertEquals(1, itemWriter.getOutput().size()); - itemWriter.write(Arrays.asList(new String[] {"2","3"})); + itemWriter.write(Arrays.asList("2","3")); assertEquals(3, itemWriter.getOutput().size()); } public static class CustomItemWriter implements ItemWriter { + private List output = TransactionAwareProxyFactory.createTransactionalList(); - List output = TransactionAwareProxyFactory.createTransactionalList(); - + @Override public void write(List items) throws Exception { output.addAll(items); } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/ErrorLogTasklet.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/ErrorLogTasklet.java index 762014af16..14baba97e9 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/ErrorLogTasklet.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/ErrorLogTasklet.java @@ -1,9 +1,22 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.common; import javax.sql.DataSource; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; import org.springframework.batch.core.ExitStatus; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.StepExecution; @@ -13,6 +26,7 @@ import org.springframework.batch.repeat.RepeatStatus; import org.springframework.jdbc.core.JdbcOperations; import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -20,17 +34,13 @@ * @since 2.0 */ public class ErrorLogTasklet implements Tasklet, StepExecutionListener { - - protected final Log logger = LogFactory.getLog(getClass()); - private JdbcOperations jdbcTemplate; - private String jobName; - private StepExecution stepExecution; - private String stepName; + @Nullable + @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { Assert.notNull(this.stepName, "Step name not set. Either this class was not registered as a listener " + "or the key 'stepName' was not found in the Job's ExecutionContext."); @@ -58,6 +68,7 @@ public void setDataSource(DataSource dataSource) { this.jdbcTemplate = new JdbcTemplate(dataSource); } + @Override public void beforeStep(StepExecution stepExecution) { this.jobName = stepExecution.getJobExecution().getJobInstance().getJobName().trim(); this.stepName = (String) stepExecution.getJobExecution().getExecutionContext().get("stepName"); @@ -65,8 +76,9 @@ public void beforeStep(StepExecution stepExecution) { stepExecution.getJobExecution().getExecutionContext().remove("stepName"); } + @Nullable + @Override public ExitStatus afterStep(StepExecution stepExecution) { return null; } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/ExceptionThrowingItemReaderProxyTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/ExceptionThrowingItemReaderProxyTests.java index 48e4644d49..4f50780d61 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/ExceptionThrowingItemReaderProxyTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/ExceptionThrowingItemReaderProxyTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.common; import static org.junit.Assert.assertEquals; @@ -23,12 +38,13 @@ public void tearDown() throws Exception { RepeatSynchronizationManager.clear(); } + @SuppressWarnings("serial") @Test public void testProcess() throws Exception { //create module and set item processor and iteration count - ExceptionThrowingItemReaderProxy itemReader = new ExceptionThrowingItemReaderProxy(); - itemReader.setDelegate(new ListItemReader(new ArrayList() {{ + ExceptionThrowingItemReaderProxy itemReader = new ExceptionThrowingItemReaderProxy<>(); + itemReader.setDelegate(new ListItemReader<>(new ArrayList() {{ add("a"); add("b"); add("c"); diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/OutputFileListenerTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/OutputFileListenerTests.java index ef1df48466..1eb8d21ea9 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/OutputFileListenerTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/OutputFileListenerTests.java @@ -1,20 +1,34 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.common; -import static org.junit.Assert.assertEquals; - import org.junit.Test; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.StepExecution; -public class OutputFileListenerTests { +import static org.junit.Assert.assertEquals; +public class OutputFileListenerTests { private OutputFileListener listener = new OutputFileListener(); private StepExecution stepExecution = new StepExecution("foo", new JobExecution(0L), 1L); @Test public void testCreateOutputNameFromInput() { listener.createOutputNameFromInput(stepExecution); - assertEquals("{outputFile=file:./target/output/foo.csv}", stepExecution.getExecutionContext().toString()); + assertEquals("{outputFile=file:./build/output/foo.csv}", stepExecution.getExecutionContext().toString()); } @Test @@ -40,5 +54,4 @@ public void testSetInputKeyName() { listener.createOutputNameFromInput(stepExecution); assertEquals("bar.csv", stepExecution.getExecutionContext().getString("outputFile")); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/SkipCheckingDecider.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/SkipCheckingDecider.java index be69db1959..01f738dd25 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/SkipCheckingDecider.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/SkipCheckingDecider.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.common; import org.springframework.batch.core.ExitStatus; @@ -5,10 +20,12 @@ import org.springframework.batch.core.StepExecution; import org.springframework.batch.core.job.flow.FlowExecutionStatus; import org.springframework.batch.core.job.flow.JobExecutionDecider; +import org.springframework.lang.Nullable; public class SkipCheckingDecider implements JobExecutionDecider { - public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) { + @Override + public FlowExecutionStatus decide(JobExecution jobExecution, @Nullable StepExecution stepExecution) { if (!stepExecution.getExitStatus().getExitCode().equals( ExitStatus.FAILED.getExitCode()) && stepExecution.getSkipCount() > 0) { @@ -17,4 +34,4 @@ public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepE return new FlowExecutionStatus(ExitStatus.COMPLETED.getExitCode()); } } -} \ No newline at end of file +} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/SkipCheckingListener.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/SkipCheckingListener.java index 9c92010f9b..a76553a8cb 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/SkipCheckingListener.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/SkipCheckingListener.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.common; import org.apache.commons.logging.Log; diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/StagingItemReaderTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/StagingItemReaderTests.java index c24fa8f4ba..6afee11351 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/StagingItemReaderTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/StagingItemReaderTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.common; import static org.junit.Assert.assertEquals; @@ -30,7 +45,6 @@ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration() public class StagingItemReaderTests { - private JdbcOperations jdbcTemplate; @Autowired @@ -54,7 +68,7 @@ public void onSetUpBeforeTransaction() throws Exception { StepExecution stepExecution = new StepExecution("stepName", new JobExecution(new JobInstance(jobId, "testJob"), new JobParameters())); writer.beforeStep(stepExecution); - writer.write(Arrays.asList(new String[] { "FOO", "BAR", "SPAM", "BUCKET" })); + writer.write(Arrays.asList("FOO", "BAR", "SPAM", "BUCKET")); reader.beforeStep(stepExecution); } @@ -67,8 +81,7 @@ public void onTearDownAfterTransaction() throws Exception { @Transactional @Test public void testReaderWithProcessorUpdatesProcessIndicator() throws Exception { - - long id = jdbcTemplate.queryForLong("SELECT MIN(ID) from BATCH_STAGING where JOB_ID=?", jobId); + long id = jdbcTemplate.queryForObject("SELECT MIN(ID) from BATCH_STAGING where JOB_ID=?", Long.class, jobId); String before = jdbcTemplate.queryForObject("SELECT PROCESSED from BATCH_STAGING where ID=?", String.class, id); assertEquals(StagingItemWriter.NEW, before); @@ -77,14 +90,13 @@ public void testReaderWithProcessorUpdatesProcessIndicator() throws Exception { String item = wrapper.getItem(); assertEquals("FOO", item); - StagingItemProcessor updater = new StagingItemProcessor(); + StagingItemProcessor updater = new StagingItemProcessor<>(); updater.setJdbcTemplate(jdbcTemplate); updater.process(wrapper); String after = jdbcTemplate.queryForObject("SELECT PROCESSED from BATCH_STAGING where ID=?", String.class, id); assertEquals(StagingItemWriter.DONE, after); - } @Transactional @@ -92,18 +104,19 @@ public void testReaderWithProcessorUpdatesProcessIndicator() throws Exception { public void testUpdateProcessIndicatorAfterCommit() throws Exception { TransactionTemplate txTemplate = new TransactionTemplate(transactionManager); txTemplate.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW); - txTemplate.execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus transactionStatus) { + txTemplate.execute(new TransactionCallback() { + @Override + public Void doInTransaction(TransactionStatus transactionStatus) { try { testReaderWithProcessorUpdatesProcessIndicator(); } catch (Exception e) { - fail("Unxpected Exception: " + e); + fail("Unexpected Exception: " + e); } return null; } }); - long id = jdbcTemplate.queryForLong("SELECT MIN(ID) from BATCH_STAGING where JOB_ID=?", jobId); + long id = jdbcTemplate.queryForObject("SELECT MIN(ID) from BATCH_STAGING where JOB_ID=?", Long.class, jobId); String before = jdbcTemplate.queryForObject("SELECT PROCESSED from BATCH_STAGING where ID=?", String.class, id); assertEquals(StagingItemWriter.DONE, before); @@ -112,14 +125,14 @@ public Object doInTransaction(TransactionStatus transactionStatus) { @Transactional @Test public void testReaderRollsBackProcessIndicator() throws Exception { - TransactionTemplate txTemplate = new TransactionTemplate(transactionManager); txTemplate.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW); - final Long idToUse = (Long) txTemplate.execute(new TransactionCallback() { - public Object doInTransaction(TransactionStatus transactionStatus) { + final Long idToUse = txTemplate.execute(new TransactionCallback() { + @Override + public Long doInTransaction(TransactionStatus transactionStatus) { - long id = jdbcTemplate.queryForLong("SELECT MIN(ID) from BATCH_STAGING where JOB_ID=?", jobId); + long id = jdbcTemplate.queryForObject("SELECT MIN(ID) from BATCH_STAGING where JOB_ID=?", Long.class, jobId); String before = jdbcTemplate.queryForObject("SELECT PROCESSED from BATCH_STAGING where ID=?", String.class, id); assertEquals(StagingItemWriter.NEW, before); @@ -136,6 +149,5 @@ public Object doInTransaction(TransactionStatus transactionStatus) { String after = jdbcTemplate.queryForObject("SELECT PROCESSED from BATCH_STAGING where ID=?", String.class, idToUse); assertEquals(StagingItemWriter.NEW, after); - } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/StagingItemWriterTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/StagingItemWriterTests.java index 1fccfce0b7..649d0d25f2 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/StagingItemWriterTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/common/StagingItemWriterTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -38,7 +38,6 @@ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration public class StagingItemWriterTests { - private JdbcOperations jdbcTemplate; @Autowired @@ -59,10 +58,9 @@ public void onSetUpBeforeTransaction() throws Exception { @Transactional @Test public void testProcessInsertsNewItem() throws Exception { - int before = jdbcTemplate.queryForInt("SELECT COUNT(*) from BATCH_STAGING"); + int before = jdbcTemplate.queryForObject("SELECT COUNT(*) from BATCH_STAGING", Integer.class); writer.write(Collections.singletonList("FOO")); - int after = jdbcTemplate.queryForInt("SELECT COUNT(*) from BATCH_STAGING"); + int after = jdbcTemplate.queryForObject("SELECT COUNT(*) from BATCH_STAGING", Integer.class); assertEquals(before + 1, after); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/config/JobRunnerConfiguration.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/config/JobRunnerConfiguration.java index b9da8b6c8e..9b4779a7f0 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/config/JobRunnerConfiguration.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/config/JobRunnerConfiguration.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/football/internal/JdbcGameDaoIntegrationTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/football/internal/JdbcGameDaoIntegrationTests.java index de6a8020d6..77107e8b34 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/football/internal/JdbcGameDaoIntegrationTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/football/internal/JdbcGameDaoIntegrationTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,11 +26,12 @@ import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; + import org.springframework.batch.sample.domain.football.Game; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcOperations; import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.transaction.annotation.Transactional; @@ -42,11 +43,8 @@ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = {"/data-source-context.xml"}) public class JdbcGameDaoIntegrationTests { - private JdbcGameDao gameDao; - private Game game = new Game(); - private JdbcOperations jdbcTemplate; @Autowired @@ -59,7 +57,6 @@ public void setDataSource(DataSource dataSource) { @Before public void onSetUpBeforeTransaction() throws Exception { - game.setId("XXXXX00"); game.setYear(1996); game.setTeam("mia"); @@ -75,12 +72,10 @@ public void onSetUpBeforeTransaction() throws Exception { game.setReceptions(1); game.setReceptionYards(16); game.setTotalTd(2); - } @Transactional @Test public void testWrite() { - gameDao.write(Collections.singletonList(game)); Game tempGame = jdbcTemplate.queryForObject("SELECT * FROM GAMES where PLAYER_ID=? AND YEAR_NO=?", @@ -88,10 +83,9 @@ public void testWrite() { assertEquals(tempGame, game); } - private static class GameRowMapper implements ParameterizedRowMapper { - + private static class GameRowMapper implements RowMapper { + @Override public Game mapRow(ResultSet rs, int arg1) throws SQLException { - if (rs == null) { return null; } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerDaoIntegrationTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerDaoIntegrationTests.java index 2730acdeef..253ecba66c 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerDaoIntegrationTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerDaoIntegrationTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -41,18 +41,13 @@ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = {"/data-source-context.xml"}) public class JdbcPlayerDaoIntegrationTests { - private JdbcPlayerDao playerDao; - private Player player; - private static final String GET_PLAYER = "SELECT * from PLAYERS"; - private JdbcOperations jdbcTemplate; @Autowired public void init(DataSource dataSource) { - this.jdbcTemplate = new JdbcTemplate(dataSource); playerDao = new JdbcPlayerDao(); playerDao.setDataSource(dataSource); @@ -64,24 +59,19 @@ public void init(DataSource dataSource) { player.setPosition("QB"); player.setBirthYear(1975); player.setDebutYear(1998); - } - @Before public void onSetUpInTransaction() throws Exception { - jdbcTemplate.execute("delete from PLAYERS"); - } - @Transactional @Test + @Test + @Transactional public void testSavePlayer(){ - playerDao.savePlayer(player); - - jdbcTemplate.query(GET_PLAYER, new RowCallbackHandler(){ - + jdbcTemplate.query(GET_PLAYER, new RowCallbackHandler() { + @Override public void processRow(ResultSet rs) throws SQLException { assertEquals(rs.getString("PLAYER_ID"), "AKFJDL00"); assertEquals(rs.getString("LAST_NAME"), "Doe"); @@ -92,5 +82,4 @@ public void processRow(ResultSet rs) throws SQLException { } }); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerSummaryDaoIntegrationTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerSummaryDaoIntegrationTests.java index 04ca13dffc..f313bd680f 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerSummaryDaoIntegrationTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/football/internal/JdbcPlayerSummaryDaoIntegrationTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -39,16 +39,12 @@ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = { "/data-source-context.xml" }) public class JdbcPlayerSummaryDaoIntegrationTests { - private JdbcPlayerSummaryDao playerSummaryDao; - private PlayerSummary summary; - private JdbcOperations jdbcTemplate; @Autowired public void init(DataSource dataSource) { - this.jdbcTemplate = new JdbcTemplate(dataSource); playerSummaryDao = new JdbcPlayerSummaryDao(); playerSummaryDao.setDataSource(dataSource); @@ -66,27 +62,21 @@ public void init(DataSource dataSource) { summary.setReceptions(0); summary.setReceptionYards(0); summary.setTotalTd(0); - } @Before public void onSetUpInTransaction() throws Exception { - jdbcTemplate.execute("delete from PLAYER_SUMMARY"); - } - @Transactional @Test + @Transactional public void testWrite() { - playerSummaryDao.write(Collections.singletonList(summary)); PlayerSummary testSummary = jdbcTemplate.queryForObject("SELECT * FROM PLAYER_SUMMARY", new PlayerSummaryMapper()); assertEquals(summary, testSummary); - } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/multiline/AggregateItemFieldSetMapperTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/multiline/AggregateItemFieldSetMapperTests.java index 48fdc5311a..af2ca6c6b8 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/multiline/AggregateItemFieldSetMapperTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/multiline/AggregateItemFieldSetMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.multiline; import static org.junit.Assert.assertEquals; @@ -9,11 +24,9 @@ import org.springframework.batch.item.file.mapping.FieldSetMapper; import org.springframework.batch.item.file.transform.DefaultFieldSet; import org.springframework.batch.item.file.transform.FieldSet; -import org.springframework.batch.sample.domain.multiline.AggregateItemFieldSetMapper; public class AggregateItemFieldSetMapperTests { - - private AggregateItemFieldSetMapper mapper = new AggregateItemFieldSetMapper(); + private AggregateItemFieldSetMapper mapper = new AggregateItemFieldSetMapper<>(); @Test public void testDefaultBeginRecord() throws Exception { @@ -53,12 +66,11 @@ public void testMandatoryProperties() throws Exception { @Test public void testDelegate() throws Exception { mapper.setDelegate(new FieldSetMapper() { + @Override public String mapFieldSet(FieldSet fs) { return "foo"; } }); assertEquals("foo", mapper.mapFieldSet(new DefaultFieldSet(new String[] { "FOO" })).getItem()); } - - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/multiline/AggregateItemReaderTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/multiline/AggregateItemReaderTests.java index b3f61bb75e..1b427a17da 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/multiline/AggregateItemReaderTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/multiline/AggregateItemReaderTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.multiline; import static org.junit.Assert.*; @@ -6,22 +21,19 @@ import org.junit.Before; import org.junit.Test; import org.springframework.batch.item.ItemReader; -import org.springframework.batch.sample.domain.multiline.AggregateItem; -import org.springframework.batch.sample.domain.multiline.AggregateItemReader; +import org.springframework.lang.Nullable; public class AggregateItemReaderTests { - private ItemReader> input; - private AggregateItemReader provider; @Before public void setUp() { - // create mock for input input = new ItemReader>() { - private int count = 0; + @Nullable + @Override public AggregateItem read() { switch (count++) { case 0: @@ -29,7 +41,7 @@ public AggregateItem read() { case 1: case 2: case 3: - return new AggregateItem("line"); + return new AggregateItem<>("line"); case 4: return AggregateItem.getFooter(); default: @@ -38,17 +50,15 @@ public AggregateItem read() { } }; - // create provider - provider = new AggregateItemReader(); + + provider = new AggregateItemReader<>(); provider.setItemReader(input); } @Test public void testNext() throws Exception { - // read object Object result = provider.read(); - // it should be collection of 3 strings "line" Collection lines = (Collection) result; assertEquals(3, lines.size()); @@ -56,9 +66,6 @@ public void testNext() throws Exception { assertEquals("line", line); } - // read object again - it should return null assertNull(provider.read()); - } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/multiline/AggregateItemTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/multiline/AggregateItemTests.java index 58669e8474..2578818cb3 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/multiline/AggregateItemTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/multiline/AggregateItemTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -20,14 +20,12 @@ import static org.junit.Assert.fail; import org.junit.Test; -import org.springframework.batch.sample.domain.multiline.AggregateItem; /** * @author Dave Syer * */ public class AggregateItemTests { - /** * Test method for {@link org.springframework.batch.sample.domain.multiline.AggregateItem#getFooter()}. */ @@ -65,5 +63,4 @@ public void testEndRecordHasNoItem() throws Exception { // expected } } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/AddressFieldSetMapperTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/AddressFieldSetMapperTests.java index 251689b05e..a90b2c7bd2 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/AddressFieldSetMapperTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/AddressFieldSetMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order; import org.springframework.batch.item.file.mapping.FieldSetMapper; @@ -7,7 +22,6 @@ import org.springframework.batch.sample.support.AbstractFieldSetMapperTests; public class AddressFieldSetMapperTests extends AbstractFieldSetMapperTests { - private static final String ADDRESSEE = "Jan Hrach"; private static final String ADDRESS_LINE_1 = "Plynarenska 7c"; private static final String ADDRESS_LINE_2 = ""; @@ -16,6 +30,7 @@ public class AddressFieldSetMapperTests extends AbstractFieldSetMapperTests { private static final String COUNTRY = "Slovakia"; private static final String ZIP_CODE = "80000"; + @Override protected Object expectedDomainObject() { Address address = new Address(); address.setAddressee(ADDRESSEE); @@ -28,6 +43,7 @@ protected Object expectedDomainObject() { return address; } + @Override protected FieldSet fieldSet() { String[] tokens = new String[] { ADDRESSEE, ADDRESS_LINE_1, ADDRESS_LINE_2, CITY, STATE, COUNTRY, ZIP_CODE }; String[] columnNames = new String[] { AddressFieldSetMapper.ADDRESSEE_COLUMN, @@ -38,6 +54,7 @@ protected FieldSet fieldSet() { return new DefaultFieldSet(tokens, columnNames); } + @Override protected FieldSetMapper
        fieldSetMapper() { return new AddressFieldSetMapper(); } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/BillingFieldSetMapperTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/BillingFieldSetMapperTests.java index 3370d1975c..4b50aeb78e 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/BillingFieldSetMapperTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/BillingFieldSetMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order; import org.springframework.batch.item.file.mapping.FieldSetMapper; @@ -7,10 +22,10 @@ import org.springframework.batch.sample.support.AbstractFieldSetMapperTests; public class BillingFieldSetMapperTests extends AbstractFieldSetMapperTests { - private static final String PAYMENT_ID = "777"; private static final String PAYMENT_DESC = "My last penny"; + @Override protected Object expectedDomainObject() { BillingInfo bInfo = new BillingInfo(); bInfo.setPaymentDesc(PAYMENT_DESC); @@ -18,6 +33,7 @@ protected Object expectedDomainObject() { return bInfo; } + @Override protected FieldSet fieldSet() { String[] tokens = new String[] { PAYMENT_ID, PAYMENT_DESC }; String[] columnNames = new String[] { BillingFieldSetMapper.PAYMENT_TYPE_ID_COLUMN, @@ -25,8 +41,8 @@ protected FieldSet fieldSet() { return new DefaultFieldSet(tokens, columnNames); } + @Override protected FieldSetMapper fieldSetMapper() { return new BillingFieldSetMapper(); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/CustomerFieldSetMapperTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/CustomerFieldSetMapperTests.java index 2a5d060373..8b8a5514c8 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/CustomerFieldSetMapperTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/CustomerFieldSetMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order; import org.springframework.batch.item.file.mapping.FieldSetMapper; @@ -7,7 +22,6 @@ import org.springframework.batch.sample.support.AbstractFieldSetMapperTests; public class CustomerFieldSetMapperTests extends AbstractFieldSetMapperTests { - private static final boolean BUSINESS_CUSTOMER = false; private static final String FIRST_NAME = "Jan"; private static final String LAST_NAME = "Hrach"; @@ -16,6 +30,7 @@ public class CustomerFieldSetMapperTests extends AbstractFieldSetMapperTests { private static final long REG_ID = 1; private static final boolean VIP = true; + @Override protected Object expectedDomainObject() { Customer cs = new Customer(); cs.setBusinessCustomer(BUSINESS_CUSTOMER); @@ -28,6 +43,7 @@ protected Object expectedDomainObject() { return cs; } + @Override protected FieldSet fieldSet() { String[] tokens = new String[] { Customer.LINE_ID_NON_BUSINESS_CUST, FIRST_NAME, LAST_NAME, MIDDLE_NAME, CustomerFieldSetMapper.TRUE_SYMBOL, String.valueOf(REG_ID), CustomerFieldSetMapper.TRUE_SYMBOL }; @@ -39,8 +55,8 @@ protected FieldSet fieldSet() { return new DefaultFieldSet(tokens, columnNames); } + @Override protected FieldSetMapper fieldSetMapper() { return new CustomerFieldSetMapper(); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/HeaderFieldSetMapperTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/HeaderFieldSetMapperTests.java index 0adf78be9a..a31f536b5a 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/HeaderFieldSetMapperTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/HeaderFieldSetMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order; import java.util.Calendar; @@ -9,10 +24,10 @@ import org.springframework.batch.sample.support.AbstractFieldSetMapperTests; public class HeaderFieldSetMapperTests extends AbstractFieldSetMapperTests { - private static final long ORDER_ID = 1; private static final String DATE = "2007-01-01"; + @Override protected Object expectedDomainObject() { Order order = new Order(); Calendar calendar = Calendar.getInstance(); @@ -23,6 +38,7 @@ protected Object expectedDomainObject() { return order; } + @Override protected FieldSet fieldSet() { String[] tokens = new String[] { String.valueOf(ORDER_ID), DATE }; String[] columnNames = new String[] { HeaderFieldSetMapper.ORDER_ID_COLUMN, @@ -30,8 +46,8 @@ protected FieldSet fieldSet() { return new DefaultFieldSet(tokens, columnNames); } + @Override protected FieldSetMapper fieldSetMapper() { return new HeaderFieldSetMapper(); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/OrderItemFieldSetMapperTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/OrderItemFieldSetMapperTests.java index b023f692df..6f1cb16d7d 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/OrderItemFieldSetMapperTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/OrderItemFieldSetMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order; import java.math.BigDecimal; @@ -9,7 +24,6 @@ import org.springframework.batch.sample.support.AbstractFieldSetMapperTests; public class OrderItemFieldSetMapperTests extends AbstractFieldSetMapperTests { - private static final BigDecimal DISCOUNT_AMOUNT = new BigDecimal("1"); private static final BigDecimal DISCOUNT_PERC = new BigDecimal("2"); private static final BigDecimal HANDLING_PRICE = new BigDecimal("3"); @@ -19,6 +33,7 @@ public class OrderItemFieldSetMapperTests extends AbstractFieldSetMapperTests { private static final BigDecimal SHIPPING_PRICE = new BigDecimal("7"); private static final BigDecimal TOTAL_PRICE = new BigDecimal("8"); + @Override protected Object expectedDomainObject() { LineItem item = new LineItem(); item.setDiscountAmount(DISCOUNT_AMOUNT); @@ -32,6 +47,7 @@ protected Object expectedDomainObject() { return item; } + @Override protected FieldSet fieldSet() { String[] tokens = new String[] { String.valueOf(DISCOUNT_AMOUNT), String.valueOf(DISCOUNT_PERC), String.valueOf(HANDLING_PRICE), String.valueOf(ITEM_ID), String.valueOf(PRICE), @@ -44,8 +60,8 @@ protected FieldSet fieldSet() { return new DefaultFieldSet(tokens, columnNames); } + @Override protected FieldSetMapper fieldSetMapper() { return new OrderItemFieldSetMapper(); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/OrderItemReaderTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/OrderItemReaderTests.java index 59c534231e..aca273d26a 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/OrderItemReaderTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/OrderItemReaderTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order; import static org.junit.Assert.assertEquals; @@ -7,10 +22,7 @@ import static org.mockito.Mockito.mock; import static org.mockito.Mockito.when; -import java.util.Iterator; - import org.junit.Before; -import org.junit.Ignore; import org.junit.Test; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.file.mapping.FieldSetMapper; @@ -19,23 +31,20 @@ import org.springframework.batch.sample.domain.order.internal.OrderItemReader; public class OrderItemReaderTests { - private OrderItemReader provider; - private ItemReader
        input; - @SuppressWarnings("unchecked") @Before + @SuppressWarnings("unchecked") public void setUp() { - - input = (ItemReader
        ) mock(ItemReader.class); + input = mock(ItemReader.class); provider = new OrderItemReader(); provider.setFieldSetReader(input); } /* - * OrderItemProvider is resposible for retrieving validated value object + * OrderItemProvider is responsible for retrieving validated value object * from input source. OrderItemProvider.next(): - reads lines from the input * source - returned as fieldsets - pass fieldsets to the mapper - mapper * will create value object - pass value object to validator - returns @@ -44,12 +53,9 @@ public void setUp() { * In testNext method we are going to test these responsibilities. So we * need create mock objects for input source, mapper and validator. */ - @Ignore //TODO mockito fix - @SuppressWarnings("unchecked") @Test + @SuppressWarnings("unchecked") public void testNext() throws Exception { - - // create fieldsets and set return values for input source FieldSet headerFS = new DefaultFieldSet(new String[] { Order.LINE_ID_HEADER }); FieldSet customerFS = new DefaultFieldSet(new String[] { Customer.LINE_ID_NON_BUSINESS_CUST }); FieldSet billingFS = new DefaultFieldSet(new String[] { Address.LINE_ID_BILLING_ADDR }); @@ -60,19 +66,9 @@ public void testNext() throws Exception { FieldSet footerFS = new DefaultFieldSet(new String[] { Order.LINE_ID_FOOTER, "100", "3", "3" }, new String[] { "ID", "TOTAL_PRICE", "TOTAL_LINE_ITEMS", "TOTAL_ITEMS" }); - when(input.read()).thenReturn(headerFS); - when(input.read()).thenReturn(customerFS); - when(input.read()).thenReturn(billingFS); - when(input.read()).thenReturn(shippingFS); - when(input.read()).thenReturn(billingInfoFS); - when(input.read()).thenReturn(shippingInfoFS); - when(input.read()).thenReturn(itemFS); - when(input.read()).thenReturn(footerFS); - when(input.read()).thenReturn(null); -// replay(input); -// input.read(); - - // create value objects + when(input.read()).thenReturn(headerFS, customerFS, billingFS, shippingFS, billingInfoFS, + shippingInfoFS, itemFS, itemFS, itemFS, footerFS, null); + Order order = new Order(); Customer customer = new Customer(); Address billing = new Address(); @@ -81,10 +77,8 @@ public void testNext() throws Exception { ShippingInfo shippingInfo = new ShippingInfo(); LineItem item = new LineItem(); - // create mock mapper @SuppressWarnings("rawtypes") FieldSetMapper mapper = mock(FieldSetMapper.class); - // set how mapper should respond - set return values for mapper when(mapper.mapFieldSet(headerFS)).thenReturn(order); when(mapper.mapFieldSet(customerFS)).thenReturn(customer); when(mapper.mapFieldSet(billingFS)).thenReturn(billing); @@ -93,7 +87,6 @@ public void testNext() throws Exception { when(mapper.mapFieldSet(shippingInfoFS)).thenReturn(shippingInfo); when(mapper.mapFieldSet(itemFS)).thenReturn(item); - // set-up provider: set mappers provider.setAddressMapper(mapper); provider.setBillingMapper(mapper); provider.setCustomerMapper(mapper); @@ -101,31 +94,25 @@ public void testNext() throws Exception { provider.setItemMapper(mapper); provider.setShippingMapper(mapper); - // call tested method Object result = provider.read(); - // verify result assertNotNull(result); - // verify whether order is constructed correctly - // Order object should contain same instances as returned by mapper Order o = (Order) result; assertEquals(o, order); assertEquals(o.getCustomer(), customer); - // is it non-bussines customer assertFalse(o.getCustomer().isBusinessCustomer()); assertEquals(o.getBillingAddress(), billing); assertEquals(o.getShippingAddress(), shipping); assertEquals(o.getBilling(), billingInfo); assertEquals(o.getShipping(), shippingInfo); - // there should be 3 line items + assertEquals(3, o.getLineItems().size()); - for (Iterator i = o.getLineItems().iterator(); i.hasNext();) { - assertEquals(i.next(), item); + + for (LineItem lineItem : o.getLineItems()) { + assertEquals(lineItem, item); } - // try to retrieve next object - nothing should be returned assertNull(provider.read()); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/ShippingFieldSetMapperTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/ShippingFieldSetMapperTests.java index 7310f43c3b..2d054dd162 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/ShippingFieldSetMapperTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/ShippingFieldSetMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.order; import org.springframework.batch.item.file.mapping.FieldSetMapper; @@ -7,11 +22,11 @@ import org.springframework.batch.sample.support.AbstractFieldSetMapperTests; public class ShippingFieldSetMapperTests extends AbstractFieldSetMapperTests { - private static final String SHIPPER_ID = "1"; private static final String SHIPPING_INFO = "most interesting and informative shipping info ever"; private static final String SHIPPING_TYPE_ID = "X"; + @Override protected Object expectedDomainObject() { ShippingInfo info = new ShippingInfo(); info.setShipperId(SHIPPER_ID); @@ -20,6 +35,7 @@ protected Object expectedDomainObject() { return info; } + @Override protected FieldSet fieldSet() { String[] tokens = new String[] { SHIPPER_ID, SHIPPING_INFO, SHIPPING_TYPE_ID }; String[] columnNames = new String[] { ShippingFieldSetMapper.SHIPPER_ID_COLUMN, @@ -27,8 +43,8 @@ protected FieldSet fieldSet() { return new DefaultFieldSet(tokens, columnNames); } + @Override protected FieldSetMapper fieldSetMapper() { return new ShippingFieldSetMapper(); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/FutureDateFunctionTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/FutureDateFunctionTests.java deleted file mode 100644 index 3348f8adcc..0000000000 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/FutureDateFunctionTests.java +++ /dev/null @@ -1,67 +0,0 @@ -package org.springframework.batch.sample.domain.order.internal.valang; - -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - -import java.util.Date; - -import org.junit.Before; -import org.junit.Test; -import org.springmodules.validation.valang.functions.Function; - -public class FutureDateFunctionTests { - - private FutureDateFunction function; - private Function argument; - - @Before - public void setUp() { - argument = mock(Function.class); - - //create function - function = new FutureDateFunction(new Function[] {argument}, 0, 0); - - } - - @Test - public void testFunctionWithNonDateValue() { - - //set-up mock argument - set return value to non Date value - when(argument.getResult(null)).thenReturn(this); - - //call tested method - exception is expected because non date value - try { - function.doGetResult(null); - fail("Exception was expected."); - } catch (Exception e) { - assertTrue(true); - } - - } - - @Test - public void testFunctionWithFutureDate() throws Exception { - - //set-up mock argument - set return value to future Date - when(argument.getResult(null)).thenReturn(new Date(Long.MAX_VALUE)); - - //vefify result - should be true because of future date - assertTrue((Boolean) function.doGetResult(null)); - - } - - @Test - public void testFunctionWithPastDate() throws Exception { - - //set-up mock argument - set return value to future Date - when(argument.getResult(null)).thenReturn(new Date(0)); - - //vefify result - should be false because of past date - assertFalse((Boolean) function.doGetResult(null)); - - } - -} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/TotalOrderItemsFunctionTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/TotalOrderItemsFunctionTests.java deleted file mode 100644 index 01edd1766e..0000000000 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/TotalOrderItemsFunctionTests.java +++ /dev/null @@ -1,83 +0,0 @@ -package org.springframework.batch.sample.domain.order.internal.valang; - -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; -import static org.junit.Assert.fail; - -import java.util.ArrayList; -import java.util.List; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.Function; - -public class TotalOrderItemsFunctionTests { - - private TotalOrderItemsFunction function; - private Function argument2; - - @Before - public void setUp() { - //create mock for first argument - set count to 3 - Function argument1 = mock(Function.class); - when(argument1.getResult(null)).thenReturn(3); - - - argument2 = mock(Function.class); - - //create function - function = new TotalOrderItemsFunction(new Function[] {argument1, argument2}, 0, 0); - - } - - @Test - public void testFunctionWithNonListValue() { - - when(argument2.getResult(null)).thenReturn(this); - - //call tested method - exception is expected because non list value - try { - function.doGetResult(null); - fail("Exception was expected."); - } catch (Exception e) { - assertTrue(true); - } - - } - - @Test - public void testFunctionWithCorrectItemCount() throws Exception { - - //create list with correct item count - LineItem item = new LineItem(); - item.setQuantity(3); - List list = new ArrayList(); - list.add(item); - - when(argument2.getResult(null)).thenReturn(list); - - //vefify result - assertTrue((Boolean) function.doGetResult(null)); - - } - - @Test - public void testFunctionWithIncorrectItemCount() throws Exception { - - //create list with incorrect item count - LineItem item = new LineItem(); - item.setQuantity(99); - List list = new ArrayList(); - list.add(item); - - when(argument2.getResult(null)).thenReturn(list); - - //vefify result - assertFalse((Boolean) function.doGetResult(null)); - - } - -} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateDiscountsFunctionTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateDiscountsFunctionTests.java deleted file mode 100644 index ed483349ca..0000000000 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateDiscountsFunctionTests.java +++ /dev/null @@ -1,172 +0,0 @@ -package org.springframework.batch.sample.domain.order.internal.valang; - -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - -import java.math.BigDecimal; -import java.util.ArrayList; -import java.util.List; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.Function; - -public class ValidateDiscountsFunctionTests { - - private ValidateDiscountsFunction function; - private Function argument; - - @Before - public void setUp() { - - argument = mock(Function.class); - - //create function - function = new ValidateDiscountsFunction(new Function[] {argument}, 0, 0); - - } - - @Test - public void testDiscountPercentageMin() throws Exception { - - //create line item with correct discount percentage and zero discount amount - LineItem item = new LineItem(); - item.setDiscountPerc(new BigDecimal(1.0)); - item.setDiscountAmount(new BigDecimal(0.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all discount percentages are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with negative percentage - item = new LineItem(); - item.setDiscountPerc(new BigDecimal(-1.0)); - item.setDiscountAmount(new BigDecimal(0.0)); - items.add(item); - - //verify result - should be false - second item has invalid discount percentage - assertFalse((Boolean) function.doGetResult(null)); - - } - - @Test - public void testDiscountPercentageMax() throws Exception { - - //create line item with correct discount percentage and zero discount amount - LineItem item = new LineItem(); - item.setDiscountPerc(new BigDecimal(99.0)); - item.setDiscountAmount(new BigDecimal(0.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all discount percentages are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with discount percentage above 100 - item = new LineItem(); - item.setDiscountPerc(new BigDecimal(101.0)); - item.setDiscountAmount(new BigDecimal(0.0)); - items.add(item); - - //verify result - should be false - second item has invalid discount percentage - assertFalse((Boolean) function.doGetResult(null)); - - } - - @Test - public void testDiscountPriceMin() throws Exception { - - //create line item with correct discount amount and zero discount percentage - LineItem item = new LineItem(); - item.setDiscountPerc(new BigDecimal(0.0)); - item.setDiscountAmount(new BigDecimal(10.0)); - item.setPrice(new BigDecimal(100.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all discount amounts are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with negative discount amount - item = new LineItem(); - item.setDiscountPerc(new BigDecimal(0.0)); - item.setDiscountAmount(new BigDecimal(-1.0)); - item.setPrice(new BigDecimal(100.0)); - items.add(item); - - //verify result - should be false - second item has invalid discount amount - assertFalse((Boolean) function.doGetResult(null)); - - } - - @Test - public void testDiscountPriceMax() throws Exception { - - //create line item with correct discount amount and zero discount percentage - LineItem item = new LineItem(); - item.setDiscountPerc(new BigDecimal(0.0)); - item.setDiscountAmount(new BigDecimal(99.0)); - item.setPrice(new BigDecimal(100.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all discount amounts are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with discount amount above item price - item = new LineItem(); - item.setDiscountPerc(new BigDecimal(0.0)); - item.setDiscountAmount(new BigDecimal(101.0)); - item.setPrice(new BigDecimal(100.0)); - items.add(item); - - //verify result - should be false - second item has invalid discount amount - assertFalse((Boolean) function.doGetResult(null)); - - } - - @Test - public void testBothDiscountValuesNonZero() throws Exception { - - //create line item with non-zero discount amount and non-zero discount percentage - LineItem item = new LineItem(); - item.setDiscountPerc(new BigDecimal(10.0)); - item.setDiscountAmount(new BigDecimal(99.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be false - only one of the discount values is empty - assertFalse((Boolean) function.doGetResult(null)); - - } - -} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateHandlingPricesFunctionTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateHandlingPricesFunctionTests.java deleted file mode 100644 index 30ddb8086b..0000000000 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateHandlingPricesFunctionTests.java +++ /dev/null @@ -1,85 +0,0 @@ -package org.springframework.batch.sample.domain.order.internal.valang; - -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - -import java.math.BigDecimal; -import java.util.ArrayList; -import java.util.List; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.Function; - -public class ValidateHandlingPricesFunctionTests { - - private ValidateHandlingPricesFunction function; - private Function argument; - - @Before - public void setUp() { - argument = mock(Function.class); - - //create function - function = new ValidateHandlingPricesFunction(new Function[] {argument}, 0, 0); - - } - - @Test - public void testHandlingPriceMin() throws Exception { - - //create line item with correct handling price - LineItem item = new LineItem(); - item.setHandlingPrice(new BigDecimal(1.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all handling prices are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with negative handling price - item = new LineItem(); - item.setHandlingPrice(new BigDecimal(-1.0)); - items.add(item); - - //verify result - should be false - second item has invalid handling price - assertFalse((Boolean) function.doGetResult(null)); - - } - - @Test - public void testHandlingPriceMax() throws Exception { - - //create line item with correct handling price - LineItem item = new LineItem(); - item.setHandlingPrice(new BigDecimal(99999999.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all handling prices are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with handling price above allowed max - item = new LineItem(); - item.setHandlingPrice(new BigDecimal(100000000.0)); - items.add(item); - - //verify result - should be false - second item has invalid handling price - assertFalse((Boolean) function.doGetResult(null)); - - } - -} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateIdsFunctionTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateIdsFunctionTests.java deleted file mode 100644 index c341049d05..0000000000 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateIdsFunctionTests.java +++ /dev/null @@ -1,83 +0,0 @@ -package org.springframework.batch.sample.domain.order.internal.valang; - -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - -import java.util.ArrayList; -import java.util.List; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.Function; - -public class ValidateIdsFunctionTests { - - private ValidateIdsFunction function; - private Function argument; - - @Before - public void setUp() { - argument = mock(Function.class); - - //create function - function = new ValidateIdsFunction(new Function[] {argument}, 0, 0); - } - - @Test - public void testIdMin() throws Exception { - - //create line item with correct item id - LineItem item = new LineItem(); - item.setItemId(1); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all ids are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with negative id - item = new LineItem(); - item.setItemId(-1); - items.add(item); - - //verify result - should be false - second item has invalid id - assertFalse((Boolean) function.doGetResult(null)); - - } - - @Test - public void testIdMax() throws Exception { - - //create line item with correct item id - LineItem item = new LineItem(); - item.setItemId(9999999999L); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all item ids are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with item id above allowed max - item = new LineItem(); - item.setItemId(10000000000L); - items.add(item); - - //verify result - should be false - second item has invalid item id - assertFalse((Boolean) function.doGetResult(null)); - - } - -} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidatePricesFunctionTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidatePricesFunctionTests.java deleted file mode 100644 index 34251cb615..0000000000 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidatePricesFunctionTests.java +++ /dev/null @@ -1,84 +0,0 @@ -package org.springframework.batch.sample.domain.order.internal.valang; - -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - -import java.math.BigDecimal; -import java.util.ArrayList; -import java.util.List; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.Function; - -public class ValidatePricesFunctionTests { - - private ValidatePricesFunction function; - private Function argument; - - @Before - public void setUp() { - - argument = mock(Function.class); - - //create function - function = new ValidatePricesFunction(new Function[] {argument}, 0, 0); - - } - - @Test - public void testItemPriceMin() throws Exception { - - //create line item with correct item price - LineItem item = new LineItem(); - item.setPrice(new BigDecimal(1.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all item prices are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with negative item price - item = new LineItem(); - item.setPrice(new BigDecimal(-1.0)); - items.add(item); - - //verify result - should be false - second item has invalid item price - assertFalse((Boolean) function.doGetResult(null)); - } - - @Test - public void testItemPriceMax() throws Exception { - - //create line item with correct item price - LineItem item = new LineItem(); - item.setPrice(new BigDecimal(99999999.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all item prices are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with item price above allowed max - item = new LineItem(); - item.setPrice(new BigDecimal(100000000.0)); - items.add(item); - - //verify result - should be false - second item has invalid item price - assertFalse((Boolean) function.doGetResult(null)); - } - -} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateQuantitiesFunctionTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateQuantitiesFunctionTests.java deleted file mode 100644 index 0da1bc5973..0000000000 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateQuantitiesFunctionTests.java +++ /dev/null @@ -1,82 +0,0 @@ -package org.springframework.batch.sample.domain.order.internal.valang; - -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - -import java.util.ArrayList; -import java.util.List; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.Function; - -public class ValidateQuantitiesFunctionTests { - - private ValidateQuantitiesFunction function; - private Function argument; - - @Before - public void setUp() { - - argument = mock(Function.class); - - //create function - function = new ValidateQuantitiesFunction(new Function[] {argument}, 0, 0); - - } - - @Test - public void testQuantityMin() throws Exception { - - //create line item with correct item quantity - LineItem item = new LineItem(); - item.setQuantity(1); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all quantities are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with negative quantity - item = new LineItem(); - item.setQuantity(-1); - items.add(item); - - //verify result - should be false - second item has invalid quantity - assertFalse((Boolean) function.doGetResult(null)); - } - - @Test - public void testQuantityMax() throws Exception { - - //create line item with correct item quantity - LineItem item = new LineItem(); - item.setQuantity(9999); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all item quantities are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with item quantity above allowed max - item = new LineItem(); - item.setQuantity(10000); - items.add(item); - - //verify result - should be false - second item has invalid item quantity - assertFalse((Boolean) function.doGetResult(null)); - } -} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateShippingPricesFunctionTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateShippingPricesFunctionTests.java deleted file mode 100644 index b7a3b5b803..0000000000 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateShippingPricesFunctionTests.java +++ /dev/null @@ -1,85 +0,0 @@ -package org.springframework.batch.sample.domain.order.internal.valang; - -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - -import java.math.BigDecimal; -import java.util.ArrayList; -import java.util.List; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.Function; - -public class ValidateShippingPricesFunctionTests { - - private ValidateShippingPricesFunction function; - private Function argument; - - @Before - public void setUp() { - - argument = mock(Function.class); - - //create function - function = new ValidateShippingPricesFunction(new Function[] {argument}, 0, 0); - - } - - @Test - public void testShippingPriceMin() throws Exception { - - //create line item with correct shipping price - LineItem item = new LineItem(); - item.setShippingPrice(new BigDecimal(1.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all shipping prices are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with negative shipping price - item = new LineItem(); - item.setShippingPrice(new BigDecimal(-1.0)); - items.add(item); - - //verify result - should be false - second item has invalid shipping price - assertFalse((Boolean) function.doGetResult(null)); - - } - - @Test - public void testShippingPriceMax() throws Exception { - - //create line item with correct shipping price - LineItem item = new LineItem(); - item.setShippingPrice(new BigDecimal(99999999.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all shipping prices are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with shipping price above allowed max - item = new LineItem(); - item.setShippingPrice(new BigDecimal(100000000.0)); - items.add(item); - - //verify result - should be false - second item has invalid shipping price - assertFalse((Boolean) function.doGetResult(null)); - - } -} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateTotalPricesFunctionTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateTotalPricesFunctionTests.java deleted file mode 100644 index 6267415000..0000000000 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/valang/ValidateTotalPricesFunctionTests.java +++ /dev/null @@ -1,138 +0,0 @@ -package org.springframework.batch.sample.domain.order.internal.valang; - -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - -import java.math.BigDecimal; -import java.util.ArrayList; -import java.util.List; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.sample.domain.order.LineItem; -import org.springmodules.validation.valang.functions.Function; - -public class ValidateTotalPricesFunctionTests { - - private ValidateTotalPricesFunction function; - private Function argument; - - @Before - public void setUp() { - - argument = mock(Function.class); - - //create function - function = new ValidateTotalPricesFunction(new Function[] {argument}, 0, 0); - - } - - @Test - public void testTotalPriceMin() throws Exception { - - //create line item with correct total price - LineItem item = new LineItem(); - item.setDiscountAmount(new BigDecimal(0.0)); - item.setDiscountPerc(new BigDecimal(0.0)); - item.setHandlingPrice(new BigDecimal(0.0)); - item.setShippingPrice(new BigDecimal(0.0)); - item.setPrice(new BigDecimal(1.0)); - item.setQuantity(1); - item.setTotalPrice(new BigDecimal(1.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all total prices are correct - assertTrue((Boolean) function.doGetResult(null)); - - //now add line item with negative item price - item = new LineItem(); - item.setTotalPrice(new BigDecimal(-1.0)); - items.add(item); - - //verify result - should be false - second item has invalid total price - assertFalse((Boolean) function.doGetResult(null)); - } - - @Test - public void testTotalPriceMax() throws Exception { - - //create line item with correct total price - LineItem item = new LineItem(); - item.setDiscountAmount(new BigDecimal(0.0)); - item.setDiscountPerc(new BigDecimal(0.0)); - item.setHandlingPrice(new BigDecimal(0.0)); - item.setShippingPrice(new BigDecimal(0.0)); - item.setPrice(new BigDecimal(99999999.0)); - item.setQuantity(1); - item.setTotalPrice(new BigDecimal(99999999.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all total prices are correct - assertEquals(true, function.doGetResult(null)); - - //now add line item with total price above allowed max - item = new LineItem(); - item.setTotalPrice(new BigDecimal(100000000.0)); - items.add(item); - - //verify result - should be false - second item has invalid total price - assertFalse((Boolean) function.doGetResult(null)); - - } - - @Test - public void testTotalPriceCalculation() throws Exception { - - //create line item - LineItem item = new LineItem(); - item.setDiscountAmount(new BigDecimal(5.0)); - item.setDiscountPerc(new BigDecimal(0.0)); - item.setHandlingPrice(new BigDecimal(1.0)); - item.setShippingPrice(new BigDecimal(2.0)); - item.setPrice(new BigDecimal(250.0)); - item.setQuantity(1); - item.setTotalPrice(new BigDecimal(248.0)); - - //add it to line items list - List items = new ArrayList(); - items.add(item); - - //set return value for mock argument - when(argument.getResult(null)).thenReturn(items); - - //verify result - should be true - all total prices are correct - assertEquals(true, function.doGetResult(null)); - - //now add line item with incorrect total price - item = new LineItem(); - item.setDiscountAmount(new BigDecimal(5.0)); - item.setDiscountPerc(new BigDecimal(0.0)); - item.setHandlingPrice(new BigDecimal(1.0)); - item.setShippingPrice(new BigDecimal(2.0)); - item.setPrice(new BigDecimal(250.0)); - item.setQuantity(1); - item.setTotalPrice(new BigDecimal(253.0)); - - items.add(item); - - //verify result - should be false - second item has incorrect total price - assertFalse((Boolean) function.doGetResult(null)); - - } - -} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/validator/OrderValidatorTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/validator/OrderValidatorTests.java new file mode 100644 index 0000000000..cc2d88f715 --- /dev/null +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/order/internal/validator/OrderValidatorTests.java @@ -0,0 +1,355 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.domain.order.internal.validator; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +import java.math.BigDecimal; +import java.util.ArrayList; +import java.util.Date; +import java.util.List; + +import org.junit.Before; +import org.junit.Test; +import org.springframework.batch.sample.domain.order.Address; +import org.springframework.batch.sample.domain.order.BillingInfo; +import org.springframework.batch.sample.domain.order.Customer; +import org.springframework.batch.sample.domain.order.LineItem; +import org.springframework.batch.sample.domain.order.Order; +import org.springframework.batch.sample.domain.order.ShippingInfo; +import org.springframework.validation.BeanPropertyBindingResult; +import org.springframework.validation.Errors; + +public class OrderValidatorTests { + + private OrderValidator orderValidator; + + @Before + public void setUp() throws Exception { + orderValidator = new OrderValidator(); + } + + @Test + public void testSupports() { + assertTrue(orderValidator.supports(Order.class)); + } + + @Test + public void testNotAnOrder() { + String notAnOrder = "order"; + Errors errors = new BeanPropertyBindingResult(notAnOrder, "validOrder"); + + orderValidator.validate(notAnOrder, errors); + + assertEquals(1, errors.getAllErrors().size()); + assertEquals("Incorrect type", errors.getAllErrors().get(0).getCode()); + + errors = new BeanPropertyBindingResult(notAnOrder, "validOrder"); + + orderValidator.validate(null, errors); + assertEquals(0, errors.getAllErrors().size()); + } + + @Test + public void testValidOrder() { + Order order = new Order(); + order.setOrderId(-5); + order.setOrderDate(new Date(new Date().getTime() + 1000000000L)); + order.setTotalLines(10); + order.setLineItems(new ArrayList<>()); + + Errors errors = new BeanPropertyBindingResult(order, "validOrder"); + + orderValidator.validateOrder(order, errors); + + assertEquals(3, errors.getAllErrors().size()); + assertEquals("error.order.id", errors.getFieldError("orderId").getCode()); + assertEquals("error.order.date.future", errors.getFieldError("orderDate").getCode()); + assertEquals("error.order.lines.badcount", errors.getFieldError("totalLines").getCode()); + + order = new Order(); + order.setOrderId(Long.MAX_VALUE); + order.setOrderDate(new Date(new Date().getTime() - 1000)); + order.setTotalLines(0); + List items = new ArrayList<>(); + items.add(new LineItem()); + order.setLineItems(items); + + errors = new BeanPropertyBindingResult(order, "validOrder"); + + orderValidator.validateOrder(order, errors); + + assertEquals(2, errors.getAllErrors().size()); + assertEquals("error.order.id", errors.getFieldError("orderId").getCode()); + assertEquals("error.order.lines.badcount", errors.getFieldError("totalLines").getCode()); + + order = new Order(); + order.setOrderId(5L); + order.setOrderDate(new Date(new Date().getTime() - 1000)); + order.setTotalLines(1); + items = new ArrayList<>(); + items.add(new LineItem()); + order.setLineItems(items); + + errors = new BeanPropertyBindingResult(order, "validOrder"); + + orderValidator.validateOrder(order, errors); + + assertEquals(0, errors.getAllErrors().size()); + } + + @Test + public void testValidCustomer() { + Order order = new Order(); + Customer customer = new Customer(); + customer.setRegistered(false); + customer.setBusinessCustomer(true); + order.setCustomer(customer); + + Errors errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validateCustomer(customer, errors); + + assertEquals(2, errors.getAllErrors().size()); + assertEquals("error.customer.registration", errors.getFieldError("customer.registered").getCode()); + assertEquals("error.customer.companyname", errors.getFieldError("customer.companyName").getCode()); + + customer = new Customer(); + customer.setRegistered(true); + customer.setBusinessCustomer(false); + customer.setRegistrationId(Long.MIN_VALUE); + order.setCustomer(customer); + + errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validateCustomer(customer, errors); + + assertEquals(3, errors.getAllErrors().size()); + assertEquals("error.customer.firstname", errors.getFieldError("customer.firstName").getCode()); + assertEquals("error.customer.lastname", errors.getFieldError("customer.lastName").getCode()); + assertEquals("error.customer.registrationid", errors.getFieldError("customer.registrationId").getCode()); + + customer = new Customer(); + customer.setRegistered(true); + customer.setBusinessCustomer(false); + customer.setRegistrationId(Long.MAX_VALUE); + order.setCustomer(customer); + + errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validateCustomer(customer, errors); + + assertEquals(3, errors.getAllErrors().size()); + assertEquals("error.customer.firstname", errors.getFieldError("customer.firstName").getCode()); + assertEquals("error.customer.lastname", errors.getFieldError("customer.lastName").getCode()); + assertEquals("error.customer.registrationid", errors.getFieldError("customer.registrationId").getCode()); + + customer = new Customer(); + customer.setRegistered(true); + customer.setBusinessCustomer(true); + customer.setCompanyName("Acme Inc"); + customer.setRegistrationId(5L); + order.setCustomer(customer); + + errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validateCustomer(customer, errors); + + assertEquals(0, errors.getAllErrors().size()); + + customer = new Customer(); + customer.setRegistered(true); + customer.setBusinessCustomer(false); + customer.setFirstName("John"); + customer.setLastName("Doe"); + customer.setRegistrationId(5L); + order.setCustomer(customer); + + errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validateCustomer(customer, errors); + + assertEquals(0, errors.getAllErrors().size()); + } + + @Test + public void testValidAddress() { + Order order = new Order(); + + Errors errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validateAddress(null, errors, "billingAddress"); + + assertEquals(0, errors.getAllErrors().size()); + + Address address = new Address(); + order.setBillingAddress(address); + + orderValidator.validateAddress(address, errors, "billingAddress"); + + assertEquals(4, errors.getAllErrors().size()); + assertEquals("error.baddress.addrline1.length", errors.getFieldError("billingAddress.addrLine1").getCode()); + assertEquals("error.baddress.city.length", errors.getFieldError("billingAddress.city").getCode()); + assertEquals("error.baddress.zipcode.length", errors.getFieldError("billingAddress.zipCode").getCode()); + assertEquals("error.baddress.country.length", errors.getFieldError("billingAddress.country").getCode()); + + address = new Address(); + address.setAddressee("1234567890123456789012345678901234567890123456789012345678901234567890"); + address.setAddrLine1("123456789012345678901234567890123456789012345678901234567890"); + address.setAddrLine2("123456789012345678901234567890123456789012345678901234567890"); + address.setCity("1234567890123456789012345678901234567890"); + address.setZipCode("1234567890"); + address.setState("1234567890"); + address.setCountry("123456789012345678901234567890123456789012345678901234567890"); + order.setBillingAddress(address); + + errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validateAddress(address, errors, "billingAddress"); + + assertEquals(8, errors.getAllErrors().size()); + assertEquals("error.baddress.addresse.length", errors.getFieldError("billingAddress.addressee").getCode()); + assertEquals("error.baddress.addrline1.length", errors.getFieldError("billingAddress.addrLine1").getCode()); + assertEquals("error.baddress.addrline2.length", errors.getFieldError("billingAddress.addrLine2").getCode()); + assertEquals("error.baddress.city.length", errors.getFieldError("billingAddress.city").getCode()); + assertEquals("error.baddress.state.length", errors.getFieldError("billingAddress.state").getCode()); + assertEquals("error.baddress.zipcode.length", errors.getFieldErrors("billingAddress.zipCode").get(0).getCode()); + assertEquals("error.baddress.zipcode.format", errors.getFieldErrors("billingAddress.zipCode").get(1).getCode()); + assertEquals("error.baddress.country.length", errors.getFieldError("billingAddress.country").getCode()); + + address = new Address(); + address.setAddressee("John Doe"); + address.setAddrLine1("123 4th Street"); + address.setCity("Chicago"); + address.setState("IL"); + address.setZipCode("60606"); + address.setCountry("United States"); + order.setBillingAddress(address); + + errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validateAddress(address, errors, "billingAddress"); + + assertEquals(0, errors.getAllErrors().size()); + } + + @Test + public void testValidPayment() { + Order order = new Order(); + BillingInfo info = new BillingInfo(); + info.setPaymentId("INVALID"); + info.setPaymentDesc("INVALID"); + order.setBilling(info); + + Errors errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validatePayment(info, errors); + + assertEquals(2, errors.getAllErrors().size()); + assertEquals("error.billing.type", errors.getFieldError("billing.paymentId").getCode()); + assertEquals("error.billing.desc", errors.getFieldError("billing.paymentDesc").getCode()); + + info = new BillingInfo(); + info.setPaymentId("VISA"); + info.setPaymentDesc("ADFI-1234567890"); + order.setBilling(info); + + errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validatePayment(info, errors); + + assertEquals(0, errors.getAllErrors().size()); + } + + @Test + public void testValidShipping() { + Order order = new Order(); + ShippingInfo info = new ShippingInfo(); + info.setShipperId("INVALID"); + info.setShippingTypeId("INVALID"); + order.setShipping(info); + + Errors errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validateShipping(info, errors); + + assertEquals(2, errors.getAllErrors().size()); + assertEquals("error.shipping.shipper", errors.getFieldError("shipping.shipperId").getCode()); + assertEquals("error.shipping.type", errors.getFieldError("shipping.shippingTypeId").getCode()); + + info = new ShippingInfo(); + info.setShipperId("FEDX"); + info.setShippingTypeId("EXP"); + info.setShippingInfo("12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890"); + order.setShipping(info); + + errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validateShipping(info, errors); + + assertEquals(1, errors.getAllErrors().size()); + assertEquals("error.shipping.shippinginfo.length", errors.getFieldError("shipping.shippingInfo").getCode()); + + info = new ShippingInfo(); + info.setShipperId("FEDX"); + info.setShippingTypeId("EXP"); + info.setShippingInfo("Info"); + order.setShipping(info); + + errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validateShipping(info, errors); + + assertEquals(0, errors.getAllErrors().size()); + } + + @Test + public void testValidLineItems() { + Order order = new Order(); + List lineItems = new ArrayList<>(); + lineItems.add(buildLineItem(-5, 5.00, 0, 0, 2, 3, 3, 30)); + lineItems.add(buildLineItem(Long.MAX_VALUE, 5.00, 0, 0, 2, 3, 3, 30)); + lineItems.add(buildLineItem(6, -5.00, 0, 0, 2, 3, 3, 0)); + lineItems.add(buildLineItem(6, Integer.MAX_VALUE, 0, 0, 2, 3, 3, 30)); + lineItems.add(buildLineItem(6, 5.00, 900, 0, 2, 3, 3, 30)); + lineItems.add(buildLineItem(6, 5.00, -90, 0, 2, 3, 3, 30)); + lineItems.add(buildLineItem(6, 5.00, 10, 20, 2, 3, 3, 30)); + lineItems.add(buildLineItem(6, 5.00, 0, -10, 2, 3, 3, 30)); + lineItems.add(buildLineItem(6, 5.00, 0, 50, 2, 3, 3, 30)); + lineItems.add(buildLineItem(6, 5.00, 0, 0, -2, 3, 3, 30)); + lineItems.add(buildLineItem(6, 5.00, 0, 0, Long.MAX_VALUE, 3, 3, 30)); + lineItems.add(buildLineItem(6, 5.00, 0, 0, 2, -3, 3, 30)); + lineItems.add(buildLineItem(6, 5.00, 0, 0, 2, Long.MAX_VALUE, 3, 30)); + lineItems.add(buildLineItem(6, 5.00, 0, 0, 2, 3, -3, 30)); + lineItems.add(buildLineItem(6, 5.00, 0, 0, 2, 3, Integer.MAX_VALUE, 30)); + lineItems.add(buildLineItem(6, 5.00, 0, 0, 2, 3, 3, -5)); + lineItems.add(buildLineItem(6, 5.00, 0, 0, 2, 3, 3, Integer.MAX_VALUE)); + order.setLineItems(lineItems); + + Errors errors = new BeanPropertyBindingResult(order, "validOrder"); + orderValidator.validateLineItems(lineItems, errors); + + assertEquals(7, errors.getAllErrors().size()); + assertEquals("error.lineitems.id", errors.getFieldErrors("lineItems").get(0).getCode()); + assertEquals("error.lineitems.price", errors.getFieldErrors("lineItems").get(1).getCode()); + assertEquals("error.lineitems.discount", errors.getFieldErrors("lineItems").get(2).getCode()); + assertEquals("error.lineitems.shipping", errors.getFieldErrors("lineItems").get(3).getCode()); + assertEquals("error.lineitems.handling", errors.getFieldErrors("lineItems").get(4).getCode()); + assertEquals("error.lineitems.quantity", errors.getFieldErrors("lineItems").get(5).getCode()); + assertEquals("error.lineitems.totalprice", errors.getFieldErrors("lineItems").get(6).getCode()); + } + + private LineItem buildLineItem(long itemId, double price, int discountPercentage, int discountAmount, long shippingPrice, long handlingPrice, int qty, int totalPrice) { + LineItem invalidId = new LineItem(); + invalidId.setItemId(itemId); + invalidId.setPrice(new BigDecimal(price)); + invalidId.setDiscountPerc(new BigDecimal(discountPercentage)); + invalidId.setDiscountAmount(new BigDecimal(discountAmount)); + invalidId.setShippingPrice(new BigDecimal(shippingPrice)); + invalidId.setHandlingPrice(new BigDecimal(handlingPrice)); + invalidId.setQuantity(qty); + invalidId.setTotalPrice(new BigDecimal(totalPrice)); + return invalidId; + } +} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/CompositeCustomerUpdateLineTokenizerTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/CompositeCustomerUpdateLineTokenizerTests.java index b44cf6f534..49f2da26af 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/CompositeCustomerUpdateLineTokenizerTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/CompositeCustomerUpdateLineTokenizerTests.java @@ -1,5 +1,17 @@ -/** - * +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ package org.springframework.batch.sample.domain.trade; @@ -10,31 +22,28 @@ import org.springframework.batch.item.file.transform.DefaultFieldSet; import org.springframework.batch.item.file.transform.FieldSet; import org.springframework.batch.item.file.transform.LineTokenizer; +import org.springframework.lang.Nullable; /** * @author Lucas Ward * */ public class CompositeCustomerUpdateLineTokenizerTests { - - StubLineTokenizer customerTokenizer; - FieldSet customerFieldSet = new DefaultFieldSet(null); - StubLineTokenizer footerTokenizer; - FieldSet footerFieldSet = new DefaultFieldSet(null); - CompositeCustomerUpdateLineTokenizer compositeTokenizer; + private StubLineTokenizer customerTokenizer; + private FieldSet customerFieldSet = new DefaultFieldSet(null); + private FieldSet footerFieldSet = new DefaultFieldSet(null); + private CompositeCustomerUpdateLineTokenizer compositeTokenizer; @Before public void init(){ customerTokenizer = new StubLineTokenizer(customerFieldSet); - footerTokenizer = new StubLineTokenizer(footerFieldSet); compositeTokenizer = new CompositeCustomerUpdateLineTokenizer(); compositeTokenizer.setCustomerTokenizer(customerTokenizer); - compositeTokenizer.setFooterTokenizer(footerTokenizer); + compositeTokenizer.setFooterTokenizer(new StubLineTokenizer(footerFieldSet)); } @Test public void testCustomerAdd() throws Exception{ - String customerAddLine = "AFDASFDASFDFSA"; FieldSet fs = compositeTokenizer.tokenize(customerAddLine); assertEquals(customerFieldSet, fs); @@ -43,7 +52,6 @@ public void testCustomerAdd() throws Exception{ @Test public void testCustomerDelete() throws Exception{ - String customerAddLine = "DFDASFDASFDFSA"; FieldSet fs = compositeTokenizer.tokenize(customerAddLine); assertEquals(customerFieldSet, fs); @@ -52,7 +60,6 @@ public void testCustomerDelete() throws Exception{ @Test public void testCustomerUpdate() throws Exception{ - String customerAddLine = "UFDASFDASFDFSA"; FieldSet fs = compositeTokenizer.tokenize(customerAddLine); assertEquals(customerFieldSet, fs); @@ -61,22 +68,20 @@ public void testCustomerUpdate() throws Exception{ @Test(expected=IllegalArgumentException.class) public void testInvalidLine() throws Exception{ - String invalidLine = "INVALID"; compositeTokenizer.tokenize(invalidLine); } - - - private static class StubLineTokenizer implements LineTokenizer{ + private static class StubLineTokenizer implements LineTokenizer{ private final FieldSet fieldSetToReturn; private String tokenizedLine; public StubLineTokenizer(FieldSet fieldSetToReturn) { this.fieldSetToReturn = fieldSetToReturn; } - - public FieldSet tokenize(String line) { + + @Override + public FieldSet tokenize(@Nullable String line) { this.tokenizedLine = line; return fieldSetToReturn; } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/CustomerUpdateProcessorTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/CustomerUpdateProcessorTests.java index 0a699bfe6d..250f24d6c1 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/CustomerUpdateProcessorTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/CustomerUpdateProcessorTests.java @@ -1,5 +1,17 @@ -/** - * +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ package org.springframework.batch.sample.domain.trade; @@ -21,10 +33,9 @@ * */ public class CustomerUpdateProcessorTests { - - CustomerDao customerDao; - InvalidCustomerLogger logger; - CustomerUpdateProcessor processor; + private CustomerDao customerDao; + private InvalidCustomerLogger logger; + private CustomerUpdateProcessor processor; @Before public void init(){ @@ -37,16 +48,14 @@ public void init(){ @Test public void testSuccessfulAdd() throws Exception{ - - CustomerUpdate customerUpdate = new CustomerUpdate(ADD, "test customer", new BigDecimal(232.2)); + CustomerUpdate customerUpdate = new CustomerUpdate(ADD, "test customer", new BigDecimal("232.2")); when(customerDao.getCustomerByName("test customer")).thenReturn(null); assertEquals(customerUpdate, processor.process(customerUpdate)); } @Test public void testInvalidAdd() throws Exception{ - - CustomerUpdate customerUpdate = new CustomerUpdate(ADD, "test customer", new BigDecimal(232.2)); + CustomerUpdate customerUpdate = new CustomerUpdate(ADD, "test customer", new BigDecimal("232.2")); when(customerDao.getCustomerByName("test customer")).thenReturn(new CustomerCredit()); logger.log(customerUpdate); assertNull("Processor should return null", processor.process(customerUpdate)); @@ -54,27 +63,23 @@ public void testInvalidAdd() throws Exception{ @Test public void testDelete() throws Exception{ - //delete should never work, therefore, ensure it fails fast. - CustomerUpdate customerUpdate = new CustomerUpdate(DELETE, "test customer", new BigDecimal(232.2)); + CustomerUpdate customerUpdate = new CustomerUpdate(DELETE, "test customer", new BigDecimal("232.2")); logger.log(customerUpdate); assertNull("Processor should return null", processor.process(customerUpdate)); } @Test public void testSuccessfulUpdate() throws Exception{ - - CustomerUpdate customerUpdate = new CustomerUpdate(UPDATE, "test customer", new BigDecimal(232.2)); + CustomerUpdate customerUpdate = new CustomerUpdate(UPDATE, "test customer", new BigDecimal("232.2")); when(customerDao.getCustomerByName("test customer")).thenReturn(new CustomerCredit()); assertEquals(customerUpdate, processor.process(customerUpdate)); } @Test public void testInvalidUpdate() throws Exception{ - - CustomerUpdate customerUpdate = new CustomerUpdate(UPDATE, "test customer", new BigDecimal(232.2)); + CustomerUpdate customerUpdate = new CustomerUpdate(UPDATE, "test customer", new BigDecimal("232.2")); when(customerDao.getCustomerByName("test customer")).thenReturn(null); logger.log(customerUpdate); assertNull("Processor should return null", processor.process(customerUpdate)); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/TradeTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/TradeTests.java index 91b4a2bd03..aba24b005e 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/TradeTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/TradeTests.java @@ -1,19 +1,33 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.trade; -import static org.junit.Assert.*; - import java.math.BigDecimal; import org.junit.Test; -public class TradeTests { +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +public class TradeTests { @Test public void testEquality(){ - - Trade trade1 = new Trade("isin", 1, new BigDecimal(1.1), "customer1"); - Trade trade1Clone = new Trade("isin", 1, new BigDecimal(1.1), "customer1"); - Trade trade2 = new Trade("isin", 1, new BigDecimal(2.3), "customer2"); + Trade trade1 = new Trade("isin", 1, new BigDecimal("1.1"), "customer1"); + Trade trade1Clone = new Trade("isin", 1, new BigDecimal("1.1"), "customer1"); + Trade trade2 = new Trade("isin", 1, new BigDecimal("2.3"), "customer2"); assertEquals(trade1, trade1Clone); assertFalse(trade1.equals(trade2)); diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditIncreaseProcessorTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditIncreaseProcessorTests.java index af9832f80a..1d643c3cfa 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditIncreaseProcessorTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditIncreaseProcessorTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.trade.internal; import static org.junit.Assert.assertEquals; @@ -8,12 +23,11 @@ import org.springframework.batch.sample.domain.trade.CustomerCredit; /** - * Tests for {@link CustomerCreditItemWriter}. + * Tests for {@link CustomerCreditIncreaseProcessor}. * * @author Robert Kasanicky */ public class CustomerCreditIncreaseProcessorTests { - private CustomerCreditIncreaseProcessor tested = new CustomerCreditIncreaseProcessor(); /* @@ -21,8 +35,7 @@ public class CustomerCreditIncreaseProcessorTests { */ @Test public void testProcess() throws Exception { - - final BigDecimal oldCredit = new BigDecimal(10.54); + final BigDecimal oldCredit = new BigDecimal("10.54"); CustomerCredit customerCredit = new CustomerCredit(); customerCredit.setCredit(oldCredit); diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditRowMapperTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditRowMapperTests.java index cb1e0a1a8a..70db7ebc05 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditRowMapperTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditRowMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.trade.internal; import static org.mockito.Mockito.when; @@ -10,16 +25,14 @@ import org.springframework.batch.sample.support.AbstractRowMapperTests; import org.springframework.jdbc.core.RowMapper; -public class CustomerCreditRowMapperTests extends AbstractRowMapperTests { +public class CustomerCreditRowMapperTests extends AbstractRowMapperTests { - /** - * - */ private static final int ID = 12; private static final String CUSTOMER = "Jozef Mak"; - private static final BigDecimal CREDIT = new BigDecimal(0.1); + private static final BigDecimal CREDIT = new BigDecimal("0.1"); - protected Object expectedDomainObject() { + @Override + protected CustomerCredit expectedDomainObject() { CustomerCredit credit = new CustomerCredit(); credit.setId(ID); credit.setCredit(CREDIT); @@ -27,14 +40,15 @@ protected Object expectedDomainObject() { return credit; } - protected RowMapper rowMapper() { + @Override + protected RowMapper rowMapper() { return new CustomerCreditRowMapper(); } + @Override protected void setUpResultSetMock(ResultSet rs) throws SQLException { when(rs.getInt(CustomerCreditRowMapper.ID_COLUMN)).thenReturn(ID); when(rs.getString(CustomerCreditRowMapper.NAME_COLUMN)).thenReturn(CUSTOMER); when(rs.getBigDecimal(CustomerCreditRowMapper.CREDIT_COLUMN)).thenReturn(CREDIT); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdatePreparedStatementSetterTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdatePreparedStatementSetterTests.java index 5c072bc266..b73ef42225 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdatePreparedStatementSetterTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdatePreparedStatementSetterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdateProcessorTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdateProcessorTests.java index c533c47b94..4b6b409ba3 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdateProcessorTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerCreditUpdateProcessorTests.java @@ -1,7 +1,21 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.trade.internal; import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; import java.math.BigDecimal; import java.util.Collections; @@ -12,16 +26,14 @@ import org.springframework.batch.sample.domain.trade.CustomerCreditDao; public class CustomerCreditUpdateProcessorTests { - private CustomerCreditDao dao; private CustomerCreditUpdateWriter writer; private static final double CREDIT_FILTER = 355.0; @Before public void setUp() { - //create mock writer dao = mock(CustomerCreditDao.class); - //create processor, set writer and credit filter + writer = new CustomerCreditUpdateWriter(); writer.setDao(dao); writer.setCreditFilter(CREDIT_FILTER); @@ -29,25 +41,15 @@ public void setUp() { @Test public void testProcess() throws Exception { - - //set-up mock writer - no writer's method should be called - - //create credit and set it to same value as credit filter CustomerCredit credit = new CustomerCredit(); credit.setCredit(new BigDecimal(CREDIT_FILTER)); - //call tested method + writer.write(Collections.singletonList(credit)); - //verify method calls - no method should be called - //because credit is not greater then credit filter - - //change credit to be greater than credit filter + credit.setCredit(new BigDecimal(CREDIT_FILTER + 1)); - //reset and set-up writer - write method is expected to be called + dao.writeCredit(credit); - //call tested method writer.write(Collections.singletonList(credit)); - } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerUpdateProcessorTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerUpdateProcessorTests.java index 5660dbe684..e03ef59f35 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerUpdateProcessorTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/CustomerUpdateProcessorTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.trade.internal; import static org.junit.Assert.assertEquals; @@ -11,28 +26,23 @@ import org.springframework.batch.sample.domain.trade.Trade; public class CustomerUpdateProcessorTests { - @Test public void testProcess() { - - //create trade object Trade trade = new Trade(); trade.setCustomer("testCustomerName"); - trade.setPrice(new BigDecimal(123.0)); + trade.setPrice(new BigDecimal("123.0")); - //create dao CustomerDebitDao dao = new CustomerDebitDao() { + @Override public void write(CustomerDebit customerDebit) { assertEquals("testCustomerName", customerDebit.getName()); - assertEquals(new BigDecimal(123.0), customerDebit.getDebit()); + assertEquals(new BigDecimal("123.0"), customerDebit.getDebit()); } }; - //create processor and set dao CustomerUpdateWriter processor = new CustomerUpdateWriter(); processor.setDao(dao); - //call tested method - see asserts in dao.write() method processor.write(Collections.singletonList(trade)); } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/FlatFileCustomerCreditDaoTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/FlatFileCustomerCreditDaoTests.java index 265ea6c608..f33ef301e0 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/FlatFileCustomerCreditDaoTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/FlatFileCustomerCreditDaoTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -28,17 +28,13 @@ import org.springframework.batch.sample.domain.trade.CustomerCredit; public class FlatFileCustomerCreditDaoTests { - private ResourceLifecycleItemWriter output; private FlatFileCustomerCreditDao writer; @Before public void setUp() throws Exception { - - //create mock for OutputSource output = mock(ResourceLifecycleItemWriter.class); - //create new writer writer = new FlatFileCustomerCreditDao(); writer.setItemWriter(output); } @@ -46,45 +42,34 @@ public void setUp() throws Exception { @Test public void testOpen() throws Exception { ExecutionContext executionContext = new ExecutionContext(); - //set-up outputSource mock + output.open(executionContext); - //call tested method writer.open(executionContext); - } @Test public void testClose() throws Exception{ - - //set-up outputSource mock output.close(); - //call tested method writer.close(); - } @Test public void testWrite() throws Exception { - - //Create and set-up CustomerCredit CustomerCredit credit = new CustomerCredit(); credit.setCredit(new BigDecimal(1)); credit.setName("testName"); - //set separator writer.setSeparator(";"); - //set-up OutputSource mock output.write(Collections.singletonList("testName;1")); output.open(new ExecutionContext()); - //call tested method writer.writeCredit(credit); } - private interface ResourceLifecycleItemWriter extends ItemWriter, ItemStream{ - + private interface ResourceLifecycleItemWriter extends ItemWriter, ItemStream { + } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/GeneratingItemReaderTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/GeneratingItemReaderTests.java index 489b5a79a8..24f7778c0b 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/GeneratingItemReaderTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/GeneratingItemReaderTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.trade.internal; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/ItemTrackingTradeItemWriter.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/ItemTrackingTradeItemWriter.java index 992e2fc36c..f5c870b7cd 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/ItemTrackingTradeItemWriter.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/ItemTrackingTradeItemWriter.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.trade.internal; import java.io.IOException; @@ -12,11 +27,8 @@ import org.springframework.jdbc.core.JdbcTemplate; public class ItemTrackingTradeItemWriter implements ItemWriter { - - private List items = new ArrayList(); - + private List items = new ArrayList<>(); private String writeFailureISIN; - private JdbcOperations jdbcTemplate; public void setDataSource(DataSource dataSource) { @@ -27,30 +39,27 @@ public void setWriteFailureISIN(String writeFailureISIN) { this.writeFailureISIN = writeFailureISIN; } - public void setItems(List items) { - this.items = items; - } - public List getItems() { return items; } - public void clearItems() { - this.items.clear(); - } - + @Override public void write(List items) throws Exception { - List newItems = new ArrayList(); + List newItems = new ArrayList<>(); + for (Trade t : items) { if (t.getIsin().equals(this.writeFailureISIN)) { throw new IOException("write failed"); } + newItems.add(t); + if (jdbcTemplate != null) { jdbcTemplate.update("UPDATE TRADE set VERSION=? where ID=? and version=?", t.getVersion() + 1, t .getId(), t.getVersion()); } } + this.items.addAll(newItems); } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDaoTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDaoTests.java index 794e496b42..084a86fddd 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDaoTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDaoTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -37,7 +37,6 @@ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration() public class JdbcCustomerDebitDaoTests { - private JdbcOperations jdbcTemplate; @Autowired @@ -48,27 +47,23 @@ public void setDataSource(DataSource dataSource) { this.jdbcTemplate = new JdbcTemplate(dataSource); } - @Transactional @Test + @Test + @Transactional public void testWrite() { - - //insert customer credit jdbcTemplate.execute("INSERT INTO CUSTOMER VALUES (99, 0, 'testName', 100)"); - //create customer debit CustomerDebit customerDebit = new CustomerDebit(); customerDebit.setName("testName"); customerDebit.setDebit(BigDecimal.valueOf(5)); - //call writer writer.write(customerDebit); - //verify customer credit jdbcTemplate.query("SELECT name, credit FROM CUSTOMER WHERE name = 'testName'", new RowCallbackHandler() { + @Override public void processRow(ResultSet rs) throws SQLException { assertEquals(95, rs.getLong("credit")); } }); - } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/JdbcTradeWriterTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/JdbcTradeWriterTests.java index d1ce736cc8..7ad44436b2 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/JdbcTradeWriterTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/JdbcTradeWriterTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -26,6 +26,7 @@ import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.batch.sample.domain.trade.Trade; +import org.springframework.beans.factory.InitializingBean; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.jdbc.core.JdbcOperations; @@ -38,38 +39,37 @@ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = {"/data-source-context.xml"}) -public class JdbcTradeWriterTests { - +public class JdbcTradeWriterTests implements InitializingBean { private JdbcOperations jdbcTemplate; - private JdbcTradeDao writer; + private AbstractDataFieldMaxValueIncrementer incrementer; @Autowired public void setDataSource(DataSource dataSource) { this.jdbcTemplate = new JdbcTemplate(dataSource); this.writer = new JdbcTradeDao(); this.writer.setDataSource(dataSource); - } @Autowired public void setIncrementer(@Qualifier("incrementerParent") AbstractDataFieldMaxValueIncrementer incrementer) { incrementer.setIncrementerName("TRADE_SEQ"); - this.writer.setIncrementer(incrementer); + this.incrementer = incrementer; } - @Transactional @Test + @Test + @Transactional public void testWrite() { - Trade trade = new Trade(); trade.setCustomer("testCustomer"); trade.setIsin("5647238492"); - trade.setPrice(new BigDecimal(Double.toString(99.69))); + trade.setPrice(new BigDecimal("99.69")); trade.setQuantity(5); writer.writeTrade(trade); - jdbcTemplate.query("SELECT * FROM TRADE WHERE ISIN = '5647238492'", new RowCallbackHandler() { + jdbcTemplate.query("SELECT * FROM TRADE WHERE ISIN = '5647238492'", new RowCallbackHandler() { + @Override public void processRow(ResultSet rs) throws SQLException { assertEquals("testCustomer", rs.getString("CUSTOMER")); assertEquals(new BigDecimal(Double.toString(99.69)), rs.getBigDecimal("PRICE")); @@ -77,4 +77,9 @@ public void processRow(ResultSet rs) throws SQLException { } }); } + + @Override + public void afterPropertiesSet() throws Exception { + this.writer.setIncrementer(incrementer); + } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/TradeFieldSetMapperTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/TradeFieldSetMapperTests.java index 780762acdf..7508fd6754 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/TradeFieldSetMapperTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/TradeFieldSetMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.trade.internal; import java.math.BigDecimal; @@ -9,15 +24,12 @@ import org.springframework.batch.sample.support.AbstractFieldSetMapperTests; public class TradeFieldSetMapperTests extends AbstractFieldSetMapperTests { - private static final String CUSTOMER = "Mike Tomcat"; - private static final BigDecimal PRICE = new BigDecimal(1.3); - private static final long QUANTITY = 7; - private static final String ISIN = "fj893gnsalX"; + @Override protected Object expectedDomainObject() { Trade trade = new Trade(); trade.setIsin(ISIN); @@ -27,6 +39,7 @@ protected Object expectedDomainObject() { return trade; } + @Override protected FieldSet fieldSet() { String[] tokens = new String[4]; tokens[TradeFieldSetMapper.ISIN_COLUMN] = ISIN; @@ -37,8 +50,8 @@ protected FieldSet fieldSet() { return new DefaultFieldSet(tokens); } + @Override protected FieldSetMapper fieldSetMapper() { return new TradeFieldSetMapper(); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/TradeProcessorTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/TradeProcessorTests.java index a42ae2cef4..efb82828ee 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/TradeProcessorTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/TradeProcessorTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.trade.internal; import static org.mockito.Mockito.mock; @@ -10,30 +25,23 @@ import org.springframework.batch.sample.domain.trade.TradeDao; public class TradeProcessorTests { - private TradeDao writer; private TradeWriter processor; @Before public void setUp() { - - //create mock writer writer = mock(TradeDao.class); - //create processor processor = new TradeWriter(); processor.setDao(writer); } @Test public void testProcess() { - Trade trade = new Trade(); - //set-up mock writer + writer.writeTrade(trade); - //call tested method processor.write(Collections.singletonList(trade)); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/TradeRowMapperTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/TradeRowMapperTests.java index 5d896af4ac..05a96f1778 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/TradeRowMapperTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/domain/trade/internal/TradeRowMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.domain.trade.internal; import static org.mockito.Mockito.when; @@ -10,26 +25,30 @@ import org.springframework.batch.sample.support.AbstractRowMapperTests; import org.springframework.jdbc.core.RowMapper; -public class TradeRowMapperTests extends AbstractRowMapperTests { +public class TradeRowMapperTests extends AbstractRowMapperTests { private static final String ISIN = "jsgk342"; private static final long QUANTITY = 0; - private static final BigDecimal PRICE = new BigDecimal(1.1); + private static final BigDecimal PRICE = new BigDecimal("1.1"); private static final String CUSTOMER = "Martin Hrancok"; - protected Object expectedDomainObject() { + @Override + protected Trade expectedDomainObject() { Trade trade = new Trade(); trade.setIsin(ISIN); trade.setQuantity(QUANTITY); trade.setPrice(PRICE); trade.setCustomer(CUSTOMER); + return trade; } - protected RowMapper rowMapper() { + @Override + protected RowMapper rowMapper() { return new TradeRowMapper(); } + @Override protected void setUpResultSetMock(ResultSet rs) throws SQLException { when(rs.getLong(TradeRowMapper.ID_COLUMN)).thenReturn(12L); when(rs.getString(TradeRowMapper.ISIN_COLUMN)).thenReturn(ISIN); @@ -38,5 +57,4 @@ protected void setUpResultSetMock(ResultSet rs) throws SQLException { when(rs.getString(TradeRowMapper.CUSTOMER_COLUMN)).thenReturn(CUSTOMER); when(rs.getInt(TradeRowMapper.VERSION_COLUMN)).thenReturn(0); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/AbstractIoSampleTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/AbstractIoSampleTests.java index 09239912b4..1db3050b00 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/AbstractIoSampleTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/AbstractIoSampleTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.iosample; import static org.junit.Assert.assertEquals; @@ -88,7 +103,7 @@ protected JobParameters getUniqueJobParameters() { */ private List getCredits(ItemReader reader) throws Exception { CustomerCredit credit; - List result = new ArrayList(); + List result = new ArrayList<>(); while ((credit = reader.read()) != null) { result.add(credit); } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/DelimitedFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/DelimitedFunctionalTests.java index 4896032685..072e888810 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/DelimitedFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/DelimitedFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -39,7 +39,7 @@ public class DelimitedFunctionalTests extends AbstractIoSampleTests { @Override protected void pointReaderToOutput(ItemReader reader) { JobParameters jobParameters = new JobParametersBuilder(super.getUniqueJobParameters()).addString("inputFile", - "file:./target/test-outputs/delimitedOutput.csv").toJobParameters(); + "file:./build/test-outputs/delimitedOutput.csv").toJobParameters(); StepExecution stepExecution = MetaDataInstanceFactory.createStepExecution(jobParameters); StepSynchronizationManager.close(); StepSynchronizationManager.register(stepExecution); @@ -49,7 +49,7 @@ protected void pointReaderToOutput(ItemReader reader) { protected JobParameters getUniqueJobParameters() { return new JobParametersBuilder(super.getUniqueJobParameters()).addString("inputFile", "data/iosample/input/delimited.csv").addString("outputFile", - "file:./target/test-outputs/delimitedOutput.csv").toJobParameters(); + "file:./build/test-outputs/delimitedOutput.csv").toJobParameters(); } } \ No newline at end of file diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/FixedLengthFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/FixedLengthFunctionalTests.java index 3755548227..30b2e7e561 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/FixedLengthFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/FixedLengthFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,7 +34,7 @@ public class FixedLengthFunctionalTests extends AbstractIoSampleTests { @Override protected void pointReaderToOutput(ItemReader reader) { JobParameters jobParameters = new JobParametersBuilder(super.getUniqueJobParameters()).addString("inputFile", - "file:./target/test-outputs/fixedLengthOutput.txt").toJobParameters(); + "file:./build/test-outputs/fixedLengthOutput.txt").toJobParameters(); StepExecution stepExecution = MetaDataInstanceFactory.createStepExecution(jobParameters); StepSynchronizationManager.close(); StepSynchronizationManager.register(stepExecution); @@ -44,7 +44,7 @@ protected void pointReaderToOutput(ItemReader reader) { protected JobParameters getUniqueJobParameters() { return new JobParametersBuilder(super.getUniqueJobParameters()).addString("inputFile", "data/iosample/input/fixedLength.txt").addString("outputFile", - "file:./target/test-outputs/fixedLengthOutput.txt").toJobParameters(); + "file:./build/test-outputs/fixedLengthOutput.txt").toJobParameters(); } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/HibernateFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/HibernateFunctionalTests.java index 2d3a3e4bee..4f0b13be0d 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/HibernateFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/HibernateFunctionalTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.iosample; import org.junit.runner.RunWith; diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/IbatisFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/IbatisFunctionalTests.java deleted file mode 100644 index 8a69722bd6..0000000000 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/IbatisFunctionalTests.java +++ /dev/null @@ -1,18 +0,0 @@ -package org.springframework.batch.sample.iosample; - -import org.junit.runner.RunWith; -import org.springframework.batch.item.ItemReader; -import org.springframework.batch.sample.domain.trade.CustomerCredit; -import org.springframework.test.context.ContextConfiguration; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; - -@RunWith(SpringJUnit4ClassRunner.class) -@ContextConfiguration(locations = "/jobs/iosample/ibatis.xml") -public class IbatisFunctionalTests extends AbstractIoSampleTests { - - @Override - protected void pointReaderToOutput(ItemReader reader) { - // no-op - } - -} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/JdbcCursorFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/JdbcCursorFunctionalTests.java index d808d9a670..72b3d9766f 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/JdbcCursorFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/JdbcCursorFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/JdbcPagingFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/JdbcPagingFunctionalTests.java index 57d57ae6e6..ca2c3da51e 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/JdbcPagingFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/JdbcPagingFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/JpaFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/JpaFunctionalTests.java index e454e02e0e..fd87575bc9 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/JpaFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/JpaFunctionalTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.iosample; import org.junit.runner.RunWith; diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/MultiLineFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/MultiLineFunctionalTests.java index bec139e0d7..0277ef2a7e 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/MultiLineFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/MultiLineFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,7 +34,7 @@ "/job-runner-context.xml" }) public class MultiLineFunctionalTests { - private static final String OUTPUT_FILE = "target/test-outputs/multiLineOutput.txt"; + private static final String OUTPUT_FILE = "build/test-outputs/multiLineOutput.txt"; private static final String INPUT_FILE = "src/main/resources/data/iosample/input/multiLine.txt"; diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/MultiRecordTypeFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/MultiRecordTypeFunctionalTests.java index eb47858502..75ffc01fbe 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/MultiRecordTypeFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/MultiRecordTypeFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -34,7 +34,7 @@ "/job-runner-context.xml" }) public class MultiRecordTypeFunctionalTests { - private static final String OUTPUT_FILE = "target/test-outputs/multiRecordTypeOutput.txt"; + private static final String OUTPUT_FILE = "build/test-outputs/multiRecordTypeOutput.txt"; private static final String INPUT_FILE = "src/main/resources/data/iosample/input/multiRecordType.txt"; diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/MultiResourceFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/MultiResourceFunctionalTests.java index 120658ffbf..a1776ca12a 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/MultiResourceFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/MultiResourceFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -38,7 +38,7 @@ public class MultiResourceFunctionalTests extends AbstractIoSampleTests { @Override protected void pointReaderToOutput(ItemReader reader) { JobParameters jobParameters = new JobParametersBuilder(super.getUniqueJobParameters()).addString( - "input.file.path", "file:target/test-outputs/multiResourceOutput.csv.*").toJobParameters(); + "input.file.path", "file:build/test-outputs/multiResourceOutput.csv.*").toJobParameters(); StepExecution stepExecution = MetaDataInstanceFactory.createStepExecution(jobParameters); StepSynchronizationManager.close(); StepSynchronizationManager.register(stepExecution); @@ -48,7 +48,7 @@ protected void pointReaderToOutput(ItemReader reader) { protected JobParameters getUniqueJobParameters() { JobParametersBuilder builder = new JobParametersBuilder(super.getUniqueJobParameters()); return builder.addString("input.file.path", "classpath:data/iosample/input/delimited*.csv").addString( - "output.file.path", "file:target/test-outputs/multiResourceOutput.csv").toJobParameters(); + "output.file.path", "file:build/test-outputs/multiResourceOutput.csv").toJobParameters(); } } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/RepositoryFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/RepositoryFunctionalTests.java index 4bf4f32165..3d53f32a92 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/RepositoryFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/RepositoryFunctionalTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.iosample; import org.junit.runner.RunWith; diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/TwoJobInstancesDelimitedFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/TwoJobInstancesDelimitedFunctionalTests.java index 9e871ba6b2..198aa56887 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/TwoJobInstancesDelimitedFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/TwoJobInstancesDelimitedFunctionalTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -49,7 +49,6 @@ @ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/ioSampleJob.xml", "/jobs/iosample/delimited.xml" }) public class TwoJobInstancesDelimitedFunctionalTests { - @Autowired private JobLauncher launcher; @@ -74,15 +73,13 @@ public void testLaunchJobTwice() throws Exception { } private void verifyOutput(int expected) throws Exception { - JobParameters jobParameters = new JobParametersBuilder().addString("inputFile", - "file:./target/test-outputs/delimitedOutput.csv").toJobParameters(); + "file:./build/test-outputs/delimitedOutput.csv").toJobParameters(); StepExecution stepExecution = MetaDataInstanceFactory.createStepExecution(jobParameters); int count = StepScopeTestUtils.doInStepScope(stepExecution, new Callable() { - + @Override public Integer call() throws Exception { - int count = 0; readerStream.open(new ExecutionContext()); @@ -96,18 +93,14 @@ public Integer call() throws Exception { readerStream.close(); } return count; - } - }); assertEquals(expected, count); - } protected JobParameters getJobParameters(String fileName) { return new JobParametersBuilder().addLong("timestamp", new Date().getTime()).addString("inputFile", fileName) - .addString("outputFile", "file:./target/test-outputs/delimitedOutput.csv").toJobParameters(); + .addString("outputFile", "file:./build/test-outputs/delimitedOutput.csv").toJobParameters(); } - -} \ No newline at end of file +} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/TwoJobInstancesPagingFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/TwoJobInstancesPagingFunctionalTests.java index 8b0d7cef37..e261267a09 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/TwoJobInstancesPagingFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/TwoJobInstancesPagingFunctionalTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2012 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -45,7 +45,6 @@ @ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/ioSampleJob.xml", "/jobs/iosample/jdbcPaging.xml" }) public class TwoJobInstancesPagingFunctionalTests { - @Autowired private JobLauncher launcher; @@ -61,11 +60,11 @@ public void setDataSource(DataSource dataSource) { @Test public void testLaunchJobTwice() throws Exception { - int first = jdbcTemplate.queryForInt("select count(0) from CUSTOMER where credit>1000"); + int first = jdbcTemplate.queryForObject("select count(0) from CUSTOMER where credit>1000", Integer.class); JobExecution jobExecution = launcher.run(this.job, getJobParameters(1000.)); assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); assertEquals(first, jobExecution.getStepExecutions().iterator().next().getWriteCount()); - int second = jdbcTemplate.queryForInt("select count(0) from CUSTOMER where credit>1000000"); + int second = jdbcTemplate.queryForObject("select count(0) from CUSTOMER where credit>1000000", Integer.class); assertNotSame("The number of records above the threshold did not change", first, second); jobExecution = launcher.run(this.job, getJobParameters(1000000.)); assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); @@ -76,5 +75,4 @@ protected JobParameters getJobParameters(double amount) { return new JobParametersBuilder().addLong("timestamp", new Date().getTime()).addDouble("credit", amount) .toJobParameters(); } - -} \ No newline at end of file +} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/XmlFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/XmlFunctionalTests.java index 6aae8c449a..9dbae1350f 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/XmlFunctionalTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/XmlFunctionalTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/DelegatingTradeLineAggregator.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/DelegatingTradeLineAggregator.java index 2ee5cd3df7..422e7158c9 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/DelegatingTradeLineAggregator.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/DelegatingTradeLineAggregator.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -28,6 +28,7 @@ public class DelegatingTradeLineAggregator implements LineAggregator { private LineAggregator tradeLineAggregator; private LineAggregator customerLineAggregator; + @Override public String aggregate(Object item) { if (item instanceof Trade) { return this.tradeLineAggregator.aggregate((Trade) item); diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/MultiLineTradeItemReader.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/MultiLineTradeItemReader.java index 9d4ad95a95..b040bdf567 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/MultiLineTradeItemReader.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/MultiLineTradeItemReader.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -23,6 +23,7 @@ import org.springframework.batch.item.file.FlatFileItemReader; import org.springframework.batch.item.file.transform.FieldSet; import org.springframework.batch.sample.domain.trade.Trade; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -30,16 +31,17 @@ * @since 2.0 */ public class MultiLineTradeItemReader implements ItemReader, ItemStream { - private FlatFileItemReader
        delegate; /** * @see org.springframework.batch.item.ItemReader#read() */ + @Nullable + @Override public Trade read() throws Exception { Trade t = null; - for (FieldSet line = null; (line = this.delegate.read()) != null;) { + for (FieldSet line; (line = this.delegate.read()) != null;) { String prefix = line.readString(0); if (prefix.equals("BEGIN")) { t = new Trade(); // Record must start with 'BEGIN' @@ -66,14 +68,17 @@ public void setDelegate(FlatFileItemReader
        delegate) { this.delegate = delegate; } + @Override public void close() throws ItemStreamException { this.delegate.close(); } + @Override public void open(ExecutionContext executionContext) throws ItemStreamException { this.delegate.open(executionContext); } + @Override public void update(ExecutionContext executionContext) throws ItemStreamException { this.delegate.update(executionContext); } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/MultiLineTradeItemWriter.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/MultiLineTradeItemWriter.java index b82df72f83..86ee192192 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/MultiLineTradeItemWriter.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/MultiLineTradeItemWriter.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,17 +31,19 @@ * @since 2.0 */ public class MultiLineTradeItemWriter implements ItemWriter, ItemStream { - private FlatFileItemWriter delegate; + @Override public void write(List items) throws Exception { - List lines = new ArrayList(); + List lines = new ArrayList<>(); + for (Trade t : items) { lines.add("BEGIN"); lines.add("INFO," + t.getIsin() + "," + t.getCustomer()); lines.add("AMNT," + t.getQuantity() + "," + t.getPrice()); lines.add("END"); } + this.delegate.write(lines); } @@ -49,14 +51,17 @@ public void setDelegate(FlatFileItemWriter delegate) { this.delegate = delegate; } + @Override public void close() throws ItemStreamException { this.delegate.close(); } + @Override public void open(ExecutionContext executionContext) throws ItemStreamException { this.delegate.open(executionContext); } + @Override public void update(ExecutionContext executionContext) throws ItemStreamException { this.delegate.update(executionContext); } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/TradeCustomerItemWriter.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/TradeCustomerItemWriter.java index 019156e466..b363029d8d 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/TradeCustomerItemWriter.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/iosample/internal/TradeCustomerItemWriter.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -32,6 +32,7 @@ public class TradeCustomerItemWriter implements ItemWriter { private TradeDao dao; private int count; + @Override public void write(List items) throws Exception { for (CustomerCredit c : items) { Trade t = new Trade("ISIN" + count++, 100, new BigDecimal("1.50"), c.getName()); diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/jmx/JobExecutionNotificationPublisherTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/jmx/JobExecutionNotificationPublisherTests.java index f4255a466c..4c92dc1186 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/jmx/JobExecutionNotificationPublisherTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/jmx/JobExecutionNotificationPublisherTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2008 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -33,21 +33,22 @@ * */ public class JobExecutionNotificationPublisherTests { - JobExecutionNotificationPublisher publisher = new JobExecutionNotificationPublisher(); @Test public void testRepeatOperationsOpenUsed() throws Exception { - final List list = new ArrayList(); + final List list = new ArrayList<>(); + publisher.setNotificationPublisher(new NotificationPublisher() { + @Override public void sendNotification(Notification notification) throws UnableToSendNotificationException { list.add(notification); } }); + publisher.onApplicationEvent(new SimpleMessageApplicationEvent(this, "foo")); assertEquals(1, list.size()); String message = list.get(0).getMessage(); assertTrue("Message does not contain 'foo': ", message.indexOf("foo") > 0); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/jsr352/JsrConfigSampleTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/jsr352/JsrConfigSampleTests.java new file mode 100644 index 0000000000..de52531d0f --- /dev/null +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/jsr352/JsrConfigSampleTests.java @@ -0,0 +1,104 @@ +/* + * Copyright 2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.jsr352; + +import org.junit.Before; +import org.junit.Test; + +import javax.batch.operations.JobOperator; +import javax.batch.runtime.BatchRuntime; +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.JobExecution; +import java.util.Properties; + +import static org.junit.Assert.assertTrue; + +/** + *

        + * Test cases to run JSR-352 configuration samples. + *

        + * + * @since 3.0 + * @author Chris Schaefer + */ +public class JsrConfigSampleTests { + private Properties properties = new Properties(); + + @Before + public void setup() { + properties.setProperty("remoteServiceURL", "https://api.example.com/"); + } + + /** + *

        + * Use inline class names as batch artifact references. + *

        + */ + @Test + public void inlineConfigSampleTest() { + JobOperator jobOperator = BatchRuntime.getJobOperator(); + Long executionId = jobOperator.start("inlineConfigSample", properties); + + BatchStatus batchStatus = waitForJobComplete(jobOperator, executionId); + assertTrue(BatchStatus.COMPLETED.equals(batchStatus)); + } + + /** + *

        + * Use batch artifact references defined in batch.xml. + *

        + */ + @Test + public void batchXmlConfigSampleTest() { + JobOperator jobOperator = BatchRuntime.getJobOperator(); + Long executionId = jobOperator.start("batchXmlConfigSample", properties); + + BatchStatus batchStatus = waitForJobComplete(jobOperator, executionId); + assertTrue(BatchStatus.COMPLETED.equals(batchStatus)); + } + + /** + *

        + * Use batch artifact references defined via Spring beans. + *

        + */ + @Test + public void springConfigSampleTest() { + JobOperator jobOperator = BatchRuntime.getJobOperator(); + Long executionId = jobOperator.start("springConfigSampleContext", properties); + + BatchStatus batchStatus = waitForJobComplete(jobOperator, executionId); + assertTrue(BatchStatus.COMPLETED.equals(batchStatus)); + } + + // JSR jobs run async + private BatchStatus waitForJobComplete(JobOperator jobOperator, long executionId) { + JobExecution execution = jobOperator.getJobExecution(executionId); + + BatchStatus curBatchStatus = execution.getBatchStatus(); + + while(true) { + if(curBatchStatus == BatchStatus.STOPPED || curBatchStatus == BatchStatus.COMPLETED || curBatchStatus == BatchStatus.FAILED) { + break; + } + + execution = jobOperator.getJobExecution(executionId); + curBatchStatus = execution.getBatchStatus(); + } + + return curBatchStatus; + } +} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/launch/RemoteLauncherTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/launch/RemoteLauncherTests.java index 45ea7a5b26..6f366adba8 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/launch/RemoteLauncherTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/launch/RemoteLauncherTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -24,7 +24,6 @@ import java.util.Date; import java.util.List; -import javax.management.MBeanServerConnection; import javax.management.MalformedObjectNameException; import org.apache.commons.logging.Log; @@ -44,23 +43,20 @@ * */ public class RemoteLauncherTests { - private static Log logger = LogFactory.getLog(RemoteLauncherTests.class); - - private static List errors = new ArrayList(); - + private static List errors = new ArrayList<>(); private static JobOperator launcher; - private static JobLoader loader; - static private Thread thread; @Test public void testConnect() throws Exception { String message = errors.isEmpty() ? "" : errors.get(0).getMessage(); + if (!errors.isEmpty()) { fail(message); } + assertTrue(isConnected()); } @@ -68,12 +64,12 @@ public void testConnect() throws Exception { public void testLaunchBadJob() throws Exception { assertEquals(0, errors.size()); assertTrue(isConnected()); + try { launcher.start("foo", "time=" + (new Date().getTime())); fail("Expected RuntimeException"); } catch (RuntimeException e) { - // expected; String message = e.getMessage(); assertTrue("Wrong message: " + message, message.contains("NoSuchJobException")); } @@ -89,18 +85,20 @@ public void testAvailableJobs() throws Exception { @Test public void testPauseJob() throws Exception { final int SLEEP_INTERVAL = 600; + assertTrue(isConnected()); assertTrue(launcher.getJobNames().contains("loopJob")); + long executionId = launcher.start("loopJob", ""); // sleep long enough to avoid race conditions (serializable tx isolation // doesn't work with HSQL) Thread.sleep(SLEEP_INTERVAL); - // assertEquals(1, launcher.getRunningExecutions("loopJob").size()); + launcher.stop(executionId); Thread.sleep(SLEEP_INTERVAL); - // assertEquals(0, launcher.getRunningExecutions("loopJob").size()); + logger.debug(launcher.getSummary(executionId)); long resumedId = launcher.restart(executionId); assertNotSame("Picked up the same execution after pause and resume", executionId, resumedId); @@ -109,14 +107,12 @@ public void testPauseJob() throws Exception { launcher.stop(resumedId); Thread.sleep(SLEEP_INTERVAL); - // assertEquals(0, launcher.getRunningExecutions("loopJob").size()); logger.debug(launcher.getSummary(resumedId)); long resumeId2 = launcher.restart(resumedId); assertNotSame("Picked up the same execution after pause and resume", executionId, resumeId2); Thread.sleep(SLEEP_INTERVAL); launcher.stop(resumeId2); - } /* @@ -127,20 +123,23 @@ public void testPauseJob() throws Exception { @BeforeClass public static void setUp() throws Exception { System.setProperty("com.sun.management.jmxremote", ""); + thread = new Thread(new Runnable() { + @Override public void run() { try { JobRegistryBackgroundJobRunner.main("adhoc-job-launcher-context.xml", "jobs/adhocLoopJob.xml"); } catch (Exception e) { - // e.printStackTrace(); logger.error(e); errors.add(e); } } }); + thread.start(); int count = 0; + while (!isConnected() && count++ < 10) { Thread.sleep(1000); } @@ -153,21 +152,24 @@ public static void cleanUp() { private static boolean isConnected() throws Exception { boolean connected = false; + if (!JobRegistryBackgroundJobRunner.getErrors().isEmpty()) { throw JobRegistryBackgroundJobRunner.getErrors().get(0); } + if (launcher == null) { MBeanServerConnectionFactoryBean connectionFactory = new MBeanServerConnectionFactoryBean(); + try { launcher = (JobOperator) getMBean(connectionFactory, "spring:service=batch,bean=jobOperator", JobOperator.class); loader = (JobLoader) getMBean(connectionFactory, "spring:service=batch,bean=jobLoader", JobLoader.class); } catch (MBeanServerNotFoundException e) { - // ignore return false; } } + try { launcher.getJobNames(); connected = loader.getConfigurations().size() > 0; @@ -184,9 +186,8 @@ private static Object getMBean(MBeanServerConnectionFactoryBean connectionFactor MBeanProxyFactoryBean factory = new MBeanProxyFactoryBean(); factory.setObjectName(objectName); factory.setProxyInterface(interfaceType); - factory.setServer((MBeanServerConnection) connectionFactory.getObject()); + factory.setServer(connectionFactory.getObject()); factory.afterPropertiesSet(); return factory.getObject(); } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/metrics/BatchMetricsTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/metrics/BatchMetricsTests.java new file mode 100644 index 0000000000..e9fea17f62 --- /dev/null +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/metrics/BatchMetricsTests.java @@ -0,0 +1,274 @@ +/* + * Copyright 2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.metrics; + +import java.time.Duration; +import java.time.LocalDateTime; +import java.time.ZoneOffset; +import java.time.temporal.ChronoUnit; +import java.util.Arrays; +import java.util.Date; +import java.util.List; + +import io.micrometer.core.instrument.Meter; +import io.micrometer.core.instrument.Metrics; +import org.junit.Test; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.core.metrics.BatchMetrics; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemWriter; +import org.springframework.batch.item.support.ListItemReader; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.context.ApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +public class BatchMetricsTests { + + private static final int EXPECTED_SPRING_BATCH_METRICS = 6; + + @Test + public void testCalculateDuration() { + LocalDateTime startTime = LocalDateTime.now(); + LocalDateTime endTime = startTime + .plus(2, ChronoUnit.HOURS) + .plus(31, ChronoUnit.MINUTES) + .plus(12, ChronoUnit.SECONDS) + .plus(42, ChronoUnit.MILLIS); + + Duration duration = BatchMetrics.calculateDuration(toDate(startTime), toDate(endTime)); + Duration expectedDuration = Duration.ofMillis(42).plusSeconds(12).plusMinutes(31).plusHours(2); + assertEquals(expectedDuration, duration); + } + + @Test + public void testCalculateDurationWhenNoStartTime() { + Duration duration = BatchMetrics.calculateDuration(null, toDate(LocalDateTime.now())); + assertNull(duration); + } + + @Test + public void testCalculateDurationWhenNoEndTime() { + Duration duration = BatchMetrics.calculateDuration(toDate(LocalDateTime.now()), null); + assertNull(duration); + } + + private Date toDate(LocalDateTime localDateTime) { + return Date.from(localDateTime.toInstant(ZoneOffset.UTC)); + } + + @Test + public void testFormatValidDuration() { + Duration duration = Duration.ofMillis(42).plusSeconds(12).plusMinutes(31).plusHours(2); + String formattedDuration = BatchMetrics.formatDuration(duration); + assertEquals("2h31m12s42ms", formattedDuration); + } + + @Test + public void testFormatValidDurationWithoutHours() { + Duration duration = Duration.ofMillis(42).plusSeconds(12).plusMinutes(31); + String formattedDuration = BatchMetrics.formatDuration(duration); + assertEquals("31m12s42ms", formattedDuration); + } + + @Test + public void testFormatValidDurationWithoutMinutes() { + Duration duration = Duration.ofMillis(42).plusSeconds(12); + String formattedDuration = BatchMetrics.formatDuration(duration); + assertEquals("12s42ms", formattedDuration); + } + + @Test + public void testFormatValidDurationWithoutSeconds() { + Duration duration = Duration.ofMillis(42); + String formattedDuration = BatchMetrics.formatDuration(duration); + assertEquals("42ms", formattedDuration); + } + + @Test + public void testFormatNegativeDuration() { + Duration duration = Duration.ofMillis(-1); + String formattedDuration = BatchMetrics.formatDuration(duration); + assertTrue(formattedDuration.isEmpty()); + } + + @Test + public void testFormatZeroDuration() { + String formattedDuration = BatchMetrics.formatDuration(Duration.ZERO); + assertTrue(formattedDuration.isEmpty()); + } + + @Test + public void testFormatNullDuration() { + String formattedDuration = BatchMetrics.formatDuration(null); + assertTrue(formattedDuration.isEmpty()); + } + + @Test + public void testBatchMetrics() throws Exception { + // given + ApplicationContext context = new AnnotationConfigApplicationContext(MyJobConfiguration.class); + JobLauncher jobLauncher = context.getBean(JobLauncher.class); + Job job = context.getBean(Job.class); + + // when + JobExecution jobExecution = jobLauncher.run(job, new JobParameters()); + + // then + assertEquals(ExitStatus.COMPLETED, jobExecution.getExitStatus()); + List meters = Metrics.globalRegistry.getMeters(); + assertTrue(meters.size() >= EXPECTED_SPRING_BATCH_METRICS); + + try { + Metrics.globalRegistry.get("spring.batch.job") + .tag("name", "job") + .tag("status", "COMPLETED") + .timer(); + } catch (Exception e) { + fail("There should be a meter of type TIMER named spring.batch.job " + + "registered in the global registry: " + e.getMessage()); + } + + try { + Metrics.globalRegistry.get("spring.batch.job.active") + .longTaskTimer(); + } catch (Exception e) { + fail("There should be a meter of type LONG_TASK_TIMER named spring.batch.job.active" + + " registered in the global registry: " + e.getMessage()); + } + + try { + Metrics.globalRegistry.get("spring.batch.step") + .tag("name", "step1") + .tag("job.name", "job") + .tag("status", "COMPLETED") + .timer(); + } catch (Exception e) { + fail("There should be a meter of type TIMER named spring.batch.step" + + " registered in the global registry: " + e.getMessage()); + } + + try { + Metrics.globalRegistry.get("spring.batch.step") + .tag("name", "step2") + .tag("job.name", "job") + .tag("status", "COMPLETED") + .timer(); + } catch (Exception e) { + fail("There should be a meter of type TIMER named spring.batch.step" + + " registered in the global registry: " + e.getMessage()); + } + + try { + Metrics.globalRegistry.get("spring.batch.item.read") + .tag("job.name", "job") + .tag("step.name", "step2") + .tag("status", "SUCCESS") + .timer(); + } catch (Exception e) { + fail("There should be a meter of type TIMER named spring.batch.item.read" + + " registered in the global registry: " + e.getMessage()); + } + + try { + Metrics.globalRegistry.get("spring.batch.item.process") + .tag("job.name", "job") + .tag("step.name", "step2") + .tag("status", "SUCCESS") + .timer(); + } catch (Exception e) { + fail("There should be a meter of type TIMER named spring.batch.item.process" + + " registered in the global registry: " + e.getMessage()); + } + + try { + Metrics.globalRegistry.get("spring.batch.chunk.write") + .tag("job.name", "job") + .tag("step.name", "step2") + .tag("status", "SUCCESS") + .timer(); + } catch (Exception e) { + fail("There should be a meter of type TIMER named spring.batch.chunk.write" + + " registered in the global registry: " + e.getMessage()); + } + } + + @Configuration + @EnableBatchProcessing + static class MyJobConfiguration { + + private JobBuilderFactory jobBuilderFactory; + private StepBuilderFactory stepBuilderFactory; + + public MyJobConfiguration(JobBuilderFactory jobBuilderFactory, StepBuilderFactory stepBuilderFactory) { + this.jobBuilderFactory = jobBuilderFactory; + this.stepBuilderFactory = stepBuilderFactory; + } + + @Bean + public Step step1() { + return stepBuilderFactory.get("step1") + .tasklet((contribution, chunkContext) -> RepeatStatus.FINISHED) + .build(); + } + + @Bean + public ItemReader itemReader() { + return new ListItemReader<>(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)); + } + + @Bean + public ItemWriter itemWriter() { + return items -> { + for (Integer item : items) { + System.out.println("item = " + item); + } + }; + } + + @Bean + public Step step2() { + return stepBuilderFactory.get("step2") + .chunk(5) + .reader(itemReader()) + .writer(itemWriter()) + .build(); + } + + @Bean + public Job job() { + return jobBuilderFactory.get("job") + .start(step1()) + .next(step2()) + .build(); + } + } +} diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/quartz/JobLauncherDetailsTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/quartz/JobLauncherDetailsTests.java index 162192313b..d74db4aec6 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/quartz/JobLauncherDetailsTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/quartz/JobLauncherDetailsTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -29,7 +29,9 @@ import org.quartz.JobDetail; import org.quartz.JobExecutionContext; import org.quartz.Scheduler; -import org.quartz.SimpleTrigger; +import org.quartz.impl.JobDetailImpl; +import org.quartz.impl.JobExecutionContextImpl; +import org.quartz.impl.triggers.SimpleTriggerImpl; import org.quartz.spi.TriggerFiredBundle; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; @@ -40,30 +42,31 @@ import org.springframework.batch.core.launch.NoSuchJobException; import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.lang.Nullable; /** * @author Dave Syer * */ public class JobLauncherDetailsTests { - private JobLauncherDetails details = new JobLauncherDetails(); - private TriggerFiredBundle firedBundle; - - private List list = new ArrayList(); + private List list = new ArrayList<>(); @Before public void setUp() throws Exception { details.setJobLauncher(new JobLauncher() { + @Override public JobExecution run(org.springframework.batch.core.Job job, JobParameters jobParameters) throws JobExecutionAlreadyRunningException, JobRestartException { list.add(jobParameters); return null; } }); + details.setJobLocator(new JobLocator() { - public org.springframework.batch.core.Job getJob(String name) throws NoSuchJobException { + @Override + public org.springframework.batch.core.Job getJob(@Nullable String name) throws NoSuchJobException { list.add(name); return new StubJob("foo"); } @@ -71,7 +74,7 @@ public org.springframework.batch.core.Job getJob(String name) throws NoSuchJobEx } private JobExecutionContext createContext(JobDetail jobDetail) { - firedBundle = new TriggerFiredBundle(jobDetail, new SimpleTrigger(), null, false, new Date(), new Date(), new Date(), new Date()); + firedBundle = new TriggerFiredBundle(jobDetail, new SimpleTriggerImpl(), null, false, new Date(), new Date(), new Date(), new Date()); return new StubJobExecutionContext(); } @@ -81,7 +84,7 @@ private JobExecutionContext createContext(JobDetail jobDetail) { */ @Test public void testExecuteWithNoJobParameters() { - JobDetail jobDetail = new JobDetail(); + JobDetail jobDetail = new JobDetailImpl(); JobExecutionContext context = createContext(jobDetail); details.executeInternal(context); assertEquals(2, list.size()); @@ -95,7 +98,7 @@ public void testExecuteWithNoJobParameters() { */ @Test public void testExecuteWithJobName() { - JobDetail jobDetail = new JobDetail(); + JobDetail jobDetail = new JobDetailImpl(); jobDetail.getJobDataMap().put(JobLauncherDetails.JOB_NAME, "FOO"); JobExecutionContext context = createContext(jobDetail); details.executeInternal(context); @@ -109,7 +112,7 @@ public void testExecuteWithJobName() { */ @Test public void testExecuteWithSomeJobParameters() { - JobDetail jobDetail = new JobDetail(); + JobDetail jobDetail = new JobDetailImpl(); jobDetail.getJobDataMap().put("foo", "bar"); JobExecutionContext context = createContext(jobDetail); details.executeInternal(context); @@ -124,7 +127,7 @@ public void testExecuteWithSomeJobParameters() { */ @Test public void testExecuteWithJobNameAndParameters() { - JobDetail jobDetail = new JobDetail(); + JobDetail jobDetail = new JobDetailImpl(); jobDetail.getJobDataMap().put(JobLauncherDetails.JOB_NAME, "FOO"); jobDetail.getJobDataMap().put("foo", "bar"); JobExecutionContext context = createContext(jobDetail); @@ -141,7 +144,7 @@ public void testExecuteWithJobNameAndParameters() { */ @Test public void testExecuteWithJobNameAndComplexParameters() { - JobDetail jobDetail = new JobDetail(); + JobDetail jobDetail = new JobDetailImpl(); jobDetail.getJobDataMap().put(JobLauncherDetails.JOB_NAME, "FOO"); jobDetail.getJobDataMap().put("foo", this); JobExecutionContext context = createContext(jobDetail); @@ -153,40 +156,43 @@ public void testExecuteWithJobNameAndComplexParameters() { assertEquals(0, parameters.getParameters().size()); } - private final class StubJobExecutionContext extends JobExecutionContext { - + @SuppressWarnings("serial") + private final class StubJobExecutionContext extends JobExecutionContextImpl { private StubJobExecutionContext() { super(mock(Scheduler.class), firedBundle, mock(Job.class)); } - } private static class StubJob implements org.springframework.batch.core.Job { - private final String name; public StubJob(String name) { this.name = name; } + @Override public void execute(JobExecution execution) { } + @Nullable + @Override public JobParametersIncrementer getJobParametersIncrementer() { return null; } + + @Override public JobParametersValidator getJobParametersValidator() { return null; } + @Override public String getName() { return name; } + @Override public boolean isRestartable() { return false; } - } - } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/AbstractFieldSetMapperTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/AbstractFieldSetMapperTests.java index 6d6fdc9d02..7e5bbe5546 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/AbstractFieldSetMapperTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/AbstractFieldSetMapperTests.java @@ -1,11 +1,27 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.support; -import static org.junit.Assert.assertEquals; - import org.junit.Test; + import org.springframework.batch.item.file.mapping.FieldSetMapper; import org.springframework.batch.item.file.transform.FieldSet; +import static org.junit.Assert.assertEquals; + /** * Encapsulates basic logic for testing custom {@link FieldSetMapper} implementations. * @@ -34,7 +50,6 @@ public abstract class AbstractFieldSetMapperTests { /** * Regular usage scenario. * Assumes the domain object implements sensible equals(Object other) - * @throws Exception */ @Test public void testRegularUse() throws Exception { diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/AbstractRowMapperTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/AbstractRowMapperTests.java index 79594c3635..de4a42ca0d 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/AbstractRowMapperTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/AbstractRowMapperTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.support; import static org.mockito.Mockito.mock; @@ -13,8 +28,9 @@ * Encapsulates logic for testing custom {@link RowMapper} implementations. * * @author Robert Kasanicky + * @param the item type */ -public abstract class AbstractRowMapperTests { +public abstract class AbstractRowMapperTests { // row number should be irrelevant private static final int IGNORED_ROW_NUMBER = 0; @@ -26,12 +42,12 @@ public abstract class AbstractRowMapperTests { * @return Expected result of mapping the mock ResultSet by the * mapper being tested. */ - abstract protected Object expectedDomainObject(); + abstract protected T expectedDomainObject(); /** * @return RowMapper implementation that is being tested. */ - abstract protected RowMapper rowMapper(); + abstract protected RowMapper rowMapper(); /* * Define the behaviour of mock ResultSet. diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/ItemTrackingItemWriterTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/ItemTrackingItemWriterTests.java index 48b4d654e5..ce248d47b6 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/ItemTrackingItemWriterTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/ItemTrackingItemWriterTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/RetrySampleItemWriterTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/RetrySampleItemWriterTests.java index 6b12d76cdb..6af18df00b 100644 --- a/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/RetrySampleItemWriterTests.java +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/support/RetrySampleItemWriterTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.sample.support; import static org.junit.Assert.assertEquals; @@ -15,7 +30,7 @@ */ public class RetrySampleItemWriterTests { - private RetrySampleItemWriter processor = new RetrySampleItemWriter(); + private RetrySampleItemWriter processor = new RetrySampleItemWriter<>(); /* * Processing throws exception on 2nd and 3rd call. diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/validation/ValidationSampleFunctionalTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/validation/ValidationSampleFunctionalTests.java new file mode 100644 index 0000000000..d7250e0304 --- /dev/null +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/validation/ValidationSampleFunctionalTests.java @@ -0,0 +1,66 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.sample.validation; + +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.launch.JobLauncher; +import org.springframework.batch.item.support.ListItemWriter; +import org.springframework.batch.sample.validation.domain.Person; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringRunner; + +/** + * @author Mahmoud Ben Hassine + */ +@RunWith(SpringRunner.class) +@ContextConfiguration(classes = {ValidationSampleConfiguration.class}) +public class ValidationSampleFunctionalTests { + + @Autowired + private Job job; + + @Autowired + private JobLauncher jobLauncher; + + @Autowired + private ListItemWriter listItemWriter; + + @Test + public void testItemValidation() throws Exception { + // given + JobParameters jobParameters = new JobParameters(); + + // when + JobExecution jobExecution = this.jobLauncher.run(this.job, jobParameters); + + // then + Assert.assertEquals(ExitStatus.COMPLETED.getExitCode(), jobExecution.getExitStatus().getExitCode()); + List writtenItems = this.listItemWriter.getWrittenItems(); + Assert.assertEquals(1, writtenItems.size()); + Assert.assertEquals("foo", writtenItems.get(0).getName()); + } +} diff --git a/spring-batch-samples/src/test/resources/job-runner-context.xml b/spring-batch-samples/src/test/resources/job-runner-context.xml index 5e56c1b71c..a2dc389565 100644 --- a/spring-batch-samples/src/test/resources/job-runner-context.xml +++ b/spring-batch-samples/src/test/resources/job-runner-context.xml @@ -1,10 +1,10 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd+http://www.springframework.org/schema/context%20https://www.springframework.org/schema/context/spring-context.xsd"> - + diff --git a/spring-batch-samples/src/test/resources/org/springframework/batch/item/json/trades.json b/spring-batch-samples/src/test/resources/org/springframework/batch/item/json/trades.json new file mode 100644 index 0000000000..d5ebcef261 --- /dev/null +++ b/spring-batch-samples/src/test/resources/org/springframework/batch/item/json/trades.json @@ -0,0 +1,5 @@ +[ + {"isin":"123","quantity":5,"price":10.5,"customer":"foo","id":1,"version":0}, + {"isin":"456","quantity":10,"price":20.5,"customer":"bar","id":2,"version":0}, + {"isin":"789","quantity":15,"price":30.5,"customer":"baz","id":3,"version":0} +] diff --git a/spring-batch-samples/src/test/resources/org/springframework/batch/sample/JobStepFunctionalTests-context.xml b/spring-batch-samples/src/test/resources/org/springframework/batch/sample/JobStepFunctionalTests-context.xml index 9add530d85..5df2b33cc9 100644 --- a/spring-batch-samples/src/test/resources/org/springframework/batch/sample/JobStepFunctionalTests-context.xml +++ b/spring-batch-samples/src/test/resources/org/springframework/batch/sample/JobStepFunctionalTests-context.xml @@ -1,7 +1,6 @@ - + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> @@ -9,5 +8,4 @@ - diff --git a/spring-batch-samples/src/test/resources/org/springframework/batch/sample/common/ColumnRangePartitionerTests-context.xml b/spring-batch-samples/src/test/resources/org/springframework/batch/sample/common/ColumnRangePartitionerTests-context.xml index ed39a3495e..6df7cd7d0c 100644 --- a/spring-batch-samples/src/test/resources/org/springframework/batch/sample/common/ColumnRangePartitionerTests-context.xml +++ b/spring-batch-samples/src/test/resources/org/springframework/batch/sample/common/ColumnRangePartitionerTests-context.xml @@ -1,10 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> - \ No newline at end of file diff --git a/spring-batch-samples/src/test/resources/org/springframework/batch/sample/common/StagingItemReaderTests-context.xml b/spring-batch-samples/src/test/resources/org/springframework/batch/sample/common/StagingItemReaderTests-context.xml index 1962da9440..9c09e635e1 100644 --- a/spring-batch-samples/src/test/resources/org/springframework/batch/sample/common/StagingItemReaderTests-context.xml +++ b/spring-batch-samples/src/test/resources/org/springframework/batch/sample/common/StagingItemReaderTests-context.xml @@ -1,10 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> - - \ No newline at end of file + diff --git a/spring-batch-samples/src/test/resources/org/springframework/batch/sample/common/StagingItemWriterTests-context.xml b/spring-batch-samples/src/test/resources/org/springframework/batch/sample/common/StagingItemWriterTests-context.xml index 1962da9440..36af206577 100644 --- a/spring-batch-samples/src/test/resources/org/springframework/batch/sample/common/StagingItemWriterTests-context.xml +++ b/spring-batch-samples/src/test/resources/org/springframework/batch/sample/common/StagingItemWriterTests-context.xml @@ -1,10 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> - \ No newline at end of file diff --git a/spring-batch-samples/src/test/resources/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDaoTests-context.xml b/spring-batch-samples/src/test/resources/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDaoTests-context.xml index 9e6004179e..5f4086490c 100644 --- a/spring-batch-samples/src/test/resources/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDaoTests-context.xml +++ b/spring-batch-samples/src/test/resources/org/springframework/batch/sample/domain/trade/internal/JdbcCustomerDebitDaoTests-context.xml @@ -1,11 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans.xsd"> - \ No newline at end of file diff --git a/spring-batch-samples/template.mf b/spring-batch-samples/template.mf deleted file mode 100755 index 0d55c0183c..0000000000 --- a/spring-batch-samples/template.mf +++ /dev/null @@ -1,11 +0,0 @@ -Manifest-Version: 1.0 -Bundle-SymbolicName: org.springframework.batch.samples -Bundle-Name: Spring Batch Samples -Bundle-Vendor: Spring -Bundle-Version: ${version} -Bundle-ManifestVersion: 2 -Ignored-Existing-Headers: - Import-Package, - Export-Package -Import-Template: - *;version="0" diff --git a/spring-batch-test/.springBeans b/spring-batch-test/.springBeans index 4f4cc55226..2f255ff8a9 100644 --- a/spring-batch-test/.springBeans +++ b/spring-batch-test/.springBeans @@ -1,44 +1,52 @@ - - - 1 - - - - - - - src/test/resources/data-source-context.xml - src/test/resources/simple-job-launcher-context.xml - src/test/resources/org/springframework/batch/sample/config/common-context.xml - src/test/resources/jobs/sampleFlowJob.xml - src/test/resources/jobs/sampleSimpleJob.xml - src/test/resources/jobs/sample-steps.xml - src/test/resources/job-runner-context.xml - src/test/resources/org/springframework/batch/test/StepScopeTestExecutionListenerIntegrationTests-context.xml - - - - - true - false - - src/test/resources/data-source-context.xml - src/test/resources/org/springframework/batch/sample/config/common-context.xml - src/test/resources/simple-job-launcher-context.xml - src/test/resources/jobs/sampleSimpleJob.xml - src/test/resources/jobs/sample-steps.xml - - - - - true - false - - src/test/resources/data-source-context.xml - src/test/resources/jobs/sampleFlowJob.xml - src/test/resources/org/springframework/batch/sample/config/common-context.xml - src/test/resources/simple-job-launcher-context.xml - - - - + + + 1 + + + + + + + src/test/resources/data-source-context.xml + src/test/resources/simple-job-launcher-context.xml + src/test/resources/org/springframework/batch/sample/config/common-context.xml + src/test/resources/jobs/sampleFlowJob.xml + src/test/resources/jobs/sampleSimpleJob.xml + src/test/resources/jobs/sample-steps.xml + src/test/resources/job-runner-context.xml + src/test/resources/org/springframework/batch/test/StepScopeTestExecutionListenerIntegrationTests-context.xml + src/test/resources/org/springframework/batch/test/JobScopeTestExecutionListenerIntegrationTests-context.xml + src/test/resources/org/springframework/batch/test/JobScopeTestExecutionListenerTests-context.xml + src/test/resources/org/springframework/batch/test/StepScopeTestExecutionListenerTests-context.xml + java:org.springframework.batch.test.StepScopeAnnotatedListenerIntegrationTests$TestConfig + + + + + true + false + + src/test/resources/data-source-context.xml + src/test/resources/org/springframework/batch/sample/config/common-context.xml + src/test/resources/simple-job-launcher-context.xml + src/test/resources/jobs/sampleSimpleJob.xml + src/test/resources/jobs/sample-steps.xml + + + + + + + true + false + + src/test/resources/data-source-context.xml + src/test/resources/jobs/sampleFlowJob.xml + src/test/resources/org/springframework/batch/sample/config/common-context.xml + src/test/resources/simple-job-launcher-context.xml + + + + + + diff --git a/spring-batch-test/pom.xml b/spring-batch-test/pom.xml deleted file mode 100755 index 946f62f2bd..0000000000 --- a/spring-batch-test/pom.xml +++ /dev/null @@ -1,78 +0,0 @@ - - - 4.0.0 - spring-batch-test - Test - jar - Domain for batch job testing - - org.springframework.batch - spring-batch-parent - 2.2.2.BUILD-SNAPSHOT - ../spring-batch-parent - - - - org.springframework.batch - spring-batch-core - ${project.version} - - - org.aspectj - aspectjrt - true - - - junit - junit - compile - - - org.springframework - spring-test - compile - - - org.springframework - spring-jdbc - - - commons-io - commons-io - - - commons-dbcp - commons-dbcp - - - org.hsqldb - hsqldb - - - commons-collections - commons-collections - - - - - - org.codehaus.mojo - emma-maven-plugin - 1.0-alpha-1 - - - - - - - - org.apache.maven.plugins - maven-surefire-plugin - - junit:junit - - - - - - diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/AbstractJobTests.java b/spring-batch-test/src/main/java/org/springframework/batch/test/AbstractJobTests.java index 00dbb07a81..c18af29c41 100644 --- a/spring-batch-test/src/main/java/org/springframework/batch/test/AbstractJobTests.java +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/AbstractJobTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -67,6 +67,7 @@ * * @deprecated (from 2.1) use {@link JobLauncherTestUtils} instead */ +@Deprecated public abstract class AbstractJobTests implements ApplicationContextAware { /** Logger */ @@ -125,7 +126,8 @@ protected JobLauncher getJobLauncher() { * Launch the entire job, including all steps. * * @return JobExecution, so that the test can validate the exit status - * @throws Exception + * + * @throws Exception is thrown if error occurs. */ protected JobExecution launchJob() throws Exception { return this.launchJob(this.getUniqueJobParameters()); @@ -134,9 +136,10 @@ protected JobExecution launchJob() throws Exception { /** * Launch the entire job, including all steps * - * @param jobParameters + * @param jobParameters parameters for the job * @return JobExecution, so that the test can validate the exit status - * @throws Exception + * + * @throws Exception is thrown if error occurs. */ protected JobExecution launchJob(JobParameters jobParameters) throws Exception { return getJobLauncher().run(this.job, jobParameters); @@ -147,7 +150,7 @@ protected JobExecution launchJob(JobParameters jobParameters) throws Exception { * current timestamp, to ensure that the job instance will be unique. */ protected JobParameters getUniqueJobParameters() { - Map parameters = new HashMap(); + Map parameters = new HashMap<>(); parameters.put("timestamp", new JobParameter(new Date().getTime())); return new JobParameters(parameters); } diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/AssertFile.java b/spring-batch-test/src/main/java/org/springframework/batch/test/AssertFile.java index ea5bbad2c6..86ae7afce0 100644 --- a/spring-batch-test/src/main/java/org/springframework/batch/test/AssertFile.java +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/AssertFile.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,12 +16,12 @@ package org.springframework.batch.test; +import static org.junit.Assert.assertEquals; + import java.io.BufferedReader; import java.io.File; import java.io.FileReader; -import junit.framework.Assert; - import org.springframework.core.io.Resource; /** @@ -39,11 +39,11 @@ public static void assertFileEquals(File expected, File actual) throws Exception int lineNum = 1; for (String expectedLine = null; (expectedLine = expectedReader.readLine()) != null; lineNum++) { String actualLine = actualReader.readLine(); - Assert.assertEquals("Line number " + lineNum + " does not match.", expectedLine, actualLine); + assertEquals("Line number " + lineNum + " does not match.", expectedLine, actualLine); } String actualLine = actualReader.readLine(); - Assert.assertEquals("More lines than expected. There should not be a line number " + lineNum + ".", null, + assertEquals("More lines than expected. There should not be a line number " + lineNum + ".", null, actualLine); } finally { @@ -53,7 +53,7 @@ public static void assertFileEquals(File expected, File actual) throws Exception } public static void assertFileEquals(Resource expected, Resource actual) throws Exception { - AssertFile.assertFileEquals(expected.getFile(), actual.getFile()); + assertFileEquals(expected.getFile(), actual.getFile()); } public static void assertLineCount(int expectedLineCount, File file) throws Exception { @@ -63,7 +63,7 @@ public static void assertLineCount(int expectedLineCount, File file) throws Exce while (expectedReader.readLine() != null) { lineCount++; } - Assert.assertEquals(expectedLineCount, lineCount); + assertEquals(expectedLineCount, lineCount); } finally { expectedReader.close(); @@ -71,6 +71,6 @@ public static void assertLineCount(int expectedLineCount, File file) throws Exce } public static void assertLineCount(int expectedLineCount, Resource resource) throws Exception { - AssertFile.assertLineCount(expectedLineCount, resource.getFile()); + assertLineCount(expectedLineCount, resource.getFile()); } } diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/DataSourceInitializer.java b/spring-batch-test/src/main/java/org/springframework/batch/test/DataSourceInitializer.java index 0f4ddfd3c9..1fa840dce6 100755 --- a/spring-batch-test/src/main/java/org/springframework/batch/test/DataSourceInitializer.java +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/DataSourceInitializer.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -17,11 +17,17 @@ package org.springframework.batch.test; import java.io.IOException; +import java.net.URI; +import java.nio.charset.StandardCharsets; +import java.nio.file.FileSystemNotFoundException; +import java.nio.file.FileSystems; +import java.nio.file.Files; +import java.nio.file.Paths; +import java.util.Collections; import java.util.List; import javax.sql.DataSource; -import org.apache.commons.io.IOUtils; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.beans.factory.BeanInitializationException; @@ -32,7 +38,6 @@ import org.springframework.dao.DataAccessException; import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jdbc.datasource.DataSourceTransactionManager; -import org.springframework.transaction.TransactionStatus; import org.springframework.transaction.support.TransactionCallback; import org.springframework.transaction.support.TransactionTemplate; import org.springframework.util.Assert; @@ -41,7 +46,7 @@ /** * Wrapper for a {@link DataSource} that can run scripts on start up and shut - * down. Us as a bean definition

        + * down. Use as a bean definition
        * * Run this class to initialize a database in a running server process. * Make sure the server is running first by launching the "hsql-server" from the @@ -50,7 +55,9 @@ * database and start again. * * @author Dave Syer - * + * @author Drummond Dawson + * @author Mahmoud Ben Hassine + * */ public class DataSourceInitializer implements InitializingBean, DisposableBean { @@ -69,18 +76,20 @@ public class DataSourceInitializer implements InitializingBean, DisposableBean { /** * Main method as convenient entry point. * - * @param args + * @param args arguments to be passed to main. */ + @SuppressWarnings("resource") public static void main(String... args) { new ClassPathXmlApplicationContext(ClassUtils.addResourcePathToPackagePath(DataSourceInitializer.class, DataSourceInitializer.class.getSimpleName() + "-context.xml")); } - @Override + @Override public void destroy() { - if (destroyScripts==null) return; - for (int i = 0; i < destroyScripts.length; i++) { - Resource destroyScript = destroyScripts[i]; + if (this.destroyScripts == null) { + return; + } + for (Resource destroyScript : this.destroyScripts) { try { doExecuteScript(destroyScript); } @@ -95,71 +104,83 @@ public void destroy() { } } - @Override - public void afterPropertiesSet() throws Exception { - Assert.notNull(dataSource); + @Override + public void afterPropertiesSet() { + Assert.notNull(this.dataSource, "A DataSource is required"); initialize(); } private void initialize() { - if (!initialized) { + if (!this.initialized) { destroy(); - if (initScripts != null) { - for (int i = 0; i < initScripts.length; i++) { - Resource initScript = initScripts[i]; + if (this.initScripts != null) { + for (Resource initScript : this.initScripts) { doExecuteScript(initScript); } } - initialized = true; + this.initialized = true; } } private void doExecuteScript(final Resource scriptResource) { - if (scriptResource == null || !scriptResource.exists()) + if (scriptResource == null || !scriptResource.exists()) { return; - TransactionTemplate transactionTemplate = new TransactionTemplate(new DataSourceTransactionManager(dataSource)); - transactionTemplate.execute(new TransactionCallback() { - - @Override - @SuppressWarnings("unchecked") - public Object doInTransaction(TransactionStatus status) { - JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); - String[] scripts; - try { - scripts = StringUtils.delimitedListToStringArray(stripComments(IOUtils.readLines(scriptResource - .getInputStream())), ";"); - } - catch (IOException e) { - throw new BeanInitializationException("Cannot load script from [" + scriptResource + "]", e); - } - for (int i = 0; i < scripts.length; i++) { - String script = scripts[i].trim(); - if (StringUtils.hasText(script)) { - try { - jdbcTemplate.execute(script); + } + TransactionTemplate transactionTemplate = new TransactionTemplate(new DataSourceTransactionManager(this.dataSource)); + transactionTemplate.execute((TransactionCallback) status -> { + JdbcTemplate jdbcTemplate = new JdbcTemplate(this.dataSource); + String[] scripts; + try { + scripts = StringUtils + .delimitedListToStringArray(stripComments(getScriptLines(scriptResource)), ";"); + } + catch (IOException e) { + throw new BeanInitializationException("Cannot load script from [" + scriptResource + "]", e); + } + for (String script : scripts) { + String trimmedScript = script.trim(); + if (StringUtils.hasText(trimmedScript)) { + try { + jdbcTemplate.execute(trimmedScript); + } + catch (DataAccessException e) { + if (this.ignoreFailedDrop && trimmedScript.toLowerCase().startsWith("drop")) { + logger.debug("DROP script failed (ignoring): " + trimmedScript); } - catch (DataAccessException e) { - if (ignoreFailedDrop && script.toLowerCase().startsWith("drop")) { - logger.debug("DROP script failed (ignoring): " + script); - } - else { - throw e; - } + else { + throw e; } } } - return null; } - + return null; }); } + private List getScriptLines(Resource scriptResource) throws IOException { + URI uri = scriptResource.getURI(); + initFileSystem(uri); + return Files.readAllLines(Paths.get(uri), StandardCharsets.UTF_8); + } + + private void initFileSystem(URI uri) throws IOException { + try { + FileSystems.getFileSystem(uri); + } + catch (FileSystemNotFoundException e) { + FileSystems.newFileSystem(uri, Collections.emptyMap()); + } + catch (IllegalArgumentException e) { + FileSystems.getDefault(); + } + } + private String stripComments(List list) { - StringBuffer buffer = new StringBuffer(); + StringBuilder buffer = new StringBuilder(); for (String line : list) { if (!line.startsWith("//") && !line.startsWith("--")) { - buffer.append(line + "\n"); + buffer.append(line).append("\n"); } } return buffer.toString(); diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/ExecutionContextTestUtils.java b/spring-batch-test/src/main/java/org/springframework/batch/test/ExecutionContextTestUtils.java index a907eedaea..0e0a668a0c 100644 --- a/spring-batch-test/src/main/java/org/springframework/batch/test/ExecutionContextTestUtils.java +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/ExecutionContextTestUtils.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2009 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -22,25 +22,29 @@ import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.StepExecution; import org.springframework.batch.item.ExecutionContext; +import org.springframework.lang.Nullable; /** * Convenience class for accessing {@link ExecutionContext} values from job and * step executions. * * @author Dave Syer + * @author Mahmoud Ben Hassine * @since 2.1.4 * */ public class ExecutionContextTestUtils { @SuppressWarnings("unchecked") + @Nullable public static T getValueFromJob(JobExecution jobExecution, String key) { return (T) jobExecution.getExecutionContext().get(key); } + @Nullable public static T getValueFromStepInJob(JobExecution jobExecution, String stepName, String key) { StepExecution stepExecution = null; - List stepNames = new ArrayList(); + List stepNames = new ArrayList<>(); for (StepExecution candidate : jobExecution.getStepExecutions()) { String name = candidate.getStepName(); stepNames.add(name); @@ -58,6 +62,7 @@ public static T getValueFromStepInJob(JobExecution jobExecution, String step } @SuppressWarnings("unchecked") + @Nullable public static T getValueFromStep(StepExecution stepExecution, String key) { return (T) stepExecution.getExecutionContext().get(key); } diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/JobLauncherTestUtils.java b/spring-batch-test/src/main/java/org/springframework/batch/test/JobLauncherTestUtils.java index 9bdd78ab10..5fc3dec28f 100644 --- a/spring-batch-test/src/main/java/org/springframework/batch/test/JobLauncherTestUtils.java +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/JobLauncherTestUtils.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -35,6 +35,7 @@ import org.springframework.batch.item.ExecutionContext; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.ApplicationContext; +import org.springframework.lang.Nullable; /** *

        @@ -62,12 +63,13 @@ * @author Lucas Ward * @author Dan Garrette * @author Dave Syer + * @author Mahmoud Ben Hassine * @since 2.1 */ public class JobLauncherTestUtils { - private static final long JOB_PARAMETER_MAXIMUM = 1000000; - + private static final long JOB_PARAMETER_MAXIMUM = 1000000; + /** Logger */ protected final Log logger = LogFactory.getLog(getClass()); @@ -135,7 +137,7 @@ public JobLauncher getJobLauncher() { * Launch the entire job, including all steps. * * @return JobExecution, so that the test can validate the exit status - * @throws Exception + * @throws Exception thrown if error occurs launching the job. */ public JobExecution launchJob() throws Exception { return this.launchJob(this.getUniqueJobParameters()); @@ -144,9 +146,9 @@ public JobExecution launchJob() throws Exception { /** * Launch the entire job, including all steps * - * @param jobParameters + * @param jobParameters instance of {@link JobParameters}. * @return JobExecution, so that the test can validate the exit status - * @throws Exception + * @throws Exception thrown if error occurs launching the job. */ public JobExecution launchJob(JobParameters jobParameters) throws Exception { return getJobLauncher().run(this.job, jobParameters); @@ -157,8 +159,8 @@ public JobExecution launchJob(JobParameters jobParameters) throws Exception { * current timestamp, to ensure that the job instance will be unique. */ public JobParameters getUniqueJobParameters() { - Map parameters = new HashMap(); - parameters.put("random", new JobParameter((long) (Math.random() * JOB_PARAMETER_MAXIMUM))); + Map parameters = new HashMap<>(); + parameters.put("random", new JobParameter((long) (Math.random() * JOB_PARAMETER_MAXIMUM))); return new JobParameters(parameters); } @@ -223,7 +225,7 @@ public JobExecution launchStep(String stepName, JobParameters jobParameters) { * loaded into the Job ExecutionContext prior to launching the step. * @return JobExecution */ - public JobExecution launchStep(String stepName, JobParameters jobParameters, ExecutionContext jobExecutionContext) { + public JobExecution launchStep(String stepName, JobParameters jobParameters, @Nullable ExecutionContext jobExecutionContext) { if (!(job instanceof StepLocator)) { throw new UnsupportedOperationException("Cannot locate step from a Job that is not a StepLocator: job=" + job.getName() + " does not implement StepLocator"); diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/JobRepositoryTestUtils.java b/spring-batch-test/src/main/java/org/springframework/batch/test/JobRepositoryTestUtils.java index 2ae8555f26..6f0584f199 100644 --- a/spring-batch-test/src/main/java/org/springframework/batch/test/JobRepositoryTestUtils.java +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/JobRepositoryTestUtils.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,10 +36,12 @@ import org.springframework.batch.core.repository.JobRestartException; import org.springframework.batch.core.repository.dao.AbstractJdbcBatchMetadataDao; import org.springframework.beans.factory.InitializingBean; +import org.springframework.beans.factory.annotation.Autowired; import org.springframework.dao.DataAccessException; import org.springframework.jdbc.core.JdbcOperations; import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.jdbc.core.simple.ParameterizedRowMapper; +import org.springframework.jdbc.core.RowMapper; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; /** @@ -49,6 +51,7 @@ * the transaction. * * @author Dave Syer + * @author Mahmoud Ben Hassine */ public class JobRepositoryTestUtils extends AbstractJdbcBatchMetadataDao implements InitializingBean { @@ -59,7 +62,7 @@ public class JobRepositoryTestUtils extends AbstractJdbcBatchMetadataDao impleme Long count = 0L; @Override - public JobParameters getNext(JobParameters parameters) { + public JobParameters getNext(@Nullable JobParameters parameters) { return new JobParameters(Collections.singletonMap("count", new JobParameter(count++))); } @@ -95,6 +98,7 @@ public JobRepositoryTestUtils(JobRepository jobRepository, DataSource dataSource setDataSource(dataSource); } + @Autowired public final void setDataSource(DataSource dataSource) { jdbcTemplate = new JdbcTemplate(dataSource); } @@ -109,6 +113,7 @@ public void setJobParametersIncrementer(JobParametersIncrementer jobParametersIn /** * @param jobRepository the jobRepository to set */ + @Autowired public void setJobRepository(JobRepository jobRepository) { this.jobRepository = jobRepository; } @@ -123,11 +128,14 @@ public void setJobRepository(JobRepository jobRepository) { * @param count the required number of instances of {@link JobExecution} to * create * @return a collection of {@link JobExecution} - * @throws Exception if there is a problem in the {@link JobRepository} + * + * @throws JobExecutionAlreadyRunningException thrown if Job is already running. + * @throws JobRestartException thrown if Job is not restartable. + * @throws JobInstanceAlreadyCompleteException thrown if Job Instance is already complete. */ public List createJobExecutions(String jobName, String[] stepNames, int count) throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException { - List list = new ArrayList(); + List list = new ArrayList<>(); JobParameters jobParameters = new JobParameters(); for (int i = 0; i < count; i++) { JobExecution jobExecution = jobRepository.createJobExecution(jobName, jobParametersIncrementer @@ -147,7 +155,9 @@ public List createJobExecutions(String jobName, String[] stepNames * @param count the required number of instances of {@link JobExecution} to * create * @return a collection of {@link JobExecution} - * @throws Exception if there is a problem in the {@link JobRepository} + * @throws JobExecutionAlreadyRunningException thrown if Job is already running. + * @throws JobRestartException thrown if Job is not restartable. + * @throws JobInstanceAlreadyCompleteException thrown if Job Instance is already complete. */ public List createJobExecutions(int count) throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException { @@ -166,7 +176,7 @@ public void removeJobExecutions(Collection list) throws DataAccess for (JobExecution jobExecution : list) { List stepExecutionIds = jdbcTemplate.query( getQuery("select STEP_EXECUTION_ID from %PREFIX%STEP_EXECUTION where JOB_EXECUTION_ID=?"), - new ParameterizedRowMapper() { + new RowMapper() { @Override public Long mapRow(ResultSet rs, int rowNum) throws SQLException { return rs.getLong(1); diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/JobScopeTestExecutionListener.java b/spring-batch-test/src/main/java/org/springframework/batch/test/JobScopeTestExecutionListener.java new file mode 100644 index 0000000000..e7855637c7 --- /dev/null +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/JobScopeTestExecutionListener.java @@ -0,0 +1,187 @@ +/* + * Copyright 2006-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test; + +import java.lang.reflect.Method; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.scope.context.JobContext; +import org.springframework.batch.core.scope.context.JobSynchronizationManager; +import org.springframework.batch.item.adapter.HippyMethodInvoker; +import org.springframework.test.context.TestContext; +import org.springframework.test.context.TestExecutionListener; +import org.springframework.util.ReflectionUtils; +import org.springframework.util.ReflectionUtils.MethodCallback; + +/** + * A {@link TestExecutionListener} that sets up job-scope context for + * dependency injection into unit tests. A {@link JobContext} will be created + * for the duration of a test method and made available to any dependencies that + * are injected. The default behaviour is just to create a {@link JobExecution} with fixed properties. Alternatively it + * can be provided by the test case as a + * factory methods returning the correct type. Example: + * + *

        + * @ContextConfiguration
        + * @TestExecutionListeners({ DependencyInjectionTestExecutionListener.class, JobScopeTestExecutionListener.class })
        + * @RunWith(SpringJUnit4ClassRunner.class)
        + * public class JobScopeTestExecutionListenerIntegrationTests {
        + * 
        + * 	// A job-scoped dependency configured in the ApplicationContext
        + * 	@Autowired
        + * 	private ItemReader<String> reader;
        + * 
        + * 	public JobExecution getJobExecution() {
        + * 		JobExecution execution = MetaDataInstanceFactory.createJobExecution();
        + * 		execution.getExecutionContext().putString("foo", "bar");
        + * 		return execution;
        + * 	}
        + * 
        + * 	@Test
        + * 	public void testJobScopedReader() {
        + * 		// Job context is active here so the reader can be used,
        + * 		// and the job execution context will contain foo=bar...
        + * 		assertNotNull(reader.read());
        + * 	}
        + * 
        + * }
        + * 
        + * + * @author Dave Syer + * @author Jimmy Praet + */ +public class JobScopeTestExecutionListener implements TestExecutionListener { + + private static final String JOB_EXECUTION = JobScopeTestExecutionListener.class.getName() + ".JOB_EXECUTION"; + + /** + * Set up a {@link JobExecution} as a test context attribute. + * + * @param testContext the current test context + * @throws Exception if there is a problem + * @see TestExecutionListener#prepareTestInstance(TestContext) + */ + @Override + public void prepareTestInstance(TestContext testContext) throws Exception { + JobExecution jobExecution = getJobExecution(testContext); + if (jobExecution != null) { + testContext.setAttribute(JOB_EXECUTION, jobExecution); + } + } + + /** + * @param testContext the current test context + * @throws Exception if there is a problem + * @see TestExecutionListener#beforeTestMethod(TestContext) + */ + @Override + public void beforeTestMethod(org.springframework.test.context.TestContext testContext) throws Exception { + if (testContext.hasAttribute(JOB_EXECUTION)) { + JobExecution jobExecution = (JobExecution) testContext.getAttribute(JOB_EXECUTION); + JobSynchronizationManager.register(jobExecution); + } + + } + + /** + * @param testContext the current test context + * @throws Exception if there is a problem + * @see TestExecutionListener#afterTestMethod(TestContext) + */ + @Override + public void afterTestMethod(TestContext testContext) throws Exception { + if (testContext.hasAttribute(JOB_EXECUTION)) { + JobSynchronizationManager.close(); + } + } + + /* + * Support for Spring 3.0 (empty). + */ + @Override + public void afterTestClass(TestContext testContext) throws Exception { + } + + /* + * Support for Spring 3.0 (empty). + */ + @Override + public void beforeTestClass(TestContext testContext) throws Exception { + } + + /** + * Discover a {@link JobExecution} as a field in the test case or create + * one if none is available. + * + * @param testContext the current test context + * @return a {@link JobExecution} + */ + protected JobExecution getJobExecution(TestContext testContext) { + + Object target = testContext.getTestInstance(); + + ExtractorMethodCallback method = new ExtractorMethodCallback(JobExecution.class, "getJobExecution"); + ReflectionUtils.doWithMethods(target.getClass(), method); + if (method.getName() != null) { + HippyMethodInvoker invoker = new HippyMethodInvoker(); + invoker.setTargetObject(target); + invoker.setTargetMethod(method.getName()); + try { + invoker.prepare(); + return (JobExecution) invoker.invoke(); + } + catch (Exception e) { + throw new IllegalArgumentException("Could not create job execution from method: " + method.getName(), + e); + } + } + + return MetaDataInstanceFactory.createJobExecution(); + } + + /** + * Look for a method returning the type provided, preferring one with the + * name provided. + */ + private final class ExtractorMethodCallback implements MethodCallback { + private String preferredName; + + private final Class preferredType; + + private Method result; + + public ExtractorMethodCallback(Class preferredType, String preferredName) { + super(); + this.preferredType = preferredType; + this.preferredName = preferredName; + } + + public String getName() { + return result == null ? null : result.getName(); + } + + @Override + public void doWith(Method method) throws IllegalArgumentException, IllegalAccessException { + Class type = method.getReturnType(); + if (preferredType.isAssignableFrom(type)) { + if (result == null || method.getName().equals(preferredName)) { + result = method; + } + } + } + } + +} diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/JobScopeTestUtils.java b/spring-batch-test/src/main/java/org/springframework/batch/test/JobScopeTestUtils.java new file mode 100644 index 0000000000..6c54e35da9 --- /dev/null +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/JobScopeTestUtils.java @@ -0,0 +1,45 @@ +/* + * Copyright 2006-2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test; + +import java.util.concurrent.Callable; + +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.scope.JobScope; +import org.springframework.batch.core.scope.context.JobSynchronizationManager; + +/** + * Utility class for creating and manipulating {@link JobScope} in unit tests. + * This is useful when you want to use the Spring test support and inject + * dependencies into your test case that happen to be job scoped in the + * application context. + * + * @author Dave Syer + * @author Jimmy Praet + */ +public class JobScopeTestUtils { + + public static T doInJobScope(JobExecution jobExecution, Callable callable) throws Exception { + try { + JobSynchronizationManager.register(jobExecution); + return callable.call(); + } + finally { + JobSynchronizationManager.close(); + } + } + +} diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/JsrTestUtils.java b/spring-batch-test/src/main/java/org/springframework/batch/test/JsrTestUtils.java new file mode 100644 index 0000000000..12e2033388 --- /dev/null +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/JsrTestUtils.java @@ -0,0 +1,129 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test; + +import javax.batch.operations.JobOperator; +import javax.batch.runtime.BatchRuntime; +import javax.batch.runtime.BatchStatus; +import javax.batch.runtime.JobExecution; +import javax.batch.runtime.Metric; +import javax.batch.runtime.StepExecution; +import java.util.Date; +import java.util.Properties; +import java.util.concurrent.TimeoutException; + +import org.springframework.lang.Nullable; + +/** + * Provides testing utilities to execute JSR-352 jobs and block until they are complete (since all JSR-352 based jobs + * are executed asynchronously). + * + * @author Michael Minella + * @author Mahmoud Ben Hassine + * @since 3.0 + */ +public class JsrTestUtils { + + private static JobOperator operator; + + static { + operator = BatchRuntime.getJobOperator(); + } + + private JsrTestUtils() {} + + /** + * Executes a job and waits for it's status to be any of {@link BatchStatus#STOPPED}, + * {@link BatchStatus#COMPLETED}, or {@link BatchStatus#FAILED}. If the job does not + * reach one of those statuses within the given timeout, a {@link java.util.concurrent.TimeoutException} is + * thrown. + * + * @param jobName the name of the job. + * @param properties job parameters to be associated with the job. + * @param timeout maximum amount of time to wait in milliseconds. + * @return the {@link JobExecution} for the final state of the job + * @throws java.util.concurrent.TimeoutException if the timeout occurs + */ + public static JobExecution runJob(String jobName, Properties properties, long timeout) throws TimeoutException{ + long executionId = operator.start(jobName, properties); + JobExecution execution = operator.getJobExecution(executionId); + + Date curDate = new Date(); + BatchStatus curBatchStatus = execution.getBatchStatus(); + + while(true) { + if(curBatchStatus == BatchStatus.STOPPED || curBatchStatus == BatchStatus.COMPLETED || curBatchStatus == BatchStatus.FAILED) { + break; + } + + if(new Date().getTime() - curDate.getTime() > timeout) { + throw new TimeoutException("Job processing did not complete in time"); + } + + execution = operator.getJobExecution(executionId); + curBatchStatus = execution.getBatchStatus(); + } + return execution; + } + + /** + * Restarts a job and waits for it's status to be any of {@link BatchStatus#STOPPED}, + * {@link BatchStatus#COMPLETED}, or {@link BatchStatus#FAILED}. If the job does not + * reach one of those statuses within the given timeout, a {@link java.util.concurrent.TimeoutException} is + * thrown. + * + * @param executionId the id of the job execution to restart. + * @param properties job parameters to be associated with the job. + * @param timeout maximum amount of time to wait in milliseconds. + * @return the {@link JobExecution} for the final state of the job + * @throws java.util.concurrent.TimeoutException if the timeout occurs + */ + public static JobExecution restartJob(long executionId, Properties properties, long timeout) throws TimeoutException { + long restartId = operator.restart(executionId, properties); + JobExecution execution = operator.getJobExecution(restartId); + + Date curDate = new Date(); + BatchStatus curBatchStatus = execution.getBatchStatus(); + + while(true) { + if(curBatchStatus == BatchStatus.STOPPED || curBatchStatus == BatchStatus.COMPLETED || curBatchStatus == BatchStatus.FAILED) { + break; + } + + if(new Date().getTime() - curDate.getTime() > timeout) { + throw new TimeoutException("Job processing did not complete in time"); + } + + execution = operator.getJobExecution(restartId); + curBatchStatus = execution.getBatchStatus(); + } + return execution; + } + + @Nullable + public static Metric getMetric(StepExecution stepExecution, Metric.MetricType type) { + Metric[] metrics = stepExecution.getMetrics(); + + for (Metric metric : metrics) { + if(metric.getType() == type) { + return metric; + } + } + + return null; + } + +} diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/MetaDataInstanceFactory.java b/spring-batch-test/src/main/java/org/springframework/batch/test/MetaDataInstanceFactory.java index de2d72f63d..5be0d82185 100644 --- a/spring-batch-test/src/main/java/org/springframework/batch/test/MetaDataInstanceFactory.java +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/MetaDataInstanceFactory.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -138,7 +138,7 @@ public static JobExecution createJobExecution(String jobName, Long instanceId, L */ public static JobExecution createJobExecution(String jobName, Long instanceId, Long executionId, JobParameters jobParameters) { - return new JobExecution(createJobInstance(jobName, instanceId), executionId, jobParameters); + return new JobExecution(createJobInstance(jobName, instanceId), executionId, jobParameters, null); } /** @@ -166,9 +166,10 @@ public static StepExecution createStepExecution(String stepName, Long executionI /** * Create a {@link StepExecution} with the parameters provided. * - * @param stepName the stepName for the {@link StepExecution} - * @param executionId the id for the {@link StepExecution} - * @return a {@link StepExecution} with the given {@link JobExecution} + * @param jobExecution instance of {@link JobExecution}. + * @param stepName the name for the {@link StepExecution}. + * @param executionId the id for the {@link StepExecution}. + * @return a {@link StepExecution} with the given {@link JobExecution}. */ public static StepExecution createStepExecution(JobExecution jobExecution, String stepName, Long executionId) { StepExecution stepExecution = jobExecution.createStepExecution(stepName); diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/StepRunner.java b/spring-batch-test/src/main/java/org/springframework/batch/test/StepRunner.java index a529b38d33..7452543765 100755 --- a/spring-batch-test/src/main/java/org/springframework/batch/test/StepRunner.java +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/StepRunner.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2018 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -40,6 +40,7 @@ import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.JobRestartException; import org.springframework.batch.item.ExecutionContext; +import org.springframework.lang.Nullable; /** * Utility class for executing steps outside of a {@link Job}. This is useful in @@ -61,6 +62,7 @@ * * @author Dan Garrette * @author Lucas Ward + * @author Mahmoud Ben Hassine * @since 2.0 * @see SimpleJob */ @@ -100,7 +102,7 @@ public JobExecution launchStep(Step step) { * loaded into the Job ExecutionContext prior to launching the step. * @return JobExecution */ - public JobExecution launchStep(Step step, ExecutionContext jobExecutionContext) { + public JobExecution launchStep(Step step, @Nullable ExecutionContext jobExecutionContext) { return this.launchStep(step, this.makeUniqueJobParameters(), jobExecutionContext); } @@ -126,7 +128,7 @@ public JobExecution launchStep(Step step, JobParameters jobParameters) { * loaded into the Job ExecutionContext prior to launching the step. * @return JobExecution */ - public JobExecution launchStep(Step step, JobParameters jobParameters, final ExecutionContext jobExecutionContext) { + public JobExecution launchStep(Step step, JobParameters jobParameters, @Nullable final ExecutionContext jobExecutionContext) { // // Create a fake job // @@ -134,7 +136,7 @@ public JobExecution launchStep(Step step, JobParameters jobParameters, final Exe job.setName("TestJob"); job.setJobRepository(this.jobRepository); - List stepsToExecute = new ArrayList(); + List stepsToExecute = new ArrayList<>(); stepsToExecute.add(step); job.setSteps(stepsToExecute); @@ -188,7 +190,7 @@ private JobExecution launchJob(Job job, JobParameters jobParameters) { * current timestamp, to ensure that the job instance will be unique */ private JobParameters makeUniqueJobParameters() { - Map parameters = new HashMap(); + Map parameters = new HashMap<>(); parameters.put("timestamp", new JobParameter(new Date().getTime())); return new JobParameters(parameters); } diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/StepScopeTestExecutionListener.java b/spring-batch-test/src/main/java/org/springframework/batch/test/StepScopeTestExecutionListener.java index c0783f49ac..7a08e5b3e0 100644 --- a/spring-batch-test/src/main/java/org/springframework/batch/test/StepScopeTestExecutionListener.java +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/StepScopeTestExecutionListener.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2010 the original author or authors. + * Copyright 2006-2014 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -61,11 +61,14 @@ * * * @author Dave Syer - * + * @author Chris Schaefer */ public class StepScopeTestExecutionListener implements TestExecutionListener { - private static final String STEP_EXECUTION = StepScopeTestExecutionListener.class.getName() + ".STEP_EXECUTION"; + private static final String SET_ATTRIBUTE_METHOD_NAME = "setAttribute"; + private static final String HAS_ATTRIBUTE_METHOD_NAME = "hasAttribute"; + private static final String GET_ATTRIBUTE_METHOD_NAME = "getAttribute"; + private static final String GET_TEST_INSTANCE_METHOD = "getTestInstance"; /** * Set up a {@link StepExecution} as a test context attribute. @@ -74,11 +77,13 @@ public class StepScopeTestExecutionListener implements TestExecutionListener { * @throws Exception if there is a problem * @see TestExecutionListener#prepareTestInstance(TestContext) */ - @Override + @Override public void prepareTestInstance(TestContext testContext) throws Exception { StepExecution stepExecution = getStepExecution(testContext); + if (stepExecution != null) { - testContext.setAttribute(STEP_EXECUTION, stepExecution); + Method method = TestContext.class.getMethod(SET_ATTRIBUTE_METHOD_NAME, String.class, Object.class); + ReflectionUtils.invokeMethod(method, testContext, STEP_EXECUTION, stepExecution); } } @@ -87,13 +92,17 @@ public void prepareTestInstance(TestContext testContext) throws Exception { * @throws Exception if there is a problem * @see TestExecutionListener#beforeTestMethod(TestContext) */ - @Override - public void beforeTestMethod(org.springframework.test.context.TestContext testContext) throws Exception { - if (testContext.hasAttribute(STEP_EXECUTION)) { - StepExecution stepExecution = (StepExecution) testContext.getAttribute(STEP_EXECUTION); + @Override + public void beforeTestMethod(TestContext testContext) throws Exception { + Method hasAttributeMethod = TestContext.class.getMethod(HAS_ATTRIBUTE_METHOD_NAME, String.class); + Boolean hasAttribute = (Boolean) ReflectionUtils.invokeMethod(hasAttributeMethod, testContext, STEP_EXECUTION); + + if (hasAttribute) { + Method method = TestContext.class.getMethod(GET_ATTRIBUTE_METHOD_NAME, String.class); + StepExecution stepExecution = (StepExecution) ReflectionUtils.invokeMethod(method, testContext, STEP_EXECUTION); + StepSynchronizationManager.register(stepExecution); } - } /** @@ -101,9 +110,12 @@ public void beforeTestMethod(org.springframework.test.context.TestContext testCo * @throws Exception if there is a problem * @see TestExecutionListener#afterTestMethod(TestContext) */ - @Override + @Override public void afterTestMethod(TestContext testContext) throws Exception { - if (testContext.hasAttribute(STEP_EXECUTION)) { + Method method = TestContext.class.getMethod(HAS_ATTRIBUTE_METHOD_NAME, String.class); + Boolean hasAttribute = (Boolean) ReflectionUtils.invokeMethod(method, testContext, STEP_EXECUTION); + + if (hasAttribute) { StepSynchronizationManager.close(); } } @@ -111,14 +123,14 @@ public void afterTestMethod(TestContext testContext) throws Exception { /* * Support for Spring 3.0 (empty). */ - @Override + @Override public void afterTestClass(TestContext testContext) throws Exception { } /* * Support for Spring 3.0 (empty). */ - @Override + @Override public void beforeTestClass(TestContext testContext) throws Exception { } @@ -130,8 +142,14 @@ public void beforeTestClass(TestContext testContext) throws Exception { * @return a {@link StepExecution} */ protected StepExecution getStepExecution(TestContext testContext) { + Object target; - Object target = testContext.getTestInstance(); + try { + Method method = TestContext.class.getMethod(GET_TEST_INSTANCE_METHOD); + target = ReflectionUtils.invokeMethod(method, testContext); + } catch (NoSuchMethodException e) { + throw new IllegalStateException("No such method " + GET_TEST_INSTANCE_METHOD + " on provided TestContext", e); + } ExtractorMethodCallback method = new ExtractorMethodCallback(StepExecution.class, "getStepExecution"); ReflectionUtils.doWithMethods(target.getClass(), method); @@ -173,7 +191,7 @@ public String getName() { return result == null ? null : result.getName(); } - @Override + @Override public void doWith(Method method) throws IllegalArgumentException, IllegalAccessException { Class type = method.getReturnType(); if (preferredType.isAssignableFrom(type)) { @@ -183,5 +201,4 @@ public void doWith(Method method) throws IllegalArgumentException, IllegalAccess } } } - } diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/StepScopeTestUtils.java b/spring-batch-test/src/main/java/org/springframework/batch/test/StepScopeTestUtils.java index 5232bfebea..23656940dc 100644 --- a/spring-batch-test/src/main/java/org/springframework/batch/test/StepScopeTestUtils.java +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/StepScopeTestUtils.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/context/BatchTestContextCustomizer.java b/spring-batch-test/src/main/java/org/springframework/batch/test/context/BatchTestContextCustomizer.java new file mode 100644 index 0000000000..1fa068193d --- /dev/null +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/context/BatchTestContextCustomizer.java @@ -0,0 +1,54 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test.context; + +import org.springframework.batch.test.JobLauncherTestUtils; +import org.springframework.batch.test.JobRepositoryTestUtils; +import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; +import org.springframework.beans.factory.support.BeanDefinitionRegistry; +import org.springframework.beans.factory.support.RootBeanDefinition; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.test.context.ContextCustomizer; +import org.springframework.test.context.MergedContextConfiguration; +import org.springframework.util.Assert; + +/** + * {@link ContextCustomizer} implementation that adds batch test utility classes + * ({@link JobLauncherTestUtils} and {@link JobRepositoryTestUtils}) as beans in + * the test context. + * + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public class BatchTestContextCustomizer implements ContextCustomizer { + + private static final String JOB_LAUNCHER_TEST_UTILS_BEAN_NAME = "jobLauncherTestUtils"; + private static final String JOB_REPOSITORY_TEST_UTILS_BEAN_NAME = "jobRepositoryTestUtils"; + + @Override + public void customizeContext(ConfigurableApplicationContext context, MergedContextConfiguration mergedConfig) { + ConfigurableListableBeanFactory beanFactory = context.getBeanFactory(); + Assert.isInstanceOf(BeanDefinitionRegistry.class, beanFactory, + "The bean factory must be an instance of BeanDefinitionRegistry"); + BeanDefinitionRegistry registry = (BeanDefinitionRegistry) beanFactory; + + registry.registerBeanDefinition(JOB_LAUNCHER_TEST_UTILS_BEAN_NAME, + new RootBeanDefinition(JobLauncherTestUtils.class)); + registry.registerBeanDefinition(JOB_REPOSITORY_TEST_UTILS_BEAN_NAME, + new RootBeanDefinition(JobRepositoryTestUtils.class)); + } + +} diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/context/BatchTestContextCustomizerFactory.java b/spring-batch-test/src/main/java/org/springframework/batch/test/context/BatchTestContextCustomizerFactory.java new file mode 100644 index 0000000000..dbedc88955 --- /dev/null +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/context/BatchTestContextCustomizerFactory.java @@ -0,0 +1,41 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test.context; + +import java.util.List; + +import org.springframework.core.annotation.AnnotatedElementUtils; +import org.springframework.test.context.ContextConfigurationAttributes; +import org.springframework.test.context.ContextCustomizer; +import org.springframework.test.context.ContextCustomizerFactory; + +/** + * Factory for {@link BatchTestContextCustomizer}. + * + * @author Mahmoud Ben Hassine + * @since 4.1 + */ +public class BatchTestContextCustomizerFactory implements ContextCustomizerFactory { + + @Override + public ContextCustomizer createContextCustomizer(Class testClass, List configAttributes) { + if (AnnotatedElementUtils.hasAnnotation(testClass, SpringBatchTest.class)) { + return new BatchTestContextCustomizer(); + } + return null; + } + +} diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/context/SpringBatchTest.java b/spring-batch-test/src/main/java/org/springframework/batch/test/context/SpringBatchTest.java new file mode 100644 index 0000000000..c0761b7bbc --- /dev/null +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/context/SpringBatchTest.java @@ -0,0 +1,98 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test.context; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Inherited; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +import org.springframework.batch.test.JobLauncherTestUtils; +import org.springframework.batch.test.JobRepositoryTestUtils; +import org.springframework.batch.test.JobScopeTestExecutionListener; +import org.springframework.batch.test.StepScopeTestExecutionListener; +import org.springframework.test.context.TestExecutionListeners; + +/** + * Annotation that can be specified on a test class that runs Spring Batch based tests. + * Provides the following features over the regular Spring TestContext Framework: + *
          + *
        • Registers a {@link JobLauncherTestUtils} bean with the + * {@link BatchTestContextCustomizer#JOB_LAUNCHER_TEST_UTILS_BEAN_NAME} which can be used + * in tests for launching jobs and steps. + *
        • + *
        • Registers a {@link JobRepositoryTestUtils} bean + * with the {@link BatchTestContextCustomizer#JOB_REPOSITORY_TEST_UTILS_BEAN_NAME} + * which can be used in tests setup to create or remove job executions. + *
        • + *
        • Registers the {@link StepScopeTestExecutionListener} and {@link JobScopeTestExecutionListener} + * as test execution listeners which are required to test step/job scoped beans. + *
        • + *
        + *

        + * A typical usage of this annotation is like: + * + *

        + * @RunWith(SpringRunner.class)
        + * @SpringBatchTest
        + * @ContextConfiguration(classes = MyBatchJobConfiguration.class)
        + * public class MyBatchJobTests {
        + *
        + *    @@Autowired
        + *    private JobLauncherTestUtils jobLauncherTestUtils;
        + *
        + *    @@Autowired
        + *    private JobRepositoryTestUtils jobRepositoryTestUtils;
        + *
        + *    @Before
        + *    public void clearJobExecutions() {
        + *       this.jobRepositoryTestUtils.removeJobExecutions();
        + *    }
        + *
        + *    @Test
        + *    public void testMyJob() throws Exception {
        + *       // given
        + *       JobParameters jobParameters = this.jobLauncherTestUtils.getUniqueJobParameters();
        + *
        + *       // when
        + *       JobExecution jobExecution = this.jobLauncherTestUtils.launchJob(jobParameters);
        + *
        + *       // then
        + *       Assert.assertEquals(ExitStatus.COMPLETED, jobExecution.getExitStatus());
        + *    }
        + *
        + * }
        + * 
        + * + * @author Mahmoud Ben Hassine + * @since 4.1 + * @see JobLauncherTestUtils + * @see JobRepositoryTestUtils + * @see StepScopeTestExecutionListener + * @see JobScopeTestExecutionListener + */ +@Target(ElementType.TYPE) +@Retention(RetentionPolicy.RUNTIME) +@Documented +@Inherited +@TestExecutionListeners( + listeners = {StepScopeTestExecutionListener.class, JobScopeTestExecutionListener.class}, + mergeMode = TestExecutionListeners.MergeMode.MERGE_WITH_DEFAULTS +) +public @interface SpringBatchTest { +} diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/context/package-info.java b/spring-batch-test/src/main/java/org/springframework/batch/test/context/package-info.java new file mode 100644 index 0000000000..32d298c926 --- /dev/null +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/context/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * APIs for the configuration of Spring Batch test support. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.test.context; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-test/src/main/java/org/springframework/batch/test/package-info.java b/spring-batch-test/src/main/java/org/springframework/batch/test/package-info.java new file mode 100644 index 0000000000..747af4e482 --- /dev/null +++ b/spring-batch-test/src/main/java/org/springframework/batch/test/package-info.java @@ -0,0 +1,25 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Utility classes for batch job/step testing. + * + * @author Mahmoud Ben Hassine + */ +@NonNullApi +package org.springframework.batch.test; + +import org.springframework.lang.NonNullApi; diff --git a/spring-batch-test/src/main/resources/META-INF/spring.factories b/spring-batch-test/src/main/resources/META-INF/spring.factories new file mode 100644 index 0000000000..98e6e569c6 --- /dev/null +++ b/spring-batch-test/src/main/resources/META-INF/spring.factories @@ -0,0 +1,4 @@ +# Batch ContextCustomizerFactory implementation for the Spring TestContext Framework +# +org.springframework.test.context.ContextCustomizerFactory= \ +org.springframework.batch.test.context.BatchTestContextCustomizerFactory diff --git a/spring-batch-test/src/site/apt/changelog.apt b/spring-batch-test/src/site/apt/changelog.apt deleted file mode 100644 index 9525fae9e7..0000000000 --- a/spring-batch-test/src/site/apt/changelog.apt +++ /dev/null @@ -1,15 +0,0 @@ -Changelog: Spring Batch Test - -* 2.0.0.RC1 - * BATCH-1049: Add compile scope to spring-test in spring-batch-test - * BATCH-1048: Add site docos for the Test project - -* 2.0.0.M4 - - * BATCH-1045: Move DataSourceInitializer to the test project's "main" folder - * BATCH-1032: Modify AbstractJobTests in test project to be able to launch FlowJob steps individually - -* 2.0.0.M3 - - * BATCH-933: AssertFile class for checking equality of files in unit tests - * BATCH-903: Create test project diff --git a/spring-batch-test/src/site/apt/index.apt b/spring-batch-test/src/site/apt/index.apt deleted file mode 100644 index 70e9beec4e..0000000000 --- a/spring-batch-test/src/site/apt/index.apt +++ /dev/null @@ -1,22 +0,0 @@ -Spring Batch Test - -* Overview - - This module contains code for facilitating the testing of batch jobs. The project - includes the following classes: - -*----+----+ -| <> | <> | -*----+----+ -| AbstractJobTests | End-to-end batch job testing as well as testing individual steps | -*----+----+ -| AssertFile | Checking equality of flat files | -*----+----+ -| DataSourceInitializer | Wrapper for a DataSource that can run scripts on start up and shut down | -*----+----+ -| JobRepositoryTestUtils | Create and remove JobExecution instances from a database | -*----+----+ -| MetaDataInstanceFactory | Create test instances of JobExecution, JobInstance and StepExecution | -*----+----+ -| StepRunner | Execute steps outside of a Job | -*----+----+ diff --git a/spring-batch-test/src/site/site.xml b/spring-batch-test/src/site/site.xml deleted file mode 100644 index 4aac57281e..0000000000 --- a/spring-batch-test/src/site/site.xml +++ /dev/null @@ -1,17 +0,0 @@ - - - - - - - - - - - - - - - - - diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/AbstractSampleJobTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/AbstractSampleJobTests.java index 64e08e8649..23cf140b1e 100644 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/AbstractSampleJobTests.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/AbstractSampleJobTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.test; import static org.junit.Assert.assertEquals; @@ -91,7 +106,7 @@ public void testStepLaunchJobContextEntry() { } private void verifyTasklet(int id) { - assertEquals(id, jdbcTemplate.queryForInt("SELECT ID from TESTS where NAME = 'SampleTasklet" + id + "'")); + assertEquals(id, jdbcTemplate.queryForObject("SELECT ID from TESTS where NAME = 'SampleTasklet" + id + "'", Integer.class).intValue()); } } diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/AssertFileTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/AssertFileTests.java index afb3fcaa68..4078f51b47 100644 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/AssertFileTests.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/AssertFileTests.java @@ -1,9 +1,24 @@ +/* + * Copyright 2008-2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.test; -import static junit.framework.Assert.assertTrue; -import static junit.framework.Assert.fail; -import junit.framework.AssertionFailedError; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; +import org.junit.ComparisonFailure; import org.junit.Test; import org.springframework.core.io.FileSystemResource; @@ -27,7 +42,7 @@ public void testAssertEquals_notEqual() throws Exception { executeAssertEquals("input1.txt", "input2.txt"); fail(); } - catch (AssertionFailedError e) { + catch (ComparisonFailure e) { assertTrue(e.getMessage().startsWith("Line number 3 does not match.")); } } @@ -38,7 +53,7 @@ public void testAssertEquals_tooLong() throws Exception { executeAssertEquals("input3.txt", "input1.txt"); fail(); } - catch (AssertionFailedError e) { + catch (AssertionError e) { assertTrue(e.getMessage().startsWith("More lines than expected. There should not be a line number 4.")); } } @@ -49,7 +64,7 @@ public void testAssertEquals_tooShort() throws Exception { executeAssertEquals("input1.txt", "input3.txt"); fail(); } - catch (AssertionFailedError e) { + catch (AssertionError e) { assertTrue(e.getMessage().startsWith("Line number 4 does not match.")); } } @@ -65,7 +80,7 @@ public void testAssertEquals_blank_tooLong() throws Exception { executeAssertEquals("blank.txt", "input1.txt"); fail(); } - catch (AssertionFailedError e) { + catch (AssertionError e) { assertTrue(e.getMessage().startsWith("More lines than expected. There should not be a line number 1.")); } } @@ -76,7 +91,7 @@ public void testAssertEquals_blank_tooShort() throws Exception { executeAssertEquals("input1.txt", "blank.txt"); fail(); } - catch (AssertionFailedError e) { + catch (AssertionError e) { assertTrue(e.getMessage().startsWith("Line number 1 does not match.")); } } diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/ExecutionContextTestUtilsTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/ExecutionContextTestUtilsTests.java index a5e1849a99..a5a5edb9ff 100644 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/ExecutionContextTestUtilsTests.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/ExecutionContextTestUtilsTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/JobLauncherTestUtilsTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/JobLauncherTestUtilsTests.java new file mode 100644 index 0000000000..3920f1ce85 --- /dev/null +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/JobLauncherTestUtilsTests.java @@ -0,0 +1,91 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test; + +import org.junit.Test; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepContribution; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.core.scope.context.ChunkContext; +import org.springframework.batch.core.step.tasklet.Tasklet; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.ApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.lang.Nullable; + +import static org.junit.Assert.assertEquals; + +/** + * @author mminella + */ +public class JobLauncherTestUtilsTests { + + @Test + public void testStepExecutionWithJavaConfig() { + ApplicationContext context = new AnnotationConfigApplicationContext(TestJobConfiguration.class); + + JobLauncherTestUtils testUtils = context.getBean(JobLauncherTestUtils.class); + + JobExecution execution = testUtils.launchStep("step1"); + + assertEquals(ExitStatus.COMPLETED, execution.getExitStatus()); + } + + @Configuration + @EnableBatchProcessing + public static class TestJobConfiguration { + + @Autowired + public JobBuilderFactory jobBuilderFactory; + + @Autowired + public StepBuilderFactory stepBuilderFactory; + + @Bean + public Step step() { + return stepBuilderFactory.get("step1").tasklet(new Tasklet() { + @Nullable + @Override + public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { + return null; + } + }).build(); + } + + @Bean + public Job job() { + return jobBuilderFactory.get("job").flow(step()).end().build(); + } + + @Bean + public JobLauncherTestUtils testUtils() { + JobLauncherTestUtils jobLauncherTestUtils = new JobLauncherTestUtils(); + jobLauncherTestUtils.setJob(job()); + + return jobLauncherTestUtils; + } + } + +} diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/JobRepositoryTestUtilsTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/JobRepositoryTestUtilsTests.java index 701aaad249..a915f9147a 100644 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/JobRepositoryTestUtilsTests.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/JobRepositoryTestUtilsTests.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2007 the original author or authors. + * Copyright 2006-2019 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -33,6 +33,7 @@ import org.springframework.batch.core.repository.JobRepository; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.lang.Nullable; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.test.jdbc.JdbcTestUtils; @@ -94,7 +95,7 @@ public void testCreateJobExecutions() throws Exception { @Test public void testRemoveJobExecutionsWithSameJobInstance() throws Exception { utils = new JobRepositoryTestUtils(jobRepository, dataSource); - List list = new ArrayList(); + List list = new ArrayList<>(); JobExecution jobExecution = jobRepository.createJobExecution("job", new JobParameters()); jobExecution.setEndTime(new Date()); list.add(jobExecution); @@ -135,7 +136,7 @@ public void testCreateJobExecutionsWithIncrementer() throws Exception { utils = new JobRepositoryTestUtils(jobRepository, dataSource); utils.setJobParametersIncrementer(new JobParametersIncrementer() { @Override - public JobParameters getNext(JobParameters parameters) { + public JobParameters getNext(@Nullable JobParameters parameters) { return new JobParametersBuilder().addString("foo","bar").toJobParameters(); } }); diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/JobScopeTestExecutionListenerIntegrationTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/JobScopeTestExecutionListenerIntegrationTests.java new file mode 100644 index 0000000000..2eea7c3c92 --- /dev/null +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/JobScopeTestExecutionListenerIntegrationTests.java @@ -0,0 +1,64 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemStream; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.TestExecutionListeners; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; +import org.springframework.test.context.support.DependencyInjectionTestExecutionListener; + +/** + * @author Dave Syer + * @author Mahmoud Ben Hassine + * @since 2.1 + */ +@ContextConfiguration +@TestExecutionListeners({ DependencyInjectionTestExecutionListener.class, JobScopeTestExecutionListener.class }) +@RunWith(SpringJUnit4ClassRunner.class) +public class JobScopeTestExecutionListenerIntegrationTests { + + @Autowired + private ItemReader reader; + + @Autowired + private ItemStream stream; + + public JobExecution getJobExecution() { + // Assert that dependencies are already injected... + assertNotNull(reader); + // Then create the execution for the job scope... + JobExecution execution = MetaDataInstanceFactory.createJobExecution(); + execution.getExecutionContext().putString("input.file", "classpath:/org/springframework/batch/test/simple.txt"); + return execution; + } + + @Test + public void testJob() throws Exception { + stream.open(new ExecutionContext()); + assertEquals("foo", reader.read()); + } + +} diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/JobScopeTestExecutionListenerTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/JobScopeTestExecutionListenerTests.java new file mode 100644 index 0000000000..6db1ea9756 --- /dev/null +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/JobScopeTestExecutionListenerTests.java @@ -0,0 +1,118 @@ +/* + * Copyright 2013 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; + +import org.junit.Test; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParametersBuilder; +import org.springframework.batch.core.scope.context.JobContext; +import org.springframework.batch.core.scope.context.JobSynchronizationManager; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.TestContext; +import org.springframework.test.context.TestContextManager; + +/** + * @author Dave Syer + * @since 2.1 + */ +@ContextConfiguration +public class JobScopeTestExecutionListenerTests { + + private JobScopeTestExecutionListener listener = new JobScopeTestExecutionListener(); + + @Test + public void testDefaultJobContext() throws Exception { + TestContext testContext = getTestContext(new Object()); + listener.prepareTestInstance(testContext); + listener.beforeTestMethod(testContext); + JobContext context = JobSynchronizationManager.getContext(); + assertNotNull(context); + listener.afterTestMethod(testContext); + assertNull(JobSynchronizationManager.getContext()); + } + + @Test + public void testWithJobExecutionFactory() throws Exception { + testExecutionContext(new WithJobExecutionFactory()); + } + + @Test + public void testWithParameters() throws Exception { + testJobParameters(new WithJobExecutionFactory()); + } + + private void testExecutionContext(Object target) throws Exception { + TestContext testContext = getTestContext(target); + listener.prepareTestInstance(testContext); + try { + listener.beforeTestMethod(testContext); + JobContext context = JobSynchronizationManager.getContext(); + assertNotNull(context); + assertEquals("bar", context.getJobExecutionContext().get("foo")); + } + finally { + listener.afterTestMethod(testContext); + } + assertNull(JobSynchronizationManager.getContext()); + } + + private void testJobParameters(Object target) throws Exception { + TestContext testContext = getTestContext(target); + listener.prepareTestInstance(testContext); + try { + listener.beforeTestMethod(testContext); + JobContext context = JobSynchronizationManager.getContext(); + assertNotNull(context); + assertEquals("spam", context.getJobParameters().get("foo")); + } + finally { + listener.afterTestMethod(testContext); + } + assertNull(JobSynchronizationManager.getContext()); + } + + @SuppressWarnings("unused") + private static class WithJobExecutionFactory { + public JobExecution getJobExecution() { + JobExecution jobExecution = MetaDataInstanceFactory.createJobExecution("job", 11L, 123L, + new JobParametersBuilder().addString("foo", "spam").toJobParameters()); + jobExecution.getExecutionContext().putString("foo", "bar"); + return jobExecution; + } + } + + private TestContext getTestContext(Object target) throws Exception { + return new MockTestContextManager(target, getClass()).getContext(); + } + + private final class MockTestContextManager extends TestContextManager { + + private MockTestContextManager(Object target, Class testClass) throws Exception { + super(testClass); + prepareTestInstance(target); + } + + public TestContext getContext() { + return getTestContext(); + } + + } + +} diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/MetaDataInstanceFactoryTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/MetaDataInstanceFactoryTests.java index ac31e4dba7..6c505c6082 100644 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/MetaDataInstanceFactoryTests.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/MetaDataInstanceFactoryTests.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/SampleFlowJobTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/SampleFlowJobTests.java index 7e0da03d37..1ab97987c5 100644 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/SampleFlowJobTests.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/SampleFlowJobTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.test; import org.junit.runner.RunWith; diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/SampleSimpleJobTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/SampleSimpleJobTests.java index f8f4e86ee3..916cab60e8 100644 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/SampleSimpleJobTests.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/SampleSimpleJobTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2009 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.test; import org.junit.runner.RunWith; diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/SampleStepTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/SampleStepTests.java index daca86a871..c5ce02aaea 100755 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/SampleStepTests.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/SampleStepTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.test; import static org.junit.Assert.*; @@ -50,7 +65,7 @@ public void tearDown() { public void testTasklet() { Step step = (Step) context.getBean("s2"); assertEquals(BatchStatus.COMPLETED, stepRunner.launchStep(step).getStatus()); - assertEquals(2, jdbcTemplate.queryForInt("SELECT ID from TESTS where NAME = 'SampleTasklet2'")); + assertEquals(2, jdbcTemplate.queryForObject("SELECT ID from TESTS where NAME = 'SampleTasklet2'", Integer.class).intValue()); } @Override diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/SpringBatchTestTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/SpringBatchTestTests.java new file mode 100644 index 0000000000..302e66a7fa --- /dev/null +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/SpringBatchTestTests.java @@ -0,0 +1,150 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test; + +import java.util.Arrays; +import javax.sql.DataSource; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.core.configuration.annotation.JobScope; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepScope; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.support.ListItemReader; +import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.batch.test.context.SpringBatchTest; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringRunner; + +/** + * Test cases for usage of {@link SpringBatchTest} annotation. + * + * @author Mahmoud Ben Hassine + */ +@RunWith(SpringRunner.class) +@SpringBatchTest +@ContextConfiguration(classes = SpringBatchTestTests.JobConfiguration.class) +public class SpringBatchTestTests { + + @Autowired + private JobLauncherTestUtils jobLauncherTestUtils; + + @Autowired + private JobRepositoryTestUtils jobRepositoryTestUtils; + + @Autowired + private ItemReader stepScopedItemReader; + + @Autowired + private ItemReader jobScopedItemReader; + + @Before + public void setUp() { + this.jobRepositoryTestUtils.removeJobExecutions(); + } + + public StepExecution getStepExecution() { + StepExecution execution = MetaDataInstanceFactory.createStepExecution(); + execution.getExecutionContext().putString("input.data", "foo,bar"); + return execution; + } + + public JobExecution getJobExecution() { + JobExecution execution = MetaDataInstanceFactory.createJobExecution(); + execution.getExecutionContext().putString("input.data", "foo,bar"); + return execution; + } + + @Test + public void testStepScopedItemReader() throws Exception { + Assert.assertEquals("foo", this.stepScopedItemReader.read()); + Assert.assertEquals("bar", this.stepScopedItemReader.read()); + Assert.assertNull(this.stepScopedItemReader.read()); + } + + @Test + public void testJobScopedItemReader() throws Exception { + Assert.assertEquals("foo", this.jobScopedItemReader.read()); + Assert.assertEquals("bar", this.jobScopedItemReader.read()); + Assert.assertNull(this.jobScopedItemReader.read()); + } + + @Test + public void testJob() throws Exception { + // when + JobExecution jobExecution = this.jobLauncherTestUtils.launchJob(); + + // then + Assert.assertEquals(ExitStatus.COMPLETED, jobExecution.getExitStatus()); + } + + @Configuration + @EnableBatchProcessing + public static class JobConfiguration { + + @Autowired + private JobBuilderFactory jobBuilderFactory; + + @Autowired + private StepBuilderFactory stepBuilderFactory; + + @Bean + public DataSource dataSource() { + return new EmbeddedDatabaseBuilder() + .setType(EmbeddedDatabaseType.HSQL) + .addScript("/org/springframework/batch/core/schema-drop-hsqldb.sql") + .addScript("/org/springframework/batch/core/schema-hsqldb.sql") + .build(); + } + + @Bean + @StepScope + public ItemReader stepScopedItemReader(@Value("#{stepExecutionContext['input.data']}") String data) { + return new ListItemReader<>(Arrays.asList(data.split(","))); + } + + @Bean + @JobScope + public ItemReader jobScopedItemReader(@Value("#{jobExecutionContext['input.data']}") String data) { + return new ListItemReader<>(Arrays.asList(data.split(","))); + } + + @Bean + public Job job() { + return this.jobBuilderFactory.get("job") + .start(this.stepBuilderFactory.get("step") + .tasklet((contribution, chunkContext) -> RepeatStatus.FINISHED) + .build()) + .build(); + } + } +} diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/StepScopeAnnotatedListenerIntegrationTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/StepScopeAnnotatedListenerIntegrationTests.java new file mode 100644 index 0000000000..f84aeeebda --- /dev/null +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/StepScopeAnnotatedListenerIntegrationTests.java @@ -0,0 +1,160 @@ +/* + * Copyright 2014-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test; + +import static org.junit.Assert.assertEquals; + +import java.util.ArrayList; +import java.util.List; + +import javax.sql.DataSource; + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.batch.core.ExitStatus; +import org.springframework.batch.core.Job; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.Step; +import org.springframework.batch.core.StepExecution; +import org.springframework.batch.core.annotation.AfterStep; +import org.springframework.batch.core.annotation.BeforeStep; +import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; +import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; +import org.springframework.batch.core.configuration.annotation.StepScope; +import org.springframework.batch.item.ItemProcessor; +import org.springframework.batch.item.ItemReader; +import org.springframework.batch.item.ItemWriter; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder; +import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType; +import org.springframework.lang.Nullable; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +@ContextConfiguration +@RunWith(SpringJUnit4ClassRunner.class) +public class StepScopeAnnotatedListenerIntegrationTests { + + @Autowired + JobLauncherTestUtils jobLauncherTestUtils; + + @Test + public void test() { + JobExecution jobExecution = jobLauncherTestUtils.launchStep("step-under-test"); + + assertEquals(ExitStatus.COMPLETED, jobExecution.getExitStatus()); + } + + public static class StatefulItemReader implements ItemReader { + + private List list; + + @BeforeStep + public void initializeState(StepExecution stepExecution) { + this.list = new ArrayList<>(); + } + + @AfterStep + public ExitStatus exploitState(StepExecution stepExecution) { + System.out.println("******************************"); + System.out.println(" READING RESULTS : " + list.size()); + + return stepExecution.getExitStatus(); + } + + @Nullable + @Override + public String read() throws Exception { + this.list.add("some stateful reading information"); + if (list.size() < 10) { + return "value " + list.size(); + } + return null; + } + } + + @Configuration + @EnableBatchProcessing + public static class TestConfig { + @Autowired + private JobBuilderFactory jobBuilder; + @Autowired + private StepBuilderFactory stepBuilder; + + @Bean + JobLauncherTestUtils jobLauncherTestUtils() { + return new JobLauncherTestUtils(); + } + + @Bean + public DataSource dataSource() { + EmbeddedDatabaseBuilder embeddedDatabaseBuilder = new EmbeddedDatabaseBuilder(); + return embeddedDatabaseBuilder.addScript("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql") + .addScript("classpath:org/springframework/batch/core/schema-hsqldb.sql") + .setType(EmbeddedDatabaseType.HSQL) + .build(); + } + + @Bean + public Job jobUnderTest() { + return jobBuilder.get("job-under-test") + .start(stepUnderTest()) + .build(); + } + + @Bean + public Step stepUnderTest() { + return stepBuilder.get("step-under-test") + .chunk(1) + .reader(reader()) + .processor(processor()) + .writer(writer()) + .build(); + } + + @Bean + @StepScope + public StatefulItemReader reader() { + return new StatefulItemReader(); + } + + @Bean + public ItemProcessor processor() { + return new ItemProcessor() { + + @Nullable + @Override + public String process(String item) throws Exception { + return item; + } + }; + } + + @Bean + public ItemWriter writer() { + return new ItemWriter() { + + @Override + public void write(List items) + throws Exception { + } + }; + } + } +} diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/StepScopeTestExecutionListenerIntegrationTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/StepScopeTestExecutionListenerIntegrationTests.java index 863158f4c6..a39f161fbb 100644 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/StepScopeTestExecutionListenerIntegrationTests.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/StepScopeTestExecutionListenerIntegrationTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.test; import static org.junit.Assert.assertEquals; @@ -17,6 +32,7 @@ /** * @author Dave Syer + * @author Mahmoud Ben Hassine * @since 2.1 */ @ContextConfiguration @@ -30,7 +46,7 @@ public class StepScopeTestExecutionListenerIntegrationTests { @Autowired private ItemStream stream; - public StepExecution getStepExection() { + public StepExecution getStepExecution() { // Assert that dependencies are already injected... assertNotNull(reader); // Then create the execution for the step scope... diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/StepScopeTestExecutionListenerTests.java b/spring-batch-test/src/test/java/org/springframework/batch/test/StepScopeTestExecutionListenerTests.java index cae61a8fb3..acf8898d5c 100644 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/StepScopeTestExecutionListenerTests.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/StepScopeTestExecutionListenerTests.java @@ -1,3 +1,18 @@ +/* + * Copyright 2010 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.test; import static org.junit.Assert.assertEquals; diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/common/LogAdvice.java b/spring-batch-test/src/test/java/org/springframework/batch/test/common/LogAdvice.java index 146045badd..75bd8be74a 100755 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/common/LogAdvice.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/common/LogAdvice.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -36,7 +36,7 @@ public class LogAdvice { */ public void doBasicLogging(JoinPoint pjp) throws Throwable { Object[] args = pjp.getArgs(); - StringBuffer output = new StringBuffer(); + StringBuilder output = new StringBuilder(); output.append(pjp.getTarget().getClass().getName()).append(": "); output.append(pjp.toShortString()).append(": "); diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/context/BatchTestContextCustomizerFactoryTest.java b/spring-batch-test/src/test/java/org/springframework/batch/test/context/BatchTestContextCustomizerFactoryTest.java new file mode 100644 index 0000000000..9b3422405f --- /dev/null +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/context/BatchTestContextCustomizerFactoryTest.java @@ -0,0 +1,68 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test.context; + +import java.util.Collections; +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; + +import org.springframework.test.context.ContextConfigurationAttributes; +import org.springframework.test.context.ContextCustomizer; + +/** + * @author Mahmoud Ben Hassine + */ +public class BatchTestContextCustomizerFactoryTest { + + private BatchTestContextCustomizerFactory factory = new BatchTestContextCustomizerFactory(); + + @Test + public void testCreateContextCustomizer_whenAnnotationIsPresent() { + // given + Class testClass = MyJobTest.class; + List configAttributes = Collections.emptyList(); + + // when + ContextCustomizer contextCustomizer = this.factory.createContextCustomizer(testClass, configAttributes); + + // then + Assert.assertNotNull(contextCustomizer); + } + + @Test + public void testCreateContextCustomizer_whenAnnotationIsAbsent() { + // given + Class testClass = MyOtherJobTest.class; + List configAttributes = Collections.emptyList(); + + // when + ContextCustomizer contextCustomizer = this.factory.createContextCustomizer(testClass, configAttributes); + + // then + Assert.assertNull(contextCustomizer); + } + + @SpringBatchTest + private static class MyJobTest { + + } + + private static class MyOtherJobTest { + + } +} diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/context/BatchTestContextCustomizerTest.java b/spring-batch-test/src/test/java/org/springframework/batch/test/context/BatchTestContextCustomizerTest.java new file mode 100644 index 0000000000..29ec878976 --- /dev/null +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/context/BatchTestContextCustomizerTest.java @@ -0,0 +1,66 @@ +/* + * Copyright 2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.test.context; + +import org.junit.Assert; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; +import org.mockito.Mockito; + +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.support.GenericApplicationContext; +import org.springframework.test.context.MergedContextConfiguration; + +/** + * @author Mahmoud Ben Hassine + */ +public class BatchTestContextCustomizerTest { + + @Rule + public ExpectedException expectedException = ExpectedException.none(); + + private BatchTestContextCustomizer contextCustomizer = new BatchTestContextCustomizer(); + + @Test + public void testCustomizeContext() { + // given + ConfigurableApplicationContext context = new GenericApplicationContext(); + MergedContextConfiguration mergedConfig = Mockito.mock(MergedContextConfiguration.class); + + // when + this.contextCustomizer.customizeContext(context, mergedConfig); + + // then + Assert.assertTrue(context.containsBean("jobLauncherTestUtils")); + Assert.assertTrue(context.containsBean("jobRepositoryTestUtils")); + } + + @Test + public void testCustomizeContext_whenBeanFactoryIsNotAnInstanceOfBeanDefinitionRegistry() { + // given + ConfigurableApplicationContext context = Mockito.mock(ConfigurableApplicationContext.class); + MergedContextConfiguration mergedConfig = Mockito.mock(MergedContextConfiguration.class); + this.expectedException.expect(IllegalArgumentException.class); + this.expectedException.expectMessage("The bean factory must be an instance of BeanDefinitionRegistry"); + + // when + this.contextCustomizer.customizeContext(context, mergedConfig); + + // then + // expected exception + } +} diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/jmx/JobExecutionNotificationPublisher.java b/spring-batch-test/src/test/java/org/springframework/batch/test/jmx/JobExecutionNotificationPublisher.java index be53867021..48750e1dee 100755 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/jmx/JobExecutionNotificationPublisher.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/jmx/JobExecutionNotificationPublisher.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -31,7 +31,7 @@ * @author Dave Syer * @since 1.0 */ -public class JobExecutionNotificationPublisher implements ApplicationListener, NotificationPublisherAware { +public class JobExecutionNotificationPublisher implements ApplicationListener, NotificationPublisherAware { protected static final Log logger = LogFactory.getLog(JobExecutionNotificationPublisher.class); @@ -57,12 +57,10 @@ public void setNotificationPublisher(NotificationPublisher notificationPublisher * @see ApplicationListener#onApplicationEvent(ApplicationEvent) */ @Override - public void onApplicationEvent(ApplicationEvent applicationEvent) { - if (applicationEvent instanceof SimpleMessageApplicationEvent) { - String message = applicationEvent.toString(); - logger.info(message); - publish(message); - } + public void onApplicationEvent(SimpleMessageApplicationEvent applicationEvent) { + String message = applicationEvent.toString(); + logger.info(message); + publish(message); } /** diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/jmx/SimpleMessageApplicationEvent.java b/spring-batch-test/src/test/java/org/springframework/batch/test/jmx/SimpleMessageApplicationEvent.java index e62f2f35e5..70d8812d6e 100755 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/jmx/SimpleMessageApplicationEvent.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/jmx/SimpleMessageApplicationEvent.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2012 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.test.jmx; import org.springframework.context.ApplicationEvent; diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/jmx/StepExecutionApplicationEventAdvice.java b/spring-batch-test/src/test/java/org/springframework/batch/test/jmx/StepExecutionApplicationEventAdvice.java index 727491f6e1..30bfd72ee5 100755 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/jmx/StepExecutionApplicationEventAdvice.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/jmx/StepExecutionApplicationEventAdvice.java @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/sample/LoggingTasklet.java b/spring-batch-test/src/test/java/org/springframework/batch/test/sample/LoggingTasklet.java index 07f9256b82..803bc3f150 100644 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/sample/LoggingTasklet.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/sample/LoggingTasklet.java @@ -1,3 +1,18 @@ +/* + * Copyright 2012-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.test.sample; import org.apache.commons.logging.Log; @@ -6,6 +21,7 @@ import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; +import org.springframework.lang.Nullable; public class LoggingTasklet implements Tasklet { @@ -17,7 +33,8 @@ public LoggingTasklet(int id) { this.id = id; } - @Override + @Nullable + @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { logger.info("tasklet executing: id=" + id); return RepeatStatus.FINISHED; diff --git a/spring-batch-test/src/test/java/org/springframework/batch/test/sample/SampleTasklet.java b/spring-batch-test/src/test/java/org/springframework/batch/test/sample/SampleTasklet.java index e9a75575a4..f27dada351 100755 --- a/spring-batch-test/src/test/java/org/springframework/batch/test/sample/SampleTasklet.java +++ b/spring-batch-test/src/test/java/org/springframework/batch/test/sample/SampleTasklet.java @@ -1,3 +1,18 @@ +/* + * Copyright 2008-2019 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.batch.test.sample; import org.springframework.batch.core.JobExecution; @@ -10,6 +25,7 @@ import org.springframework.batch.repeat.RepeatStatus; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.lang.Nullable; public class SampleTasklet implements Tasklet { @@ -26,7 +42,8 @@ public SampleTasklet(int id) { this.id = id; } - @Override + @Nullable + @Override public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { this.jdbcTemplate.update("insert into TESTS(ID, NAME) values (?, 'SampleTasklet" + id + "')", id); diff --git a/spring-batch-test/src/test/resources/batch-hsql.properties b/spring-batch-test/src/test/resources/batch-hsql.properties index 16fad3d5ba..24d9bd62d2 100755 --- a/spring-batch-test/src/test/resources/batch-hsql.properties +++ b/spring-batch-test/src/test/resources/batch-hsql.properties @@ -1,7 +1,7 @@ # Placeholders batch.* # for HSQLDB: batch.jdbc.driver=org.hsqldb.jdbcDriver -batch.jdbc.url=jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true +batch.jdbc.url=jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true;hsqldb.tx=mvcc # use this one for a separate server process so you can inspect the results # (or add it to system properties with -D to override at run time). # batch.jdbc.url=jdbc:hsqldb:hsql://localhost:9005/samples diff --git a/spring-batch-test/src/test/resources/data-source-context.xml b/spring-batch-test/src/test/resources/data-source-context.xml index bbbb4e6db9..271a882e81 100755 --- a/spring-batch-test/src/test/resources/data-source-context.xml +++ b/spring-batch-test/src/test/resources/data-source-context.xml @@ -1,8 +1,7 @@ + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> @@ -17,7 +16,7 @@ - + diff --git a/spring-batch-test/src/test/resources/job-runner-context.xml b/spring-batch-test/src/test/resources/job-runner-context.xml index 5e56c1b71c..f21b39219f 100644 --- a/spring-batch-test/src/test/resources/job-runner-context.xml +++ b/spring-batch-test/src/test/resources/job-runner-context.xml @@ -1,8 +1,8 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd+http://www.springframework.org/schema/context%20https://www.springframework.org/schema/context/spring-context-3.1.xsd"> diff --git a/spring-batch-test/src/test/resources/jobs/sample-steps.xml b/spring-batch-test/src/test/resources/jobs/sample-steps.xml index 918b388513..25265f1b75 100644 --- a/spring-batch-test/src/test/resources/jobs/sample-steps.xml +++ b/spring-batch-test/src/test/resources/jobs/sample-steps.xml @@ -4,10 +4,10 @@ xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd - http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd + http://www.springframework.org/schema/batch https://www.springframework.org/schema/batch/spring-batch-2.2.xsd"> diff --git a/spring-batch-test/src/test/resources/jobs/sampleFlowJob.xml b/spring-batch-test/src/test/resources/jobs/sampleFlowJob.xml index 8f65adbc50..9e532d44a9 100644 --- a/spring-batch-test/src/test/resources/jobs/sampleFlowJob.xml +++ b/spring-batch-test/src/test/resources/jobs/sampleFlowJob.xml @@ -6,11 +6,11 @@ xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans-%20%20%20%20%20%20%20http://www.springframework.org/schema/beans/spring-beans-3.1.xsd+%20%20%20%20%20%20%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd%20%20%20%20%20%20%20%20http://www.springframework.org/schema/batch-%20%20%20%20%20%20%20http://www.springframework.org/schema/batch/spring-batch-2.2.xsd+%20%20%20%20%20%20%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd%20%20%20%20%20%20%20%20http://www.springframework.org/schema/aop-%20http://www.springframework.org/schema/aop/spring-aop-3.1.xsd"> + https://www.springframework.org/schema/aop/spring-aop-3.1.xsd"> diff --git a/spring-batch-test/src/test/resources/jobs/sampleSimpleJob.xml b/spring-batch-test/src/test/resources/jobs/sampleSimpleJob.xml index a40bb271f8..18d334b70e 100644 --- a/spring-batch-test/src/test/resources/jobs/sampleSimpleJob.xml +++ b/spring-batch-test/src/test/resources/jobs/sampleSimpleJob.xml @@ -3,9 +3,9 @@ xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" - http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd - http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.1.xsd - http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> + http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-3.1.xsd + http://www.springframework.org/schema/aop https://www.springframework.org/schema/aop/spring-aop-3.1.xsd + http://www.springframework.org/schema/tx https://www.springframework.org/schema/tx/spring-tx-3.1.xsd"> diff --git a/spring-batch-test/src/test/resources/log4j.properties b/spring-batch-test/src/test/resources/log4j.properties index 5e0f6839e2..25230c0a1f 100644 --- a/spring-batch-test/src/test/resources/log4j.properties +++ b/spring-batch-test/src/test/resources/log4j.properties @@ -1,7 +1,7 @@ log4j.rootCategory=INFO, stdout -log4j.appender.stdout=org.apache.log4j.ConsoleAppender -log4j.appender.stdout.layout=org.apache.log4j.PatternLayout +log4j.appender.stdout=org.apache.logging.log4j.core.appender.ConsoleAppender +log4j.appender.stdout.layout=org.apache.logging.log4j.core.layout.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %t %c{2} - %m%n log4j.category.org.apache.activemq=ERROR diff --git a/spring-batch-test/src/test/resources/org/springframework/batch/sample/config/common-context.xml b/spring-batch-test/src/test/resources/org/springframework/batch/sample/config/common-context.xml index d725c26e63..13a3f45adf 100755 --- a/spring-batch-test/src/test/resources/org/springframework/batch/sample/config/common-context.xml +++ b/spring-batch-test/src/test/resources/org/springframework/batch/sample/config/common-context.xml @@ -1,11 +1,7 @@ - + diff --git a/spring-batch-test/src/test/resources/org/springframework/batch/test/JobScopeTestExecutionListenerIntegrationTests-context.xml b/spring-batch-test/src/test/resources/org/springframework/batch/test/JobScopeTestExecutionListenerIntegrationTests-context.xml new file mode 100644 index 0000000000..23ad441f54 --- /dev/null +++ b/spring-batch-test/src/test/resources/org/springframework/batch/test/JobScopeTestExecutionListenerIntegrationTests-context.xml @@ -0,0 +1,21 @@ + + + + + + + + + + + + + + + + + + + diff --git a/spring-batch-test/src/test/resources/org/springframework/batch/test/JobScopeTestExecutionListenerTests-context.xml b/spring-batch-test/src/test/resources/org/springframework/batch/test/JobScopeTestExecutionListenerTests-context.xml new file mode 100644 index 0000000000..52b7107fa2 --- /dev/null +++ b/spring-batch-test/src/test/resources/org/springframework/batch/test/JobScopeTestExecutionListenerTests-context.xml @@ -0,0 +1,5 @@ + + + + \ No newline at end of file diff --git a/spring-batch-test/src/test/resources/org/springframework/batch/test/StepScopeTestExecutionListenerIntegrationTests-context.xml b/spring-batch-test/src/test/resources/org/springframework/batch/test/StepScopeTestExecutionListenerIntegrationTests-context.xml index 325009f69d..9f7a71c641 100644 --- a/spring-batch-test/src/test/resources/org/springframework/batch/test/StepScopeTestExecutionListenerIntegrationTests-context.xml +++ b/spring-batch-test/src/test/resources/org/springframework/batch/test/StepScopeTestExecutionListenerIntegrationTests-context.xml @@ -1,7 +1,7 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd+http://www.springframework.org/schema/batch%20https://www.springframework.org/schema/batch/spring-batch-2.2.xsd"> diff --git a/spring-batch-test/src/test/resources/org/springframework/batch/test/StepScopeTestExecutionListenerTests-context.xml b/spring-batch-test/src/test/resources/org/springframework/batch/test/StepScopeTestExecutionListenerTests-context.xml index c30a604c0d..0cc6252aa2 100644 --- a/spring-batch-test/src/test/resources/org/springframework/batch/test/StepScopeTestExecutionListenerTests-context.xml +++ b/spring-batch-test/src/test/resources/org/springframework/batch/test/StepScopeTestExecutionListenerTests-context.xml @@ -1,5 +1,5 @@ + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> diff --git a/spring-batch-test/src/test/resources/simple-job-launcher-context.xml b/spring-batch-test/src/test/resources/simple-job-launcher-context.xml index 73720e8bdd..54bdec8a49 100755 --- a/spring-batch-test/src/test/resources/simple-job-launcher-context.xml +++ b/spring-batch-test/src/test/resources/simple-job-launcher-context.xml @@ -1,11 +1,7 @@ + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.springframework.org/schema/beans%20https://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> @@ -39,7 +35,7 @@ diff --git a/spring-batch-test/template.mf b/spring-batch-test/template.mf deleted file mode 100644 index 8287064be5..0000000000 --- a/spring-batch-test/template.mf +++ /dev/null @@ -1,39 +0,0 @@ -Manifest-Version: 1.0 -Bundle-SymbolicName: org.springframework.batch.test -Bundle-Name: Spring Batch Test -Bundle-Vendor: Spring -Bundle-Version: ${version} -Bundle-ManifestVersion: 2 -Ignored-Existing-Headers: - Import-Package, - Export-Package -Import-Template: - com.ibatis.sqlmap.*;version="[2.3.0, 3.0.0)";resolution:=optional, - javax.jms.*;version=1.1.0;resolution:=optional, - javax.xml.stream.*;version=1.2.0;resolution:=optional, - javax.persistence.*;version="[1.0.0,2.0.0)";resolution:=optional, - org.aopalliance.*;version="[1.0.0, 2.0.0)";resolution:=optional, - org.apache.commons.logging.*;version="[1.1.0, 2.0.0)", - org.apache.commons.io.*;version="[1.4.0, 2.0.0)";resolution:=optional, - org.junit;version="[4.4.0,5.0.0)";resolution:=optional, - junit.framework;version="[4.4.0,5.0.0)";resolution:=optional, - org.hibernate.*;version="[3.2.5.ga, 3.3.0.ga)";resolution:=optional, - org.springframework.batch.*;version="[2.0.0, 3.0.0)", - org.springframework.aop.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.aop.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.beans.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.core.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.dao.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.test.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.context.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.jdbc.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.jms.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.orm.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.transaction.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.util.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.validation.*;version="[2.5.5, 4.0.0)";resolution:=optional, - org.springframework.oxm.*;version="[1.5.0, 2.0.0)";resolution:=optional, - org.springframework.xml.*;version="[1.5.0, 2.0.0)";resolution:=optional, - javax.sql.*;version="0", - javax.xml.namespace.*;version="0", - javax.xml.transform.*;version="0" diff --git a/src/api/overview.html b/src/api/overview.html new file mode 100644 index 0000000000..a14718f5f4 --- /dev/null +++ b/src/api/overview.html @@ -0,0 +1,17 @@ + + +

        + This document is the API specification for the Spring Batch +

        +
        +

        + For further API reference and developer documentation, see the + Spring + Batch reference documentation. + That documentation contains more detailed, developer-targeted + descriptions, with conceptual overviews, definitions of terms, + workarounds, and working code examples. +

        +
        + + diff --git a/src/assembly/license.txt b/src/assembly/license.txt index 261eeb9e9f..20e4bd8566 100644 --- a/src/assembly/license.txt +++ b/src/assembly/license.txt @@ -1,6 +1,6 @@ Apache License Version 2.0, January 2004 - http://www.apache.org/licenses/ + https://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION @@ -192,7 +192,7 @@ you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, diff --git a/src/assembly/notice.txt b/src/assembly/notice.txt index b45a527d0b..38732336c9 100644 --- a/src/assembly/notice.txt +++ b/src/assembly/notice.txt @@ -4,13 +4,13 @@ ====================================================================== This product includes software developed by - the Apache Software Foundation (http://www.apache.org). + the Apache Software Foundation (https://www.apache.org). The end-user documentation included with a redistribution, if any, must include the following acknowledgement: "This product includes software developed by the Spring Framework - Project (http://www.springframework.org)." + Project (https://www.spring.io)." Alternatively, this acknowledgement may appear in the software itself, if and wherever such third-party acknowledgements normally appear. diff --git a/src/dist/license.txt b/src/dist/license.txt new file mode 100644 index 0000000000..e84d62a49e --- /dev/null +++ b/src/dist/license.txt @@ -0,0 +1,279 @@ + Apache License + Version 2.0, January 2004 + https://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + +======================================================================= + +SPRING FRAMEWORK ${version} SUBCOMPONENTS: + +Spring Framework ${version} includes a number of subcomponents +with separate copyright notices and license terms. The product that +includes this file does not necessarily use all the open source +subcomponents referred to below. Your use of the source +code for these subcomponents is subject to the terms and +conditions of the following licenses. + + +>>> ASM 4.0 (org.ow2.asm:asm:4.0, org.ow2.asm:asm-commons:4.0): + +Copyright (c) 2000-2011 INRIA, France Telecom +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: + +1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + +3. Neither the name of the copyright holders nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF +THE POSSIBILITY OF SUCH DAMAGE. + +Copyright (c) 1999-2009, OW2 Consortium + + +>>> CGLIB 3.0 (cglib:cglib:3.0): + +Per the LICENSE file in the CGLIB JAR distribution downloaded from +https://sourceforge.net/projects/cglib/files/cglib3/3.0/cglib-3.0.jar/download, +CGLIB 3.0 is licensed under the Apache License, version 2.0, the text of which +is included above. + + +======================================================================= + +To the extent any open source subcomponents are licensed under the EPL and/or +other similar licenses that require the source code and/or modifications to +source code to be made available (as would be noted above), you may obtain a +copy of the source code corresponding to the binaries for such open source +components and modifications thereto, if any, (the "Source Files"), by +downloading the Source Files from https://www.springsource.org/download, or by +sending a request, with your name and address to: + + Pivotal, Inc., 875 Howard St, + San Francisco, CA 94103 + United States of America + +or email info@gopivotal.com. All such requests should clearly specify: + + OPEN SOURCE FILES REQUEST + Attention General Counsel + +Pivotal shall mail a copy of the Source Files to you on a CD or equivalent +physical medium. This offer to obtain a copy of the Source Files is valid for +three years from the date you acquired this Software product. diff --git a/src/dist/notice.txt b/src/dist/notice.txt new file mode 100644 index 0000000000..80430f4a36 --- /dev/null +++ b/src/dist/notice.txt @@ -0,0 +1,11 @@ +Spring Batch ${version} +Copyright (c) 2002-${copyright} Pivotal, Inc. + +This product is licensed to you under the Apache License, Version 2.0 +(the "License"). You may not use this product except in compliance with +the License. + +This product may include a number of subcomponents with separate +copyright notices and license terms. Your use of the source code for +these subcomponents is subject to the terms and conditions of the +subcomponent's license, as noted in the license.txt file. diff --git a/src/dist/readme.txt b/src/dist/readme.txt new file mode 100644 index 0000000000..67d3a7bfe1 --- /dev/null +++ b/src/dist/readme.txt @@ -0,0 +1,14 @@ +Spring Batch version ${version} +===================================================================================== + +To find out what has changed since earlier releases, see the 'Change Log' section at +https://jira.springsource.org/browse/BATCH + +Please consult the documentation located within the 'docs/spring-batch-reference' +directory of this release and also visit the official Spring Batch home at +https://projects.spring.io/spring-batch/ + +There you will find links to the forum, issue tracker, and other resources. + +See https://github.com/spring-projects/spring-batch#readme for additional +information including instructions on building from source. diff --git a/src/docbkx/resources/xsl/html_chunk.xsl b/src/docbkx/resources/xsl/html_chunk.xsl index 2ff1692f32..878c3cea4b 100644 --- a/src/docbkx/resources/xsl/html_chunk.xsl +++ b/src/docbkx/resources/xsl/html_chunk.xsl @@ -73,11 +73,11 @@ @@ -187,8 +187,8 @@ - Sponsored by SpringSource + Sponsored by Pivotal diff --git a/src/it/settings.xml b/src/it/settings.xml deleted file mode 100644 index a492aab1c3..0000000000 --- a/src/it/settings.xml +++ /dev/null @@ -1,35 +0,0 @@ - - - - - it-repo - - true - - - - local.central - @localRepositoryUrl@ - - true - - - true - - - - - - local.central - @localRepositoryUrl@ - - true - - - true - - - - - - \ No newline at end of file diff --git a/src/site/apt/articles.apt b/src/site/apt/articles.apt deleted file mode 100644 index 6f2ec23579..0000000000 --- a/src/site/apt/articles.apt +++ /dev/null @@ -1,28 +0,0 @@ - ------ - Spring Batch in the Media - ------ - Dave Syer - ------ - November 2007 - -Spring Batch In the Media - - * http://www.theserverside.com/tt/articles/article.tss?l=SpringBatchOverview - - * http://www.theserverside.com/news/thread.tss?thread_id=47506#242493 - - * http://blog.springsource.com/main/2007/05/07/spring-batch - - * http://blog.decaresystems.ie/index.php/2007/04/12/spring-batch/ - - * http://www.itweek.co.uk/itweek/news/2189502/accenture-launches-batch - - * http://www.theserverside.com/tt/articles/article.tss?l=SpringBatchOverview - - * http://www.springframework.org/node/698 - - * http://www.infoq.com/interviews/johnson-spring-portfolio - - * http://www.infoq.com/news/2008/06/spring-batch - - * http://blog.springsource.com/main/2008/05/30/running-a-spring-batch-job-in-the-springsource-aplication-platform diff --git a/src/site/apt/batch-principles-guidelines.apt b/src/site/apt/batch-principles-guidelines.apt deleted file mode 100644 index eecbde5771..0000000000 --- a/src/site/apt/batch-principles-guidelines.apt +++ /dev/null @@ -1,40 +0,0 @@ - ------ - Spring Batch Overview - ------ - Scott Wintermute - ------ - May 2007 - -General Principles and Guidelines for Batch Architectures - - The following are a number of key principles, guidelines, and general considerations to take into consideration when building a batch solution. - - * A batch architecture typically affects on-line architecture and vice versa. Design with both architectures and environments in mind using common building blocks when possible. - - * Simplify as much as possible and avoid building complex logical structures in single batch applications. - - * Process data as close to where the data physically resides as possible or vice versa (i.e., keep your data where your processing occurs). - - * Minimize system resource use, especially I/O. Perform as many operations as possible in internal memory. - - * Review application I/O (analyze SQL statements) to ensure that unnecessary physical I/O is avoided. In particular, the following four common flaws need to be looked for: - - ** Reading data for every transaction when the data could be read once and kept cached or in the working storage; - - ** Rereading data for a transaction where the data was read earlier in the same transaction; - - ** Causing unnecessary table or index scans; - - ** Not specifying key values in the WHERE clause of an SQL statement. - - * Do not do things twice in a batch run. For instance, if you need data summarization for reporting purposes, increment stored totals if possible when data is being initially processed, so your reporting application does not have to reprocess the same data. - - * Allocate enough memory at the beginning of a batch application to avoid time-consuming reallocation during the process. - - * Always assume the worst with regard to data integrity. Insert adequate checks and record validation to maintain data integrity. - - * Implement checksums for internal validation where possible. For example, flat files should have a trailer record telling the total of records in the file and an aggregate of the key fields. - - * Plan and execute stress tests as early as possible in a production-like environment with realistic data volumes. - - * In large batch systems backups can be challenging, especially if the system is running concurrent with on-line on a 24-7 basis. Database backups are typically well taken care of in the on-line design, but file backups should be considered to be just as important. If the system depends on flat files, file backup procedures should not only be in place and documented, but regularly tested as well. diff --git a/src/site/apt/batch-processing-strategies.apt b/src/site/apt/batch-processing-strategies.apt deleted file mode 100644 index 233959d779..0000000000 --- a/src/site/apt/batch-processing-strategies.apt +++ /dev/null @@ -1,196 +0,0 @@ - ------ - Batch Processing Strategy - ------ - Scott Wintermute, Wayne Lund - ------ - May 2007 - -Batch Processing Strategies - - To help design and implement batch systems, basic batch application building blocks and patterns should be provided to the designers and programmers in form of sample structure charts and code shells. When starting to design a batch job, the business logic should be decomposed into a series of steps which can be implemented using the following standard building blocks: - - * <> For each type of file supplied by or generated to an external system, a conversion application will need to be created to convert the transaction records supplied into a standard format required for processing. This type of batch application can partly or entirely consist of translation utility modules (see Basic Batch Services). - - * <> Validation applications ensure that all input/output records are correct and consistent. Validation is typically based on file headers and trailers, checksums and validation algorithms as well as record level cross-checks. - - * <> An application that reads a set of records from a database or input file, selects records based on predefined rules, and writes the records to an output file. - - * <> An application that reads records from a database or an input file, and makes changes to a database or an output file driven by the data found in each input record. - - * <> An application that performs processing on input transactions from an extract or a validation application. The processing will usually involve reading a database to obtain data required for processing, potentially updating the database and creating records for output processing. - - * <> Applications reading an input file, restructures data from this record according to a standard format, and produces an output file for printing or transmission to another program or system. - - Additionally a basic application shell should be provided for business logic that cannot be built using the previously mentioned building blocks. - - In addition to the main building blocks, each application may use one or more of standard utility steps, such as: - - * Sort - A Program that reads an input file and produces an output file where records have been re-sequenced according to a sort key field in the records. Sorts are usually performed by standard system utilities. - - * Split - A program that reads a single input file, and writes each record to one of several output files based on a field value. Splits can be tailored or performed by parameter-driven standard system utilities. - - * Merge - A program that reads records from multiple input files and produces one output file with combined data from the input files. Merges can be tailored or performed by parameter-driven standard system utilities. - - Batch applications can additionally be categorized by their input source: - - * Database-driven applications are driven by rows or values retrieved from the database. - - * File-driven applications are driven by records or values retrieved from a file. - - * Message-driven applications are driven by messages retrieved from a message queue. - - The foundation of any batch system is the processing strategy. Factors affecting the selection of the strategy include: estimated batch system volume, concurrency with on-line or with another batch systems, available batch windows (and with more enterprises wanting to be up and running 24x7, this leaves no obvious batch windows). - - Typical processing options for batch are: - - * Normal processing in a batch window during off-line - - * Concurrent batch / on-line processing - - * Parallel processing of many different batch runs or jobs at the same time - - * Partitioning (i.e. processing of many instances of the same job at the same time) - - * A combination of these - - The order in the list above reflects the implementation complexity, processing in a batch window being the easiest and partitioning the most complex to implement. - - Some or all of these options may be supported by a commercial scheduler. - - In the following section these processing options are discussed in more detail. It is important to notice that the commit and locking strategy adopted by batch processes will be dependent on the type of processing performed, and as a rule of thumb and the on-line locking strategy should also use the same principles. Therefore, the batch architecture cannot be simply an afterthought when designing an overall architecture. - - The locking strategy can use only normal database locks, or an additional custom locking service can be implemented in the architecture. The locking service would track database locking (for example by storing the necessary information in a dedicated db-table) and give or deny permissions to the application programs requesting a db operation. Retry logic could also be implemented by this architecture to avoid aborting a batch job in case of a lock situation. - - <<1. Normal processing in a batch window>> - For simple batch processes running in a separate batch window, where the data being updated is not required by on-line users or other batch processes, concurrency is not an issue and a single commit can be done at the end of the batch run. - - In most cases a more robust approach is more appropriate. A thing to keep in mind is that batch systems have a tendency to grow as time goes by, both in terms of complexity and the data volumes they will handle. If no locking strategy is in place and the system still relies on a single commit point, modifying the batch programs can be painful. Therefore, even with the simplest batch systems, consider the need for commit logic for restart-recovery options as well as the information concerning the more complex cases below. - - <<2. Concurrent batch / on-line processing>> - Batch applications processing data that can simultaneously be updated by on-line users, should not lock any data (either in the database or in files) which could be required by on-line users for more than a few seconds. Also updates should be committed to the database at the end of every few transaction. This minimizes the portion of data that is unavailable to other processes and the elapsed time the data is unavailable. - - Another option to minimize physical locking is to have a logical row-level locking implemented using either an Optimistic Locking Pattern or a Pessimistic Locking Pattern. - - * Optimistic locking assumes a low likelihood of record contention. It typically means inserting a timestamp column in each database table used concurrently by both batch and on-line processing. When an application fetches a row for processing, it also fetches the timestamp. As the application then tries to update the processed row, the update uses the original timestamp in the WHERE clause. If the timestamp matches, the data and the timestamp will be updated successfully. If the timestamp does not match, this indicates that another application has updated the same row between the fetch and the update attempt and therefore the update cannot be performed. - - * Pessimistic locking is any locking strategy that assumes there is a high likelihood of record contention and therefore either a physical or logical lock needs to be obtained at retrieval time. One type of pessimistic logical locking uses a dedicated lock-column in the database table. When an application retrieves the row for update, it sets a flag in the lock column. With the flag in place, other applications attempting to retrieve the same row will logically fail. When the application that set the flag updates the row, it also clears the flag, enabling the row to be retrieved by other applications. Please note, that the integrity of data must be maintained also between the initial fetch and the setting of the flag, for example by using db locks (e.g., SELECT FOR UPDATE). Note also that this method suffers from the same downside as physical locking except that it is somewhat easier to manage building a time-out mechanism that will get the lock released if the user goes to lunch while the record is locked. - - These patterns are not necessarily suitable for batch processing, but they might be used for concurrent batch and on-line processing (e.g. in cases where the database doesn't support row-level locking). As a general rule, optimistic locking is more suitable for on-line applications, while pessimistic locking is more suitable for batch applications. Whenever logical locking is used, the same scheme must be used for all applications accessing data entities protected by logical locks. - - Note that both of these solutions only address locking a single record. Often we may need to lock a logically related group of records. With physical locks, you have to manage these very carefully in order to avoid potential deadlocks. With logical locks, it is usually best to build a logical lock manager that understands the logical record groups you want to protect and can ensure that locks are coherent and non-deadlocking. This logical lock manager usually uses its own tables for lock management, contention reporting, time-out mechanism, etc. - - <<3. Parallel Processing>> - Parallel processing allows multiple batch runs / jobs to run in parallel to minimize the total elapsed batch processing time. This is not a problem as long as the jobs are not sharing the same files, db-tables or index spaces. If they do, this service should be implemented using partitioned data. Another option is to build an architecture module for maintaining interdependencies using a control table. A control table should contain a row for each shared resource and whether it is in use by an application or not. The batch architecture or the application in a parallel job would then retrieve information from that table to determine if it can get access to the resource it needs or not. - - If the data access is not a problem, parallel processing can be implemented through the use of additional threads to process in parallel. In the mainframe environment, parallel job classes have traditionally been used, in order to ensure adequate CPU time for all the processes. Regardless, the solution has to be robust enough to ensure time slices for all the running processes. - - Other key issues in parallel processing include load balancing and the availability of general system resources such as files, database buffer pools etc. Also note that the control table itself can easily become a critical resource. - - <<4. Partitioning>> - Using partitioning allows multiple versions of large batch applications to run concurrently. The purpose of this is to reduce the elapsed time required to process long batch jobs. Processes which can be successfully partitioned are those where the input file can be split and/or the main database tables partitioned to allow the application to run against different sets of data. - - In addition, processes which are partitioned must be designed to only process their assigned data set. A partitioning architecture has to be closely tied to the database design and the database partitioning strategy. Please note, that the database partitioning doesn't necessarily mean physical partitioning of the database, although in most cases this is advisable. The following picture illustrates the partitioning approach: - -[images/partitioned.png] - - - The architecture should be flexible enough to allow dynamic configuration of the number of partitions. Both automatic and user controlled configuration should be considered. Automatic configuration may be based on parameters such as the input file size and/or the number of input records. - - <<4.1 Partitioning Approaches>> - The following lists some of the possible partitioning approaches. Selecting a partitioning approach has to be done on a case-by-case basis. - - <1. Fixed and Even Break-Up of Record Set> - - This involves breaking the input record set into an even number of portions (e.g. 10, where each portion will have exactly 1/10th of the entire record set). Each portion is then processed by one instance of the batch/extract application. - - In order to use this approach, preprocessing will be required to split the recordset up. The result of this split will be a lower and upper bound placement number which can be used as input to the batch/extract application in order to restrict its processing to its portion alone. - - Preprocessing could be a large overhead as it has to calculate and determine the bounds of each portion of the record set. - - <2. Breakup by a Key Column> - - This involves breaking up the input record set by a key column such as a location code, and assigning data from each key to a batch instance. In order to achieve this, column values can either be - - <3. Assigned to a batch instance via a partitioning table (see below for details).> - - <4. Assigned to a batch instance by a portion of the value (e.g. values 0000-0999, 1000 - 1999, etc.)> - - Under option 1, addition of new values will mean a manual reconfiguration of the batch/extract to ensure that the new value is added to a particular instance. - - Under option 2, this will ensure that all values are covered via an instance of the batch job. However, the number of values processed by one instance is dependent on the distribution of column values (i.e. there may be a large number of locations in the 0000-0999 range, and few in the 1000-1999 range). Under this option, the data range should be designed with partitioning in mind. - - Under both options, the optimal even distribution of records to batch instances cannot be realized. There is no dynamic configuration of the number of batch instances used. - - <5. Breakup by Views> - - This approach is basically breakup by a key column, but on the database level. It involves breaking up the recordset into views. These views will be used by each instance of the batch application during its processing. The breakup will be done by grouping the data. - - With this option, each instance of a batch application will have to be configured to hit a particular view (instead of the master table). Also, with the addition of new data values, this new group of data will have to be included into a view. There is no dynamic configuration capability, as a change in the number of instances will result in a change to the views. - - <6. Addition of a Processing Indicator> - - This involves the addition of a new column to the input table, which acts as an indicator. As a preprocessing step, all indicators would be marked to non-processed. During the record fetch stage of the batch application, records are read on the condition that that record is marked non-processed, and once they are read (with lock), they are marked processing. When that record is completed, the indicator is updated to either complete or error. Many instances of a batch application can be started without a change, as the additional column ensures that a record is only processed once. - - With this option, I/O on the table increases dynamically. In the case of an updating batch application, this impact is reduced, as a write will have to occur anyway. - - <7. Extract Table to a Flat File> - - This involves the extraction of the table into a file. This file can then be split into multiple segments and used as input to the batch instances. - - With this option, the additional overhead of extracting the table into a file, and splitting it, may cancel out the effect of multi-partitioning. Dynamic configuration can be achieved via changing the file splitting script. - - <8. Use of a Hashing Column> - - This scheme involves the addition of a hash column (key/index) to the database tables used to retrieve the driver record. This hash column will have an indicator to determine which instance of the batch application will process this particular row. For example, if there are three batch instances to be started, then an indicator of 'A' will mark that row for processing by instance 1, an indicator of 'B' will mark that row for processing by instance 2, etc. - - The procedure used to retrieve the records would then have an additional WHERE clause to select all rows marked by a particular indicator. The inserts in this table would involve the addition of the marker field, which would be defaulted to one of the instances (e.g. 'A'). - - A simple batch application would be used to update the indicators such as to redistribute the load between the different instances. When a sufficiently large number of new rows have been added, this batch can be run (anytime, except in the batch window) to redistribute the new rows to other instances. - - Additional instances of the batch application only require the running of the batch application as above to redistribute the indicators to cater for a new number of instances. - - - <<4.2 Database and Application design Principles>> - - An architecture that supports multi-partitioned applications which run against partitioned database tables using the key column approach, should include a central partition repository for storing partition parameters. This provides flexibility and ensures maintainability. The repository will generally consist of a single table known as the partition table. - - Information stored in the partition table will be static and in general should be maintained by the DBA. The table should consist of one row of information for each partition of a multi-partitioned application. The table should have columns for: Program ID Code, Partition Number (Logical ID of the partition), Low Value of the db key column for this partition, High Value of the db key column for this partition. - - On program start-up the program id and partition number should be passed to the application from the architecture (Control Processing Tasklet). These variables are used to read the partition table, to determine what range of data the application is to process (if a key column approach is used). In addition the partition number must be used throughout the processing to: - - * Add to the output files/database updates in order for the merge process to work properly - - * Report normal processing to the batch log and any errors that occur during execution to the architecture error handler - - <<4.3 Minimizing Deadlocks>> - When applications run in parallel or partitioned, contention in database resources and deadlocks may occur. It is critical that the database design team eliminates potential contention situations as far as possible as part of the database design. - - Also ensure that the database index tables are designed with deadlock prevention and performance in mind. - - Deadlocks or hot spots often occur in administration or architecture tables such as log tables, control tables, and lock tables. The implications of these should be taken into account as well. A realistic stress test is crucial for identifying the possible bottlenecks in the architecture. - - To minimize the impact of conflicts on data, the architecture should provide services such as wait-and-retry intervals when attaching to a database or when encountering a deadlock. This means a built-in mechanism to react to certain database return codes and instead of issuing an immediate error handling, waiting a predetermined amount of time and retrying the database operation. - - <<4.4 Parameter Passing and Validation>> - - The partition architecture should be relatively transparent to application developers. The architecture should perform all tasks associated with running the application in a partitioned mode including: - - * Retrieve partition parameters before application start-up - - * Validate partition parameters before application start-up - - * Pass parameters to application at start-up - - The validation should include checks to ensure that: - - * the application has sufficient partitions to cover the whole data range - - * there are no gaps between partitions - - If the database is partitioned, some additional validation may be necessary to ensure that a single partition does not span database partitions. - - Also the architecture should take into consideration the consolidation of partitions. Key questions include: - - * Must all the partitions be finished before going into the next job step? - - * What happens if one of the partitions aborts? diff --git a/src/site/apt/building.apt b/src/site/apt/building.apt deleted file mode 100644 index 8d14e5cc90..0000000000 --- a/src/site/apt/building.apt +++ /dev/null @@ -1,377 +0,0 @@ - ------ - Building Spring Batch - ------ - Dave Syer - ------ - April 2007 - -Building Spring Batch - - Spring Batch is organised as a reactor build in Maven (m2). To - build from the command line use - -+--- -$ mvn install -+--- - - or the goal of your choice (compile, test, etc.). This builds the - artifact (e.g. jar file) from the project in the current directory, - and deploys it to you local m2 repo at - <<<${user.home}/.m2/repository>>>. If there are any dependency resolution - problems try - -+--- -$ mvn install -P bootstrap -+--- - - which enables some additional, non-standard repositories (which - should not be present in ther deployed artifacts). You should only - need to do this once, because then all the dependencies will be - installed in your local repository. To get the source code (where - available) for all dependencies, you can use - -+--- -$ mvn dependency:sources -P bootstrap -+--- - - See below for instructions on how - to build the documentation and web site. - - By default the whole project (including subprojects) will be built - using Maven's "reactor" plugin. This can be expensive. To build - only one module, cd to that directory first. Or at the top level - use -N (for non-recursive) to exclude subprojects. - -+--- -$ mvn -N install -+--- - -* Skipping Tests - - The profile <> skips all the tests, so - -+--- -mvn -o install -P fast -+--- - - is the quickest way to update your local repo (assuming the tests - are OK). It is equivalent of setting <<<-Dmaven.test.skip=true>>>. - -* Running Individual Tests - - The standard way to do this with Maven is -Dtest= with the class name (not fully qualified), e.g. - -+--- -$ mvn test -Dtest=FootballJobFunctionalTests -+--- - - In the samples you can also add additional system properties, which will be used to override bean properties. This can be done with an argLine property, e.g. - -+--- -$ mvn test -Dtest=FootballJobFunctionalTests -DargLine='-Dplayer.file.name=player.csv -Dgames.file.name=games.csv' -+--- - - or by specifying forkMode=never (in which case the test is run in the same process as Maven): - -+--- -$ mvn test -DforkMode=never -Dtest=FootballJobFunctionalTests -Dplayer.file.name=player.csv -Dgames.file.name=games.csv -Djob.commit.interval=50 -+--- - -* Eclipse IDE - - Our policy is to commit Eclipse (and only Eclipse) meta data to - source control. This will work out of the box for you if you use - the (excellent) Q4E Eclipse-plugin - (http://q4e.googlecode.com/svn/trunk/updatesite). With this plugin, - each of the reactor modules at the top level builds independently - and feeds changes into other projects in your workspace. It is not - recommended to use the Maven Eclipse plugin (<<>>) because it cannot track dependencies across the - Eclipse workspace. It will also create Eclipse meta-data every time - you run it, conflicting with the version under source control. - -* Dependencies - - If you get multiple versions of the same jar across projects, or a - jar is appearing in the classpath that you don't think is necessary, - look into the dependency structure and try and exclude it from - wherever it is being transitively included. To see the dependencies - for a project look in the site for the dependency report. - Alternatively (very useful for quickly locating a rogue jar) use - -+--- -$ mvn -P snapshots dependency:tree -+--- - - We use the "snapshots" profile here so that we get a snapshot of the - dependency plugin (older versions did not have the tree goal, but - newer versions are not stable enough to use in production). - -* Subversion and Line Endings - - Please use - -+--- -*.xml = svn:eol-style=LF -*.sql = svn:eol-style=LF -*.txt = svn:eol-style=LF -*.java = svn:eol-style=LF -*.apt = svn:eol-style=LF -*.properties = svn:eol-style=LF -+--- - - in your <<<~/.subversion>>> (or <<\Application Data\Subversion/config>>>). If anyone forgets to do that then the property can be recursively set using Tortoise (type in the property key and value and use the recursive checkbox). - -* Documentation - - With the exception of reference docs, please put content in the - project that it is most closely associated with. Here is a - {{{./sitemap.html}site map}} to help you decide. - -** Quotidian Web Content - - Maven allows you to choose from a range of source format for - building web content. For Spring Batch we prefer the "almost plain - text" version. See files under <<>> in all the projects - for examples, and also refer to the - {{{http://maven.apache.org/guides/mini/guide-apt-format.html}Apt - Format Guide}} on the Maven website. - - N.B. you put .apt source files in a subdirectory called <<>>, - but they are moved to the top level when the site is built. Thus - <<>> becomes <<>>. - -*** Using emacs to edit .apt files - - Because the .apt format relies on indentation in plain text files, - the emacs auto-fill feature in text mode makes editing very - convenient. Put this in your .emacs - -+--- -(setq auto-mode-alist (cons '("\\.apt\\'" . text-mode) auto-mode-alist)) -+--- - - Then use <<>> to auto-fill the current paragraph. Emacs - adjusts the indentation of all the lines to match the first one (or - the first two if the second is different. - - If anyone knows how to do this with Eclipse or other editors, let us - know and we'll put a note here. - -** Reference Guide - - The <<>> project is reserved for reference guides in the - normal Spring docbook format. Each chapter of the reference guide - is in a separate xml file under <<>>. - The easiest way to work with the reference guide is to cd to the - <<>> module, and run Maven from there. - - Use the DTD with a validating XML editor (e.g. Eclipse) to explore - the docbook format. Also look at existing examples in Spring Batch - and in the Core Spring Framework source code. - - [Section numbers] There is no need to explicitly create section numbers in the - XML - this is done for you by the build when everything is stitched - together into a book. - - [Source location] You put docbook .xml source files in a - subdirectory called <<>>, but they are moved to the top - level when the site is built. Thus - <<>> becomes - <<>>. - - [XMLMind] If you use {{{http://www.xmlmind.com}XMLMind}} to edit the - reference guide add the following line to - <<</addon/config/docbook/common.incl>>>: - -+--- - -+--- - -** Adding a new chapter to the Reference Guide - - Here is a skeleton chapter including the DTD to get you started on a - new chapter. - -+--- - - - - Chapter Title -
        - Introduction - -
        -
        -+--- - - Create a file with the template above, and put it in - <<>>. Use lower case, dash separated file names - (XML style), e.g. <<>>. - - Add the chapter to the master book in <<>> using - -+--- - -+--- - -* Adding graphics - - Put (e.g.) PNG image content in <<>>, and - then refer to the file using the <<>> directory prefix. - -** In .apt - - With no whitespace add the image name in square brackets (\[\]): - -+--- -[images/MyFigure.png] Caption content here is not rendered by default -in a browser (it's the ALT content)... -+--- - -** In docbook - - Use the \ element: - -+--- - - - - - - - - - - Figure 1: the figure caption... - - - -+--- - -* Program Listings in Docbook (Including XML) - - Use CDATA to save you from having to use the HTML escapes for all - the special characters. E.g. - -+--- - -]]> - -+--- - -* Dynamic Editing - - To see your changes to web site content as soon as you have typed - it, use - -+--- -mvn site:run -+--- - - and go to http://localhost:8080. - - In a project with unit tests, you can skip the tests and go straight - to the documentation using - -+--- -mvn -o site:run -P fast -+--- - - If you are offline, or want to speed things up a bit, the "-o" stops - Maven from trying to resolve dependencies on the internet. - - Use -N to build only the current project, not subprojects, So this - is pretty useful at the top level: - -+--- -mvn -N -o site:run -P fast -+--- - - In the <<>> project the docbook reference guide shows up at - http://localhost:8080/reference/*.html, where * is the name of an - xml file with a chapter in it. There is no link to these pages on - the site because the real docbook generated output is much nicer, - but this is still pretty useful for debugging and dynamic - editing. - - Note that the formatting is a bit limited compared to the whole - docbook stylesheet - Maven uses Doxia to squish all of docbook into - some simple wiki-like formatting rules. In particular it can't - generate the index page in the format we need it, so you may see - errors from <<>> if you visit that page. One of the - features is that the <<<\>>> syntax we use to build the - index and table of contents in the docbook-generated pages does not - work. Images are another problem. Use the generated content from - <<>> to view these artifacts. - -* Building and deploying the web site - - There is a bug in the m2 reactor (MNG-740) which means that we have - to install the parent pom to the local repo first. - - So do it this way: - -+--- -$ mvn install -P fast -$ mvn -P staging clean site site:deploy -+--- - - Remove "-P staging" to deploy to the real website (requires ssh - access to static.springframework.org). - - The "-P staging" is to deploy to <<>>, so we - don't get accidental updates to the site. To test the site contents - navigate with your browser to that directory. The site:stage goal - deos not work properly for this build: all the subprojects are not - integrated into the staging site, so use site:deploy instead. - - The static website content is not deleted during the deployment - process - merely replaced. If you need to clean everything up from - scratch you need to delete the contents on the server as well - (using ssh). - -Problems? - - Make sure your source code is up to date. Delete everything from - your local Spring Batch repo - <<<${user.home}/.m2/repository/org/springframework/batch>>>. If - necessary, delete a project or directory and update from SVN again. - - Try - -+--- -$ mvn install -+--- - - or - -+--- -$ mvn clean install -+--- - - or - -+--- -$ mvn clean install -P fast -+--- - - from the top level, and - -+--- -$ mvn -U ... -+--- - - from wherever you are (top level or sub-project). The latter will - update any older plugins you have in your local Maven repository. - Some people have had trouble building the web site without this. - - If you get <<>> e.g. building the site, use - MAVEN_OPTS to boost the heap size (on the command line if you have a - sensible shell): - -+--- -$ MAVEN_OPTS=-Xmx256m mvn site -+--- diff --git a/src/site/apt/cases/async.apt b/src/site/apt/cases/async.apt deleted file mode 100644 index 2255d5ba33..0000000000 --- a/src/site/apt/cases/async.apt +++ /dev/null @@ -1,159 +0,0 @@ - ------ - Asynchronous Chunk Processing Use Case - ------ - Dave Syer - ------ - January 2007 - -Use Case: Asynchronous Chunk Processing - -* Goal - - Increased the efficiency of chunk processing by having it execute - asynchronously: in multiple threads. Maintain transactional - intergrity of the chunk. - -* Scope - - * All chunks might conceivably benefit from parallel processing, so - we don't want any unnecessary restrictions on the batch operation, - or its implementation. A should be possible for Client to write a - batch operation without reference to the fact that it might run in - an asynchronous chunk. - -* Preconditions - - * Input data exists with non-trivial size: chunks contain more than - one record. - - * Batch processing of a record is slow, or can be delayed, so that - the asynchronous processing can take longer than launching the - threads. - - * A chunk can be made to fail after at least one record is - processed. - -* Success - - * A chunk is processed and the results inspected to verify that all - records were processed. - - * Transactional behaviour is verified by rolling back a chunk and - verifying that no records were processed. - -* Description - - The vanilla case proceeds as for normal {{{./chunks.html}chunk - processing}}, but: - - [[1]] Within a chunk, Container processes records in parallel. - - [[1]] At the end of a chunk, Container waits for the last record - to be processed (with a timeout if the wait is long). - -* Variations - -** Rollback on Failure - - If there is an exception in one of the record processing threads, - the whole chunk should roll back: - - [[1]] Client throws exception in record processing. - - [[1]] Container catahes exception and attempts to abort other - running processes. - - [[1]] Container waits for running processes to abort (or finish - normally, but preferably to abort). - - [[1]] Container propagates the exception and signals transaction to - rollback. - -** Timeout - - If there is a timeout during a chunk, it might happen before the - chunk has finished, or while waiting for the processes to complete - before exiting. - - [[1]] At end of chunk, Container is waiting for all processes to - finish. It times out, according to a parameter set by the - Operator. - - [[1]] Container does not start any new processes, and attempts to - abort running processes. - - [[1]] Container waits for running processes to abort (or finish - normally, but preferably to abort). - - [[1]] Container throws a time out exception and signals chunk - transaction to rollback. - -* Implementation - - * The implementation of this use case could be tricky in the general - case. In particular, the transactional nature is going to be hard - or impossible to maintain across multiple threads without the - individual processes being aware of the transaction, and (perhaps) - without global (XA) transaction support. - - A "normal" local transaction is thread bound - i.e. it only executes - in one thread. If the code inside the transaction creates new - threads, then they might not finish processing before the parent - exits and the transaction wants to finish. The transaction needs to - wait for the sub-processes before committing, or (more difficult) - rolling back. The rollback case basically forces us to a model of - one transaction per thread, and therefore to one transaction per - data item in a concurrent environment. - - Otherwise some transactional semantics might be respected in a - parallel process, but others certainly will not be because - synchronizations and resources are managed at the level of the - thread where the transaction started. If the transaction manager is - a local one (not XA) there is little hope even that the datasource - resource would be the same for all the parallel threads and the - parent method. - - If we use a global transaction manager to make the parallel - processes transactional, how will they know which transaction to - participate in? There could be many active chunks, and each would - have its own threads - how would each one be able to guide its child - processes to participate in the same transaction? - - * Beware a framework that extracts data from an <<>> - before executing the business logic (e.g. in a - <<>>). It is not enough to allow concurrent - processing but simply insist that the individual records are - processed transactionally because the <<>> will then - not be able to participate in the transaction - its next record has - already been passed to the consumer when the transaction starts, so - if there is a rollback then the record is lost. - - This is the origin of the signature: - -+--- -public interface ItemReader { - Object next(); -} -+--- - - There is no peeking and no iterator-style <<>>. If there - is a processing problem, transactional clients of the - <<>> throw an exception the provider's - <<>> has been called, but in the same thread (so that - transactional semantics are preserved and the data provider reverts - to its previous state). - - This means that in the callback interface also picks up an - <<>> return type - -+--- -public interface RepeatCallback { - Object doInIteration(BatchContext context); -} -+--- - - so we can return an object, which is null when the processing has - finished. - - In the end we decided against the <<>> return type and went - with an exit status to signal for no more processing. diff --git a/src/site/apt/cases/chunks.apt b/src/site/apt/cases/chunks.apt deleted file mode 100644 index 0f3e129d77..0000000000 --- a/src/site/apt/cases/chunks.apt +++ /dev/null @@ -1,208 +0,0 @@ - ------ - Commit Periodically Use Case - ------ - Dave Syer - ------ - January 2007 - -Use Case: Commit Batch Process Periodically - -* Goal - - Read a file line-by-line and process into database inserts, for - example using the Jdbc API. Commit periodically, and if there is a - fault where the database transaction rolls back, then the file - reader is reset to the place it was after the last successful - commit. - - To develop a batch process to achieve the goal above should be as - simple a process as possible. The more that can be done with simple - POJOs and Spring configuration the better. - -* Scope - - To keep things simple for now, assume that: - - * All lines in the input file are in the same format and each line - generates a single database insert (or a fixed number). - - * The file is read synchronously by a single consumer. - -* Preconditions - - * A file exists in the right format, with a sufficiently large - number of lines to be realistic. - - * A mechanism exists to force a rollback at a non-trivial position - (not during the first commit), but produce a successful operation - on the second try. - - * A framework for retry exists, so that the case above can be - tested. - -* Success - - Integration test confirms that - - * All data are processed and records inserted successfully. - - * When a rollback occurs and the retry is successful, the complete - dataset is processed (same result as successful run). - - * Batch operations can be implemented without framework code (or - with minimal dependencies, e.g. through interfaces). Launching - the batch might require access to framework code. - -* Description - - The vanilla successful batch use case proceeds as follows: - - [[1]] Container starts a transaction. - - [[1]] Container makes resources available, e.g. opens file and - creates <<>> for it. - - [[1]] Client reads a line from the file, and converts it to a - database statement, then runs it. - - [[1]] Container increments counter. - - [[1]] Repeat previous two steps until a counter is equal to chunk - size. - - [[1]] Container commits database transaction. - - [[1]] Repeat chunk processing until input source is exhausted. - -* Variations - -** Non-fatal Chunk Failure - - If there is an unrecoverable database exception during execution of - client code: - - [[1]] Container rolls back current transaction. - - [[1]] Container resets input source to the point it was at before - failure. - - [[1]] Container retries chunk. - -** Fatal Chunk Failure - - If there is an error in the input data in the middle of a chunk - (could be manifested as database exception, e.g. uniqueness - exception, or nullable exception): - - [[1]] Container rolls back current transaction. - - [[1]] Container terminates batch and notifies client of precise - details, including the line number of error, and the last line - that was committed (last of the previous chunk). - - There is no need to reset the input source because the error is - fatal. - - To restart: - - [[1]] Operator truncates the input file so the completed chunks - are not repeated. - - [[1]] Operator fixes bad line (if there was one), and starts the - batch process wit hthe same parameters. - - Variations on this theme are also necessary, e.g. a tolerance for a - small number of bad records in the input data. - -* Implementation - - * The concept of a batch iterator seems relevant here (see also the - {{{./simple.html}simple}} use case). The iterator could be more than - just a loop that might terminate early: here it could also manage - the file cursor on the input source. In this design there is a - <<>> interface that can take care of termination and - iteration (e.g. iterator-like method signatures). - - * Another design idea (more encapsulated and more in keeping with - existing Spring practice) is to make the data source transaction - aware, and for the client use it like a database resource, through a - template. In this case there is a <<>>. The - <<>> needs to be aware of the data source template, so - that it can terminate when the data is exhausted. - - In this version of events there are two kinds of resource in play. - The transaction itself, and the data sources that are aware of the - transaction. The comparison with <<>> - and <<>> is obvious. The client is often completely - unaware of the transaction manager, which is applied through an - interceptor, whereas the data source is used explicitly with its own - API through a template. The Client can concentrate on his domain, - and not be concerned with infrastructure or resource handling. - - * The analogy with <<>> is even stronger. If the input - data came from JMS instead of a file, we would hardly have to do - anything to implement very robust chunking. JMS is the obvious best - practice and already provides all the transactional semantics we - need for chunking - simply roll back a transaction and the records - processed return to the message system for delivery to the next - consumer. Bad records can be sent to a bad message queue for - independent processing. JMS might ssem like overkill for a lot of - batch processes, but it is tempting to say that if the robustness is - needed then the we should take that as a sign that installing and - configuring JMS is worth the extra effort. - - * Naturally we do not want to insist that the client code is aware - of the transaction that is surrounding it - this would be the normal - practice familiar from the Spring programming model. Should a - client need access to transaction-scoped resources, the usual way to - do that is to wrap the transactional resource (data source etc.) in - a proxy that uses a synchronization, or a more generic thread-bound - resource (using <<>>). The aim - is to retain this separation in a batch operation. The batch - framework itself might provide some of these synchronizations. - - * The {{{./simple.html}Simple Batch Repeat}} is actually a pretty good - model for the chunk processing in this use case. This observation - leads to another: that a batch of chunks is a nested (or composed) - batch - the outer termination policy is dependent only on the data - source having further records to process, the inner one is a simple - iterator (with a check for empty data). A simplified programming - model for this is - -+--- -RepeatCallback chunkCallback = new RepeatCallback() { - - public boolean doInIteration(RepeatContext context) { - - int count = 0; - - do { - - Object result = callback.doWithRepeat(context); - - } while (result!=null && count++>>). The termination policy depends - only on a data source eventually returning null. - - * N.B. the chunkSize can be dynamic. E.g., if the chunk is long - during a nightime batch window, and short when the window is over, - in case the batch has to be terminated. - - * Chunking can also be implemented simply in an - <<>>. The handler just buffers records up to a - chunk size, and then executes them all in one step (which might be - transactional). This is easier to implement, and easier to - configure for the clients, but cannot easily be made both concurrent - and transactional. diff --git a/src/site/apt/cases/file-to-file.apt b/src/site/apt/cases/file-to-file.apt deleted file mode 100644 index 53a6bc45d9..0000000000 --- a/src/site/apt/cases/file-to-file.apt +++ /dev/null @@ -1,89 +0,0 @@ - ------ - Copy File to File - ------ - Dave Syer - ------ - January 2007 - -Use Case: Copy File to File - -* Goal - - Read a file line-by-line and process into a file in a different - format (possibly different number of lines). Commit periodically - and in the event of an error both data sources (input and output) - rollback to the last known good point. - -* Scope - - To keep things simple for now, assume that: - - * All lines in the file are in the same format and the final - output is an aggregate. - - * The files are read and written synchronously by a single - consumer. - - * This use case requires two kinds of transactional file source. - One is read-only and the other is write-only. Only one consumer - can use the write-only source at a time. - -* Preconditions - - * An input file exists in the right format, with a sufficiently - large number of lines to be realistic. - -* Success - - Integration test confirms that - - * All data are processed and output produced successfully. - -* Description - - Very similar to the use case {{{./chunks.html}Copy File to - Database}}, but involving transactional access to an output source - which is a file. Also we are introducing the idea of an aggregate - function for the output. - - The vanilla successful case proceeds as in the file to database - version, except that: - - [[1]] A successful chunk results in a line in an intermediate file - output source. - - [[1]] After all chunks are successfully processed the intermediate - file is itself processed in a single transaction to complete the - aggregate. The output is itself sent to an output channel - (e.g. database or file). - -* Variations - - * Chunk failure variations proceed as in the use case - {{{./chunks.html}Copy File to Database}}. In the case of a - restart after fatal failure, the intermediate output file need does - not need to be reset or re-created. - -* Implementation - - * The write-only file source is new in this use case. It has a - similar flavour to the read-only version, but also has more serious - implications for implementation and usage. Since a file system is - not inherently transactional, when we create the write-only data - source we are assuming that consumers will play by the rules, - principally that there is only one consumer at a time. - - * With some external limitations the write-only file source can be - implemented so that within a single JVM it will behave like a - transactional database datasource. We can provide a - <<>> that hides the resource acquisition and - release, and interacts with an existing transaction to provide the - transactional behaviour that is required. - - * File-based transactional resources are a lot like messaging - clients. We can send a message (write a line) through a sender - client, and receive a message (read a line) through a consumer - client. In the case of a transaction rollback, all sent messages - are guaranteed not to reach consumers, and all received messages are - returned to the queue. Maybe ActiveMQ has a file transport already? - Mule definitely does, but it isn't transactional. diff --git a/src/site/apt/cases/index.apt b/src/site/apt/cases/index.apt deleted file mode 100644 index c544aa4cb6..0000000000 --- a/src/site/apt/cases/index.apt +++ /dev/null @@ -1,113 +0,0 @@ - ------ - Use Cases - ------ - Dave Syer - ------ - January 2007 - -Use Cases for Spring Batch - - These are more like scenarios or flows than real use cases in formal - UML terms, but they serve a useful purpose as both. We don't want - to be over formal, and probably code is being written and tested at - the same time as these use cases. But there are many stakeholders - in this project, and use cases are a useful resource to make sure - they are all agreed on scope and certain implementation details. - - * {{{./simple.html}Simple Batch Repeat}} - - * {{{./retry.html}Automatic Retry After Failure}} - - * {{{./chunks.html}Commit Batch Process Periodically}}: chunk - processing. - - * {{{./async.html}Asynchronous Chunk Processing}}: parallel - processing within a chunk. - - * {{{./file-to-file.html}Copy File to File in a Batch}} - - * {{{./parallel.html}Massively Parallel Batch Processing}}. Spring - Batch 1.0 does not contain any implementations of this use case, - but it is quite feasible to implement them using the framework as - a starting point. 1.1 has some prototype code under the Integration - module. - - * {{{./restart.html}Manual Restart After Failure}} - - * {{{./steps.html}Sequential Processing of Dependent Steps}} - - * {{{./partial.html}Partial Processing}}: skip records (e.g. on rollback). - - * Whole-Batch Transaction - transactional support for the whole - batch, not just chunks. Quite a common requirement, but not - always practical using normal transaction support. May require a - staging area, and a decision after it is full about whether to - copy it in one big batch (e.g. using native database tools) or - chunk it (e.g. if it is now in a form for which chunk failure is - easier to deal with). - - * {{{./scheduled.html}Scheduled Processing}}: Batch Jobs controlled - by scheduler (e.g. start, stop, suspend, kill). Spring Batch does - not intend to implement the scheduler concerns, but needs - to provide enough information that a scheduler can act - appropriately. - - * Non-Sequential Processing of Steps (Conditional Branching) - - * {{{./pause.html}Pause and Resume Job Execution}} - - -* Actors - - The following actors are involved in the use cases (Container and - Client being the most common / important). - -** Client or Business Domain - - Code written by the batch developer. - - One aim us that the client is a POJO - the batch behaviour, boundary - conditions, transactions etc. can be dealt with by the Container in - such as way that the client does not need to know about them. The - client may have access to framework abstractions, like templated - data sources (<<>> etc.), but these should work the - same whether they are in a batch or not. - -** Container - - An application that converts user requests for batch jobs into - running processes. Container concerns are robustness, traceability, - manageability. - -** Framework - - The Framework is the infrastructure code that the Container depends - on, and possibly spi implementations where knowledge of the - non-business logic resides. - - The Framework provides two kinds of infrastruture (as per usual - Spring cornerstones and ): - - * For cross-cutting concerns there are interceptors that can be - wrapped around client code without it needing any knowledge of the - Framework at all. An existing parallel is with transaction - support - the client code can use <<>> - directly, but does not always need to. - - * Concrete abstractions that allow access to resources in a - uniform way without needing to know the details of how they are - provided (e.g. partitioned). Client code can use these - abstractions like it would a use a <<>>. - -** Operator - - The batch operator is not a developer. Tools are provided for the - Operator to be able to stop and start a batch, and to monitor the - progress and status of on ongoing or finished batch. - -** Business User - - The Operator has technical skills, e.g. a member of an application - support team, but may need help with business-related decisions. - For instance if input data are bad, he would not expect to be able - to fix them alone because they might be bad for a business reason. diff --git a/src/site/apt/cases/parallel.apt b/src/site/apt/cases/parallel.apt deleted file mode 100644 index c61cacf2ab..0000000000 --- a/src/site/apt/cases/parallel.apt +++ /dev/null @@ -1,278 +0,0 @@ - ------ - Parallel Processing Use Case - ------ - Dave Syer - ------ - January 2007 - -Use Case: Massively Parallel Batch Processing - -* Goal - - Support efficient processing of really large batch jobs (100K - - 1000K records) through parallel processing, across multiple - processes or physical or virtual machines. The goals of other use - cases should not be compromised, e.g. we need to be able to start - and stop a batch job easily (for non developer), and trace the - progress and failure points of a batch. The client code should not - be aware of whether the processing is parallel or serial. - -* Scope - - * Any batch operation that reads data item-by-item from an input - source is capable of being scaled up by parallelizing. - - * The initial implementation might concentrate on multiple threads - in a single process. Ultimately we need to be able to support - multiple processes each one running in an application server - (e.g. so that jobs that require EJBs can be used). - -* Preconditions - - * A data source with multiple chunks (commitable units) - more chunks - than parallel processes. - - * A way for the framework to launch parallel processes. - -* Success - - * A batch completes successfully, and the results are verified. - - * A batch fails in one of the nodes, and when restarted processes - the remaining records. - -* Description - - [[1]] Framework splits input data into partitions. - - [[1]] Framework sends input data (or references to them) to - processing nodes. - - [[1]] Processing nodes act independently, converting the input data - and sending it transactionally to output source (as per normal - single process batch). - - [[1]] Framework collects status data from individual nodes for - reporting and auditing. - - [[1]] When all nodes are complete Framework decides that batch is - complete finishes processing. - -* Variations - - Two failure cases can be distinguished, bad input data on a node and - an internal node failure have different implications for how to - proceed. In both cases, however - - [[1]] Framework catches exception and classifies it. Rolls back - current transaction to preserve state of data (input and output). - - [[1]] Framework saves state for restart from last known good - point, including a pointer to the next input record. - - Then if a processing node detects bad data in the input source, it - cannot be restarted or re-distributed because the data need to be - modified for a successful outcome. - - [[1]] Framework alerts Operator of the location and nature of the - failure. - - [[1]] Operator waits for batch to finish - the overall status will - be a failure, but most of the data might be consumed. - - [[1]] Operator fixes problem and restarts batch. - - [[1]] Framework does not re-process data that has already been - processed successfully. The parallel processing nodes are used as - before. - - [[1]] Batch completes normally. - - If a processing node fails unrecoverably (e.g. after retry timeout), - but with no indication that the input data were bad, then the data - can be re-used: Framework returns unprocessed input data, and - redistributes it to other nodes. - -* Implementation - - * There are actually two approaches to this problem, which are - largely complementary. - - [[1]] The model dynamically assigned chunks of items to - be processed and sends them to durable middleware. Worker - processes pick them up and process them, sending back a message - about the status. This approach works best if the dispatching is - efficient compared to the processing. - - [[1]] The approach is more like running multiple - jobs in parallel, with input data partitioned into larger pieces, - and not split any further by the dispatcher. The item reading - happens in the worker processes. This approach is necessary if - the dispatcher in the model becomes a bottle neck. - - Generally, chunking is easier to implement than partitioning, but - there are tools available for implementing both patterns - efficiently. - -** Chunking - - The messages from a dispatcher to worker processes consist of a - chunk of items - a set of items to be processed together in a single - transaction (or as the worker sees fit). The dispatcher is usually - single threaded, but this is only a restriction based on the input - data type (if it is a file it is difficult to read in parallel and - maintain restartability). Using a process indicator the dispatcher - could be reading from a database table in a multi-threaded model. - - The main restriction is that for restartability the messages between - the dispatcher and workers has to be durable (i.e. JMS or - equivalent). If there is a durable middleware there are no in - principle difficulties with this approach. - - The practicalities deserve some discussion. In particular the - dispatcher has to co-ordinate asynchronous replies from its workers, - and also has to avoid overwhelming the workers (so there should be - some throttling). As long as the middleware is durable the - dispatcher can simply wait for replies whenever it thinks there are - workers working. It needs to record this expectation in a durable - form as well, as part of an <<>> for the step. - -** Partitioning - - The hard thing about this use case is the partitioning of input (and - output) sources. Ideally, this has to be done in such a way that - the individual operations are unaware that they are participating in - a batch farm. Partitioning has to be at least partially - deterministic because restarts have to be able to ignore data that - have already been processed successfully. - - Consider two examples: a file input source and a JDBC (SQL query) - based input source. Each provides its own challenges. - -*** File Data Source - - * If each node reads the whole file there could be a performance - issue. They would all need to have instructions about which lines - to process. - - * If each record of input data is a line, this isn't so bad. Each - node can have a range of line numbers to process. The only problem - is knowing how many lines there are, and how many nodes, so that the - job can be partitionaed efficiently. - - * But if each input record can span a variable number of lines (not - that unlikely in practice), then we can't use line numbers - - * Maybe the best solution is to use middleware anyway. A single - process parses the file and sends it to a message queue, item by - item (or chunk by chunk). The integration pattern could then be a - simple Eager Consumer, assuming that all records are processed - independently. The messaging semantics would simply have to ensure - that a consumer can roll back and return the input records to a - queue for another consumer to retry. - - * For large batches a real messaging infrastructure (JMS etc.) with - guaranteed delivery would be a benefit, but might be seen as - overkill for a system that didn't otherwise require it. In this - case we could imagine the partitioning process being one of simply - dividing the input file up into smaller files, which are then - processed by individual nodes independently. The integration - pattern is then different - more like a Router. - - * What would parallel processing look like to the client? We can - make it completely transparent if we assume that the client only - ever implements <<>> and <<>>. The - client code is unaware of the partitioning of its data source. - - * Parallelisation could also take place at the level of the - <<>> - we could proxy the data provider and wrap it in - a partitioning proxy: - -+--- - - - - - ... - - - - - - - - ... - - -+--- - -*** SQL Data Source Partitioning - - * If each node is allowed to do its own query or queries to - determine the input data: - - * Each node has to be given a way to narrow the query so that they - don't all use the same data. There is no easy universal way to - achieve this, and in the general case we have to know in advance - when we are going to execute in a parallel or as a single process. - Maybe a range of primary keys would work as a special case that we - could support as a strategy. - - * Maybe we could assume that all nodes execute precisely the same - query, and then provide a way to add a cursor to the result set, - so it can be treated a bit more like a file. - - * We might be forced to use a distributed transaction to ensure - that all the nodes see the same data. This would be unfortunate, - but possibly necessary. It would be up to the client to configure - distributed transactions if that was required, otherwise the - result might be unpredictable if data can be added to an input - source while it is being read. - - * If only one query is done by the Framework and the results shared - out amongst the nodes we face the issue of how to send the data - between nodes. Performance problems might ensue. Plus (more - seriously) the individual nodes would now need a different - implementation if they were acting in a parallel cluster to the - vanilla serial processing case - a single node would do the query - and work directly with the results, whereas in a parallel - environment it would be one step removed from the actual query. - This breaks our encapsulation design goal. - - * When considering the approach to partitioning the data source - we should follow closely the discussion above on partitioning a file - input source. If the client is to remain unaware of the batch - parameters, then an interceptor looks like the best approach. - - If each node prefers to do its own query then an interceptor would - have to catch the call to a JDBC template and modify the query - dynamically. This is quite a scary thing to be doing - it might end - up with us needing to parse the SQL and add where clauses. Maybe a - client should be forced to specify (in the case of a parallel batch) - how his query should be partitioned. For example: - -+--- - - - - SELECT * from T_INPUT - - - - SELECT * from T_INPUT where ID>=? and ID - - - -+--- - - It would be an error to run a batch in parallel if the partition - query had not been provided. - - * What happens if the data source changes between failed execution - and restart? We can't legislate for that because it is outside the - realm of what can be controlled through a transaction. A restart - might produce different results than the original failed batch would - have done were it successful. diff --git a/src/site/apt/cases/partial.apt b/src/site/apt/cases/partial.apt deleted file mode 100644 index 015ff5a19a..0000000000 --- a/src/site/apt/cases/partial.apt +++ /dev/null @@ -1,154 +0,0 @@ - ------ - Partial Processing Use Case - ------ - Dave Syer - ------ - January 2007 - -Partial Processing - -* Goal - - Support partial processing of a batch, without having to interrupt - or manually restart, but enabling corrective action to be taken - after the process has finished to complete the processing of failed - records. A batch that is going to fail completely can be be - identified as soon as possible, but one which is substantially - alright can run as far as possible to prevent costly duplication. - Records that are skipped are reported in such a way that they can be - easily identified by the Operator and / or Business User and a new - batch created to finish the original goal. By the same token, in - the case of an aborted batch where a minority of records are - processed successfully first time, it should be possible to identify - the successful records and exclude them from data presented on - restart. - -* Scope - - Any batch should be configurable to support partial processing. - -* Preconditions - - * A data source with a small number of bad records exists. - -* Success - - * A test data set with a small number of bad records is run through - the batch processer and completes normally. Operator confirms - that the good recirds are all processed and then fixes and - resubmits the bad records, and confirms that they are also - correctly processed with no duplicates. - -* Description - - The vanilla flow proceeds as follows: - - [[1]] Batch processing begins as per normal (see for example - {{{./chunks.apt}chunk processing use case}}). - - [[1]] A record is processed. This step repeats until... - - [[1]] Container detects a bad record, e.g. by catching a - classified execption. - - [[1]] Container logs the exception in a way that identifies the - bad record easily and immediately to the Operator. - - [[1]] Container stores an identifier for the bad record (or the - whole record) in a location designated to the Operator for that - purpose. - - [[1]] Container determines that the batch can still succeed - despite the cumulative number or nature of bad records - the bad - record is skipped. Container goes back to normal processing, and - eventually completes the whole batch. - -* Variations - -** Abort Batch Early - - The batch cannot skip all records. After each failure the decision - about whether to coninue has to be made: - - [[1]] When a record is processed successfully, Container logs the - event in a form that can be used later to identify successful - records in case the batch is aborted. - - [[1]] Container determines that a sufficiently large fraction of - the records processed so far have failed. The faction relevant is - to be specified through configuration meta data (not specified by - business logic). - - [[1]] Container aborts the batch with a clear signal to the - Operator that it has aborted owing to an unacceptable number of - errors. - -* Implementation - - * When the decision to abort is taken, Container may have - successfully processed a small number of records and the - corresponding transactions might have committed. Those records that - were successfully processed on the first attempt are easy to - exclude from the restart, if transactional semantics are respected - by the item processing. - - * The decision to abort is based on exception classification. Each - time an item is processed, the framework needs to catch exceptions - and classify them as - - * fatal: signals an abort - rethrow. - - * transient: nominally fatal, but the operation is retryable. - - * non-fatal: signals a skip. - - The transient failure is really just a sub-type of fatal case. It - is treated differently by the {{{./retry.html}retry framework}} but - not necessarily by the vanilla batch. - - * Actually we can't decide what action to take simply on the - evidence of the current exception. What we need to do is decide, - potentially based on the whole history of exceptions in a given - batch, whether the latest one should trigger an abort. E.g. a - simple and sensible policy would be to abort if the total number of - exceptions reaches a threshold, either absolute or relative to the - number of items processed. - - * So how does it look? In the template... - -+--- -public void iterate(RepeatCallback callback) { - - ... - - try { - result = callback.doInIteration(context); - } catch (Exception e) { - handleException(e); // Maybe re-throw, maybe not... - } - - ... - -} -+--- - - If the callback was transactional it has already rolled back. If - the whole <<>> was transactional we need to rethrow - - * If the processing is asynchronous, the template has to execute in - a separate thread (see {{{./async.html}asynchronous example}}). In - this case the whole thread (i.e. the <<>>) has to be - transactional. Whoever is counting failed items needs to be - poooling information from multiple threads. - - * It may also be the role of the framework to translate exceptions - into a batch-specific hierarchy. This is not the same concern as - exception classification (as done for instance by the Spring Jdbc - and Jms templates). Exception classification might also be of - value, but the argument is not as clear cut as the existing core - templates, where there is an underlying Jave EE API checked - exception to convert. In the absence of a batch-specific exception - hierarchy definition, we could choose to leave exception translation - out of the batch framework. - - diff --git a/src/site/apt/cases/pause.apt b/src/site/apt/cases/pause.apt deleted file mode 100644 index 15b568a3b8..0000000000 --- a/src/site/apt/cases/pause.apt +++ /dev/null @@ -1,121 +0,0 @@ - ------ - Pause Resume Use Case - ------ - Dave Syer - ------ - October 2008 - -Use Case: Pause and Resume Job Execution - -* Goal - - Allow a job to pause itself and await further instructions. A - paused status indicates to a user that the job is waiting, either - for a manual signal to proceed, or for a remote worker to finish - doing something asynchronously. For instance, a job may require - manual verification of business condition before continuing - a - sanity check on critical data. Assume that a job execution could - receive hundreds of resume signals, and this is a "normal" - situation, so it does not create a horrible mess in the history of - the execution - e.g. looking like hundreds of restarts. - -* Scope - - * The instruction to pause comes from processing logic, not from an - external signal (like an interrupt). A variation where the signal - comes from outside might be a useful extension, but isn't explicitly - included here. - -* Preconditions - - * A job is configured and one of its components can send the signal to pause - - * The launching interface has the ability to resume a paused job - - * The execution meta data can be inspected to verify that a pause has occurred - -* Success - - * User launches job and verifies that it has paused at a certain point - - * User resumes job and verifies that it completes successfully. - - * The end state is indistinguishable from a successful completion of - the job in one attempt - -* Description - - The vanilla successful case proceeds as follows: - - [[1]] User launches a new job execution. - - [[1]] Framework begins processing, and successfully executes one - or more steps. - - [[1]] At the end of a step Framework encounters condition that - signals it should pause (e.g. a status flag). - - [[1]] Framework gracefully exits the job execution, marking it as - paused so that it can be identifed as such when asked to resume. - Often the framework will also be configured to notify a user that - the pause has occurred, so that some business condition can be - verified manually. - - [[1]] User requests the job execution be resumed. - - [[1]] Framework picks up where it left off, ignoring steps that - have already successfully executed and starting with the one after - the pause. - - [[1]] Job finishes processing and Framework marks it as - sucessfully completed, just as if it hadn't paused in the first - place. - -* Variations - - * The agent that causes the job to resume is not a User but a remote - worker process. - - * Two agents request a resume at the same time. One of them has to - lose (an exception is acceptable). - - * A step pauses in the middle of execution. The job picks it - up and start where it left off, just like in a restart. - - * More than one step was executing when the pause signal was - detected. Framework allows steps that are executing in process to - complete (or pause) before exiting the job execution. - - * More than one step is in a paused state when the job resumes. - Requires no special treatment from Framework: if those steps were - active when the pause reached the job level on the last run, then - they will be processed in the same way on a resume (presumably in - multiple threads). - -* Implementation - - * A new <<>>. - - * The <<>> interface may not need any more than it already has: - -+--- -public interface JobLauncher { - - public JobExecution run(Job job, JobParameters jobParameters) throws ....; - -} -+--- - - In the case that the last execution failed, we already pick up from - where we left off with a new <<>>. The only - difference now is that we don't need a new <<>>, so we - have to be careful about concurrency - what happens if two agents - try to resume the job at once. To be safe we can treat this the - same way as a restart - lock the <<>> table in the - database by setting a TX isolation attribute on the - <<>>. - - * When we resume we need to wind forward through the job execution - and look at all step executions to see if they are active. Once the - <<>> has been identified the process should be no - different to a restart. diff --git a/src/site/apt/cases/restart.apt b/src/site/apt/cases/restart.apt deleted file mode 100644 index 8f703d5ef5..0000000000 --- a/src/site/apt/cases/restart.apt +++ /dev/null @@ -1,86 +0,0 @@ - ------ - Restart Use Case - ------ - Dave Syer - ------ - January 2007 - -Use Case: Manual Restart After Failure - -* Goal - - Restart a failed or interrupted batch and have it pick up where it - left off (within limits of transaction boundaries) to save time and - resources. A key goal is that the management of the batch process - (locating a job and its input and results, starting, scheduling, - restarting) should be as easy as possible for a non-developer, like - an application support team with some business back up. - -* Scope - - Any batch should be able to restart gracefully, even if (depending - on chosen execution or client implementation) it might have to go - right back to the beginning. - -* Preconditions - - * It is possible to identify exception conditions under which a - restart will be able to carry on processing a batch from where it - left off. - - * There exists a persistent storage mechanism for the initial - conditions. - -* Success - - * Force a batch to fail, and then fix the problem and restart. See - successful completion with no duplicate results. - -* Description - - [[1]] A batch operation encounters an exception which forces the - process to stop processing. - - [[1]] Framework catches exception and classifies it. - - [[1]] Framework logs event with enough information to identify the - location of the job and the nature of the problem. - - [[1]] Framework saves initial condition from last commit point, to - enable restart to start from the last known good operation. - - [[1]] Operator fixes problem (e.g. makes missing resource available, - edits input file). - - [[1]] Operator restarts batch. - - [[1]] Framework loads initial conditions and continues processing. - -* Variations - - * Some restarts might lend themsleves to being handled automatically - - see the use case {{{./retry.html}Automatic Retry}}. - -* Implementation - - * The saving of initial conditions needs to be strategised. In some - cases saving a native serialization to a file will suffice. In - others a database might be used, or some custom serialization - (persist / rehydrate). - - * The initial condition is naturally under control of the - <<>>. The client need not know about the persistence - and rehydration. In fact explicit persistence and rehydration might - be overkill - just relying on the transaction semantics might be - adequate in a lot of cases. The <<>> would have to be - aware of the transactions, which we assume are normally demarcated - in the <<>>. Since the point at which persistence - is needed is tied to transaction commits, there may have to be some - transaction synchronization. - - * The persistence of initial conditions is a cross cutting concern. - It may lend itself (along with the application of an execution - handler generally) to being implemented as an aspect. Compare the - <<>>, where the most common usage is via an - interceptor, but occasionally the template is used directly by - client code. diff --git a/src/site/apt/cases/retry.apt b/src/site/apt/cases/retry.apt deleted file mode 100644 index 5ea21cac70..0000000000 --- a/src/site/apt/cases/retry.apt +++ /dev/null @@ -1,279 +0,0 @@ - ------ - Automatic Retry Use Case - ------ - Dave Syer - ------ - January 2007 - -Use Case: Automatic Retry - -* Goal - - Support automatic retry of an operation if it fails in certain - pre-determined ways. Client code is not aware of the details of - when and how many times to retry the operation, and various - strategies for those details are available. The decision about - whether to retry or abandon lies with the Framework, but is - parameterisable through some retry meta data. - - Retryable operations are usually transactional, but this can be - provided by a normal transaction template or interceptor - (transaction meta data are independent of the retry meta data). - -* Scope - - Any operation can be retried, but there are restrictions on nesting - transactions (normally an inner transaction needs to be - propagation=NESTED). - -* Preconditions - - An operation exists that can be forced to fail and is able to - succeed on a retry. - -* Success - - * Verify that an operation fails and then succeeds on a retry. - - * Verify that back off policy (time between retries) can be - strategised without changing client code. - - * Verify that the retry policy can be strategised, and can be used - to change the number of retry attempts depending on the type of - exception thrown in the retry block. - -* Description - - Successful retry proceeds as follows: - - [[1]] Framework executes an operation provided by Client. - - [[1]] The operation fails and Framework catches an exception, - classified as retryable. - - [[1]] Framework waits for a pre-defined back off period. The - period is not be fixed, but is strategised so that different - policies can be applied. The most common and useful policy is an - exponentially increasing back off delay, with a ceiling. - - [[1]] Framework repeats the operation. - - [[1]] Processing is successful. - - [[1]] Framework stores and / or logs statistics about the retry - for management purposes. Details? - -* Variations - - The following variations are supported. - -** Retry Failure - - A retry can fail for a number of reasons. E.g. if the number of - retries is too high, or there is a timeout, or an exception of - another sort that cannot be classified as retryable. - - [[1]] Last retry attempt fails and Framework determines that - another retry is not permitted by the current policy. - - [[1]] Framework records status for management purposes. - - [[1]] Framework throws a recognisable exception? - - [[1]] Control may return to client (if the exception was caught), - or the processing may end. - -** Transient and Non-transient Failures - - We may wish to classify exceptions into (at least) three types, and - vary the retry policy based on the classification: - - * Transient failures come from resources that are external and may - have independent lifecycles to the client process. Examples are - database deadlock, network connectivity. It is always worth - retrying on a transient failure, and normally we can keep retrying - (if not forever then for a very long time), in the belief that - eventually the resource will become available again. - - * Non-transient failures can be retried a few times. This is the - default. - - * Non-retryable failures like a configuration or input data error - should not be retried (they will always fail the same way). - -** Early Termination - - Normally client code is unaware of the Framework, but occasionally - emergency measures might be taken inside client code where all - further retry attempts are vetoed for the current block. - -** Stateful Retry - - A stateful (or external) retry is used to force a roll back of an - external message (or other data) resource, so that the message will - be re-delivered. The implementation has to be stateful so it can - remember the context for the failed message next time it is - delivered. The additional features of a stateful retry, as opposed - to a normal rollback, are that: - - * A message can be retried indefinitely or up to a set number of - times, after which an error processing route is taken. - - * A back-off delay is used at the of the retry - before any other transactional resources are enlisted. - -* {Implementation} - - * The vanilla case and most of the variations can be achieved with a - simple template approach: - -+--- -RetryTemplate retryTemplate = new RetryTemplate(); -retryTemplate.setRetryPolicy(new SimpleRetryPolicy(5)); -Object result = retryTemplate.execute(new RetryCallback() { - public Object doWithRetry(RetryContext context) throws Throwable { - // do some processing - return result; - } -}); -+--- - - * Schematically we can represent the implementation of the [retry} - template as follows: - -+--- -1 | TRY { -1.1 | do something; -2 | } FAIL { -2.1 | if (retry limit reached) { -2.2 | rethrow exception; - | } else { -2.3 | TRY(1) again; - | } - | } -+--- - - * The template has policies for back off and retry (whether or not - to retry the last exception). The example above shows the retry - policy being set to simply retry all exceptions up to a limit of 5 - times. - - * The <<>> has an API that allows clients to override - the retry policy. The context can also be accessed as a thread - local from a static convenience class, in the case that the callback - is implemented as a wrapper around a POJO. - - * External retry is the most difficult variation to implement, and - doesn't fit naturally into the template model above. Two things - depend on the retry count - back-off delay and the decision to - follow the recovery path - so it needs to be available at the - beginning of every processing block. - - We will discuss the implementation from a JMS-flavoured viewpoint, - where the current item being processed is a message. This can be - generalised to more generic data types, as long as the item can be - rejected transactionally to signal that we require it to be - re-delivered to this or another consumer. - - Consider this pattern, which is very typical: - -+--- -1 | SESSION { -2 | receive; -3 | RETRY { - | remote access; - | } - | } -+--- - - A <<>> is responsible for the RETRY(3) block. But - we can't put the same wrapper around the whole process: - -+--- -0 | RETRY { // Do not do this! -1 | SESSION { -2 | receive; -3 | RETRY { - | remote access; - | } - | } - | } -+--- - - because the receive(2) might not get the same message back on the - second and subsequent attempts (another consumer might get it, or it - might come out of order). So external retry has a different flow - - it might be a different implementation of the same interface, or a - different parameterisation of the normal retry template. - - We can break down the implementation of an external retry into steps - as follows: - -+--- -1 | SESSION { -2 | receive; -3 | TRY { -3.1 | if (already processed) { -3.2 | backoff; - | } -4 | RETRY { - | remote access; - | } -5 | } FAIL { -5.1 | if (retry limit reached) { -5.2 | recover; - | } else { -5.3 | rethrow exception; - | } - | } - | } -+--- - - Decisions (3.1) and (5.1) require knowledge of the history of - processing the current message. Note that the action on failure is - the opposite to the vanilla case {{{Implementation}above}} - if the retry - limit is not reached then we rethrow the exception. - - If the retry limit is not reached then the rethrow(5.3) causes the - SESSION(1) to roll back, and the message will be re-delivered. - RETRY(4) is a normal retry with a template. - - The retry logic is easy to implement - the hard bit is that the - policies depend on the history of the message. This requires some - special retry and back off policies that are aware of the history: - - * When a message arrives, at the beginning of the TRY(3) above, we - need to update our knowledge of its history. - - * The backoff policy can decide whether to back off immediately - when it is initialized at step (3.1). - - * The retry decision at (5.1) has to be aware of the history as - well as some simple exception classification rules. - - * If the retry cannot proceed the retry policy can take steps to - recover (5.2), e.g. send the current message to an error queue. - The exception should not propagate in this case. - - * If we fail and rethrow (5.3), then we need to store the - knowledge of the message history somewhere where another consumer - can access it. - - There is a small conundrum about what value to return from the - TRY(3) block if it ultimately fails (5.2) - a normal retry never - completes unless it is successful, but an external retry can - complete if it is unsuccessful. The obvious choice is to return - null. It probably won't matter in a messaging application anyway - because the client of the retry block probably isn't expecting - anything. It may matter if the TRY(3) block is part of a batch - because the batch template uses null as a signal that the current - batch is complete. But on the other hand it might be a good - strategy to close the batch if processing a message fails. - - With JMS there is no indication in the <<>> how many times - it has been rejected - only a flag <<>> to show - that it has failed at least once. To count the number of retries, - we have to store a global map of messages (ids) to retry counts - (within a single VM - for more than one OS process each one has to - be independent). - diff --git a/src/site/apt/cases/scheduled.apt b/src/site/apt/cases/scheduled.apt deleted file mode 100644 index 7b65ce82d1..0000000000 --- a/src/site/apt/cases/scheduled.apt +++ /dev/null @@ -1,42 +0,0 @@ - ------ - Scheduler Managed Use Case - ------ - Wayne Lund, Dave Syer - ------ - May 2007 - -Use Case: Scheduler Managed Processing - -* Goal - - Ensure that an Enterprise Scheduler can interact with the Batch Launcher to start, stop, - suspend and/or kill a batch job. - -* Scope - - * Batch jobs tends to run within carefully planned job stream - schedules. At a minimum this requires an integration between the - Batch Launcher (in the abstract) and the scheduler's control - mechanism to start and stop batch jobs and then to understand the - results of the batch job execution (e.g. COMPLETED, ABENDED, etc.) - so that subsequent actions may be taken. - - * Spring Batch does not aim to implement the scheduling concerns as - such (other tools are available for that). The framework, does need - to provide the information that such tools need to decide when to - act and what to do (e.g. exit code mapping). - -* Preconditions - - * A mechanism has been established for the scheduler to launch a batch job. This is often times - a simple unix or dos shell script. - - * A mapping of exit codes to the error code numbers that the scheduler is expecting on the exiting - of a batch job. - -* Success - - * Batch Jobs are launched and managed by scheduler - -* Description - diff --git a/src/site/apt/cases/simple.apt b/src/site/apt/cases/simple.apt deleted file mode 100644 index b24c772333..0000000000 --- a/src/site/apt/cases/simple.apt +++ /dev/null @@ -1,290 +0,0 @@ - ------ - Simple Batch Repeat Use Case - ------ - Dave Syer - ------ - January 2007 - -Use Case: Simple Batch Repeat - -* Goal - - Repeat a simple operation such as processing a data item, or a - message, up to a fixed number of times, normally with a transaction - scoped to the whole batch. Transaction resources are shared between - the operations in the batch, leading to performance benefits. - -* Scope - - The operation to be repeated: - - * Can expect to use and manage its own I/O or datastore resources, - but not necessarily transactions; - - * May need to introspect the batch status (as a variation); - - * Executes synchronously or asynchronously (as a variation). - - * Is stateless - this is not a framework restriction in principle, - but simplifies the implementation for now. See in the - {{{store}Implementation}} section below for some notes on - stateful synchronisation; - - * Should be implementable as a POJO if desired. - -* Preconditions - - Client code can locate and acquire all the resources it needs for - the batched operation, and can force transactions to rollback for - testing purposes. - -* Success - - * Verify that a successful batch executed a fixed number of times. - - * Verify that a batch completes early but successfully if an - underlying transaction times out. - - * Terminate a batch by failing one of the operations, and verify - that the preceding operations rolled back (subject to batch meta - data). - - * Execute a batch asynchronously and verify that the correct number - of operations is performed. - -* Description - - We are often interested in a specific scenario of this use case - where the batched operation is: - - * Read a message or data item from an endpoint like a JMS - Destination. - - * Do some business processing involving database reads and writes. - - The vanilla successful batch use case proceeds as follows: - - [[1]] Framework starts a batch, acquiring resources as needed and - creating a context for the execution. - - [[1]] Client provides a batch operation in the form of a source of - data items and a processor acting on the data item. - - [[1]] Framework executes batch operation. - - [[1]] Repeat the last step until the batch size is reached. - - [[1]] Framework commits the batch. All database changes are - committed and received messages removed from the endpoints. - -* Variations - -** Rollback - - If one of the operations rolls back it will throw an exception. - Normal transaction semantics determine what happens next. Usually - (in the scenario described above) there is an outer transaction for - the whole batch, which rolls back as well: all the messages remain - unsent, and all the data remain uncommitted. A retry will receive - exactly the same initial conditions. - -** Timeout - - The batch size is not fixed. The use case proceeds as above, but in - the middle of a batch operation execution: - - - [[1]] Framework determines that the batch has timed out operation - (e.g. while it was waiting for an incoming message). - - [[1]] Framework commits the batch with all operations so far - complete - possibly a smaller than normal size. - -** Asynchronous Processing - - Instead of the Framework waiting for each operation to complete it - could spin them off independently into separate threads or a work - queue. The batch still has to have a definite endpoint, so the - Framework waits for all the operations to finish or fail - before cmpleting the batch. - -** Introspection of Batch Context - - Client may wish to inspect the state of the ongoing batch operation, - and potentially force an early completion. - -* {Implementation} - - * The completion of the batch loop is handled by a policy delegate - that we can use to strategise the concept of a loop that might - complete early. This can cover both the timeout variation and the - vanilla use case flow. - - * What form should the batch template (<<>>) - interface take? We might start with something like this: - -+--- -batchTemplate.iterate(new RepeatCallback() { - - public boolean doInIteration() { - // do stuff - } - -}); -+--- - - * A nice tool for a batch operation in a callback is an iterator - through a data set or message endpoint (<<>>), coupled - with a handler for processing the item. This adds a potential - implementation of <<>> that knows about the - <<>> and adds a processor object. E.g. as an - anonymous inner class: - -+--- -final ItemProvider provider = new JmsItemProvider(); -final ItemProcessor processor = new ItemProcessor() { - public void process(Object data) { - // do something with the data (a record) - } -}; - -batchTemplate.execute(new RepeatCallback() { - - public boolean doInIteration() { - Object data = provider.next(); - if (data!=null) { - processor.process(data); - } - return data!=null; - } - -}); -+--- - - * Is a batch template with callback the best implementation? Could - we perhaps use or re-use <<>> somehow? Which is - better for the client: - -+--- -batchTemplate.iterate(new RepeatCallback() { - - public boolean doInIteration() { - // do stuff - } - -}); -+--- - - where the batch template might itself use a <<>> - internally, or - -+--- -batchTemplate.iterate(new Runnable() { - - public void run() { - // do stuff with data - }; - -}); -+--- - - where the batch template is a <<>>. Probably the - former because it is more encapsulated: it gives the framework more - freedom to implement the template in any way it needs to, e.g. to - accommodate more complicated use cases. - - * To {store} up SQL operations until the end of a batch, and take - advantage of JDBC driver efficiencies, the client needs to store - some state during the batch, and also register a transaction - synchronisation. For this kind of scenario we introduce an - interceptor framework in the template execution. The template calls - back to interceptors, which themselves can strategise clean up and - close-type behaviour: - -+--- -public class RepeatTemplate implements RepeatOperations { - - public void iterate(RepeatCallback callback) { - - // set up the batch - interceptors.open(); - - while (running) { - - // allow interceptor to pre-process and veto continuation - interceptor.before(); - - // continue only if batch is ongoing - if (running = callback.doInIteration()!=null) { - interceptor.after(); - } - - } - - // clean up or commit the whole batch - interceptor.close(); - - } -} -+--- - - The <<>> can be stateful, and can store up inserts - until the end of the batch. If the <<>> is - transactional then they will only happen if the transaction is - successful. - - This way the client can even decide to use a batch interceptor - that runs in its own transaction at the end of the batch. - - * There is no need for an overall batch timeout because the inner - operations are synchronous and have their own timeout metadata - though transaction definitions. The whole batch (outer transaction) - may still have a timeout attribute, and then there is a corner case - where the batch operations are all successful, but because they all - took a long time the whole batch rolls back because of the timeout. - - * The context of the ongoing batch is closely linked with the - completion policy. The completion policy is pluggable into the - batch template, and acts as a factory for context objects which can - then be inspected by Client in the callback. For example: - -+--- -public class RepeatTemplate implements RepeatOperations { - - public void iterate(RepeatCallback callback) { - - // set up the batch session - RepeatContext context = completionPolicy.start(); - - while (!completionPolicy.isComplete(context)) { - - // callback gets the context as an argument - callback.doInIteration(context); - - completionPolicy.update(context); - } - - } -} -+--- - - * The example above provides Client the opportunity to inspect the - context through the callback interface. If Client is a POJO, - Framework has to create a callback and wrap it, in which case there - needs to be a global accessor for the current context or session. - The template is then responsible for registering the current context - with a <<>>. E.g.client code can look - at the session and mark it as complete if desired - (c.f. <<>>): - -+--- -public Object doMyBatch() { - - // do some processing - - // something bad happened... - RepeatContext context = RepeatSynchronizationManager.getContext(); - context.setCompleteOnly(); - -} -+--- diff --git a/src/site/apt/cases/steps.apt b/src/site/apt/cases/steps.apt deleted file mode 100644 index c0340464b7..0000000000 --- a/src/site/apt/cases/steps.apt +++ /dev/null @@ -1,176 +0,0 @@ - ------ - Batch: Sequential Steps Use Case - ------ - Dave Syer - ------ - January 2007 - -Use Case: Sequential Processing of Dependent Steps - -* Goal - - Compose a batch operation from a sequence of dependent steps. - Define and implement the operation only once, and allow restart - after failure without having to change configuration, and without - having to repeat steps that were successful. - - A sub-goal is to allow the progress of a batch through the steps to - be traced accurately for reporting and auditing purposes. This - requires the steps to be uniquely identified. - -* Scope - - * Simple linear sequence of steps. Slightly more complicated - requirements can be handled by putting independent steps in a - sequence (no need for splits and joins). - -* Preconditions - - * A non-trivial sequence is defined: - - * more than one step: - - * the effects of each step can be measured. - - * The sequence can be interrupted or artificially terminated in the - second or subsequent step. - -* Success - - * A non-trivial sequence executes successfully. The progress and - success of each step can be verified by the tester. - - * The same sequence is forced to fail on second step in such a way - that the first step result is not suspected of being in error, - e.g. by interrupting it. When it is restarted the first step is not - repeated, and the sequence is successful. - - * The same sequence is forced to fail on second step in such a way - that the first step result is obviously in error, even though it - completed normally. When the batch is restarted the first step - repeated, and the sequence is successful. - -* Description - - The vanilla successful case proceeds as follows: - - [[1]] Framework logs the start of a step, uniquely indentifying - the initial conditions. - - [[1]] Framework stores internal state so that initial conditions - can be re-created in the event of a restart. - - [[1]] Step execution proceeds as per one of the other use cases - (e.g. {{{file-to-database.html}Copy File to Database}}), including - transactional behaviour. - - [[1]] Client instructs Framework to store internal state needed by - further steps (e.g. cached reference data). - - [[1]] Framework logs successful completion of step, and stores - - [[1]] Repeat for next and subsequent steps. Internal state is - passed from one state to the next. - -* Variations - -** Internal Failure of Step - - If a step fails internally, e.g. because of resource becoming - temporarily unavailable, the sequence can be restarted without - repeating the previous steps. - - [[1]] Operator fixes resource problem (e.g. starts web service). - - [[1]] Operator restarts batch with no configuration or input data - changes. - - [[1]] Framework resumes batch from the last commit point of the - failed step. - - [[1]] Sequence completes normally. - - The process above could be carried out by the framework entirely (no - need for operator intervention) if a retry policy is in effect. - -** Failure of Step Owing to Bad Initial State - - If a step fails because it receives bad data from an earlier step, - the Framework cannot recover without intervention. - - [[1]] Operator attempts to restart without doing anything to fix - the problem. - - [[1]] Framework detects bad initial state immediately and fails - fast. - - If the original problem can be located and fixed (e.g. input data - for earlier step is revised): - - [[1]] Operator restarts batch signalling to framework which step - to begin with. - - [[1]] Framework locates initial state for the first step to be - executed. - - [[1]] Framework starts execution from the beginning of the desired - state. This time the input data are different, so the sequence - can complete normally. - -* Implementation - - * The need to save state for subsequent steps leads to the - introduction of a batch context concept. And the need for - initialising restarts leads to the context being serializable, - either natively or by some pluggable strategy (this is covered in - the {{{./restart.html}Restart after Failure}} use case). - - Unfortunately, the need for {{{./parallel.html}parallel processing}} - and automatic {{{./restart.html}restart}} also makes it practically - impossible for steps to handle the context at the level of a single - thread of execution, where the client needs to implement business - logic. If a step is executing in parallel, then each node needs to - be able to restart independently, but the context needs to be a - single object that can be passed on to the next step (unless all the - steps are parallelised with the same multiplicity, which might not - be efficient in general). - - Thus batch context must be defined and managed by the template or - execution handler. - - * The requirement for steps might have implications for the - implementer of the batch operation (the client). Obviously a client - defines the sequence of steps according to the business requirement, - but ideally we would like him to be unaware of the reporting and - restart infrastructure. Maybe an array of callbacks works (the - callback interface is irrelevant, except that it accepts a context - object as an argument): - -+--- -batchTemplate.iterate(new RepeatCallback[] { - - new RepeatCallback() { - public boolean doInIteration(RepeatContext context) { - // do stuff for step one - }; - }, - - new RepeatCallback() { - public boolean doInIteration(RepeatContext context) { - // do stuff for step two - the context - // is the same... - }; - } - -}); -+--- - - Notice that there is no need for the context to be set explicitly - before executing the callback. The context is handled internally to - the batch template using an analogue of the - <<>>. - - * If we prefer that clients never need to know about batch - templates, then the code above needs to be automated. This would be - where an additional domain layer might come into play - (c.f. <<>>). diff --git a/src/site/apt/cases/template.apt b/src/site/apt/cases/template.apt deleted file mode 100644 index ed359bed6c..0000000000 --- a/src/site/apt/cases/template.apt +++ /dev/null @@ -1,22 +0,0 @@ - ------ - Template Use Case - ------ - Dave Syer - ------ - January 2007 - -Use Case: Template - -* Goal - -* Scope - -* Preconditions - -* Success - -* Description - -* Variations - -* Implementation \ No newline at end of file diff --git a/src/site/apt/downloads.apt b/src/site/apt/downloads.apt deleted file mode 100644 index d80760e1ec..0000000000 --- a/src/site/apt/downloads.apt +++ /dev/null @@ -1,133 +0,0 @@ - --------- - Downloads - --------- - Dave Syer, Ben Hale, Michael Minella - ------ - December 2007, February 2009, July 2009 - -Spring Batch Downloads - - The current GA release is <<2.2.0.RELEASE>>, the latest snapshots are <<3.0.0.BUILD-SNAPSHOT>>. The version 2.0.x and 1.x branches are now in maintenance (the last release was <<2.0.4.RELEASE>>). - -For runtime concerns and a container for running a Job as a service see the {{{http://static.springframework.org/spring-batch-admin}Spring Batch Admin}} project and the {{{http://static.springframework.org/spring-batch-admin/getting-started.html}getting started}} link there. - -* Zip Downloads - - There is a ZIP artifact containing the release JARs called <<>>. This file contains the JAR files for the release, including source code and the samples. - - * Full releases: {{{http://static.springframework.org/downloads/nightly/release-download.php?project=BATCH}here}}. - - * Milestones: {{{http://static.springframework.org/downloads/nightly/milestone-download.php?project=BATCH}here}}. - -Source code can also be browsed and downloaded at {{{http://github.com/SpringSource/spring-batch}Github}}. - -* Maven Artifacts - - Traditional "spring-*" artifacts are deployed on the Maven central repo. Full releases in this format also go in our s3 repository, browseable {{{http://shrub.appspot.com/maven.springframework.org/release/org/springframework/batch/}here}}: - -+--------------- - - spring-releases - Spring Maven RELEASE Repository - http://repo.springsource.org/release - -+--------------- - - You can see the internal and external project dependencies in the <<>> files and also in the dependency reports for each module on this website. You will probably need <<>> and <<>>. Source code is packaged in a separate jar file in the same directory, and the samples are also bundled in the same way. - - Individual dependencies can then by added like so (inside a \ element at the top level): - -+--------------- - - org.springframework.batch - spring-batch-core - 2.1.9.RELEASE - -+--------------- - - All releases are also published in the SpringSource Enterprise Repository, {{{http://www.springsource.com/repository/app/}searchable here}} and {{{http://shrub.appspot.com/repository.springsource.com/maven/bundles/release}browseable here}}. The repository is segregrated by release, milestone and snaphot releases. For full releases use these repositories: - -+--- - - com.springsource.repository.bundles.release - SpringSource Enterprise Bundle Repository - - SpringSource Bundle Releases - http://repository.springsource.com/maven/bundles/release - - false - - - - com.springsource.repository.bundles.external - SpringSource Enterprise Bundle Repository - - External Bundle Releases - http://repository.springsource.com/maven/bundles/external - - false - - -+--- - - In the SpringSource Enterprise Repository the artifacts have long, Java-package-like names: <<>>, <<>> and <<>>. They have dependencies which are in the same style, and are transitively complete within the same repository. So individual dependencies can by added like so (inside a \ element at the top level): - -+--------------- - - org.springframework.batch - org.springframework.batch.core - 2.2.0.RELEASE - -+--------------- - -* Milestone Builds - - These builds are provided for evaluation and community feedback. Spring Batch builds have a release identifier ending in "MX" where X is the milestone number, or "RCX" for a release candidate (older builds have "mX" and "rcX"). - - You can find the .ZIP downloads of the milestones {{{http://static.springframework.org/downloads/nightly/milestone-download.php?project=BATCH}here}}. - - Milestones also published in the s3 repository with Maven Central format dependencies {{{http://shrub.appspot.com/maven.springframework.org/milestone/org/springframework/batch}browseable here}}. To use them, just add the following repository to your POM (inside a \ element at the top level): - -+--------------- - - spring-s3 - Spring Maven MILESTONE Repository - http://maven.springframework.org/milestone - -+--------------- - - Then individual dependencies can then by added like so (inside a \ element at the top level): - -+--------------- - - org.springframework.batch - spring-batch-core - 2.2.0.RELEASE - -+--------------- - - Milestones are also deployed to the SpringSource Enterprise Repository ({{{http://shrub.appspot.com/repository.springsource.com/maven/bundles/milestone/}browseable here}}), so use this repository if you want to depend on artifacts in the <<>> style: - -+--- - - com.springsource.repository.bundles.milestone - SpringSource Enterprise Bundle Repository - - SpringSource Bundle Milestones - http://repository.springsource.com/maven/bundles/milestone - - false - - -+--- - - Then individual dependencies can by added like so (inside a \ element at the top level): - -+--------------- - - org.springframework.batch - org.springframework.batch.core - 2.2.0.RELEASE - -+--------------- - -Snapshot Builds - - Nightly snapshots are available through the same process with slightly different URLs. See {{{./snapshots.html}here}} for details. diff --git a/src/site/apt/features.apt b/src/site/apt/features.apt deleted file mode 100644 index 0c730f59df..0000000000 --- a/src/site/apt/features.apt +++ /dev/null @@ -1,157 +0,0 @@ - ------ - Spring Batch Features - ------ - Dave Syer - ------ - July 2007, February 2008, January 2009 - -Spring Batch Features and Roadmap - - See also {{{./migration/2.0-highlights.html}details of main themes of 2.0}}. - -* 2.0 Features - - The following features are supported by Spring Batch 2.0: - -** Optimisation and Infrastructure - - * RepeatOperations: an abstraction for grouping repeated - operations together and moving the iteration logic into the - framework. - - * RetryOperations: an abstraction for automatic retry. - - * ItemReader abstraction and implementations for flat files, xml - streaming and simple database queries. - - * Flat files are supported with fixed length and delimited records - (input and ouput). - - * Xml is supported through Spring OXM mapping between objects and Xml - elements (input and ouput). Large files are streamed, not read as a whole. - - * Database implementations of ItemReader are provided that map a row of - a ResultSet identified by a simple (single or multiple column) - primary key. - - * ItemWriter abstraction and implementations for flat files and xml - (the Sql case is just a regular Jdbc Dao). - - * ItemReader and ItemWriter implementations are generally - ItemStreams. An ItemStream provides the facility to be restored - from a persistent ExecutionContext so that jobs can fail and be - restarted in another process. - - * For modifying an item before it is written, there is the - ItemProcessor abstraction. ItemProcessor and ItemWriter are the - two most common application developer touch points. - -** Core Domain - - * Job is the root of the core domain - it is a recipe for how to - construct and run a JobInstance. - - * A Job is composed of a list of Steps (sequential step model for - job). - - * Job is also the entry point for launching a JobExecution. - - * Step is the corresponding point for a StepExecution. Step is the - main strategy for different scaling, distribution and processing - approaches. The 2.0 release contains implementations for in-process - execution (single VM), and a PartitionStep as part of an SPI for - remote execution of steps. See below (under Execution). - - * The most commonly used implementation of Step is a wrapper for an - ItemReader and an ItemWriter. There is also a special - implementation that wraps a Tasklet, which can be used to execute a - single action like a stored procedure call. - - * FactoryBeans are provided for creating Step instances with the - most common features. See in particular - FaultTolerantStepFactoryBean for a factory that provides convenient - configuration points for skips and retries. - - * Late binding of environment properties, job parameters and - execution context values into a Step when it starts. A custom - Spring Scope takes care of deferring the initialization of - components until a step is executing. - -** Job Execution and Management - - * A simple JobLauncher to launch jobs. Start a new one or restart - one that has previously failed. This can be used by a command-line - or JMX launcher to take simple input parameters and convert them to - the form required by the Core. (Examples of both are in the Samples - module.) - - * Persistence of job meta data for management and reporting - purposes: job and step identifiers, job parameters, commit counts, - rollback counts. Execution attributes (a human readable - represenation of the state of the job - can be augmented by - developers). - - * Adjustiable exception handling strategies allowing fault - tolerance through skipping bad records. - - * Concurrent execution of chunks (a chunk is a batch of items - processed in the same transaction) through the Spring TaskExecutor - abstraction. - - * Automatic retry of a chunk and recovery for items that have - exhausted their retry count. - - * Translation of job execution result into an exit code for - schedulers running the job as an OS process. - - * A set of listener callbacks that users can implement and register - with a Step to add custom behaviour like footer records. - - * Remote chunking of steps. The step proceeds as in the single JVM - case, but each chunk is passed on to the remote processes. The - remote execution is an asynchronous listener of some sort - (e.g. message-driven component or web service). Implemented using - {{{http://www.springsource.org/spring-integration}Spring - Integration}} in a Batch sub-project (spring-batch-integration). - - * Partitioning - steps execute concurrently and optionally in - separate processes. Feedback loop between consumers and producers - to prevent overflows. Spring Batch provides an SPI for Partitioning - and an implementation for local (multi-threaded, single JVM) - execution. - - * OSGi support. Deploy the Spring Batch framework as a set of OSGi - services. Deploy individual jobs or groups of jobs as additional - bundles that depend on the core. Spring Batch JAR files are also - OSGi bundles, and can be deployed easily in - {{{http://www.springsource.com/dmserver}SpringSource dm Server}}. - - * Non-sequential models for Job configuration (branching and - descision support). - -** Samples - - * A range of samples is available as a separate module. They all - use a common simple configuration and extend in various ways to show - the different features of the Execution module. - -* Roadmap (Beyond 2.0). - - * Issue tracking - a job is not finished until all issues with its - executions are resolved. Spring Batch can provide hooks to - integrate with internal issue tracking systems so that the lifetime - of a job can be properly managed. - - * Auditing. Implement hooks to monitor not only what jobs execute - and the result of the execution (as per 1.0 possibly with some - richer options for detailed outcome reports), but also who has - executed the job, what changes they made to runtime parameters. - -* SpringSource Enterprise Batch - - * The plan is for {{{http://www.springsource.com}SpringSource}} to - provide an enterprise product that deals with runtime concerns, as - opposed to programming and configuration. - - * Triggering. Other runtime concerns, like monitoring and managemtn - of jobs and historical executions. diff --git a/src/site/apt/getting-started.apt b/src/site/apt/getting-started.apt deleted file mode 100644 index c539c94fe9..0000000000 --- a/src/site/apt/getting-started.apt +++ /dev/null @@ -1,90 +0,0 @@ - ------ - Spring Batch Getting Started - ------ - Dave Syer - ------ - December 2007, June 2009 - -Spring Batch Getting Started - - A convenient way to get started quickly with Spring Batch is to run the samples which are packaged in the samples module. There is also a simple command line sample (or "archetype") which has a bare bones but complete implementation of a simpel job. The source code for the samples (and the other modules) is available either from the {{{./downloads.html}.Zip assembly}} or from {{{source-repository.html}Git}}. - -* Using SpringSource Tool Suite (STS) - - This is the quickest way to get started. It requires an internet connection for download, and access to a Maven repository (remote or local). - - * Download STS version 2.3.2.* from the {{{http://www.springsource.com/products/sts}SpringSource website}}. STS is a free Eclipse bundle with many features useful for Spring developers. - - * Go to <<Other...>>> and select <<Template Project>>> from the wizard chooser. - - * The wizard has a drop down with a list of template projects. One of them is a "Simple Spring Batch Project". Select it and follow the wizard. - - * A project is created with all dependencies and a simple input/output job configuration. It can be run using a unit test, or on the command line (see instructions in the pom.xml). - -* Using the .Zip Distribution - -** With Maven and Eclipse - - * Download the "no-dependencies" version of the distribution and unzip to create a directory <<>>>. - - * Get the m2eclipse plugin (http://m2eclipse.sonatype.org/update) - (installed in STS out of the box). If you can't or don't want to - install this plugin, you can use the - {{{http://maven.apache.org/plugins/maven-eclipse-plugin/}Maven - Eclipse Plugin}} to create the classpath entries you need. - - * Open Eclipse and create a workspace as for the non-Mavenized version. - - * Import the samples and archetype projects from the samples sub-directory in the directory you just unpacked. - - * The project should build cleanly without having to fix the dependencies. If it doesn't, make sure you are online, and maybe try building on the command line first to make sure all the dependencies are downloaded. See the {{{./building.html}building instructions}} if you run into difficulty. - - (N.B. the "archetype" is not a real Maven archetype, just a template project that can be used as a starting point for a self-contained batch job. It is the same project that can be imported into STS using the Project Template wizard.) - -** With Maven on the Command Line - - * Download the distribution as above. - - * Then run Maven in the spring-batch-samples directory, e.g. - -+--- -$ cd spring-batch-samples -$ mvn test -... -+--- - -** With Eclipse and without Maven - - Similar instructions would apply to other IDEs. - - * Download the "no-dependencies" package and unzip to create a directory <<>>>. - - * Open Eclipse and make a workspace in the directory you just created. - - * Import the <<>> project from the samples directory. - - * Find all the compile scope and non-optional runtime jar files listed in the {{{./spring-batch-core/dependencies.html}core dependencies report}} and {{{./spring-batch-infrastructure/dependencies.html}infrastructure dependencies report}}, and import them into the project. - - * Force the workspace to build (e.g. Project -> Clean...) - - * Run the unit tests in your project under src/test/java. N.B. the FootbalJobFunctionTests takes quite a long time to run. - - You can get a pretty good idea about how to set up a job by examining the unit tests in the <<>> package (in <<>>) and the configuration in <<>>. - - To launch a job from the command line instead of a unit test use the <<>> method (see Javadocs included in that class). - -* Using Maven and Git - - * Check out the Spring Batch project from Git (instructions are available {{{./source-repository.html}here}}). - - * Run Maven from the command line in the samples directory. There are additional building instructions and suggestions about what to do if it goes wrong {{{./building.html}here}}. - -* Using Gradle (for Groovy programmers) - - * There is a Groovy project template with instructions available {{{http://robokaso.github.com/Groovy-Spring-Batch-Template/}here}} - -* Migrating to 2.2.x - - As part of the update to support non-identifying job parameters ({{{https://jira.springsource.org/browse/BATCH-1412}BATCH-1412}}), the database schema for the job repository was updated. To migrate your existing job repository to the new one, you'll need to execute the appropriate migration script for your platform. This script will create the new BATCH_JOB_EXECUTION_PARAMS table and migrate the parameters from the old table to the new table. The script can be found in the <<>> package. - - diff --git a/src/site/apt/index.apt b/src/site/apt/index.apt deleted file mode 100644 index 63c986b433..0000000000 --- a/src/site/apt/index.apt +++ /dev/null @@ -1,57 +0,0 @@ - ------ - Spring Batch - ------ - Dave Syer, Scott Wintermute - ------ - March 2007, May 2007 - -Introduction - - Many applications within the enterprise domain require bulk processing to perform business operations in mission critical environments. These business operations include automated, complex processing of large volumes of information that is most efficiently processed without user interaction. These operations typically include time based events (e.g. month-end calculations, notices or correspondence), periodic application of complex business rules processed repetitively across very large data sets (e.g. insurance benefit determination or rate adjustments), or the integration of information that is received from internal and external systems that typically requires formatting, validation and processing in a transactional manner into the system of record. Batch processing is used to process billions of transactions every day for enterprises. - - Spring Batch is a lightweight, comprehensive batch framework designed to enable the development of robust batch applications vital for the daily operations of enterprise systems. Spring Batch builds upon the productivity, POJO-based development approach, and general ease of use capabilities people have come to know from the Spring Framework, while making it easy for developers to access and leverage more advanced enterprise services when necessary. - - Spring Batch provides reusable functions that are essential in processing large volumes of records, including logging/tracing, transaction management, job processing statistics, job restart, skip, and resource management. It also provides more advanced technical services and features that will enable extremely high-volume and high performance batch jobs through optimization and partitioning techniques. Simple as well as complex, high-volume batch jobs can leverage the framework in a highly scalable manner to process significant volumes of information. - - Spring Batch is part of {{{http://www.springframework.org/projects}Spring}}. For runtime concerns and a container for running a Job as a service see the {{{http://static.springframework.org/spring-batch-admin}Spring Batch Admin}} project. - -* Spring Batch Architecture - - Spring Batch is designed with extensibility and a diverse group of end users in mind. The figure below shows a sketch of the layered architecture that supports the extensibility and ease of use for end-user developers. - -[images/ExecutionEnvironment.png] Spring Batch Architecture showing Infrastructure and Execution Layers. Potential execution strategy implementations support different platforms and end-user goals from the same blocks of business logic in the Application Layer. - - Spring Batch provides an Infrastructure layer in the form of low level tools. There is also a simple execution environment, using the infrastructure in its implementation. The execution environment provides robust features for traceability and management of the batch lifecycle. A key goal is that the management of the batch process (locating a job and its input, starting, scheduling, restarting, and finally processing to created results) should be as easy as possible for developers. - - The Infrastructure provides the ability to batch operations together, and to retry an piece of work if there is an exception. Both requirements have a transactional flavour, and similar concepts are relevant (propagation, synchronisation). They also both lend themselves to the template programming model common in Spring, c.f. <<>>, <<>>, <<>>. - - The Core module is the batch-focused domain and implementation. It provides a robust set of integrated features including job processing statistics, job launch and restart to enable the management of the full lifecycle of traditional batch processing. - - A number of sample jobs are packaged in a separate Samples module to more clearly articulate the usage and capabilities of the Core module. - - The runtime dependencies of infrastructure and core are shown in the figure below. - -[images/RuntimeDependencies.png] Spring Batch runtime dependencies, showing how a client application (as per the samples) can be built in terms of other modules. - -* Roadmap - - The current production release version is specified {{{./downloads.html}here}}, and links are also provided to the latest development work. - - The framework is oriented around application developers not needing to know any details of the framework - there are a few application developer interfaces that can be used for convenient construction of data processing pipelines, but apart from that we support as close to a POJO programming model as is practical. This is similar to the approach taken in Spring Core in the area of DAO implementation. - - Spring Batch version 1.x was targeted at Java 1.4 and single-process, possibly multi-threaded execution. Spring Batch 2.0 is a Java 5 only release, using all available language features with no compromises for backward compatibility with Java 2. We think this will provide a significantly improved programming model for batch application developers. - - Framework tools for scaling to multiple processes are available in Spring Batch 2.0. These provide advanced technical services and features to enable extremely high-volume and high performance batch jobs though proven optimization and clustering techniques. An SPI is provided with a simple implementation that works in a single process (multi-threaded). Various remoting and grid technologies can be used to implement the same SPI in a multi-process, clustered environment. - - Matt Welsh's work shows that {{{http://www.eecs.harvard.edu/~mdw/proj/seda/}SEDA}} has enormous benefits over more rigid processing architectures, and messaging environments give us a lot of resilience out of the box. So we also want to enable a more SEDA flavoured execution environments, as well as supporting the more traditional ETL style approach. The key to unlocking the programming model is {{{http://www.springframework.org/spring-integration}Spring Integration}}, where the choice of transport and distribution strategy can be made as late as possible. The key to the runtime requirements of deployment and manageability is going to be with the {{{http://www.springsource.com/products/suite/applicationplatform}SpringSource Application Platform}}. The same application code could be used in principle for a standalone tool processing a small amount of data, and a massive enterprise-scale bulk-processing engine. - -* Background - - While open source software projects and associated communities have focused greater attention on web-based and SOA messaging-based architecture frameworks, there has been a notable lack of focus on reusable architecture frameworks to accommodate Java-based batch processing needs, despite continued needs to handle such processing within enterprise IT environments. The lack of a standard, reusable batch architecture has resulted in the proliferation of many one-off, in-house solutions developed within client enterprise IT functions. - - SpringSource and Accenture are collaborating to change this. Accenture's hands-on industry and technical experience in implementing batch architectures, SpringSource's depth of technical experience, and Spring's proven programming model together mark a natural and powerful partnership to create high-quality, market relevant software aimed at filling an important gap in enterprise Java. Both companies are also currently working with a number of clients solving similar problems developing Spring-based batch architecture solutions. This has provided some useful additional detail and real-life constraints helping to ensure the solution can be applied to the real-world problems posed by clients. For these reasons and many more, SpringSource and Accenture have teamed to collaborate on the development of Spring Batch. - - Accenture is contributing previously proprietary batch processing architecture frameworks -- based upon decades worth of experience in building batch architectures with the last several generations of platforms (i.e., COBOL/Mainframe, C++/Unix, and now Java/anywhere) -- to the Spring Batch project along with committer resources to drive support, enhancements, and the future roadmap. - - The collaborative effort between Accenture and SpringSource aims to promote the standardization of software processing approaches, frameworks, and tools that can be consistently leveraged by enterprise users when creating batch applications. Companies and government agencies desiring to deliver standard, proven solutions to their enterprise IT environments will benefit from Spring Batch. - diff --git a/src/site/apt/migration/1.0-m2-m3.apt b/src/site/apt/migration/1.0-m2-m3.apt deleted file mode 100644 index 0e7766e65e..0000000000 --- a/src/site/apt/migration/1.0-m2-m3.apt +++ /dev/null @@ -1,67 +0,0 @@ - ------ - Spring Batch Upgrade - ------ - Robert Kasanicky - ------ - December 2007 - -Updating Spring Batch from 1.0-m2 to 1.0-m3 - - This is a description of what needed to be done to migrate the samples from m2 to the new m3 release. - - [Tip:] if you use SpringIDE, make sure it tracks all your configuration files, -so that errors are immediately visible. - -* VARIOUS - - * OutputSource interface has been renamed to ItemWriter, implementations have been renamed correspondingly -(e.g. FlatFileOutputSource is now FlatFileItemWriter). ItemWriter no longer extends ResourceLifecycle -(which declares methods open() and close). Often you don't need to call these methods at all, in case -you do you should implement InitializingBean and DisposableBean interfaces and declare the bean -in step scope. - - * Sql renamed to Jdbc (e.g. SqlInputSource -> JdbcInputSource). - - * Database input sources split into driving and cursor packages (cursor makes a single query and iterates -over the result set, driving query for keys and then make a new query for each key). - - * ExceptionHandler interface has been changed to handle single throwable instead of a collection of throwables -and it has become responsible for deciding about step completion. See javadoc for more details. - - * FlatFileInputSources now include FieldSet mapping logic - you can inject a FieldSetMapper -into the input source, so you no longer need to handle this in ItemProvider. Therefore -FieldSetInputSource interface and FlatFileItemProvider have been removed. Where FlatFileItemProvider -was used you can use the InputSourceItemProvider and inject the mapper into the input source. - - * Validation has been removed from input sources and moved upwards to ValidatingItemProvider. - - * XML related classes have been moved from io.stax package under io.file - - * Fixed-length tokenizer accept ranges property instead of array of lengths, e.g. "2,3" -(see javadoc for more details and fixedLengthImportJob.xml for example usage). - - -* CONTAINER CONFIGURATION - - * Use the value of JobConfiguration name property as the bean id. - - * Simplest approach to update is to use the simple-container-definition.xml, -data-source-context.xml (plus hibernate-context.xml) and batch.properties -from M3 samples and replace the post-processor bean in your job xml file with -"\" - - * JobLauncher has been moved from execution.bootstrap to execution.launch - - * JobExecutor facade has been removed, it's properties are now injected directly -into SimpleJobLauncher intead of the facade itself. - - * DefaultStepExecutorFactory is now SimpleStepExecutorFactory located in simple subpackage -and needs a reference to jobRepository instead of value for StepExecutor name. - - * ScheduledJobIdentifier has a new jobKey property replaces jobRun and jobStream. - - * SimpleStepConfiguration has been moved one package up (from execution.step.simple to execution.step). - - * Transaction manager class is specified in batch.properties - - diff --git a/src/site/apt/migration/1.0-m3-m4.apt b/src/site/apt/migration/1.0-m3-m4.apt deleted file mode 100644 index f8cf4a196f..0000000000 --- a/src/site/apt/migration/1.0-m3-m4.apt +++ /dev/null @@ -1,59 +0,0 @@ - ------ - Spring Batch Upgrade - ------ - Robert Kasanicky - ------ - February 2008 - -Updating Spring Batch jobs from 1.0.0.m3 to 1.0.0.m4 framework - - This is a description of what needed to be done to migrate the samples -from m3 to the m4 release. - - [Tip:] if you use SpringIDE, make sure it tracks all your configuration -files, so that errors are immediately visible. - -* JOB CONFIGURATION - - * ItemProviderTasklet and RestartableItemProviderTasklet have been replaced -with ItemOrientedTasklet - - * ItemProvider/ItemProcessor pair is now ItemReader/ItemWriter - - * InputSource and ItemProvider interfaces have been merged into ItemReader -(ItemReaders can be composite when desired) - - * OutputSource and ItemProcessor interfaces have been merged into ItemWriter -(ItemTransformerItemWriter and CompositeItemTransformer provide support for -item transformations before writing to output) - - * JobConfiguration and Step Configuration are now called simply Job and Step. There is no JobExecutor or StepExecutor. - -* CONTAINER CONFIGURATION - - * JobIdentifier is replaced by JobParameters - - * JobLauncher#run(Job, JobParameters) now returns JobExecution which has -methods stop() and isRunnig() (previously part of JobLauncher) - - * Hibernate repository (JobDao and StepDao) has been removed - - * JobExecutor and StepExecutor are no longer explicitly declared -(Job and Step execute themselves) - - -* VARIOUS - - * Remember to update the database schema - - * Several moves and package renames easily fixed by IDE refactoring support -('organize imports' in Eclipse) - - * FieldSet is now an interface with DefaultFieldSet as provided implementation - - * ResourceLifecycle and Restartable interfaces have been removed, ItemStream -now covers their responsibilities - - * StatisticsProvider interface has been superceded by ExecutionAttributesProvider - - * BatchTransationSynchronizationManager has been removed. Use the regular TransactionSynchronizationManager if you need to register a synchronization. diff --git a/src/site/apt/migration/1.0-m4-m5.apt b/src/site/apt/migration/1.0-m4-m5.apt deleted file mode 100644 index 901bf04c3f..0000000000 --- a/src/site/apt/migration/1.0-m4-m5.apt +++ /dev/null @@ -1,49 +0,0 @@ - ------ - Spring Batch Upgrade - ------ - Robert Kasanicky - ------ - March 2008 - -Updating Spring Batch jobs from 1.0.0.m4 to 1.0.0.m5 framework - - The following is a description of what is needed to migrate the samples from m4 to the m5 release. - -* JOB CONFIGURATION - - * Tasklet's role has changed - It is no longer called iteratively, but is now dedicated for tasks that are not natural to split into read-write phases (e.g. calling stored procedure or system command). Readers and writers are now typically injected into (ItemOriented)Step. See Tasklet and TaskletStep javadoc for more details. - - * Step scope removed - lifecycle of readers and writers explicitly handled by Step. They need to be registered for the lifecycle callbacks with the step. This is automatic if using one of the new step factory beans, but only for the directly injected reader and writer. Any indirect dependencies need to be registered separately (using the streams property of the factory bean). - - * When launching from command line there is one applicaton context for job rather than parent container and child job context. CommandLineJobRunner requires only one xml file location. It is recommended to use a new ApplicationContext per job execution for other use cases involving launching multiple jobs in a single VM. The JMX demo in the samples shows this in practice. - - * SimpleFlatFileItemReader and DefaultFlatFileItemReader merged into FlatFileItemReader - - * LineAggregator is now inverse to LineTokenizer - method signature changed to 'String aggregate(FieldSet)'. - - -* CONTAINER CONFIGURATION - - * Modularized repository daos - JobInstanceDao, JobExecutionDao and StepExecutionDao instead of JobDao and StepDao. - - * Step creation handled by factories. - - -* VARIOUS - - * ItemStream now has single 'open(ExecutionContext)' rather than 'open()' and 'restoreFrom(ExecutionContext)'. - - * ItemStream no longer has mark() and reset() methods - moved to ItemReader. ItemWriter uses more intuitive clear() and flush(). - - * Interceptors renamed to Listeners e.g. RepeatInterceptor is now RepeatListener. - - * StepContext merged into ExecutionContext (previously ExecutionAttributes). Single ExecutionContext is now shared between ItemStreams instead of separate contexts being composed before save and decomposed after load. ItemStreams now have names to distinguish context keys of multiple ItemStreams of the same class. - - * Domain-oriented listeners introduced - JobListener, StepListener, ChunkListener, ItemReadListener, ItemWriteListener. - - * StepInstance removed from domain model. - - * Database schema updated. - - * JobInstance identity is now given by Job and JobParameters pair - identical JobInstances with different IDs no longer allowed (see JobInstance javadoc for more details). - diff --git a/src/site/apt/migration/1.0-m5-rc1.apt b/src/site/apt/migration/1.0-m5-rc1.apt deleted file mode 100644 index 04cdd571b6..0000000000 --- a/src/site/apt/migration/1.0-m5-rc1.apt +++ /dev/null @@ -1,35 +0,0 @@ - ------ - Spring Batch Upgrade - ------ - Robert Kasanicky - ------ - March 2008 - -Updating Spring Batch jobs from 1.0.0.m5 to 1.0.0.rc1 framework - - The following is a description of what is needed to migrate the samples from m5 to the rc1 release. The changes to existing job configurations in samples are minimal. Simply replacing the old version of the base launcher configuration should suffice. - - -* CONTAINER CONFIGURATION - - * The basic execution environment for the samples was renamed from "simple-container-definition.xml" to "simple-job-launcher-context.xml". - - * DefaultStepFactoryBean has been removed, SimpleStepFactoryBean offers basic configuration options. - - * SkipLimitStepFactoryBean adds additional skip configuration for specific exception types. - - -* VARIOUS - - * Core and Execution modules merged to Core - - * Heavy package renaming and moving (resolved cyclic dependencies) - - * all StepListener methods pass the StepExecution as argument - - * all JobListener methods pass the JobExecution as argument - - * JobListener was enhanced with onError(..) and onInterrupt(..) methods - - * InfrastructureException replaced with UnexpectedJobExecutionException - diff --git a/src/site/apt/migration/1.0-rc1-final.apt b/src/site/apt/migration/1.0-rc1-final.apt deleted file mode 100644 index 2286ed3b83..0000000000 --- a/src/site/apt/migration/1.0-rc1-final.apt +++ /dev/null @@ -1,26 +0,0 @@ - ------ - Spring Batch Upgrade - ------ - Robert Kasanicky - ------ - March 2008 - -Updating Spring Batch jobs from 1.0.0.rc1 to 1.0.0.final framework - - Migrating from rc1 to final release does not require configuration changes. Following is a list of notable imporovements and/or changes in behavior. - - * SimpleStepFactoryBean defaults to commit interval = 1 - - * SkipLimitStepFactoryBean accepts a list of fatal exceptions (java.lang.Error by default) that cause immediate step failure, regardless of skippable settings. - - * Skip and retry are no longer exclusive. - - * An exception on read does not cause transaction rollback if the exception is skippable. - - * Checked skippable exceptions are skipped correctly - - * FlatFileItemWriter buffers output and writes to file only on flush() i.e. at the end of chunk. - - * ExitStatusExceptionClassifier interface removed - SimpleExitStatusExceptionClassifier implements JobListener instead. - - * ExecutionContext is persisted for the first time before the processing of first chunk starts, so the scenario when job fails before first chunk is commited is no longer special. diff --git a/src/site/apt/migration/1.0.0-1.0.1.apt b/src/site/apt/migration/1.0.0-1.0.1.apt deleted file mode 100644 index a09b7efa4c..0000000000 --- a/src/site/apt/migration/1.0.0-1.0.1.apt +++ /dev/null @@ -1,49 +0,0 @@ -Changes in version 1.0.1 (2008-04-25) - -* Bug fixes (changes made, loosely ordered by importance) - - * StepExecutionListener#afterStep(..) is called only after successful processing, onErrorInStep(..) handles failures - - * Throwing exception in StepExecutionListener#afterStep(..) causes step to fail - - * StatefulRetryStepFactoryBean honors skip configuration (ignored it before) - - * More robust skip and retry logic with informative failures - - * JdbcCursorItemReader handles resets (rollbacks) correctly - - * JdbcCursorItemReader handles multiple restarts correctly - - * Long values mapped to JDBC Types.BIGINT instead of Types.INTEGER to avoid value truncating - - * StepExecutionResourceProxy now works correctly with FlatFileItemWriter - - * JobRepositoryBean applies table prefix consistently (including sequences) - - * HibernateCursorItemReader closes stateful session correctly - - * RetryTemplate rethrows Throwables that are neither Exception nor Error instead of ignoring - - * StepExecution#itemCount value was off by one - - * TaskletAdapter returns ExitStatus.FINISHED by default instead of ExitStatus.CONTINUABLE - - * JdbcCursorItemReader works with Derby under condition verifyCurorPosition=false - - * MySQL schema uses DATETIME instead of problematic TIMESTAMP - -* Improvements - - * Cleaned up javadocs and documentation - - * StepExecutionResourceProxy#toString delegates to the proxied Resource (once it is set) - - * More informative exception messages in SpringValidator (the invalid item is included) - - * ExecutionContext and JobParameters support default values - - * Oracle schema uses more appropriate datatypes - - * Subclasses of SimpleStepFactoryBean have access to ItemOrientedStep#chunkOperations property - - * Reusable tests for item readers, repository daos and step implementations (if you need to implement any of these) diff --git a/src/site/apt/migration/1.0.1-1.1.apt b/src/site/apt/migration/1.0.1-1.1.apt deleted file mode 100644 index d122a9bfd3..0000000000 --- a/src/site/apt/migration/1.0.1-1.1.apt +++ /dev/null @@ -1,42 +0,0 @@ -Spring Batch 1.1 Release Notes - -* Notable new features: - - * Job-level execution context (shared among all steps) - - * MultiResourceItemReader for reading multi-file input - - * SystemCommandTasklet for executing system commands - - * FlatFileItemWriter and StaxEventItemWriter support headers - - * Maven archetype - - * Modifiable transaction attributes for ItemOrientedStep - - * StepExecutionSimpleCompletionPolicy for pulling commit interval from JobParameters - - * MapJobRepositoryFactoryBean for easy setup of in-memory repository - - * StepExecution has persistent rollbackCount, readSkipCount and writeSkipCount - - * All ItemReader implementations default to returning null in the initial call to read if no data is found to process. NoWorkFoundListener can be used to fail a step if no work was done. (e.g. input was empty) - - * LineTokenizer implementations now properly enforce formatting. For example, a fixed length line must match the maximum line length defined in the provided ranges. - - * FlatFileItemWriter must be opened before it can be written to. However, open can be called multiple times before closing. - - * All ItemReader and ItemWriter implementations now default to saving state. (i.e. saveState=true by default) - -* Other - - * StatefulRetryStepFactoryBean is deprecated - use SkipLimitStepFactoryBean instead - - -* Upgrade instructions from 1.0.1 to 1.1 - - * update the database schema using the update script (or create a fresh schema) - note that 1.0.x jobs won't work with 1.1 schema - - * use the updated simple-job-launcher-context.xml (or remove TX advice for repository - it is applied automatically, and remove beans 'jobRegistry' and 'jobRegistryBeanPostProcessor') - - * jobs should run without modification diff --git a/src/site/apt/migration/1.1-1.1.1.apt b/src/site/apt/migration/1.1-1.1.1.apt deleted file mode 100644 index 13ded39771..0000000000 --- a/src/site/apt/migration/1.1-1.1.1.apt +++ /dev/null @@ -1,27 +0,0 @@ -Spring Batch 1.1.1 Release Notes - -* Bugfixes: - - * FlatFileItemReader restart broken for non-default RecordSeparatorPolicy (record > 1 line) - - * JdbcCursorItemReader fails to restart after rollback on first chunk - - * Throwing an Exception from a SkipListener caused unpredictable behavior - - * StepExecution is saved before trying to save ExecutionContext - - * DefaultFieldSet should clone the tokens before exposing them in getValues() - - -* Improvements - - * Configurable RetryPolicy in SkipLimitStepFactoryBean - - -* Documentation updates - - * Updated 'Getting Started' page - - * Fixed missing images from the html reference documentation - - * Added description how to access JobExecution and StepExecution from Tasklets \ No newline at end of file diff --git a/src/site/apt/migration/1.1-2.0-m1.apt b/src/site/apt/migration/1.1-2.0-m1.apt deleted file mode 100644 index d8238e578a..0000000000 --- a/src/site/apt/migration/1.1-2.0-m1.apt +++ /dev/null @@ -1,16 +0,0 @@ -Spring Batch 2.0-m1 Release Notes - -* Bugfixes: - - * [BATCH-737] - JdbcCursorItemReader will spin through entire resultset if numberOfProcessRows=0 - * [BATCH-732] - FlatFileItemReader doesn't take "firstLineIsHeader" flag into account when restarting * [BATCH-735] - Getting Started page should be updated * [BATCH-734] - ItemReaders and ItemWriters using Resource(s) should check for file during ItemStream#open * [BATCH-778] - MapJobRepositoryFactoryBean shouldn't require transactionManager * [BATCH-693] - Refactor samples along domain contours * [BATCH-761] - StaxEventItemWriter writes extra end document tag with Woodstox 3.2.6 * [BATCH-765] - StepExecution should be saved on every commit - * [BATCH-757] - remove DelegatingItemWriter * [BATCH-766] - Insufficient error handling in case of a missing resource for a org.springframework.batch.item.xml.StaxEventItemWriter - * [BATCH-727] - Move all POMs in trunk to 2.0.0.CI-SNAPSHOT * [BATCH-744] - restart.count is always 0 in FlatFileItemWriter * [BATCH-741] - DefaultFieldSet should clone the tokens before exposing them in getValues() - -* Improvements - - * [BATCH-671] - Upgrade JobParameters and ExecutionContext for Java 5 * [BATCH-770] - Make ItemTransformer a first class citizen and rename as ItemProcessor * [BATCH-278] - Allow FixedLengthLineAggregator to be configured with different padding/alignment for specific columns * [BATCH-709] - Change all collections to use generics * [BATCH-581] - Add filter capability to item oriented paradigm * [BATCH-743] - ExportedJobLauncher should be able to stop running batches on an individual basis * [BATCH-230] - Recoverable and ItemProvider.recover() abstractions refactored into correct place in chunk-oriented framework * [BATCH-745] - strong typing in AggregateItemReader * [BATCH-544] - Configurable RetryPolicy in SkipLimitStepFactoryBean * [BATCH-354] - Create JPA ItemReader/Writer * [BATCH-758] - Generify ExceptionClassifier and ExceptionClassifierSupport * [BATCH-300] - Improve typesafety of ItemProvider/Processor paradigm * [BATCH-753] - Listener exception handling * [BATCH-274] - Support Callable * [BATCH-712] - Upgrade ItemReaders to use Parameterized types * [BATCH-711] - Upgrade ItemWriter and implementations to use parameterized types * [BATCH-756] - Upgrade JdbcExecutionContextDao to use enums * [BATCH-672] - Upgrade samples to Java 5.0 * [BATCH-710] - Upgrade unit tests to use Spring Test Java 5 features * [BATCH-518] - clean up the *Or* repository methods * [BATCH-662] - simplify FlatFileItemReader * [BATCH-728] - Add remove(key) method to ExecutionContext class * [BATCH-570] - Job.getSteps() does not need to be exposed in the interface * [BATCH-668] - JobRepository needs to distinguish between save and update of execution context - -* Documentation updates - - * [BATCH-768] - Update documentation for M1 * [BATCH-586] - Documentation for the walkthrough of samples * [BATCH-769] - Create documentation for JPA Reader * [BATCH-689] - Document how to access StepExecution and JobExecution in Tasklet \ No newline at end of file diff --git a/src/site/apt/migration/1.1.1-1.1.2.apt b/src/site/apt/migration/1.1.1-1.1.2.apt deleted file mode 100644 index 9d7e2ef823..0000000000 --- a/src/site/apt/migration/1.1.1-1.1.2.apt +++ /dev/null @@ -1,16 +0,0 @@ -Spring Batch 1.1.2 Release Notes - -* Bugfixes: - - * STEP_EXECUTION table is now updated after every chunk - - * StaxEventItemWriter throws informative error for non-existing Resource, and only does it on open (so it will work with MultResourceItemReader) - - * In a multi-threaded step, the lock that is taken while the step executio nis being updated cannot be inadvertently released by another thread if it fails - - -* Improvements - - * StaxEventItemWriter#endDocument(..) can be overriden (allows users to fix problems with Woodstox end tag auto-insertions) - - * Javadoc included in the distribution diff --git a/src/site/apt/migration/1.1.2-1.1.3.apt b/src/site/apt/migration/1.1.2-1.1.3.apt deleted file mode 100644 index c0e79d1907..0000000000 --- a/src/site/apt/migration/1.1.2-1.1.3.apt +++ /dev/null @@ -1,56 +0,0 @@ -Release Notes - Spring Batch - Version 1.1.3 - - -* Bug Fixes - - * {{{http://jira.springframework.org/browse/BATCH-725}BATCH-725}} - The description of DelegatingItemReader in Appendix A( List of ItemReaders) - - * {{{http://jira.springframework.org/browse/BATCH-811}BATCH-811}} - StepExecutionResourceProxy should throw an exception if a job paramter key in the path isn't found - - * {{{http://jira.springframework.org/browse/BATCH-812}BATCH-812}} - StepExecutionResourceProxy should use a different JobParametersConverter - - * {{{http://jira.springframework.org/browse/BATCH-820}BATCH-820}} - Storing a Blob with JdbcExecutionContextDao in DB2 - - * {{{http://jira.springframework.org/browse/BATCH-831}BATCH-831}} - id counter in MapJobInstanceDao should be declared static - - * {{{http://jira.springframework.org/browse/BATCH-832}BATCH-832}} - DefaultTransactionAttribute is inappropriate for TaskletStep - - * {{{http://jira.springframework.org/browse/BATCH-833}BATCH-833}} - TransactionAttributes swallows Exceptions - - * {{{http://jira.springframework.org/browse/BATCH-838}BATCH-838}} - TimeoutTerminationPolicy does not terminate when eof is encountered - - * {{{http://jira.springframework.org/browse/BATCH-839}BATCH-839}} - Reference doc reference deprecated code: 'StatefulRetryStepFactoryBean' - - * {{{http://jira.springframework.org/browse/BATCH-847}BATCH-847}} - FaultTolerantChunkOrientedTasklet loses chunks when non-skippable exceptions thrown in read phase - - * {{{http://jira.springframework.org/browse/BATCH-857}BATCH-857}} - map daos need to be truly transactional for correct restart - - * {{{http://jira.springframework.org/browse/BATCH-866}BATCH-866}} - Reference manual has invalid references to org.springframework.batch.io.file package - - * {{{http://jira.springframework.org/browse/BATCH-870}BATCH-870}} - Cannot add description to empty ExitStatus - - * {{{http://jira.springframework.org/browse/BATCH-884}BATCH-884}} - SkipLimitStep does not fail if exception occurs - - * {{{http://jira.springframework.org/browse/BATCH-892}BATCH-892}} - Thread visibility issues in repeat template - - -* Improvement - - * {{{http://jira.springframework.org/browse/BATCH-802}BATCH-802}} - Incorrect metadata in fixed-length file example - - * {{{http://jira.springframework.org/browse/BATCH-809}BATCH-809}} - fixedLenghtTokenizer problem with 1.1.2 - - * {{{http://jira.springframework.org/browse/BATCH-828}BATCH-828}} - org.springframework.batch.item.file.MultiResourceItemReader should allow for no resources - - * {{{http://jira.springframework.org/browse/BATCH-841}BATCH-841}} - insufficient logging in AbstractStep - - * {{{http://jira.springframework.org/browse/BATCH-844}BATCH-844}} - Lacking of DAO's property setting in JobRepositoryFactoryBean - - * {{{http://jira.springframework.org/browse/BATCH-881}BATCH-881}} - Update meta data schema to limit primary key values to Java Long sizes - - * {{{http://jira.springframework.org/browse/BATCH-885}BATCH-885}} - Reference Doc Appendix B. Meta-Data Schema outdated - - -* Refactoring - - * {{{http://jira.springframework.org/browse/BATCH-810}BATCH-810}} - Need to add DB2MainframeSequenceMaxValueIncrementer to DefaultDataFieldMaxValueIncrementerFactory diff --git a/src/site/apt/migration/2.0-highlights.apt b/src/site/apt/migration/2.0-highlights.apt deleted file mode 100644 index 60399cf144..0000000000 --- a/src/site/apt/migration/2.0-highlights.apt +++ /dev/null @@ -1,165 +0,0 @@ - ------ - Spring Batch 2.0 Highlights - ------ - Dave Syer - ------ - January 2009 - -Spring Batch 2.0 Highlights - - Here we outline the main themes of Spring Batch 2.0, and highlight the changes from 1.x. - -* Spring Batch 2.0 Themes - - The four main themes of the new release are - - * Java 5 (mandatory) and Spring 3.0 (optional) - - * Non-sequential execution - - * Scalability - - * Configuration: annotations and XML namespace - - so we'll cover each of those areas separately and describe what they mean and the impact of the changes on Spring Batch existing users. There is more detail below for features that are already implemented, which is mostly in the first category with some enabling features in other areas. - - There are no changes to the physical layout of the project in Spring Batch 2.0.0.M2 (same old downloads, same basic layout of Java packages). We have not removed any features, but we have taken the opportunity to revise a couple of APIs, and there are some minor changes for people updating projects from 1.x. Spring Batch is immature enough and we were adding some pretty big features, so we decided a major version change was a good opportunity to have a bit of a clean out. We don't expect anyone to have any difficulty upgrading, and if you are an existing user this page will help you to get the measure of the changes. - -* Java 5 - - As you may know, Spring 3.0 is going to be the first major release of Spring to target Java 5 exclusively (I'll leave it to Juergen and Arjen to clarify that in more detail). Now that Sun has put an "{{{http://java.dzone.com/articles/rip%E2%80%A6jdk-14}End of Service Life}}" stamp on the JDK 1.4 it seems appropriate, and there are some great new features on Spring 3.0 that we want to take advantage of. - - Since Spring 3.0 took a little longer to release than we had initially hoped we did some work in 2.0.0 to provide the late binding features independent of Spring 3.0. The 2.0.0 and 2.0.1 releases of Spring Batch actually do not work with Spring 3.0, but 2.0.2 should. The expressions in step scoped beans will use the weaker, but very similar, expression language from Spring Batch instead of the full Spring EL. In a future release of Spring Batch we will be able to fully support Spring EL in late binding expressions. - -** Type Safety - - Most of the work in the 2.0.0.M1 release of Spring Batch went into converting the existing code to Java 5, taking advantage of generics and parameterised types wherever we could. This gives users of the framework a much nicer programming experience, allowing compile time checks for type safety and ultimately reducing maintenance costs for projects using Spring Batch. For instance the in the <<>>, one of the central interfaces and user extension points in Spring Batch, we now have a typesafe <<>> method: - -+--- -public interface ItemReader { - S read(); -} -+--- - - A further point to note here is that the old (1.x) framework callbacks <<>> and <<>> have gone from this interface, making it more friendly to end users, and preventing misunderstanding about what the framework requires and when. The same concerns (or mark and reset) are now handled internally in the <<>> implementations that the framework provides. - -** Chunk-oriented Processing - - Similar changes, and one slightly more radical, have also occurred in the partner <<>> interface, used by the framework for writing data: - -+--- -public interface ItemWriter { - void write(List items); -} -+--- - - The old framework callbacks for <<>> and <<>> have gone from this interface too, and to compensate for that, the <<>> method has a new signature. The bottom line here is that we have moved to a chunk-oriented processing paradigm internally to the framework. This is actually much more natural in a batch framework than the old item-oriented approach because for performance reasons we often need to buffer and flush, and the old interface made that awkward for users. Now you can do all the batching you need inside the <<>> method. - -** Step Factory Bean Changes - - A side effect of the chunk-oriented approach to processing is a change in the step factory bean implementations. The old <<>> has been renamed to <<>>, and it can still be used for most common use cases as a replacement for the old factory bean. By default it creates a step implementation that buffers input items across rollbacks, so that the new <<>> and <<>> interfaces work properly with non-transactional input sources (like files). For input sources that re-present the items after a rollback, clients have to set a flag in the factory bean (<<>> is the name of the flag as of M2) - <<>> is the only relevant reader in the framework and not many projects use it so this isn't a big change. - -** Business Processing - - A related new feature is that there is a new kid on the block in the form of the <<>>: - -+---- -public interface ItemProcessor { - T process(S item); -} -+---- - - In 1.x the transformation between input items of type <<>> and output items of type <<>> had to be hidden inside one of the other participants (usually the <<>>). Now we have genericised this concern and placed it at the same level of importance in the framework as its siblings, <<>> and <<>>. Users of 1.x might recognise the traces of the old <<>> interface here, which has now been removed. - -** A More Useful Tasklet Interface - - Many people, looking at Spring Batch 1.x, have asked "what if my business logic is not reading and writing?" To answer this question more satisfactorily we have modified the <<>> interface. In Spring Batch 1.x it is fairly bland - little more than a <<>> in fact, but in 2.0 we have given it some more flexibility and slotted it more into the mainstream of the framework (e.g. the central chunk-oriented step implementation is now implemented as a <<>>). Here is the new interface: - -+---- -public interface Tasklet { - ExitStatus execute(StepContribution contribution, - AttributeAccessor attributes); -} -+---- - - The idea is that the tasklet can now contribute more back to the enclosing step, and this makes it a much more flexible platform for implementing business logic. The <<>> was already part of the 1.x API, but it wasn't very publicly exposed. Its role is to collect updates to the current <<>> without the programmer having to worry about concurrent modifications in another thread. This also tells us that the <<>> will be called repeatedly (instead of just once per step in the 1.x framework), and so it can be used to carry out a greater range of business processing tasks. The <<>> is a chunk-scoped bag of key-value pairs. The tasklet can use this to store intermediate results that will be preserved across a rollback. - -** Late Binding of Job and Step Attributes - - Late binding of <<>> and <<>> attributes to step components is possible in a very generic way. This is a much requested feature, and we have some workarounds for special cases in 1.x, like the <<>> for binding to a filename as an input parameter to a job. A more generic solution available in Spring Batch 2.0 is to allow those step attributes to be bound to arbitrary components, by defining them in an appropriate Spring scope, e.g. - -+---- - - - - - ... - - - ... - -+---- - - The item reader needs to be bound to a file name that is only available at runtime. To do this we have declared it as scope="step" and used the Spring EL binding pattern <<<#{...}>>> to bind in a job parameter. The same pattern works with step and job level execution context attributes (binding at step execution time). - - Step scoped beans are also a good solution to the old Spring Batch problem of how to keep your steps thread safe. If a step relies on a stateful component like a <<>>, then it only has to define that component as scope="step" and the framework creates a lazy initializing proxy for it, and it will be created as needed once per step execution. There's still noting wrong with creating a new <<>> for each job execution, which is the current best practice in 1.x for keeping threads from colliding across jobs. But now step scope gives you another option, and one that is probably easier for most Spring users to get to grips with. - -* Spring 3.0 - - Spring Batch 2.0 depends on Spring 2.5.6 (the lastest stable version at the time the development of Spring Batch was in progress). The plan is to add a Spring 3.0 dependence for Spring Batch 2.1 (while keeping the option of 2.5.6). This provides some useful new features in particular in the configuration of jobs and steps using late binding with Spring Expression Language (EL), which has the same syntax as we are using for late binding in Spring Batch 2.0, but has more features and is rather more flexible. - -* Non-sequential Execution - - In 1.x the model of a job was always as a linear sequence of steps, and if one step failed, then the job failed. Although many jobs still fit that pattern so it hasn't gone away, in 2.0 we are lifting that restriction by introducing some new features. These are planned for the M3 release so the implementation details might change, but the idea is to support three features: - - * Conditional execution: branching to a different step based on the <<>> of the last one. This includes the ability to branch on a FAILED status, which implies that a step failure is no longer fatal for a job. - - * Pause execution and wait for explicit instruction to proceed. This is useful for instance where there is a business rule that forces manual intervention to check the validity of business critical data. - - * Parallel execution of multiple steps. Where steps are independent the user can specify which branches can be executed in parallel. - - These features are available through the custom XML namespace described briefly below. - -* Scalability - - Spring Batch 1.x was always intended as a single VM, possibly multi-threaded model, but we built a lot of features into it that support parallel execution in multiple processes. Many projects have successfully implemented a scalable solution relying on the quality of service features of Spring Batch to ensure that processing only happens in the correct sequence. In 2.0 we expose those features more explicitly. There are two approaches to scalability, and we support both: remote chunking, and partitioning. - -** Remote Chunking - - Remote chunking is a technique for dividing up the work of a step without any explicit knowledge of the structure of the data. Any input source can be split up dynamically by reading it in a single process (as per normal in 1.x) and sending the items as a chunk to a remote worker process. The remote process implements a listener pattern, responding to the request, processing the data and sending an asynchronous reply. The transport for the request and reply has to be durable with guaranteed delivery and a single consumer, and those features are readily available with any JMS implementation. But Spring Batch is building the remote chunking feature on top of {{{http://www.springframework.org/spring-integration}Spring Integration}}, so actually it is agnostic to the actual implementation of the message middleware. - -** Partitioning - - Partitioning is an alternative approach which in contrast depends on having some knowledge of the structure of the input data, like a range of primary keys, or the name of a file to process. The advantage of this model is that the processors of each element in a partition can act as if they are a single step in a normal Spring Batch job. They don't have to implement any special or new patterns, which makes them easy to configure and test. Partitioning in principle is more scalable than remote chunking because there is no serialization bottleneck arising from reading all the input data in one place. - - In Spring Batch 2.0 partitioning is supported by two interfaces: <<>> and <<>>. The <<>> is the one that knows about the execution fabric - it has to transmit requests to remote steps and collect the results using whatever grid or remoting technology is available. <<>> is an SPI, and we provide one implementation out of the box for local execution through a <<>>. This will be useful immediately to a number of projects we have seen where parallel processing of heavily IO bound tasks is required, since in those cases remote execution only complicates the deployment and doesn't necessarily help much with the performance. Other implementations will be specific to the execution fabric, e.g. one of the grid providers (IBM, Oracle, Terracotta, Appistry etc.), and we don't want to imply a preference for any of those over the others in Spring Batch. - - SpringSource is planning an Enterprise Batch product that will provide a full runtime solution for partitioning and remote chunking, as well as admin and scheduling concerns. - -* Configuration: annotations and XML namespace - - The idea behind using annotations to implement batch logic is by analogy with Spring @MVC. The net effect is that instead of having to implement and possibly register a bunch of interfaces (reader, writer, processor, listeners, etc.) you would just annotate a POJO and plug it into a step. There are method level annotations corresponding to the various interfaces, and parameter level annotations and factory methods corresponding to job, step and chunk level attributes (a bit like <<<@ModelParameter>>> and <<<@RequestParameter>>> in Spring @MVC. - - A XML namespace for Spring Batch makes configuration of common things even easier. For example, the <<>> we mentioned above has an XML configuration option, so a simple conditional execution might look like this: - -+---- - - - - - - - - - -+---- - - The first step ("gamesLoad") is followed immediately by "playerLoad", but if that fails, then we can go to an alternative step ("compensate") instead of finishing the job normally with the "summarize" step. The name= attribute of the <<<<step>>> element in the XML is a bean reference, so the implementation of <<>> is defined elsewhere (possibly through annotations). - -* Database Schema Changes - - There are a couple of tidying up tasks and extensions to the data model in the meta data schema. Unfortunately we cannot provide update scripts for moving from 1.x to 2.0 because the differences cannot be bridged using plain SQL. Upgrading an existing database would require the old schema to be dumped and re-loaded with a custom job. Our recommendation is to leave the old data alone and start fresh with 2.0: the benefit is that the new APIs definitely make it easier to navigate and interact with the meta data. For those of you who are new to Spring Batch, some of the key benefits of the framework are the quality of service features like restartability and idempotence (process business data once and only once). We implement these features through shared state in a relational database (in most use cases), and the definition of the data model in this database has changed slightly in 2.0. - - The main changes are to do with the storage of <<>>, which used to be centralised in one table, even though the context can be associated either with a <<>> or a <<>>. The new model will be more popular with DBAs because it makes the relationships more transparent in the DDL. We also started storing the context values in JSON, to make them easier to read and track for human users (the context for a single entity is all stored in one row in a table, instead of many). - - We have also added some more statistics for the counting and accounting of items executed and skipped, splitting out counts for total items read, processed and written at each stage. For steps (or tasklets) that do not split their execution into read, process, write, this is more comprehensive than is needed, but for the majority use case it is more appropriate than just storing an overall item count. diff --git a/src/site/apt/migration/2.0-m1-m2.apt b/src/site/apt/migration/2.0-m1-m2.apt deleted file mode 100644 index dd50527cf8..0000000000 --- a/src/site/apt/migration/2.0-m1-m2.apt +++ /dev/null @@ -1,76 +0,0 @@ -Spring Batch 2.0-m2 Release Notes - - - -* Bugfixes: - - * [BATCH-682] - Use SoftReference and/or expiry to store entries in RetryContextCache implementation(s) - * [BATCH-764] - Truncate flatfile in restart scenarios. - * [BATCH-776] - StaxEventItemWriter headers should not have to be the same type as the T that the writer is parameterised with - * [BATCH-782] - Synchronization issue in ItemOrientedStep if exception is throw in chunk processing - * [BATCH-790] - SQL error caused by changes to JdbcExecutionContextDao - * [BATCH-795] - JdbcJobExecutionDao output sorting - * [BATCH-807] - Serialization bug in JobExecution - * [BATCH-812] - StepExecutionResourceProxy should use a different JobParametersConverter - * [BATCH-813] - SingleColumnJdbcKeyGeneratorIntegrationTests fails in Eclipse - * [BATCH-814] - JobRepository should not require Step or Job (only their names) - * [BATCH-822] - New WRITE_COUNT should count items instead of chunks - * [BATCH-830] - DelegatingItemReader should be removed - * [BATCH-831] - id counter in MapJobInstanceDao should be declared static - * [BATCH-832] - DefaultTransactionAttribute is inappropriate for TaskletStep - * [BATCH-833] - TransactionAttributes swallows Exceptions - * [BATCH-835] - sql error "column ambiguously defined" in JdbcJobInstanceDao - * [BATCH-838] - TimeoutTerminationPolicy does not terminate when eof is encountered - * [BATCH-839] - Reference doc reference deprecated code: 'StatefulRetryStepFactoryBean' - * [BATCH-843] - FlatFileItemWriter handling of failure in LineAggregator - * [BATCH-847] - FaultTolerantChunkOrientedTasklet loses chunks when non-skippable exceptions thrown in read phase - * [BATCH-856] - JobExecution status isn't being updated when Step fails. - * [BATCH-865] - CompositeItemProcessor should handle null properly - -* Improvements - - * [BATCH-401] - aspect-oriented handling of job interruption logic - * [BATCH-453] - Killed batches cannot be restarted - * [BATCH-592] - RFC: buffer read items in ItemHandler instead of ItemReader - * [BATCH-640] - FieldSetMapper.mapLine() should contain the line number - * [BATCH-719] - API should provide access to original input line in flat file outside of error scenario - * [BATCH-755] - Include Javadocs in distribution - * [BATCH-762] - Change ExecutionContext storage strategy - * [BATCH-763] - Create callback for header and footer writing in xml and flat files - * [BATCH-772] - Upgrade CommandLineJobRunner to use args4j - * [BATCH-775] - Throw exception if ExecutionContext is modified during processing. - * [BATCH-779] - RFC: make FieldSetMapper and LineAggregator extend ItemProcessor - * [BATCH-785] - Fix Parallel Sample to use a batch update for process indicator - * [BATCH-792] - Automatically detect database type in JobRepositoryFactoryBean - * [BATCH-796] - typos in javadoc @links and comments - * [BATCH-802] - Incorrect metadata in fixed-length file example - * [BATCH-825] - Modify JobLauncher contract to not throw exception on job failure. - * [BATCH-826] - Create callback for header and footer reading in xml and flat files - * [BATCH-828] - org.springframework.batch.item.file.MultiResourceItemReader should allow for no resources - * [BATCH-841] - insufficient logging in AbstractStep - * [BATCH-844] - Lacking of DAO's property setting in JobRepositoryFactoryBean - * [BATCH-863] - introduce LineMapper interface to encapsulate string-to-item mapping - * [BATCH-864] - rename SkipLimitStepFactoryBean - * [BATCH-774] - Split item count into read/write/filter - * [BATCH-829] - add persistent processSkipCount to StepExecution - * [BATCH-810] - Need to add DB2MainframeSequenceMaxValueIncrementer to DefaultDataFieldMaxValueIncrementerFactory - * [BATCH-8] - Determine approach for handling file creation/update in restart scenarios - * [BATCH-220] - Chunk-oriented approach to processing - * [BATCH-575] - FlatFileItemReader default charset should mimic java's default charset behavior - * [BATCH-675] - API Analysis - * [BATCH-759] - ItemReader refactoring - * [BATCH-773] - Refactor and extend ExportedJobLauncher to JobOperator - * [BATCH-806] - Resurrect ResourceLineReader - * [BATCH-821] - remove EventSerializer/Deserializer abstractions and use Marshaller/Unmarshaller directly - * [BATCH-827] - Create Sample job for reading and writing headers and footers - * [BATCH-845] - integration tests for database autodetection - - - -* Documentation updates - - * [BATCH-767] - Update documentation for M2 - * [BATCH-823] - Confusing explanation - * [BATCH-718] - Add section on FlatFileItemReader error handling - * [BATCH-784] - Refactor testing chapter to reflect Spring Testing framework changes. - * [BATCH-850] - Create documentation for JdbcPagingItemReader and related classes diff --git a/src/site/apt/migration/2.0-m2-m3.apt b/src/site/apt/migration/2.0-m2-m3.apt deleted file mode 100644 index 00cd8812cb..0000000000 --- a/src/site/apt/migration/2.0-m2-m3.apt +++ /dev/null @@ -1 +0,0 @@ -Spring Batch 2.0-m3 Release Notes * Bug * {{{http://jira.springframework.org/browse/BATCH-837}BATCH-837}} - BIGINT datatype is not in Sybase but schema-sybase.sql is using it * {{{http://jira.springframework.org/browse/BATCH-853}BATCH-853}} - broken transactional item processing * {{{http://jira.springframework.org/browse/BATCH-857}BATCH-857}} - map daos need to be truly transactional for correct restart * {{{http://jira.springframework.org/browse/BATCH-887}BATCH-887}} - onSkipInProcess called multiple times for same item using FaultTolerantChunkOrientedTasklet * {{{http://jira.springframework.org/browse/BATCH-889}BATCH-889}} - SkipLimitStepFactoryBean should be FaultTolerantStepFactoryBean * {{{http://jira.springframework.org/browse/BATCH-892}BATCH-892}} - Thread visibility issues in repeat template * {{{http://jira.springframework.org/browse/BATCH-895}BATCH-895}} - Fix XML schema so that only top-level elements are allowed at top level (i.e. job). * {{{http://jira.springframework.org/browse/BATCH-897}BATCH-897}} - Version is not rehydrated from database in JobInstance or JobExecution * {{{http://jira.springframework.org/browse/BATCH-912}BATCH-912}} - Thread safety issue in JobRegistryBackgroundJobRunner * {{{http://jira.springframework.org/browse/BATCH-917}BATCH-917}} - Concurrent modification of execution context in tasklet step * {{{http://jira.springframework.org/browse/BATCH-921}BATCH-921}} - ExecutionContext keys are not unique enough in partition components * {{{http://jira.springframework.org/browse/BATCH-925}BATCH-925}} - StaxEventItemWriter doesn't reset restart flag on close * {{{http://jira.springframework.org/browse/BATCH-926}BATCH-926}} - vague skip limit for concurrent chunks * {{{http://jira.springframework.org/browse/BATCH-931}BATCH-931}} - Write failures don't fail immediately. * {{{http://jira.springframework.org/browse/BATCH-939}BATCH-939}} - Make step scope work with aop-auto-proxy * Improvement * {{{http://jira.springframework.org/browse/BATCH-21}BATCH-21}} - OutputResource abstraction for file / stream output * {{{http://jira.springframework.org/browse/BATCH-63}BATCH-63}} - Custom namespace for Job and related (Step, and maybe some *Reader/Writer) * {{{http://jira.springframework.org/browse/BATCH-282}BATCH-282}} - Make input parameters easier to access from ItemReaders, etc. * {{{http://jira.springframework.org/browse/BATCH-341}BATCH-341}} - Parameter generation strategy * {{{http://jira.springframework.org/browse/BATCH-783}BATCH-783}} - No logging for item oriented step in case of exception occured in chunk processing * {{{http://jira.springframework.org/browse/BATCH-797}BATCH-797}} - Refactor Map daos into another repository implementation. * {{{http://jira.springframework.org/browse/BATCH-809}BATCH-809}} - fixedLenghtTokenizer problem with 1.1.2 * {{{http://jira.springframework.org/browse/BATCH-848}BATCH-848}} - Determine best way tto support a paging iBATIS reader * {{{http://jira.springframework.org/browse/BATCH-869}BATCH-869}} - End Time of a step or a job always null when read in a StepExecutionListener or a JobExecutionListener * {{{http://jira.springframework.org/browse/BATCH-872}BATCH-872}} - RFC: Should FileWriterCallback have two methods, one for headers and one for footers? * {{{http://jira.springframework.org/browse/BATCH-878}BATCH-878}} - AbstractMethodInvokingDelegator is not abstract * {{{http://jira.springframework.org/browse/BATCH-881}BATCH-881}} - Update meta data schema to limit primary key values to Java Long sizes * {{{http://jira.springframework.org/browse/BATCH-886}BATCH-886}} - RFC: consolidate JobInstance launching logic * {{{http://jira.springframework.org/browse/BATCH-888}BATCH-888}} - skip listeners should be called when chunk is about to commit * {{{http://jira.springframework.org/browse/BATCH-894}BATCH-894}} - RFC: move ExitStatus up into Core? * {{{http://jira.springframework.org/browse/BATCH-896}BATCH-896}} - "DRY" FaultTolerantTasklet implementations * {{{http://jira.springframework.org/browse/BATCH-900}BATCH-900}} - Retreive null values from the ExecutionContext * {{{http://jira.springframework.org/browse/BATCH-907}BATCH-907}} - Integer support in ExecutionContext * {{{http://jira.springframework.org/browse/BATCH-913}BATCH-913}} - Remove the JdkConcurrent* special classes * {{{http://jira.springframework.org/browse/BATCH-930}BATCH-930}} - rename ItemSkipPolicy * {{{http://jira.springframework.org/browse/BATCH-938}BATCH-938}} - Clean up DelimitedLineAggregator implementation * New Feature * {{{http://jira.springframework.org/browse/BATCH-34}BATCH-34}} - Support for multiple I/O files in a single jobRun for a particular scheduleDate. * {{{http://jira.springframework.org/browse/BATCH-679}BATCH-679}} - Non-sequential execution * {{{http://jira.springframework.org/browse/BATCH-808}BATCH-808}} - Provide named parameter support to JdbcCursorItemReader * {{{http://jira.springframework.org/browse/BATCH-874}BATCH-874}} - Annotation support * {{{http://jira.springframework.org/browse/BATCH-903}BATCH-903}} - Create test project * {{{http://jira.springframework.org/browse/BATCH-910}BATCH-910}} - Create a sample job for JobParametersIncrementer * {{{http://jira.springframework.org/browse/BATCH-927}BATCH-927}} - Add JobParametersIncrementer to job namespace element * {{{http://jira.springframework.org/browse/BATCH-932}BATCH-932}} - Add reflection-based Field Extractor * Refactoring * {{{http://jira.springframework.org/browse/BATCH-911}BATCH-911}} - Consolidate Samples * Task * {{{http://jira.springframework.org/browse/BATCH-624}BATCH-624}} - Create contribution page * {{{http://jira.springframework.org/browse/BATCH-673}BATCH-673}} - Add new Java 5.0 features * {{{http://jira.springframework.org/browse/BATCH-798}BATCH-798}} - Tidy up chunk processing and retry * {{{http://jira.springframework.org/browse/BATCH-871}BATCH-871}} - Create FactoryBean for FlatFileItemReader * {{{http://jira.springframework.org/browse/BATCH-893}BATCH-893}} - Remove the HibernateAwareItemWriter? * {{{http://jira.springframework.org/browse/BATCH-904}BATCH-904}} - Create a sample job for non sequential execution. * {{{http://jira.springframework.org/browse/BATCH-916}BATCH-916}} - JobOperator#startNewInstance should inspect last execution status * {{{http://jira.springframework.org/browse/BATCH-922}BATCH-922}} - Why is there always one more commit than seems necessary? * Sub-task * {{{http://jira.springframework.org/browse/BATCH-264}BATCH-264}} - Dependencies among jobs * {{{http://jira.springframework.org/browse/BATCH-676}BATCH-676}} - Create sample job for 'non sequential step execution' * {{{http://jira.springframework.org/browse/BATCH-726}BATCH-726}} - Pass a resource pattern to MultiResourceItemReader as a JobParameter * {{{http://jira.springframework.org/browse/BATCH-733}BATCH-733}} - Upgrade StepExecutionResourceProxy to be able to use values from job execution context. * {{{http://jira.springframework.org/browse/BATCH-801}BATCH-801}} - Add support for passing parameters from execution context * {{{http://jira.springframework.org/browse/BATCH-816}BATCH-816}} - Make sure all dependencies are part of the Enterprise Repository * {{{http://jira.springframework.org/browse/BATCH-817}BATCH-817}} - Change project names to correspond with bundle names * {{{http://jira.springframework.org/browse/BATCH-858}BATCH-858}} - Pause / resume of Job * {{{http://jira.springframework.org/browse/BATCH-875}BATCH-875}} - Pull jobRepository.save() out of Step implementations and put it in the driving Job * {{{http://jira.springframework.org/browse/BATCH-879}BATCH-879}} - Decision states: not a regular step, just a volatile decision based on information in the JobExecution. * {{{http://jira.springframework.org/browse/BATCH-880}BATCH-880}} - Suspend after step and wait for user input * {{{http://jira.springframework.org/browse/BATCH-890}BATCH-890}} - Stop transition in XML namespace * {{{http://jira.springframework.org/browse/BATCH-891}BATCH-891}} - Create Annotations * {{{http://jira.springframework.org/browse/BATCH-905}BATCH-905}} - MultiResourceItemReader should be more dynamic in nature \ No newline at end of file diff --git a/src/site/apt/migration/2.0-m3-m4.apt b/src/site/apt/migration/2.0-m3-m4.apt deleted file mode 100644 index 3b7b941ecd..0000000000 --- a/src/site/apt/migration/2.0-m3-m4.apt +++ /dev/null @@ -1,171 +0,0 @@ -Spring Batch 2.0.0.M4 Release Notes - -* Bug - - - * {{{http://jira.springframework.org/browse/BATCH-923}BATCH-923}} - Missing dependency / repository breaks build on fresh checkout - - * {{{http://jira.springframework.org/browse/BATCH-946}BATCH-946}} - NullPointerException in MapStepExecutionDao.getStepExecutions - - * {{{http://jira.springframework.org/browse/BATCH-948}BATCH-948}} - MapJobInstanceDao.getLastJobInstances ignores jobName parameter - - * {{{http://jira.springframework.org/browse/BATCH-950}BATCH-950}} - Exception during rollback hides root cause - - * {{{http://jira.springframework.org/browse/BATCH-951}BATCH-951}} - MapJobInstanceDao.getLastJobInstances doesn't return the last job instance - - * {{{http://jira.springframework.org/browse/BATCH-952}BATCH-952}} - StagingItemReader is not restartable - - * {{{http://jira.springframework.org/browse/BATCH-954}BATCH-954}} - Failure on job stop - - * {{{http://jira.springframework.org/browse/BATCH-957}BATCH-957}} - Optional dependencies included by default with new m3 pom - - * {{{http://jira.springframework.org/browse/BATCH-959}BATCH-959}} - Get rid of compiler warnings in samples - - * {{{http://jira.springframework.org/browse/BATCH-960}BATCH-960}} - Redundant test cases in samples? - - * {{{http://jira.springframework.org/browse/BATCH-963}BATCH-963}} - ExecutionContext modifications in ItemStream.close(ExecutionContext) are not persisted - - * {{{http://jira.springframework.org/browse/BATCH-969}BATCH-969}} - FlatFileItemWriters interference in CompositeItemWriter - - * {{{http://jira.springframework.org/browse/BATCH-979}BATCH-979}} - Insert Apache license header in Java sources (where missing) - - * {{{http://jira.springframework.org/browse/BATCH-991}BATCH-991}} - Getting PSQLException: ERROR: column "serialized_context" is of type bytea but expression is of type oid - - * {{{http://jira.springframework.org/browse/BATCH-994}BATCH-994}} - BackOffPolicy is not applied for exceptions that cause rollback - - * {{{http://jira.springframework.org/browse/BATCH-995}BATCH-995}} - unclear retry configuration in FaultTolerantStepFactoryBean - - * {{{http://jira.springframework.org/browse/BATCH-996}BATCH-996}} - use default retryLimit == 1 (not 0) in *StepFactoryBean - - * {{{http://jira.springframework.org/browse/BATCH-997}BATCH-997}} - EXIT_CODE and EXIT_MESSAGE out of synch for step - - * {{{http://jira.springframework.org/browse/BATCH-998}BATCH-998}} - PDF of Reference Documentation does not contain images of figures - - * {{{http://jira.springframework.org/browse/BATCH-999}BATCH-999}} - JobExecution ExecutionContext should not be persisted by Step? - - * {{{http://jira.springframework.org/browse/BATCH-1001}BATCH-1001}} - Make jobs restartable by default - - * {{{http://jira.springframework.org/browse/BATCH-1002}BATCH-1002}} - Default behavior for a Job should be failure if a step fails - - * {{{http://jira.springframework.org/browse/BATCH-1003}BATCH-1003}} - spring-batch-2.0.xsd should not allow "listeners" element on "tasklet" - - * {{{http://jira.springframework.org/browse/BATCH-1004}BATCH-1004}} - Using namespace to define a step does not store step name - - * {{{http://jira.springframework.org/browse/BATCH-1005}BATCH-1005}} - startLimit and allowStartIfComplete cannot be set in the namespace - - * {{{http://jira.springframework.org/browse/BATCH-1006}BATCH-1006}} - Namespace does not allow for variables (ie, ${varname}) - - * {{{http://jira.springframework.org/browse/BATCH-1007}BATCH-1007}} - JobRepository default is inconsistent between job and step - - * {{{http://jira.springframework.org/browse/BATCH-1021}BATCH-1021}} - AssertFile.assertFileEquals(File,File) parameters in the wrong order - - * {{{http://jira.springframework.org/browse/BATCH-1031}BATCH-1031}} - FlatFileItemReader should identify missing resource in warning - - -* Improvement - - - * {{{http://jira.springframework.org/browse/BATCH-883}BATCH-883}} - Document JobOperator.start - - * {{{http://jira.springframework.org/browse/BATCH-919}BATCH-919}} - Clean up FaultTolerant* - - * {{{http://jira.springframework.org/browse/BATCH-945}BATCH-945}} - Add support for step and job name in late binding - - * {{{http://jira.springframework.org/browse/BATCH-964}BATCH-964}} - Add fetchSize property to JdbcPagingItemReader - - * {{{http://jira.springframework.org/browse/BATCH-967}BATCH-967}} - DRY cleanup of StepParser - - * {{{http://jira.springframework.org/browse/BATCH-968}BATCH-968}} - Refactor step/simple-task/item-task elements in the core namespace - - * {{{http://jira.springframework.org/browse/BATCH-970}BATCH-970}} - JdbcCursorItemReader executes outside of main transaction - - * {{{http://jira.springframework.org/browse/BATCH-974}BATCH-974}} - Change Tasklet interface to use StepContext rather than AttributeAccessor - - * {{{http://jira.springframework.org/browse/BATCH-985}BATCH-985}} - (Skiplistener) improve doumentation and/or provide sample for saving skipped lines - - * {{{http://jira.springframework.org/browse/BATCH-988}BATCH-988}} - MethodInvokingTaskletAdapter (via AbstractMethodInvokingDelegator) only allows specification of targetObject's declared methods - - * {{{http://jira.springframework.org/browse/BATCH-993}BATCH-993}} - Slightly confusing use of *Dao in sample configuration in documentation, where *ItemWriter would be clearer - - * {{{http://jira.springframework.org/browse/BATCH-1009}BATCH-1009}} - Automatically register ItemReadListener, ItemWriteListener and ItemProcessListener - - * {{{http://jira.springframework.org/browse/BATCH-1015}BATCH-1015}} - ItemListenerSupport should implement ItemProcessListener - - * {{{http://jira.springframework.org/browse/BATCH-1020}BATCH-1020}} - Create Loop Flow Sample - - * {{{http://jira.springframework.org/browse/BATCH-1024}BATCH-1024}} - FlowJob's start state should be the first state listed in the config - - * {{{http://jira.springframework.org/browse/BATCH-1032}BATCH-1032}} - Modify AbstractJobTests in test project to be able to launch FlowJob steps individually - - -* New Feature - - - * {{{http://jira.springframework.org/browse/BATCH-677}BATCH-677}} - Partitioning enablement (SPI). - - * {{{http://jira.springframework.org/browse/BATCH-941}BATCH-941}} - Create a sample job to highlight late binding. - - * {{{http://jira.springframework.org/browse/BATCH-958}BATCH-958}} - Batch Ibatis Update Item Writer - - * {{{http://jira.springframework.org/browse/BATCH-965}BATCH-965}} - Create non-delegating ItemWriters for JPA and Hibernate - - * {{{http://jira.springframework.org/browse/BATCH-980}BATCH-980}} - Add SystemPropertyInitializer - - * {{{http://jira.springframework.org/browse/BATCH-986}BATCH-986}} - Provide factory bean for SqlPagingQueryProvider - - * {{{http://jira.springframework.org/browse/BATCH-987}BATCH-987}} - Create JobRepositoryTestUtils - - * {{{http://jira.springframework.org/browse/BATCH-989}BATCH-989}} - Add support for named queries on HibernateCursorItemReader - - * {{{http://jira.springframework.org/browse/BATCH-1016}BATCH-1016}} - Create listener to promote items from Step ExecutionContext to Job ExecutionContext - - * {{{http://jira.springframework.org/browse/BATCH-1033}BATCH-1033}} - Create a PrefixMatchingCompositeLineMapper - - -* Refactoring - - - * {{{http://jira.springframework.org/browse/BATCH-956}BATCH-956}} - Remove pause and wait functionality - - * {{{http://jira.springframework.org/browse/BATCH-962}BATCH-962}} - refactor TaskletStep's exception handling for better clarity - - * {{{http://jira.springframework.org/browse/BATCH-982}BATCH-982}} - Update all samples to use batch namespace - - * {{{http://jira.springframework.org/browse/BATCH-983}BATCH-983}} - Remove StepExecutionResourceProxy in favor of late-binding - - * {{{http://jira.springframework.org/browse/BATCH-1012}BATCH-1012}} - BatchListenerFactoryHelper: two if-statements, identical conditions - - * {{{http://jira.springframework.org/browse/BATCH-1019}BATCH-1019}} - Create one sample without the namespace - - * {{{http://jira.springframework.org/browse/BATCH-1025}BATCH-1025}} - Move PatternMatcher to infrastructure project - - * {{{http://jira.springframework.org/browse/BATCH-1026}BATCH-1026}} - Rename ExitStatus.FINISHED to ExitStatus.COMPLETED - - * {{{http://jira.springframework.org/browse/BATCH-1045}BATCH-1045}} - Move DataSourceInitializer to the test project's "main" folder - - -* Task - - - * {{{http://jira.springframework.org/browse/BATCH-674}BATCH-674}} - Upgrade reference documentation - - * {{{http://jira.springframework.org/browse/BATCH-961}BATCH-961}} - integration tests for JobOperator - - * {{{http://jira.springframework.org/browse/BATCH-971}BATCH-971}} - add database writers to iosample - - * {{{http://jira.springframework.org/browse/BATCH-1035}BATCH-1035}} - Add Late Binding to Docs - - -* Sub-task - - - * {{{http://jira.springframework.org/browse/BATCH-53}BATCH-53}} - Aggregation of execution context when partitioning - - * {{{http://jira.springframework.org/browse/BATCH-234}BATCH-234}} - Revise documentation to reflect chunk-oriented approach - - * {{{http://jira.springframework.org/browse/BATCH-607}BATCH-607}} - FlatFileItemWriter section needs additional details - - * {{{http://jira.springframework.org/browse/BATCH-730}BATCH-730}} - HibernateCursorItemReader Parameters - - * {{{http://jira.springframework.org/browse/BATCH-920}BATCH-920}} - ChunkContext in StepContextRepeatCallback is unused - diff --git a/src/site/apt/migration/2.0-m4-rc1.apt b/src/site/apt/migration/2.0-m4-rc1.apt deleted file mode 100644 index 5bf442f2c8..0000000000 --- a/src/site/apt/migration/2.0-m4-rc1.apt +++ /dev/null @@ -1,135 +0,0 @@ -Spring Batch 2.0.0.RC1 Release Notes - -* Bug - - - * {{{http://jira.springframework.org/browse/BATCH-1089}BATCH-1089}} - Getting Started page should be updated - - * {{{http://jira.springframework.org/browse/BATCH-1088}BATCH-1088}} - NamespaceHandler no handle all XML tags - - * {{{http://jira.springframework.org/browse/BATCH-1087}BATCH-1087}} - ClassPathXmlJobRegistry does not accept patterns for resources - - * {{{http://jira.springframework.org/browse/BATCH-1086}BATCH-1086}} - JdbcJobExecutionDao.getRunningJobExecutions() ignores jobName - - * {{{http://jira.springframework.org/browse/BATCH-1085}BATCH-1085}} - Should adding next entities mean that the next attribute cannot be used? - - * {{{http://jira.springframework.org/browse/BATCH-1084}BATCH-1084}} - Change "status" attribute of and to "exit-code" - - * {{{http://jira.springframework.org/browse/BATCH-1083}BATCH-1083}} - Use RowMapper instead of ParameterizedRowMapper for public API - - * {{{http://jira.springframework.org/browse/BATCH-1082}BATCH-1082}} - If file reader is lenient about resource existing on startup, it should also check when it is closed - - * {{{http://jira.springframework.org/browse/BATCH-1081}BATCH-1081}} - Add task-executor= to - - * {{{http://jira.springframework.org/browse/BATCH-1080}BATCH-1080}} - make a top-level element - - * {{{http://jira.springframework.org/browse/BATCH-1078}BATCH-1078}} - Maven Build - ClassNotFoundException: com.springsource.util.math.Sets - - * {{{http://jira.springframework.org/browse/BATCH-1077}BATCH-1077}} - Update docs to reflect changes in statuses - - * {{{http://jira.springframework.org/browse/Update}Update}} - docs to reflect changes in statuses - - * {{{http://jira.springframework.org/browse/BATCH-1076}BATCH-1076}} - Add a test case to show how a can restart on the step it failed on - - * {{{http://jira.springframework.org/browse/BATCH-1075}BATCH-1075}} - allow-start-if-complete and start-limit should be on , not - - * {{{http://jira.springframework.org/browse/BATCH-1073}BATCH-1073}} - on element, "listeners" must be disallowed when "ref" attribute is specified - - * {{{http://jira.springframework.org/browse/BATCH-1071}BATCH-1071}} - JobInterruptedException needs to be on the fatal list - - * {{{http://jira.springframework.org/browse/BATCH-1070}BATCH-1070}} - Add 'strict' property to BeanWrapperFieldSetMapper to allow suppression of errors - - * {{{http://jira.springframework.org/browse/BATCH-1069}BATCH-1069}} - Update docs to show top-level element - - * {{{http://jira.springframework.org/browse/BATCH-1068}BATCH-1068}} - Update samples to use top-level element - - * {{{http://jira.springframework.org/browse/BATCH-1067}BATCH-1067}} - Problems with the way 'split' element handles EndStates - - * {{{http://jira.springframework.org/browse/BATCH-1066}BATCH-1066}} - Modify namespace so that if "next" attribute is used, then no transition elements will be allowed. - - * {{{http://jira.springframework.org/browse/BATCH-1064}BATCH-1064}} - Namespace end transition names don't match their corresponding statuses - - * {{{http://jira.springframework.org/browse/BATCH-1063}BATCH-1063}} - Fix image sizes - - * {{{http://jira.springframework.org/browse/BATCH-1062}BATCH-1062}} - Add namespace declaration information to docs - - * {{{http://jira.springframework.org/browse/BATCH-1061}BATCH-1061}} - FlowJob.getLastStepExecution() puts arguments into isLater() in the wrong order - - * {{{http://jira.springframework.org/browse/BATCH-1059}BATCH-1059}} - BATCH_JOB_INSTANCE.JOB_KEY ignores Date milliseconds - - * {{{http://jira.springframework.org/browse/BATCH-1057}BATCH-1057}} - Residual from 1.x left over in AbstractItemWriter - - * {{{http://jira.springframework.org/browse/BATCH-1056}BATCH-1056}} - Proofread documentation - - * {{{http://jira.springframework.org/browse/BATCH-1055}BATCH-1055}} - Add section on filtering to docs - - * {{{http://jira.springframework.org/browse/BATCH-1054}BATCH-1054}} - Maven Build - incompatible bundle manifest version - - * {{{http://jira.springframework.org/browse/BATCH-1053}BATCH-1053}} - Add getStep(String) to Job interface - - * {{{http://jira.springframework.org/browse/BATCH-1052}BATCH-1052}} - Write Common Pattern section about SystemCommandTasklet - - * {{{http://jira.springframework.org/browse/BATCH-1051}BATCH-1051}} - Write Common Pattern section about multi-line records - - * {{{http://jira.springframework.org/browse/BATCH-1050}BATCH-1050}} - SimpleJobExplorer doesn't retrieve StepExecutions of running JobExecutions using MapStepExecutionDao - - * {{{http://jira.springframework.org/browse/BATCH-1049}BATCH-1049}} - Add compile scope to spring-test in spring-batch-test - - * {{{http://jira.springframework.org/browse/BATCH-1048}BATCH-1048}} - Add site docos for the Test project - - * {{{http://jira.springframework.org/browse/BATCH-1047}BATCH-1047}} - CommandLineJobRunner should use JobOperator - - * {{{http://jira.springframework.org/browse/BATCH-1043}BATCH-1043}} - Add chapter on scaling (partitioning, etc) to docs - - * {{{http://jira.springframework.org/browse/BATCH-1041}BATCH-1041}} - Create section in documentation about launching from within a web container - - * {{{http://jira.springframework.org/browse/BATCH-1040}BATCH-1040}} - Add cross referencing to documentation - - * {{{http://jira.springframework.org/browse/BATCH-1037}BATCH-1037}} - Change namespace to set job-repository only on job - - * {{{http://jira.springframework.org/browse/BATCH-1034}BATCH-1034}} - Add StepScope to application context automatically in the parser - - * {{{http://jira.springframework.org/browse/BATCH-1030}BATCH-1030}} - FlowJob replays failed steps on restart, even if the failure did not fail the job - - * {{{http://jira.springframework.org/browse/BATCH-1028}BATCH-1028}} - JdbcCursorItemReader driverSupportsAbsolute property defaults to false - - * {{{http://jira.springframework.org/browse/BATCH-1027}BATCH-1027}} - PassThroughFieldExtractor should have a unit test - - * {{{http://jira.springframework.org/browse/BATCH-1023}BATCH-1023}} - File writing related interfaces need more javadoc - - * {{{http://jira.springframework.org/browse/BATCH-1013}BATCH-1013}} - Using the batch namespace, steps can't be defined outside of the "job" tag. - - * {{{http://jira.springframework.org/browse/BATCH-1011}BATCH-1011}} - Need distinction between "stop" and "end" transitions? - - * {{{http://jira.springframework.org/browse/BATCH-1010}BATCH-1010}} - StepFactoryBeans cleanup - - * {{{http://jira.springframework.org/browse/BATCH-1008}BATCH-1008}} - Elements in namespace are order-dependent - - * {{{http://jira.springframework.org/browse/BATCH-1000}BATCH-1000}} - Add timeout to TaskExecutorPartitionHandler - - * {{{http://jira.springframework.org/browse/BATCH-955}BATCH-955}} - Update XML schema to be tooling friendly - - * {{{http://jira.springframework.org/browse/BATCH-949}BATCH-949}} - JdbcCursorItemReader: property name for "mapper"? - - * {{{http://jira.springframework.org/browse/BATCH-942}BATCH-942}} - Obvious JobLocator implementation - - * {{{http://jira.springframework.org/browse/BATCH-909}BATCH-909}} - Turn off getWarnings() call in JdbcCursorItemReader when ignoreWarnings is true - - * {{{http://jira.springframework.org/browse/BATCH-882}BATCH-882}} - Create section in documentation highlighting key changes between 1.x and 2.0 - - * {{{http://jira.springframework.org/browse/BATCH-851}BATCH-851}} - Remove DrivingQueryItemReader support in favor of PagingItemReaders - - * {{{http://jira.springframework.org/browse/BATCH-819 Create}BATCH-819 Create}} - separate CI builds for artifacts in Maven Central and Enterprise Repository - - * {{{http://jira.springframework.org/browse/BATCH-818 Create}BATCH-818 Create}} - and test ivy and maven files needed for build - - * {{{http://jira.springframework.org/browse/BATCH-793}BATCH-793}} - Creating a new ApplicationContext per Job needs refactoring - - * {{{http://jira.springframework.org/browse/BATCH-690}BATCH-690}} - Move drop table statements to separate script - - * {{{http://jira.springframework.org/browse/BATCH-659}BATCH-659}} - unused classes deprecation/removal - - * {{{http://jira.springframework.org/browse/BATCH-618}BATCH-618}} - DefaultJobParametersConverter does not parse parameters of type double - - * {{{http://jira.springframework.org/browse/BATCH-583}BATCH-583}} - Figures missing from PDF docs - diff --git a/src/site/apt/migration/2.0-rc1-rc2.apt b/src/site/apt/migration/2.0-rc1-rc2.apt deleted file mode 100644 index da3cf5b403..0000000000 --- a/src/site/apt/migration/2.0-rc1-rc2.apt +++ /dev/null @@ -1,152 +0,0 @@ -Spring Batch 2.0.0.RC2 Release Notes - -* Issues - - * {{{http://jira.springframework.org/browse/BATCH-1165}BATCH-1165}} - Allow 'id' and 'ref' to exist together on <*-listener/> - - * {{{http://jira.springframework.org/browse/BATCH-1164}BATCH-1164}} - Putting scope="step" on a listener causes failure - - * {{{http://jira.springframework.org/browse/BATCH-1163}BATCH-1163}} - In Batch xsd, elements in should be unordered - - * {{{http://jira.springframework.org/browse/BATCH-1162}BATCH-1162}} - In Batch xsd, elements in should be unordered - - * {{{http://jira.springframework.org/browse/BATCH-1161}BATCH-1161}} - Throw error if a flow has no steps - - * {{{http://jira.springframework.org/browse/BATCH-1160}BATCH-1160}} - In Batch xsd, stepType and flowStepType should be unordered - - * {{{http://jira.springframework.org/browse/BATCH-1159}BATCH-1159}} - PROCESS_SKIP_COUNT seems not to ever get written to database - - * {{{http://jira.springframework.org/browse/BATCH-1157}BATCH-1157}} - Add max count parameter to counting item readers - - * {{{http://jira.springframework.org/browse/BATCH-1154}BATCH-1154}} - TaskletElementParser can't predict which StepFactoryBean to use - - * {{{http://jira.springframework.org/browse/BATCH-1153}BATCH-1153}} - listener element should only have an "id" when defined at the top level - - * {{{http://jira.springframework.org/browse/BATCH-1152}BATCH-1152}} - Allow comma and newline as delimiters in exception lists in namespace - - * {{{http://jira.springframework.org/browse/BATCH-1151}BATCH-1151}} - Add IDE support to in xsd - - * {{{http://jira.springframework.org/browse/BATCH-1147}BATCH-1147}} - StepExecution getFilterCount always return 0 - - * {{{http://jira.springframework.org/browse/BATCH-1145}BATCH-1145}} - Conflicts if both and have a "parent" attribute - - * {{{http://jira.springframework.org/browse/BATCH-1144}BATCH-1144}} - Create top-level element - - * {{{http://jira.springframework.org/browse/BATCH-1143}BATCH-1143}} - Standalone should not be allowed to have "tasklet" attribute and together - - * {{{http://jira.springframework.org/browse/BATCH-1142}BATCH-1142}} - In the xsd, should be moved from "flowType" to "job" - - * {{{http://jira.springframework.org/browse/BATCH-1141}BATCH-1141}} - Rename CompositeExecutionJobListener to CompositeJobExecutionListener - - * {{{http://jira.springframework.org/browse/BATCH-1139}BATCH-1139}} - Add "parent" attribute to - - * {{{http://jira.springframework.org/browse/BATCH-1138}BATCH-1138}} - Add "parent" attribute to - - * {{{http://jira.springframework.org/browse/BATCH-1136}BATCH-1136}} - Add setListeners(StepExecutionListener[]) to TaskletStep - - * {{{http://jira.springframework.org/browse/BATCH-1135}BATCH-1135}} - Create top-level element - - * {{{http://jira.springframework.org/browse/BATCH-1134}BATCH-1134}} - Create a superclass for StepListenerParser and JobExecutionListenerParser - - * {{{http://jira.springframework.org/browse/BATCH-1132}BATCH-1132}} - Add "parent" attribute to - - * {{{http://jira.springframework.org/browse/BATCH-1131}BATCH-1131}} - It is not possible to set transaction-attributes for a tasklet step - - * {{{http://jira.springframework.org/browse/BATCH-1130}BATCH-1130}} - Ensure Ordered is respected by generated listeners - - * {{{http://jira.springframework.org/browse/BATCH-1129}BATCH-1129}} - Problems with exception classifications - - * {{{http://jira.springframework.org/browse/BATCH-1128}BATCH-1128}} - Make sure WRITE_COUNT and ROLLBACK_COUNT are being updated correctly - - * {{{http://jira.springframework.org/browse/BATCH-1127}BATCH-1127}} - Add contextual line number information for exceptions thrown by FlatFileItemReader - - * {{{http://jira.springframework.org/browse/BATCH-1126}BATCH-1126}} - StepScope does not apply to twice nested inner beans - - * {{{http://jira.springframework.org/browse/BATCH-1125}BATCH-1125}} - NoWorkFoundStepExecutionListener doesn't fail the step - - * {{{http://jira.springframework.org/browse/BATCH-1124}BATCH-1124}} - Fix error message that occurs when the same annotation is used twice on one method - - * {{{http://jira.springframework.org/browse/BATCH-1123}BATCH-1123}} - ExecutionContextPromotionListener may perform promotion multiple times - - * {{{http://jira.springframework.org/browse/BATCH-1122}BATCH-1122}} - StaxEventWriter.startDocument() needs to be protected - - * {{{http://jira.springframework.org/browse/BATCH-1121}BATCH-1121}} - Documentation should cover the 'no work found' scenario - - * {{{http://jira.springframework.org/browse/BATCH-1120}BATCH-1120}} - Allow completion policy to be set on step - - * {{{http://jira.springframework.org/browse/BATCH-1119}BATCH-1119}} - afterWrite() will only be called if an exception is raised during throttling - - * {{{http://jira.springframework.org/browse/BATCH-1118}BATCH-1118}} - Listener annotation with wrong signature fails too late and too quietly - - * {{{http://jira.springframework.org/browse/BATCH-1117}BATCH-1117}} - onWriteError should be called with only the bad item - - * {{{http://jira.springframework.org/browse/BATCH-1116}BATCH-1116}} - Create section on MetaDataInstanceFactory in testing chapter - - * {{{http://jira.springframework.org/browse/BATCH-1113}BATCH-1113}} - query for JdbcJobExecutionDao.findRunningJobExecutions is broken - - * {{{http://jira.springframework.org/browse/BATCH-1112}BATCH-1112}} - Remove cycle in infrastructure database/support - - * {{{http://jira.springframework.org/browse/BATCH-1111}BATCH-1111}} - ChunkListener called before WriteListener - - * {{{http://jira.springframework.org/browse/BATCH-1110}BATCH-1110}} - Defer EntityManager flushing and clearing in JpaPagingItemReader - - * {{{http://jira.springframework.org/browse/BATCH-1109}BATCH-1109}} - Generalise PrefixMatching* to PatternMatching* - - * {{{http://jira.springframework.org/browse/BATCH-1108}BATCH-1108}} - Add composite ItemWriter/Processor based on Classifier - - * {{{http://jira.springframework.org/browse/BATCH-1107}BATCH-1107}} - Fix Date conversion in PlaceholderTargetSource - - * {{{http://jira.springframework.org/browse/BATCH-1106}BATCH-1106}} - SqlPagingQueryProviderFactoryBean ascending should default to true - - * {{{http://jira.springframework.org/browse/BATCH-1105}BATCH-1105}} - Namespace error in documentation - - * {{{http://jira.springframework.org/browse/BATCH-1104}BATCH-1104}} - Wrong stax version in spring-batch-parent-2.0.0.RC1.pom, is 1.2 should be 1.2.0 - - * {{{http://jira.springframework.org/browse/BATCH-1103}BATCH-1103}} - Add partioning sample - - * {{{http://jira.springframework.org/browse/BATCH-1102}BATCH-1102}} - Classes with "listener" annotations should be auto-registered - - * {{{http://jira.springframework.org/browse/BATCH-1101}BATCH-1101}} - Remove StepScope bean definition from samples that don't need it - - * {{{http://jira.springframework.org/browse/BATCH-1099}BATCH-1099}} - Make writer in skip sample stateful - - * {{{http://jira.springframework.org/browse/BATCH-1098}BATCH-1098}} - Check @AfterWrite is only called once per item - - * {{{http://jira.springframework.org/browse/BATCH-1097}BATCH-1097}} - Add late binding to some io samples - - * {{{http://jira.springframework.org/browse/BATCH-1096}BATCH-1096}} - Add late binding to some io samples - - * {{{http://jira.springframework.org/browse/BATCH-1093}BATCH-1093}} - Make AbstractJobTests.makeUniqueJobParameters() public - - * {{{http://jira.springframework.org/browse/BATCH-1092}BATCH-1092}} - Fix naming conventions for exit status in - - * {{{http://jira.springframework.org/browse/BATCH-1091}BATCH-1091}} - Add strict flag to file readers (flat and XML). - - * {{{http://jira.springframework.org/browse/BATCH-1072}BATCH-1072}} - Update docs to reflect attribute change in step element - - * {{{http://jira.springframework.org/browse/BATCH-1065}BATCH-1065}} - "BATCH-1011 - - * {{{http://jira.springframework.org/browse/Update documentation to explain end/fail/pause transitions" - * {{{http://jira.springframework.org/browse/BATCH-976}BATCH-976}} - Provide automatic batch database schema installation utility - - * {{{http://jira.springframework.org/browse/BATCH-937}BATCH-937}} - Make sure JobRepository can be proxied - - * {{{http://jira.springframework.org/browse/BATCH-935}BATCH-935}} - Use NumberFormat when parsing real numbers - - * {{{http://jira.springframework.org/browse/BATCH-918}BATCH-918}} - Duplicate jar files in *-with-dependencies.zip - - * {{{http://jira.springframework.org/browse/BATCH-902}BATCH-902}} - Revise FAQ on web site - - * {{{http://jira.springframework.org/browse/BATCH-794}BATCH-794}} - Align with SpringSource Enterprise Repository - - * {{{http://jira.springframework.org/browse/BATCH-632}BATCH-632}} - Move Tasklet interface from core to infrastructure - - * {{{http://jira.springframework.org/browse/BATCH-573}BATCH-573}} - Delegating streams and listeners (e.g. HibernateAwareItemWriter) should delegate those interfaces to their delegates. - - * {{{http://jira.springframework.org/browse/BATCH-501}BATCH-501}} - Core and Infrastructure still have circular dependency - - * {{{http://jira.springframework.org/browse/BATCH-470}BATCH-470}} - RFC: Can all ExceptionHandlers be replaced by a combination of making Repeat/RetryListeners more robust and rewriting as Listeners? - - * {{{http://jira.springframework.org/browse/BATCH-394}BATCH-394}} - If FieldSet is an interface it needs a factory, otherwise existing clients are tied to specific implementations - - * {{{http://jira.springframework.org/browse/BATCH-22}BATCH-22}} - Find alternative to ThreadLocal for RepeatSynchronizationManager diff --git a/src/site/apt/migration/2.0-rc2-rc3.apt b/src/site/apt/migration/2.0-rc2-rc3.apt deleted file mode 100644 index 5e28d78c7e..0000000000 --- a/src/site/apt/migration/2.0-rc2-rc3.apt +++ /dev/null @@ -1,27 +0,0 @@ -Spring Batch 2.0.0.RC3 Release Notes - -* Issues - - * {{{http://jira.springframework.org/browse/BATCH-1168}BATCH-1168}} - Bad ERROR_LOG definition in business-schema-mysql.sql - - * {{{http://jira.springframework.org/browse/BATCH-1169}BATCH-1169}} - Incorrect system property name in data-source-context.xml - - * {{{http://jira.springframework.org/browse/BATCH-1171}BATCH-1171}} - Interrupted step does not fail job. - - * {{{http://jira.springframework.org/browse/BATCH-1174}BATCH-1174}} - Late binding of jobParameters does not work if late binding expression is not preceded or trailed by string - - * {{{http://jira.springframework.org/browse/BATCH-1176}BATCH-1176}} - Update documentation section 5.2.2. Example Tasklet implementation - - * {{{http://jira.springframework.org/browse/BATCH-1178}BATCH-1178}} - with "ref=" silently ignores other attributes - - * {{{http://jira.springframework.org/browse/BATCH-1180}BATCH-1180}} - Error occurs if parent= attribute appears on inline without tasket - - * {{{http://jira.springframework.org/browse/BATCH-1181}BATCH-1181}} - element always assumes the step is a TaskletStep - - * {{{http://jira.springframework.org/browse/BATCH-1175}BATCH-1175}} - Update Validator interface for Java 5 - - * {{{http://jira.springframework.org/browse/BATCH-1183}BATCH-1183}} - Implement toString() on Step Implementations - - * {{{http://jira.springframework.org/browse/BATCH-1179}BATCH-1179}} - Remove ref= attribute from in favor of parent= - - * {{{http://jira.springframework.org/browse/BATCH-1184}BATCH-1184}} - DelegatingStep is not used. It should be removed. diff --git a/src/site/apt/migration/2.0-rc3-release.apt b/src/site/apt/migration/2.0-rc3-release.apt deleted file mode 100644 index 51b50084d8..0000000000 --- a/src/site/apt/migration/2.0-rc3-release.apt +++ /dev/null @@ -1,49 +0,0 @@ -Spring Batch 2.0.0.RELEASE Release Notes - -* Bug - - * {{{http://jira.springframework.org/browse/BATCH-1170}BATCH-1170}} - When using FixedLengthTokenizer and FixedLengthLineAggregator, can not read the record correctly with parameters like [p:columns="1-9,10-19,20-29,30-39,40-49"] in the configuration file. - - * {{{http://jira.springframework.org/browse/BATCH-1185}BATCH-1185}} - Job slows when step scope is used - - * {{{http://jira.springframework.org/browse/BATCH-1187}BATCH-1187}} - Step shouldn't exit with status=EXECUTING - - * {{{http://jira.springframework.org/browse/BATCH-1188}BATCH-1188}} - Build fails for bundlor SNAPSHOT used for Batch since it conflicts with more recent one - - * {{{http://jira.springframework.org/browse/BATCH-1192}BATCH-1192}} - Error in documentation section 4.2.3 - In-Memory Repository - - * {{{http://jira.springframework.org/browse/BATCH-1196}BATCH-1196}} - ExecutionContext not re-hydrated by JdbcJobExecutionDao - - * {{{http://jira.springframework.org/browse/BATCH-1197}BATCH-1197}} - Rerunning a job sometimes creates new job instance - - * {{{http://jira.springframework.org/browse/BATCH-1198}BATCH-1198}} - processSkipCount and filterCount mixed up - - * {{{http://jira.springframework.org/browse/BATCH-1201}BATCH-1201}} - Listener Annotations don't allow parameters to be subtypes of expected types - -* Improvement - - * {{{http://jira.springframework.org/browse/BATCH-1095}BATCH-1095}} - Throw exception if late binding fails - - * {{{http://jira.springframework.org/browse/BATCH-1150}BATCH-1150}} - Update StepExecutionPreparedStatementSetter to no longer be a listener - - * {{{http://jira.springframework.org/browse/BATCH-1172}BATCH-1172}} - Register RangeArrayPropertyEditor automatically - - * {{{http://jira.springframework.org/browse/BATCH-1177}BATCH-1177}} - Allow ValidatingItemProcessor to filter items on ValidationException - - * {{{http://jira.springframework.org/browse/BATCH-1189}BATCH-1189}} - By default, all exceptions should be fatal - - * {{{http://jira.springframework.org/browse/BATCH-1190}BATCH-1190}} - Navigation pane for Modules pages have bad links - - * {{{http://jira.springframework.org/browse/BATCH-1191}BATCH-1191}} - site:deploy - Missing distribution management information in the project - - * {{{http://jira.springframework.org/browse/BATCH-1200}BATCH-1200}} - Update JavaDocs for Listener Annotations - -* Task - - * {{{http://jira.springframework.org/browse/BATCH-834}BATCH-834}} - Review stop signal handling at JobExecution - - * {{{http://jira.springframework.org/browse/BATCH-1042}BATCH-1042}} - Add 'Whats New in 2.0" chapter to docs - - * {{{http://jira.springframework.org/browse/BATCH-1194}BATCH-1194}} - Swicth Eclipse meta data to m2eclipse - - * {{{http://jira.springframework.org/browse/BATCH-1195}BATCH-1195}} - Add "Common Pattern" for passing data from one step to another diff --git a/src/site/apt/migration/2.0.0-2.0.1.apt b/src/site/apt/migration/2.0.0-2.0.1.apt deleted file mode 100644 index 6867954062..0000000000 --- a/src/site/apt/migration/2.0.0-2.0.1.apt +++ /dev/null @@ -1,88 +0,0 @@ -Spring Batch 2.0.1.RELEASE Release Notes - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1203]}} - Correct ChunkOrientedTasklet.setBuffering javadoc - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1204]}} - Error in FieldSetMapper documentation - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1205]}} - When readCount % commitInterval == 0, commitCount is one more than it should be - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1208]}} - late-binding not being resolved in - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1210]}} - AbstractStep overwrites custom exit status for STOPPED steps - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1212]}} - Incorrect link in Documentation - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1213]}} - Defaults in xsd override parent attributes - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1214]}} - CoreNamespaceUtils.addRangePropertyEditor fails with Spring 3.0 - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1215]}} - Docs refer to non-existent class SimpleDelegatingPagingQueryProvider - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1218]}} - item streams won't get registered if ItemStream reader is used with step - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1223]}} - Maven Build - ClassNotFoundException: com.springsource.util.osgi.manifest.ManifestFactory - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1225]}} - FlatFileItemWriter and StaxEventItemWriter do not restart in the right place - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1230]}} - scope "step" does not work together with Annotation "@BeforeStep" - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1234]}} - ExitStatus.FINISHED should be ExitStatus.COMPLETED - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1236]}} - Fix namespace errors in Readers and Writers chapter - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1240]}} - ItemStream is not registered when defined in step scope - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1241]}} - SystemCommandTasklet package name is not accurate - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1244]}} - Documentation error for skip-limit - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1245]}} - StaxEventItemWriter writes extra end document tag with Woodstox 3.2.9 plus - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1249]}} - Add files to .springBeans in samples project - - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1172]}} - Register RangeArrayPropertyEditor automatically - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1202]}} - Improve error message when neither commit-interval nor chunk-completion-policy are specified on a chuck (xml namespace) - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1216]}} - Adding the possiblity to set the throttle limit in the spring batch name space - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1217]}} - Suggest use .doubleValue() value in DefaultFieldSet.readDouble(..) - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1219]}} - JmsItemReader and -Writer should check for proper settings on JmsTemplate - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1220]}} - Apply consistent debug logging in framework-provided ItemWriter implementations - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1221]}} - Use batch namespace in simple cli template / archetype - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1228]}} - Possibility to modify proxyTargetClass property of StepScope - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1233]}} - Confusing docs in section 4.1 on non-default job repo id for nested step element - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1247]}} - Clean up multilineOrderJob sample - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1248]}} - Remove default from "merge" attribute in XSD - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1250]}} - should disallow , , and if abstract=true - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1251]}} - Use standard JPQL in JPA sample - - -* New Feature - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1226]}} - Add assertLineCount method to AssertFile - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1238]}} - Add ItemProcessorAdapter similar to Item*Adapter - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1243]}} - Expose the current resource of MultiResourceItemReader - - -* Task - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-724]}} - create tests for non-default table prefix - - * {{{http://jira.springsource.org/browse/BATCH-1203}[BATCH-1211]}} - Update samples docs on website diff --git a/src/site/apt/migration/2.0.1-2.0.2.apt b/src/site/apt/migration/2.0.1-2.0.2.apt deleted file mode 100644 index 80e14c8770..0000000000 --- a/src/site/apt/migration/2.0.1-2.0.2.apt +++ /dev/null @@ -1,122 +0,0 @@ -Spring Batch 2.0.2.RELEASE Release Notes - - -* Sub-task - - *{{{http://jira.springsource.org/browse/BATCH-1327}[BATCH-1327]}} - Prevent re-processing and re-writing on rollback after non-skippable and non-retryable exception - - * {{{http://jira.springsource.org/browse/BATCH-1331}[BATCH-1331]}} - Fatal exceptions that are also marked as no-rollback - - * {{{http://jira.springsource.org/browse/BATCH-1332}[BATCH-1332]}} - Skippable exceptions on write that are also marked as no-rollback cause a rollback, and double processing of item (writer and skip listener) - - * {{{http://jira.springsource.org/browse/BATCH-1333}[BATCH-1333]}} - Support for retry of non-skippable subclasses of skippable exceptions - - * {{{http://jira.springsource.org/browse/BATCH-1334}[BATCH-1334]}} - Exception marked as no-rollback but not skippable should not cause skips - - * {{{http://jira.springsource.org/browse/BATCH-1335}[BATCH-1335]}} - Pathological cases of no-rollback-for-exceptions on framework panic exceptions - - - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1232}[BATCH-1232]}} - Sybase 12.5 compatiblity when writing to the spring batch context tables - - * {{{http://jira.springsource.org/browse/BATCH-1264}[BATCH-1264]}} - NPE in StepParserStepFactoryBean#configureTaskletStep() #289 when omitting "isolation" for <transaction-attributes> - - * {{{http://jira.springsource.org/browse/BATCH-1272}[BATCH-1272]}} - Write skips do not work in a multi-threaded step - - * {{{http://jira.springsource.org/browse/BATCH-1278}[BATCH-1278]}} - RepeatTemplate aborts early if multiple threads throw ignorable exceptions - - * {{{http://jira.springsource.org/browse/BATCH-1280}[BATCH-1280]}} - JobParserJobFactoryBean should be a singleton - - * {{{http://jira.springsource.org/browse/BATCH-1282}[BATCH-1282]}} - JobRegistryBeanPostProcessor skips jobs in XML namespace unless they are injected as dependency - - * {{{http://jira.springsource.org/browse/BATCH-1284}[BATCH-1284]}} - Partition Step Stop is incorrectly setting the BatchStatus to COMPLETED. - - * {{{http://jira.springsource.org/browse/BATCH-1287}[BATCH-1287]}} - JobRegistryBeanPostProcessor is checking for Job instead of JobParserJobFactoryBean - - * {{{http://jira.springsource.org/browse/BATCH-1289}[BATCH-1289]}} - Null pointer in CoreNamespaceUtils.rangeArrayEditorAlreadyDefined() - - * {{{http://jira.springsource.org/browse/BATCH-1301}[BATCH-1301]}} - ItemStream is not being opened correctly for multi-threaded Step when scope="step" - - * {{{http://jira.springsource.org/browse/BATCH-1304}[BATCH-1304]}} - Filter counter not incremented whenever there's a skip - - * {{{http://jira.springsource.org/browse/BATCH-1308}[BATCH-1308]}} - FixedLengthTokenizer's 'names' property may not have spaces - - * {{{http://jira.springsource.org/browse/BATCH-1313}[BATCH-1313]}} - loopFlowSample's LimitDecider returns "COMPLETE" instead of "COMPLETED" - - * {{{http://jira.springsource.org/browse/BATCH-1314}[BATCH-1314]}} - FFIW in tradeJob is pointing to classpath instead of the target - - * {{{http://jira.springsource.org/browse/BATCH-1315}[BATCH-1315]}} - Section 2.3 Configuration Enhancements contains invalid example - - * {{{http://jira.springsource.org/browse/BATCH-1318}[BATCH-1318]}} - Ensure exception classes are behaving correctly - - * {{{http://jira.springsource.org/browse/BATCH-1319}[BATCH-1319]}} - Small memory leak in StepSynchronizationManager - - - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-1262}[BATCH-1262]}} - org.springframework.batch.item.file.transform.DefaultFieldSet#readBigDecimal doesn't use supplied numberFormat - - * {{{http://jira.springsource.org/browse/BATCH-1267}[BATCH-1267]}} - Make sure jmsTemplate is not null in JmsItemReader - - * {{{http://jira.springsource.org/browse/BATCH-1268}[BATCH-1268]}} - Remove dependency on commons lang - - * {{{http://jira.springsource.org/browse/BATCH-1270}[BATCH-1270]}} - Update documentation for formatting consistency - - * {{{http://jira.springsource.org/browse/BATCH-1275}[BATCH-1275]}} - ExecutionContextPromotionListener contains duplicate assertion - - * {{{http://jira.springsource.org/browse/BATCH-1276}[BATCH-1276]}} - Change Common Pattern "Writing a Summary Footer" to use AfterWrite for updating total. - - * {{{http://jira.springsource.org/browse/BATCH-1285}[BATCH-1285]}} - Raise an exception if a step cannot be reached. - - * {{{http://jira.springsource.org/browse/BATCH-1299}[BATCH-1299]}} - Add processSkipCount in StepExecution#getSummary() - - * {{{http://jira.springsource.org/browse/BATCH-1302}[BATCH-1302]}} - Tidy up samples - remove prefixes for namespaces like beans: - - * {{{http://jira.springsource.org/browse/BATCH-1305}[BATCH-1305]}} - Add download page link to the batch home page - - * {{{http://jira.springsource.org/browse/BATCH-1310}[BATCH-1310]}} - SimpleJobExplorer should return null when a StepExecution cannot be found - - * {{{http://jira.springsource.org/browse/BATCH-1311}[BATCH-1311]}} - SimpleJobExplorer should return null when a StepExecution cannot be found - - * {{{http://jira.springsource.org/browse/BATCH-1312}[BATCH-1312]}} - Improve error messages for skip/retry configurations - - - -* New Feature - - * {{{http://jira.springsource.org/browse/BATCH-1253}[BATCH-1253]}} - Ability to throttle the number of rows returned by a HibernateCursorItemReader - - * {{{http://jira.springsource.org/browse/BATCH-1279}[BATCH-1279]}} - Add default constructor to ValidatingItemProcessor for those preferring setter injection - - * {{{http://jira.springsource.org/browse/BATCH-1288}[BATCH-1288]}} - Typo in CallableTaskletAdapter JavaDoc - - * {{{http://jira.springsource.org/browse/BATCH-1295}[BATCH-1295]}} - Provide ability to pass job's ExecutionContext into AbstractJobTests.launchStep() - - * {{{http://jira.springsource.org/browse/BATCH-1297}[BATCH-1297]}} - Add null check for FlatFileItemReader in case resource exists on close, but not on open(!) - - * {{{http://jira.springsource.org/browse/BATCH-1303}[BATCH-1303]}} - Support Spring 3.0 expression language in beans that are not step scoped - - * {{{http://jira.springsource.org/browse/BATCH-1306}[BATCH-1306]}} - DelimitedLineAggregator might skip null objects - - * {{{http://jira.springsource.org/browse/BATCH-1309}[BATCH-1309]}} - Added a 'strict' property to ExecutionContextPromotionListener to ensure the key is present - - * {{{http://jira.springsource.org/browse/BATCH-1317}[BATCH-1317]}} - Add flag to FlatFileItemWriter to delete empty files - - * {{{http://jira.springsource.org/browse/BATCH-1320}[BATCH-1320]}} - New line special character in Header - - * {{{http://jira.springsource.org/browse/BATCH-1321}[BATCH-1321]}} - Upgrade MANIFEST to allow Spring 3.0 to be used optionally - - * {{{http://jira.springsource.org/browse/BATCH-1325}[BATCH-1325]}} - Add more convenience methods to JobRepositoryTestUtils - - * {{{http://jira.springsource.org/browse/BATCH-1336}[BATCH-1336]}} - Add -restart and -next options to CommandLineJobRunner - - - -* Task - - * {{{http://jira.springsource.org/browse/BATCH-1271}[BATCH-1271]}} - Add a note about bean scope for batch parameters usage - - diff --git a/src/site/apt/migration/2.0.2-2.0.3.apt b/src/site/apt/migration/2.0.2-2.0.3.apt deleted file mode 100644 index 35f469cc90..0000000000 --- a/src/site/apt/migration/2.0.2-2.0.3.apt +++ /dev/null @@ -1,33 +0,0 @@ -Spring Batch 2.0.2.RELEASE Release Notes - - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1341}[BATCH-1341]}} - DataSourceInitializer throws ArrayOutOfBoundException when any destroyScript is specified - - * {{{http://jira.springsource.org/browse/BATCH-1345}[BATCH-1345]}} - Fix error message for when <tasklet/> has no ref= or <chunk/> - - * {{{http://jira.springsource.org/browse/BATCH-1351}[BATCH-1351]}} - An empty <*-exception-classes/> list does not override parent's list - - * {{{http://jira.springsource.org/browse/BATCH-1354}[BATCH-1354]}} - Infinite loop caused by throwing an Error from the ItemWriter of a skippable step - - * {{{http://jira.springsource.org/browse/BATCH-1355}[BATCH-1355]}} - Step scope causes type= attribute of <value/> to be ignored - - * {{{http://jira.springsource.org/browse/BATCH-1362}[BATCH-1362]}} - Threads spinning doing nothing at end of multi-threaded Step - - * {{{http://jira.springsource.org/browse/BATCH-1363}[BATCH-1363]}} - Job stopped in split state does not finish with status = STOPPED - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-1342}[BATCH-1342]}} - Check for valid parameter arguments in CommandLineJobRunner - - * {{{http://jira.springsource.org/browse/BATCH-1359}[BATCH-1359]}} - Throw helpful error from JdbcCursorItemReader if read() called before open() - - * {{{http://jira.springsource.org/browse/BATCH-1360}[BATCH-1360]}} - Throw helpful error from if Tasklet.execute() returns null - -* New Feature - - * {{{http://jira.springsource.org/browse/BATCH-1361}[BATCH-1361]}} - Support for maps in PassThroughFieldExtractor - - * {{{http://jira.springsource.org/browse/BATCH-1370}[BATCH-1370]}} - Bind to non-scalar map entry values in step scope - diff --git a/src/site/apt/migration/2.0.3-2.0.4.apt b/src/site/apt/migration/2.0.3-2.0.4.apt deleted file mode 100755 index e9072b6047..0000000000 --- a/src/site/apt/migration/2.0.3-2.0.4.apt +++ /dev/null @@ -1,19 +0,0 @@ -Spring Batch 2.1.0.M1 Release Notes - -* Bug - - * {{{http://jira.springframework.org/browse/BATCH-1392}[BATCH-1392]}} - Throttle limit is not parsed in ChunkElementParser - - * {{{http://jira.springframework.org/browse/BATCH-1397}[BATCH-1397]}} - Late binding only happens once per ApplicationContext if expression is in substring - - * {{{http://jira.springframework.org/browse/BATCH-1401}[BATCH-1401]}} - All inserts of JobId should be of Types.BIGINT - - * {{{http://jira.springframework.org/browse/BATCH-1408}[BATCH-1408]}} - SimpleJobLauncher package wrong in user guide - - * {{{http://jira.springframework.org/browse/BATCH-1410}[BATCH-1410]}} - Isolation Level in the example in user guide is not supported - - * {{{http://jira.springframework.org/browse/BATCH-1417}[BATCH-1417]}} - Error in FlatFileItemReader when RecordSeparatorPolicy.preProcess readLine returns null - - * {{{http://jira.springframework.org/browse/BATCH-1420}[BATCH-1420]}} - Late Binding only happens first time when using inner bean definition with collection property - - * {{{http://jira.springframework.org/browse/BATCH-1423}[BATCH-1423]}} - Upon job restart, step with FlatFileItemReader doesn't honor skippable-exception-classes diff --git a/src/site/apt/migration/2.0.x-2.1.0.M1.apt b/src/site/apt/migration/2.0.x-2.1.0.M1.apt deleted file mode 100644 index c2358e6ec6..0000000000 --- a/src/site/apt/migration/2.0.x-2.1.0.M1.apt +++ /dev/null @@ -1,78 +0,0 @@ -Spring Batch 2.1.0.M1 Release Notes - -* Sub-task - - * {{{http://jira.springsource.org/browse/BATCH-1298}[BATCH-1298]}} - no-rollback-exception-classes ignored by non-chunk-oriented TaskletStep - - * {{{http://jira.springsource.org/browse/BATCH-1366}[BATCH-1366]}} - Syntactic sugar for CompositeItemProcessor and CompositeItemWriter in namespace - - * {{{http://jira.springsource.org/browse/BATCH-1367}[BATCH-1367]}} - Syntactic sugar for Item*Adapter in namespace - - * {{{http://jira.springsource.org/browse/BATCH-1368}[BATCH-1368]}} - Update docs to remove syntax - - * {{{http://jira.springsource.org/browse/BATCH-1381}[BATCH-1381]}} - Update docs for and in exception lists - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1390}[BATCH-1390]}} - ExecutionContextPromotionListener erases previous step - - * {{{http://jira.springsource.org/browse/BATCH-1392}[BATCH-1392]}} - Throttle limit is not parsed in ChunkElementParser - - * {{{http://jira.springsource.org/browse/BATCH-1397}[BATCH-1397]}} - Late binding only happens once per ApplicationContext if expression is in substring - - * {{{http://jira.springsource.org/browse/BATCH-1408}[BATCH-1408]}} - SimpleJobLauncher Package wrong - - * {{{http://jira.springsource.org/browse/BATCH-1410}[BATCH-1410]}} - Isolation Level in the Example is not supported - - * {{{http://jira.springsource.org/browse/BATCH-1413}[BATCH-1413]}} - A typo in the 11.8 of reference documentation - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-630}[BATCH-630]}} - Possibility to trim input line and fields in Fieldset in LineTokenizer - - * {{{http://jira.springsource.org/browse/BATCH-742}[BATCH-742]}} - Inclusion of a ResourceItemReader that will return resources (such as files in a directory) instead of records from a single resource. - - * {{{http://jira.springsource.org/browse/BATCH-915}[BATCH-915]}} - Call open() lazily in MultiResourceItemReader to allow resources to be skipped - - * {{{http://jira.springsource.org/browse/BATCH-1323}[BATCH-1323]}} - Modify skip/retry/no-rollback exception class configurations to allow for exclude/include - - * {{{http://jira.springsource.org/browse/BATCH-1339}[BATCH-1339]}} - Move task-executor attribute up from to - - * {{{http://jira.springsource.org/browse/BATCH-1357}[BATCH-1357]}} - Allow empty , , and lists - - * {{{http://jira.springsource.org/browse/BATCH-1375}[BATCH-1375]}} - Give CompositeItemProcessor's and CompositeItemWriter's property the same name (delegates) - - * {{{http://jira.springsource.org/browse/BATCH-1404}[BATCH-1404]}} - archetype update - - * {{{http://jira.springsource.org/browse/BATCH-1409}[BATCH-1409]}} - More efficient use of pool threads in repeat template (hence multi-threaded steps) - -* New Feature - - * {{{http://jira.springsource.org/browse/BATCH-1277}[BATCH-1277]}} - Add JobParametersBuilder.addJobParameter(String key, JobParameter parameter) - - * {{{http://jira.springsource.org/browse/BATCH-1348}[BATCH-1348]}} - Allow inlining of reader/writer/processor into - - * {{{http://jira.springsource.org/browse/BATCH-1350}[BATCH-1350]}} - Add back (much simplified) StepExecutionResourceProxy - - * {{{http://jira.springsource.org/browse/BATCH-1356}[BATCH-1356]}} - Line reader for binary files - - * {{{http://jira.springsource.org/browse/BATCH-1358}[BATCH-1358]}} - Copy InfiniteLoopIncrementer into core, and rename it to RunIdIncrementer - - * {{{http://jira.springsource.org/browse/BATCH-1385}[BATCH-1385]}} - Composite elements do not honour @Order annotations - - * {{{http://jira.springsource.org/browse/BATCH-1389}[BATCH-1389]}} - Thread safety in *PagingReader - - * {{{http://jira.springsource.org/browse/BATCH-1406}[BATCH-1406]}} - Avoid deadlock with database pool and multithreaded step when throttle limit is too high. - - * {{{http://jira.springsource.org/browse/BATCH-1415}[BATCH-1415]}} - Execution context cannot be saved on Oracle (UTF8) when context length is between 2000 and 2500 characters - -* Refactoring - - * {{{http://jira.springsource.org/browse/BATCH-973}[BATCH-973]}} - Switch RetryPolicy back to Throwable instead of Exception as in 1.x - - * {{{http://jira.springsource.org/browse/BATCH-1414}[BATCH-1414]}} - Move schema scripts to a package - -* Task - - * {{{http://jira.springsource.org/browse/BATCH-1391}[BATCH-1391]}} - Tidy up skip sample. - diff --git a/src/site/apt/migration/2.1.0-2.1.1.apt b/src/site/apt/migration/2.1.0-2.1.1.apt deleted file mode 100644 index 929c1d9abc..0000000000 --- a/src/site/apt/migration/2.1.0-2.1.1.apt +++ /dev/null @@ -1,80 +0,0 @@ -Spring Batch 2.1.1 Release Notes - -* Sub-task - - * {{{http://jira.springsource.org/browse/BATCH-1520}[BATCH-1520]}} - Allow null ItemWriter as long as ItemProcessor is provided - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1503}[BATCH-1503]}} - JobExecution marked COMPLETE on failure to save step execution metadata - - * {{{http://jira.springsource.org/browse/BATCH-1507}[BATCH-1507]}} - FlowJob.getStep() only looks at state names, not step names - - * {{{http://jira.springsource.org/browse/BATCH-1510}[BATCH-1510]}} - List of stepnames incomplete for nested flow job - - * {{{http://jira.springsource.org/browse/BATCH-1513}[BATCH-1513]}} - HibernateItemReaderHelper requires queryProvider field to be an instance of AbstractHibernateQueryProvider - - * {{{http://jira.springsource.org/browse/BATCH-1522}[BATCH-1522]}} - Intermittent failure of FaultTolerantStepFactoryBean in multi-threaded test - - * {{{http://jira.springsource.org/browse/BATCH-1525}[BATCH-1525]}} - ExitStatus description can be null when re-hyrated from Oracle - - * {{{http://jira.springsource.org/browse/BATCH-1526}[BATCH-1526]}} - Memory leak in web deployments because ThreadLocal is not nulled out in ChunkMonitor - - * {{{http://jira.springsource.org/browse/BATCH-1528}[BATCH-1528]}} - Namespace context partition element requires bean with name "transactionManager" - - * {{{http://jira.springsource.org/browse/BATCH-1533}[BATCH-1533]}} - API change and is not reflected in API and documentation examples - - * {{{http://jira.springsource.org/browse/BATCH-1540}[BATCH-1540]}} - Typo in the user guide: "directlory" - - * {{{http://jira.springsource.org/browse/BATCH-1542}[BATCH-1542]}} - Thread safety in JobExecution and StepExecution collections - - * {{{http://jira.springsource.org/browse/BATCH-1545}[BATCH-1545]}} - FlatFileItemWriter logs as JdbcBatchItemWriter - - * {{{http://jira.springsource.org/browse/BATCH-1546}[BATCH-1546]}} - Some issues with pagination in Oracle - - * {{{http://jira.springsource.org/browse/BATCH-1547}[BATCH-1547]}} - ExecutionContextPromotionListener strict flag misinterpreted in listener code - - * {{{http://jira.springsource.org/browse/BATCH-1551}[BATCH-1551]}} - Db2PagingQueryProvider needs an alias in the jump to subquery - - * {{{http://jira.springsource.org/browse/BATCH-1552}[BATCH-1552]}} - Typo on Use Case page of website - -* Defect - - * {{{http://jira.springsource.org/browse/BATCH-1550}[BATCH-1550]}} - Section 5.2.1 references TaskletAdapter class, but it should be MethodInvokingTaskletAdapter - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-1530}[BATCH-1530]}} - Let job-repository assign id to itself by default. - - * {{{http://jira.springsource.org/browse/BATCH-1531}[BATCH-1531]}} - Restart of a partitioned Step should not call Partitioner - - * {{{http://jira.springsource.org/browse/BATCH-1535}[BATCH-1535]}} - Extend SQL scripts to provide exit code limit of 100 chars - - * {{{http://jira.springsource.org/browse/BATCH-1536}[BATCH-1536]}} - Negative skip count in SkipPolicy - - * {{{http://jira.springsource.org/browse/BATCH-1537}[BATCH-1537]}} - Modify JdbcPagingItemReader to use startAfterValue on restart - - * {{{http://jira.springsource.org/browse/BATCH-1541}[BATCH-1541]}} - Make MapJobRepository work for multi-threaded steps and jobs - - * {{{http://jira.springsource.org/browse/BATCH-1553}[BATCH-1553]}} - Restart of partitioned job should prevent starting multiple step execution instances for a partition - -* New Feature - - * {{{http://jira.springsource.org/browse/BATCH-1307}[BATCH-1307]}} - MapJobRepository with multi-threaded parallel steps (split in job) - - * {{{http://jira.springsource.org/browse/BATCH-1508}[BATCH-1508]}} - Make ApplicationContext ids unique and identifiable in ClasspathXmlApplicationContextsFactoryBean - - * {{{http://jira.springsource.org/browse/BATCH-1527}[BATCH-1527]}} - Add "-stopAll" to CommandLineJobRunner - - * {{{http://jira.springsource.org/browse/BATCH-1529}[BATCH-1529]}} - Explicit protection against exceptions in SkipPolicy to prevent infinite loop in RetryTemplate - - * {{{http://jira.springsource.org/browse/BATCH-1559}[BATCH-1559]}} - Ability to promote FlowStep execution context to its constituent steps - -* Refactoring - - * {{{http://jira.springsource.org/browse/BATCH-1534}[BATCH-1534]}} - JobLauncherTestUtils.setJob should take the interfacetype Job as argument rather than AbstractJob - -* Task - - * {{{http://jira.springsource.org/browse/BATCH-1548}[BATCH-1548]}} - XMLStream Documentation Error - diff --git a/src/site/apt/migration/2.1.0.M1-2.1.0.M2.apt b/src/site/apt/migration/2.1.0.M1-2.1.0.M2.apt deleted file mode 100644 index 77c3373240..0000000000 --- a/src/site/apt/migration/2.1.0.M1-2.1.0.M2.apt +++ /dev/null @@ -1,59 +0,0 @@ -Spring Batch 2.1.0.M2 Release Notes - - -Release Notes - Spring Batch - Version 2.1.0.M2 - -* Sub-task - - * {{{http://jira.springsource.org/browse/BATCH-1400}[BATCH-1400]}} - Specify schema location for imports in xsd - - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1401}[BATCH-1401]}} - All inserts of JobId should be of Types.BIGINT - - * {{{http://jira.springsource.org/browse/BATCH-1420}[BATCH-1420]}} - Late Binding only happens first time when using inner bean definition with collection property - - * {{{http://jira.springsource.org/browse/BATCH-1427}[BATCH-1427]}} - SimpleRetryExceptionHandler treats AbstractStep$FatalException as non-fatal - - * {{{http://jira.springsource.org/browse/BATCH-1432}[BATCH-1432]}} - Error in chpt. 7.1 "Multithreaded step" - - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-1395}[BATCH-1395]}} - Remove or deprecate old application context creation pattern - - * {{{http://jira.springsource.org/browse/BATCH-1398}[BATCH-1398]}} - Initialize ClassPathXmlJobRegistry only after application context is ready - - * {{{http://jira.springsource.org/browse/BATCH-1424}[BATCH-1424]}} - Documentation of annotations in section 5.1.11 needs expanding - - * {{{http://jira.springsource.org/browse/BATCH-1426}[BATCH-1426]}} - RetryTemplate calls BackOffPolicy even when retry will not occur. - - * {{{http://jira.springsource.org/browse/BATCH-1429}[BATCH-1429]}} - Allow DelimitedLineTokenizer to handle malformed file/lines gracefully. - - -* New Feature - - * {{{http://jira.springsource.org/browse/BATCH-1265}[BATCH-1265]}} - Add default value support for Dates in FieldSet - - * {{{http://jira.springsource.org/browse/BATCH-1300}[BATCH-1300]}} - Add description element to and - - * {{{http://jira.springsource.org/browse/BATCH-1399}[BATCH-1399]}} - Expose StepExecutionAggregator as a strategy interface in PartitionStep - - * {{{http://jira.springsource.org/browse/BATCH-1416}[BATCH-1416]}} - Namespace support for max varchar length in execution context and execution daos - - * {{{http://jira.springsource.org/browse/BATCH-1421}[BATCH-1421]}} - Add method to get all job names to JobExplorer - - * {{{http://jira.springsource.org/browse/BATCH-1428}[BATCH-1428]}} - Add support for lobhandler in job repository name space / factory bean - - * {{{http://jira.springsource.org/browse/BATCH-1433}[BATCH-1433]}} - IteratorItemReader - - -* Refactoring - - * {{{http://jira.springsource.org/browse/BATCH-1369}[BATCH-1369]}} - Change semantics of @AfterChunk to execute outside transaction - - * {{{http://jira.springsource.org/browse/BATCH-1419}[BATCH-1419]}} - Change is-reader-transactional-queue to reader-transactional in - - - diff --git a/src/site/apt/migration/2.1.0.M2-2.1.0.M3.apt b/src/site/apt/migration/2.1.0.M2-2.1.0.M3.apt deleted file mode 100644 index 0cf7790b89..0000000000 --- a/src/site/apt/migration/2.1.0.M2-2.1.0.M3.apt +++ /dev/null @@ -1,68 +0,0 @@ -Spring Batch 2.1.0.M3 Release Notes - - -Release Notes - Spring Batch - Version 2.1.0.M3 - -* Sub-task - - * {{{http://jira.springsource.org/browse/BATCH-1439}[BATCH-1439]}} - Make step names unique inside a job - - * {{{http://jira.springsource.org/browse/BATCH-1443}[BATCH-1443]}} - Add JobStep: a Step implementation that executes a Job - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1442}[BATCH-1442]}} - Stopping a job with two parallel steps is only stopping one of the steps - - * {{{http://jira.springsource.org/browse/BATCH-1444}[BATCH-1444]}} - ChunkMonitor warning message about stream state is inaccurate - - * {{{http://jira.springsource.org/browse/BATCH-1452}[BATCH-1452]}} - Stream closed exception when combining MultiResourceItemWriter and FlatFileItemWriter with footer callback - - * {{{http://jira.springsource.org/browse/BATCH-1453}[BATCH-1453]}} - OraclePagingQueryProvider generates wrong queries for pages - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-509}[BATCH-509]}} - Add support for dates in ExecutionContext - - * {{{http://jira.springsource.org/browse/BATCH-981}[BATCH-981]}} - Use Woodstox instead of ref impl of StaX - - * {{{http://jira.springsource.org/browse/BATCH-1281}[BATCH-1281]}} - Make ItemProcessor semantics optionally non-transactional and run-once - - * {{{http://jira.springsource.org/browse/BATCH-1316}[BATCH-1316]}} - JobParameters throws NullPointerException on missing key - - * {{{http://jira.springsource.org/browse/BATCH-1434}[BATCH-1434]}} - Corner case: job with step that repeats via a transition in a flow only executes the step once - - * {{{http://jira.springsource.org/browse/BATCH-1448}[BATCH-1448]}} - BeanWrapperFieldSetMapper should throw BindException (so the errors can be accessed) - - * {{{http://jira.springsource.org/browse/BATCH-1449}[BATCH-1449]}} - Option to disable transactional behavior of FlatFileItemWriter - -* New Feature - - * {{{http://jira.springsource.org/browse/BATCH-1246}[BATCH-1246]}} - Add support for a JSON Reader from text files which are JSON formatted - - * {{{http://jira.springsource.org/browse/BATCH-1344}[BATCH-1344]}} - Allow autowired to be disabled in AbstractJobTests - - * {{{http://jira.springsource.org/browse/BATCH-1347}[BATCH-1347]}} - Restartable look-ahead (peekable) ItemReader - - * {{{http://jira.springsource.org/browse/BATCH-1380}[BATCH-1380]}} - Make a top-level element, so it can be shared or strategised in a job - - * {{{http://jira.springsource.org/browse/BATCH-1411}[BATCH-1411]}} - Allow a Job to specify its required JobParameters - - * {{{http://jira.springsource.org/browse/BATCH-1436}[BATCH-1436]}} - Allow inner bean for configuration of tasklet - - * {{{http://jira.springsource.org/browse/BATCH-1437}[BATCH-1437]}} - Support for CallbackPreferringPlatformTransactionManager (and for native TX in WAS) - - * {{{http://jira.springsource.org/browse/BATCH-1446}[BATCH-1446]}} - Add convenience method to ExitStatus for capturing stack trace - - * {{{http://jira.springsource.org/browse/BATCH-1447}[BATCH-1447]}} - Ensure that JobLauncher fails a job if the task executor cannot run it - - * {{{http://jira.springsource.org/browse/BATCH-1454}[BATCH-1454]}} - Create a standard component that can poll for a step or job finishing - -Refactoring - - * {{{http://jira.springsource.org/browse/BATCH-1394}[BATCH-1394]}} - Migrate StepScope to Spring 3.0.0 - - * {{{http://jira.springsource.org/browse/BATCH-1450}[BATCH-1450]}} - Make Map Daos store data in instance variables (instead of static) - -* Task - - * {{{http://jira.springsource.org/browse/BATCH-1451}[BATCH-1451]}} - Upgrade Hibernate to 3.3 diff --git a/src/site/apt/migration/2.1.0.M3-2.1.0.M4.apt b/src/site/apt/migration/2.1.0.M3-2.1.0.M4.apt deleted file mode 100644 index c1b7636b09..0000000000 --- a/src/site/apt/migration/2.1.0.M3-2.1.0.M4.apt +++ /dev/null @@ -1,36 +0,0 @@ -Spring Batch 2.1.0.M4 Release Notes - - -Release Notes - Spring Batch - Version 2.1.0.M4 - -* Sub-task - - * {{{http://jira.springsource.org/browse/BATCH-1456}[BATCH-1456]}} - Detect spring-batch-2.0.xsd and raise an exception in the namespace parsers - - * {{{http://jira.springsource.org/browse/BATCH-1458}[BATCH-1458]}} - Add FlowStep: a Step implementation that executes a flow - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1460}[BATCH-1460]}} - Rownum clauses are illegal in DerbyPagingQueryProvider (plus additional fix to Oracle) - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-585}[BATCH-585]}} - Improve coverage of examples of listener implementations in user guide - - * {{{http://jira.springsource.org/browse/BATCH-924}[BATCH-924]}} - BeanWrapperFieldSetMapper could create child objects if they are null when needed - - * {{{http://jira.springsource.org/browse/BATCH-1259}[BATCH-1259]}} - Ensure that skips in the processor cause items to be reprocessed at most once. - -* New Feature - - * {{{http://jira.springsource.org/browse/BATCH-992}[BATCH-992]}} - Alternative fault-tolerant step implementation - - * {{{http://jira.springsource.org/browse/BATCH-1372}[BATCH-1372]}} - Namespace support for partitioning - - * {{{http://jira.springsource.org/browse/BATCH-1407}[BATCH-1407]}} - Integration tests for core (including multi-threaded long running tests) - -* Task - - * {{{http://jira.springsource.org/browse/BATCH-834}[BATCH-834]}} - Review stop signal handling at JobExecution - - * {{{http://jira.springsource.org/browse/BATCH-1457}[BATCH-1457]}} - remove tiger profile from archetype's pom.xml diff --git a/src/site/apt/migration/2.1.0.M4-2.1.0.RC1.apt b/src/site/apt/migration/2.1.0.M4-2.1.0.RC1.apt deleted file mode 100644 index 1c7e627bd8..0000000000 --- a/src/site/apt/migration/2.1.0.M4-2.1.0.RC1.apt +++ /dev/null @@ -1,97 +0,0 @@ -Spring Batch 2.1.0.RC1 Release Notes - -* Sub-task - - * {{{http://jira.springsource.org/browse/BATCH-873}[BATCH-873]}} - Ensure SkipPolicies are generic and extendable - - * {{{http://jira.springsource.org/browse/BATCH-929}[BATCH-929]}} - Deferrable Constraints cause unrecoverable errors - - * {{{http://jira.springsource.org/browse/BATCH-1382}[BATCH-1382]}} - Publish spring-batch-2.1.xsd - - * {{{http://jira.springsource.org/browse/BATCH-1387}[BATCH-1387]}} - Fix validation errors in core and samples - - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1371}[BATCH-1371]}} - Rename spring-batch-2.0.xsd to spring-batch-2.1.xsd - - * {{{http://jira.springsource.org/browse/BATCH-1418}[BATCH-1418]}} - Upon job restart, step with FlatFileItemReader doesn't honor skippable-exception-classes - - * {{{http://jira.springsource.org/browse/BATCH-1422}[BATCH-1422]}} - HibernateCursorItemReader causes OutOfMemoryError when skipping large sets of data - - * {{{http://jira.springsource.org/browse/BATCH-1463}[BATCH-1463]}} - Late binding broken for inner beans with Spring 3. - - * {{{http://jira.springsource.org/browse/BATCH-1467}[BATCH-1467]}} - Samples: environment switching does not work on command line (pom typo) - - * {{{http://jira.springsource.org/browse/BATCH-1469}[BATCH-1469]}} - org.apache.commons.lang.SerializationException: java.lang.ClassNotFoundException using samples in WebLogic - - * {{{http://jira.springsource.org/browse/BATCH-1471}[BATCH-1471]}} - Typo in FaultTolerantStepFactoryBean - - * {{{http://jira.springsource.org/browse/BATCH-1473}[BATCH-1473]}} - Components that create application contexts should look for *Aware in the infrastructure beans it copies down to the child context - - * {{{http://jira.springsource.org/browse/BATCH-1476}[BATCH-1476]}} - Filter counts too high when write skips happen - - * {{{http://jira.springsource.org/browse/BATCH-1477}[BATCH-1477]}} - Allow excludes (as well as includes) in retryable exceptions - - * {{{http://jira.springsource.org/browse/BATCH-1482}[BATCH-1482]}} - SaveState default value wrong in reference guide - - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-1340}[BATCH-1340]}} - Add Support for Native Queries to Jpa/Hibernate readers - - * {{{http://jira.springsource.org/browse/BATCH-1374}[BATCH-1374]}} - Mention the JobRegistryBeanPostProcessor in the reference guide - - * {{{http://jira.springsource.org/browse/BATCH-1376}[BATCH-1376]}} - CommandLineJobRunner -restart should accept executionId as parameter. - - * {{{http://jira.springsource.org/browse/BATCH-1462}[BATCH-1462]}} - MultiResourceItemReader / delegate should skip directories - - * {{{http://jira.springsource.org/browse/BATCH-1464}[BATCH-1464]}} - Allow for use of PersistenceExceptionTranslator on JpaItemWriter - - * {{{http://jira.springsource.org/browse/BATCH-1465}[BATCH-1465]}} - Unwrap exceptions in AbstractMethodInvokingDelegator - - * {{{http://jira.springsource.org/browse/BATCH-1472}[BATCH-1472]}} - Allow for use of PersistenceExceptionTranslator on JpaItemWriter - - * {{{http://jira.springsource.org/browse/BATCH-1478}[BATCH-1478]}} - Add auto registration of StepScope to all the top-level elements in XML - - -* New Feature - - * {{{http://jira.springsource.org/browse/BATCH-729}[BATCH-729]}} - Supporting stored procedures in an ItemReader - - * {{{http://jira.springsource.org/browse/BATCH-1239}[BATCH-1239]}} - Add email-sending item writer - - * {{{http://jira.springsource.org/browse/BATCH-1377}[BATCH-1377]}} - StaxEventItemWriter: Handle namespace for the root tag - - * {{{http://jira.springsource.org/browse/BATCH-1430}[BATCH-1430]}} - StaxEventItemWriter: Declare additional namespaces at the top-level element - - * {{{http://jira.springsource.org/browse/BATCH-1441}[BATCH-1441]}} - BackOffPolicy cannot be set on a parent of an XML (namespaced) 'step' - - * {{{http://jira.springsource.org/browse/BATCH-1468}[BATCH-1468]}} - StaxEventItemReader: optionally provide QName or namespace declaration for fragment element - - * {{{http://jira.springsource.org/browse/BATCH-1474}[BATCH-1474]}} - Add registry management features to JobLoader - - * {{{http://jira.springsource.org/browse/BATCH-1475}[BATCH-1475]}} - Allow a JobParametersValidator to be specified as an inner bean definition (as well as a reference) - - * {{{http://jira.springsource.org/browse/BATCH-1479}[BATCH-1479]}} - Make AbstractMethodInvokingDelegate more lax in the types of arguments it accepts - - * {{{http://jira.springsource.org/browse/BATCH-1480}[BATCH-1480]}} - Allow method adapter for Tasklet to be configured in namespace - - * {{{http://jira.springsource.org/browse/BATCH-1481}[BATCH-1481]}} - Support injection of step-scoped dependencies into unit tests - - * {{{http://jira.springsource.org/browse/BATCH-1483}[BATCH-1483]}} - Make the JobParametersValidator instance accessible from the Job interface - - * {{{http://jira.springsource.org/browse/BATCH-1484}[BATCH-1484]}} - Add a getter for JobParametersValidator in AbstractJob - - * {{{http://jira.springsource.org/browse/BATCH-1486}[BATCH-1486]}} - Add HibernatePagingItemReader - - -* Task - - * {{{http://jira.springsource.org/browse/BATCH-724}[BATCH-724]}} - create tests for non-default table prefix - - * {{{http://jira.springsource.org/browse/BATCH-1466}[BATCH-1466]}} - Migrate Spring Batch Integration into the Admin project - - * {{{http://jira.springsource.org/browse/BATCH-1470}[BATCH-1470]}} - Remove commons-lang dependency - - diff --git a/src/site/apt/migration/2.1.0.RC1-2.1.0.apt b/src/site/apt/migration/2.1.0.RC1-2.1.0.apt deleted file mode 100755 index dee47c0d1b..0000000000 --- a/src/site/apt/migration/2.1.0.RC1-2.1.0.apt +++ /dev/null @@ -1,34 +0,0 @@ -Spring Batch 2.1.0 Release Notes - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1490}[BATCH-1490]}} - StaxEventItemWriter outputs invalid xml if step handling is failed and retried when handling the first chunk of data - - * {{{http://jira.springsource.org/browse/BATCH-1494}[BATCH-1494]}} - FetchSize not accessible in HibernateCursorItemReader - - * {{{http://jira.springsource.org/browse/BATCH-1497}[BATCH-1497]}} - SqlServerPagingQueryProvider should use an alias for the inner query in a jump-to-item query - - * {{{http://jira.springsource.org/browse/BATCH-1498}[BATCH-1498]}} - JdbcPagingItemReader does not apply parameter values correctly on restart - - * {{{http://jira.springsource.org/browse/BATCH-1499}[BATCH-1499]}} - SqlServerPagingQueryProvider needs an alias in the jump to subquery - - * {{{http://jira.springsource.org/browse/BATCH-1502}[BATCH-1502]}} - HibernatePagingItemReader doesn't close sessions - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-1402}[BATCH-1402]}} - MapJobRegistry throws DuplicateJobException when the same job factory instance is registered twice - - * {{{http://jira.springsource.org/browse/BATCH-1493}[BATCH-1493]}} - Step listeners detected and invoked twice - -* New Feature - - * {{{http://jira.springsource.org/browse/BATCH-1496}[BATCH-1496]}} - Expose retry-policy in namespace configuration - -* Refactoring - - * {{{http://jira.springsource.org/browse/BATCH-1393}[BATCH-1393]}} - Migrate StepScope to Spring 3.0.0 - -* Task - - * {{{http://jira.springsource.org/browse/BATCH-1488}[BATCH-1488]}} - Tests and documentation for StoredProcedureItemReader - diff --git a/src/site/apt/migration/2.1.1-2.1.2.apt b/src/site/apt/migration/2.1.1-2.1.2.apt deleted file mode 100644 index 3702803dc4..0000000000 --- a/src/site/apt/migration/2.1.1-2.1.2.apt +++ /dev/null @@ -1,47 +0,0 @@ -Spring Batch 2.1.2 Release Notes - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1543}[BATCH-1543]}} - OrderedComposite cannot register two items with the same order - - * {{{http://jira.springsource.org/browse/BATCH-1558}[BATCH-1558]}} - Error in Programmatic Flow Decisions documentation - - * {{{http://jira.springsource.org/browse/BATCH-1564}[BATCH-1564]}} - incorrect apachemq artifactId - - * {{{http://jira.springsource.org/browse/BATCH-1566}[BATCH-1566]}} - ExecutionContext.isDirty() is not very accurate - - * {{{http://jira.springsource.org/browse/BATCH-1567}[BATCH-1567]}} - When step encounters error saving ExecutionContext it tries to stop the job but fails - - * {{{http://jira.springsource.org/browse/BATCH-1571}[BATCH-1571]}} - PostgresPagingQueryProvider generateJumpToItemQuery generates bad SQL - - * {{{http://jira.springsource.org/browse/BATCH-1573}[BATCH-1573]}} - End transition states will cause the batch job to finish with an Unknown status if the namespace prefix is used. - - * {{{http://jira.springsource.org/browse/BATCH-1574}[BATCH-1574]}} - TaskExecutor configuration ignored in 2.1 namespace for with no - - * {{{http://jira.springsource.org/browse/BATCH-1587}[BATCH-1587]}} - DefaultFieldSetFactory is not setting the numberFormat in the enhance() call - - * {{{http://jira.springsource.org/browse/BATCH-1588}[BATCH-1588]}} - Job Excecution Listener - XML Namespace parsing fails for methods named different to "beforeJob", "afterJob" - - * {{{http://jira.springsource.org/browse/BATCH-1590}[BATCH-1590]}} - OraclePagingQueryProvider.generateJumpToItemQuery generates an incorrect query - - * {{{http://jira.springsource.org/browse/BATCH-1591}[BATCH-1591]}} - Documentation: StepListener is a marker interface so it cannot be applied directly to a step - - * {{{http://jira.springsource.org/browse/BATCH-1594}[BATCH-1594]}} - StepListenerSupport implements method onErrorInStep which is not declared in any of the implemented interfaces - -* Defect - - * {{{http://jira.springsource.org/browse/BATCH-1569}[BATCH-1569]}} - MultiResourceItemReader.getCurrentResource cause java.lang.ArrayIndexOutOfBoundsException when .read() was not called - - * {{{http://jira.springsource.org/browse/BATCH-1582}[BATCH-1582]}} - DefaultStepExecutionAggregator can simply ignore null or empty input - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-1568}[BATCH-1568]}} - Implement SkipListener in StepListenerSupport - - * {{{http://jira.springsource.org/browse/BATCH-1575}[BATCH-1575]}} - Parallel job test case missing in 2.1 - - * {{{http://jira.springsource.org/browse/BATCH-1578}[BATCH-1578]}} - Mismatch between JavaDoc and enum elements in BatchStatus - - * {{{http://jira.springsource.org/browse/BATCH-1589}[BATCH-1589]}} - Have RunIDIncrementer reuse given parameters - - * {{{http://jira.springsource.org/browse/BATCH-1593}[BATCH-1593]}} - XML configuration: p-namespace does not work on inline tasklet bean element diff --git a/src/site/apt/migration/2.1.2-2.1.3.apt b/src/site/apt/migration/2.1.2-2.1.3.apt deleted file mode 100644 index 6afb34a4da..0000000000 --- a/src/site/apt/migration/2.1.2-2.1.3.apt +++ /dev/null @@ -1,59 +0,0 @@ -Spring Batch 2.1.3 Release Notes - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1572}[BATCH-1572]}} - Step not failing on org.springframework.transaction.UnexpectedRollbackException - - * {{{http://jira.springsource.org/browse/BATCH-1579}[BATCH-1579]}} - Problems with null job parameters and late binding - - * {{{http://jira.springsource.org/browse/BATCH-1597}[BATCH-1597]}} - DirectPoller only works with timeout in milliseconds - - * {{{http://jira.springsource.org/browse/BATCH-1598}[BATCH-1598]}} - JobRepositoryTestUtils delete job execution fails if there is another execution with the same job instance - - * {{{http://jira.springsource.org/browse/BATCH-1600}[BATCH-1600]}} - CommandLineJobRunner cannot stop a Job execution that was restarted - - * {{{http://jira.springsource.org/browse/BATCH-1601}[BATCH-1601]}} - The "initialized" field in org.springframework.batch.test.DataSourceInitializer shouldn't be static. - - * {{{http://jira.springsource.org/browse/BATCH-1602}[BATCH-1602]}} - Empty string JobParameter would be re-hydrated as null by Oracle - - * {{{http://jira.springsource.org/browse/BATCH-1603}[BATCH-1603]}} - MultiResourceItemReader infinite read/exception loop after a failed delegate.open() when skip policy is very lax - - * {{{http://jira.springsource.org/browse/BATCH-1605}[BATCH-1605]}} - HippyMethodInvoker candidate arguments repeated - - * {{{http://jira.springsource.org/browse/BATCH-1615}[BATCH-1615]}} - MultiResourceItemReader infinite read/exception loop after NonTransientDataAccessResourceException - - * {{{http://jira.springsource.org/browse/BATCH-1616}[BATCH-1616]}} - A custom partitioner no longer restart the job properly upon failure - - * {{{http://jira.springsource.org/browse/BATCH-1618}[BATCH-1618]}} - MultiResourceItemWriter creates an empty file if the number of item to write is a multiple of itemCountLimitPerResource - - * {{{http://jira.springsource.org/browse/BATCH-1619}[BATCH-1619]}} - BadSqlGrammarException accessing Executions page with Apache Derby 10.6 datasource - - * {{{http://jira.springsource.org/browse/BATCH-1620}[BATCH-1620]}} - FlowStep never fails - - * {{{http://jira.springsource.org/browse/BATCH-1621}[BATCH-1621]}} - Add attribute - - * {{{http://jira.springsource.org/browse/BATCH-1623}[BATCH-1623]}} - A chunk configured with processor-transactional="true" shouldn't require a retry- or skip-limit - - * {{{http://jira.springsource.org/browse/BATCH-1624}[BATCH-1624]}} - Spring Batch Website documentation - fix link to Spring Integration - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-836}[BATCH-836]}} - CompositeItemWriter should also implement ItemStream - - * {{{http://jira.springsource.org/browse/BATCH-1531}[BATCH-1531]}} - Restart of a partitioned Step should not call Partitioner - - * {{{http://jira.springsource.org/browse/BATCH-1556}[BATCH-1556]}} - JobLauncher sequence diagrams have wrong message (call) labels - - * {{{http://jira.springsource.org/browse/BATCH-1604}[BATCH-1604]}} - Update docs to describe use of Unicode characters in JobRepository - - * {{{http://jira.springsource.org/browse/BATCH-1608}[BATCH-1608]}} - Javadocs: declare that readDate might throw NullPointerException when value is empty - - * {{{http://jira.springsource.org/browse/BATCH-1609}[BATCH-1609]}} - Javadocs: DefaultFieldSet.readAndTrim does not throw a NullPointerException as stated by the Javadoc - - * {{{http://jira.springsource.org/browse/BATCH-1614}[BATCH-1614]}} - More informative log message for ClassPathXmlApplicationContextFactory.ResourceXmlApplicationContext - -* New Feature - - * {{{http://jira.springsource.org/browse/BATCH-1495}[BATCH-1495]}} - Detect possible overrun and memory issues when skip limit is not reached but reader skips so many records that the chunk never completes - - * {{{http://jira.springsource.org/browse/BATCH-1622}[BATCH-1622]}} - Support transaction propagation properly in ResourcelessTransactionManager diff --git a/src/site/apt/migration/2.1.3-2.1.4.apt b/src/site/apt/migration/2.1.3-2.1.4.apt deleted file mode 100644 index 504be97911..0000000000 --- a/src/site/apt/migration/2.1.3-2.1.4.apt +++ /dev/null @@ -1,55 +0,0 @@ -Spring Batch 2.1.4 Release Notes - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1572}[BATCH-1572]}} - Step not failing on org.springframework.transaction.UnexpectedRollbackException - - * {{{http://jira.springsource.org/browse/BATCH-1632}[BATCH-1632]}} - DefaultFieldSet#readBigDecimal(String, BigDecimal) and NumberFormatException - - * {{{http://jira.springsource.org/browse/BATCH-1633}[BATCH-1633]}} - Dependency injection problem with step scoped anonymous inner bean - - * {{{http://jira.springsource.org/browse/BATCH-1637}[BATCH-1637]}} - OraclePagingQueryProvider adds an extra column to the jump to page query - - * {{{http://jira.springsource.org/browse/BATCH-1638}[BATCH-1638]}} - Possible NullPointerException when using no-rollback - - * {{{http://jira.springsource.org/browse/BATCH-1639}[BATCH-1639]}} - Oracle jumpToItemQuery needs a tweak (again) - - * {{{http://jira.springsource.org/browse/BATCH-1640}[BATCH-1640]}} - File writers do not behave correctly on rollback - - * {{{http://jira.springsource.org/browse/BATCH-1643}[BATCH-1643]}} - Unpredictable binding in BeanWrapperFieldSetMapper because of "fuzzy" property matching - - * {{{http://jira.springsource.org/browse/BATCH-1647}[BATCH-1647]}} - Inner bean with a late binding parameter in the constructor does not work - - * {{{http://jira.springsource.org/browse/BATCH-1648}[BATCH-1648]}} - Paging query for IbatisPagingItemReader in reference documentation is incorrect - - - -* Defect - - - * {{{http://jira.springsource.org/browse/BATCH-1629}[BATCH-1629]}} - FaultTolerantChunkProcessor contains dangerous log statements - - * {{{http://jira.springsource.org/browse/BATCH-1631}[BATCH-1631]}} - Typos on Spring Batch home page - - - -* Improvement - - - * {{{http://jira.springsource.org/browse/BATCH-1627}[BATCH-1627]}} - SpringValidator - include Binding Results with the thrown ValidationException - - * {{{http://jira.springsource.org/browse/BATCH-1630}[BATCH-1630]}} - Add defensive logging conditionals in performance critical areas of the code base. - - * {{{http://jira.springsource.org/browse/BATCH-1634}[BATCH-1634]}} - Documentation: custom tableprefix for JobExplorer - - * {{{http://jira.springsource.org/browse/BATCH-1636}[BATCH-1636]}} - AutomaticJobRegistar should implement the Ordered interface - - * {{{http://jira.springsource.org/browse/BATCH-1650}[BATCH-1650]}} - Override GroupAwareJob#toString method - - - -* New Feature - - - * {{{http://jira.springsource.org/browse/BATCH-1628}[BATCH-1628]}} - Add job parameters via properties file in CommandLineJobRunner - diff --git a/src/site/apt/migration/2.1.4-2.1.5.apt b/src/site/apt/migration/2.1.4-2.1.5.apt deleted file mode 100644 index f1d6839be1..0000000000 --- a/src/site/apt/migration/2.1.4-2.1.5.apt +++ /dev/null @@ -1,5 +0,0 @@ -Spring Batch 2.1.5 Release Notes - -* Bug - - * {{{https://jira.springframework.org/browse/BATCH-1683}[BATCH-1683]}} - CommandLineJobRunner hangs waiting for input on stdin diff --git a/src/site/apt/migration/2.1.5-2.1.6.apt b/src/site/apt/migration/2.1.5-2.1.6.apt deleted file mode 100644 index 3423b3da51..0000000000 --- a/src/site/apt/migration/2.1.5-2.1.6.apt +++ /dev/null @@ -1,54 +0,0 @@ -Spring Batch 2.1.6 Release Notes - -* Sub-task - - * {{{https://jira.springframework.org/browse/BATCH-1672}[BATCH-1672]}} - When appendAllowed is true, file is not created (in the first time). - - * {{{https://jira.springframework.org/browse/BATCH-1687}[BATCH-1687]}} - Fix documentation for Step Listeners - - * {{{https://jira.springframework.org/browse/BATCH-1688}[BATCH-1688]}} - Add to and - -* Bug - - * {{{https://jira.springframework.org/browse/BATCH-1635}[BATCH-1635]}} - Spring Batch and Hibernate Search do not work together - - * {{{https://jira.springframework.org/browse/BATCH-1654}[BATCH-1654]}} - StepExecution.equals() should consider id as well as name and jobExecution - - * {{{https://jira.springframework.org/browse/BATCH-1656}[BATCH-1656]}} - Infinite loop on no-rollback-for exception when skipLimit is reached due to exception in ItemProcessor - - * {{{https://jira.springframework.org/browse/BATCH-1657}[BATCH-1657]}} - ORDER BY clause missing from paging queries for Derby - - * {{{https://jira.springframework.org/browse/BATCH-1659}[BATCH-1659]}} - FileUtils setUpOutputFile fails on NAS from linux - - * {{{https://jira.springframework.org/browse/BATCH-1670}[BATCH-1670]}} - Nested splits lead to invalid flow definition - - * {{{https://jira.springframework.org/browse/BATCH-1671}[BATCH-1671]}} - static methods are not public in ExecutionContextTestUtils - - * {{{https://jira.springframework.org/browse/BATCH-1680}[BATCH-1680]}} - Simple cli sample hangs because of wrong dependency - - * {{{https://jira.springframework.org/browse/BATCH-1681}[BATCH-1681]}} - Restarting a job that generates XML output using StaxEventItemWriter with Woodstox fails - -* Defect - - * {{{https://jira.springframework.org/browse/BATCH-1677}[BATCH-1677]}} - Error Documentation in pointcut expression declaration - - * {{{https://jira.springframework.org/browse/BATCH-1678}[BATCH-1678]}} - Inheriting from a Parent Job - -* Improvement - - * {{{https://jira.springframework.org/browse/BATCH-1661}[BATCH-1661]}} - FlatFileItemReader always logs as ERROR non data lines even though the row should be skipped - - * {{{https://jira.springframework.org/browse/BATCH-1662}[BATCH-1662]}} - FieldSetFactory inconsistent parameter ordering create method - - * {{{https://jira.springframework.org/browse/BATCH-1665}[BATCH-1665]}} - Provide MultiResourceItemReader property controlling whether or not it fails when there are "no resources to read" - - * {{{https://jira.springframework.org/browse/BATCH-1668}[BATCH-1668]}} - Check for existing transaction when job is started (and fail if present by default) - -* New Feature - - * {{{https://jira.springframework.org/browse/BATCH-1224}[BATCH-1224]}} - Appending functionality in FlatFileItemwriter - - * {{{https://jira.springframework.org/browse/BATCH-1509}[BATCH-1509]}} - Allow a step inside a partition to be specified as an inner bean definition (as well as a reference) - - * {{{https://jira.springframework.org/browse/BATCH-1514}[BATCH-1514]}} - Support for the tag in the partition element - diff --git a/src/site/apt/migration/2.1.6-2.1.7.apt b/src/site/apt/migration/2.1.6-2.1.7.apt deleted file mode 100644 index 505d009cb3..0000000000 --- a/src/site/apt/migration/2.1.6-2.1.7.apt +++ /dev/null @@ -1,36 +0,0 @@ -Spring Batch 2.1.7 Release Notes - -* Bug - - - * {{{https://jira.springframework.org/browse/BATCH-1705}[BATCH-1705]}} - CommandLineJobRunner fails if standard input not available - - * {{{https://jira.springframework.org/browse/BATCH-1707}[BATCH-1707]}} - MapJobInstanceDao.getJobInstances(String jobName, int start, int count) does not work - - * {{{https://jira.springframework.org/browse/BATCH-1709}[BATCH-1709]}} - BeanWrapperFieldSetMapper race condition in cache - - * {{{https://jira.springframework.org/browse/BATCH-1712}[BATCH-1712]}} - Inline step definitions clash if multiple instances share a TaskExecutorPartitionHandler - - * {{{https://jira.springframework.org/browse/BATCH-1717}[BATCH-1717]}} - Failure in RetryPolicy leads to infinite loop in Step - -* Improvement - - - * {{{https://jira.springframework.org/browse/BATCH-1693}[BATCH-1693]}} - Add RemoteStepExecutionAggregator to update step executions from the repository during partition processing - - * {{{https://jira.springframework.org/browse/BATCH-1713}[BATCH-1713]}} - The step execution context is not deserialized by default and no API to do it effectively - -* New Feature - - - * {{{https://jira.springframework.org/browse/BATCH-1396}[BATCH-1396]}} - Late binding of commit-interval, retry-limit, skip-limit, e.g. bound from job parameters. - - * {{{https://jira.springframework.org/browse/BATCH-1696}[BATCH-1696]}} - DelimitedLineTokenizer always trims the input data - - * {{{https://jira.springframework.org/browse/BATCH-1708}[BATCH-1708]}} - Inefficient (and unnecessary?) locking in TaskletStep and CompositeItemStream - -* Refactoring - - - * {{{https://jira.springframework.org/browse/BATCH-1532}[BATCH-1532]}} - Use Spring 3.0 OXM instead of SWS 1.5 - diff --git a/src/site/apt/migration/2.1.7-2.1.8.apt b/src/site/apt/migration/2.1.7-2.1.8.apt deleted file mode 100644 index b719f761ab..0000000000 --- a/src/site/apt/migration/2.1.7-2.1.8.apt +++ /dev/null @@ -1,39 +0,0 @@ -Spring Batch 2.1.8 Release Notes - - -* Bug - - * {{{https://jira.springframework.org/browse/BATCH-1725}[BATCH-1725]}} - SubclassClassifier should use ConcurrentHashMap - - * {{{https://jira.springframework.org/browse/BATCH-1727}[BATCH-1727]}} - Child contexts created by AutomaticJobRegistrar cannot easily use PropertyPlaceholderConfigurer - - * {{{https://jira.springframework.org/browse/BATCH-1738}[BATCH-1738]}} - StaxEventItemReader stops reading when exception occurs during unmarshalling - - * {{{https://jira.springframework.org/browse/BATCH-1739}[BATCH-1739]}} - Inheriting from parent step with skip-limit/retry-limit causes IllegalArgumentException when the inheriting bean doesn't define exception-classes. - - * {{{https://jira.springframework.org/browse/BATCH-1742}[BATCH-1742]}} - HippyMethodInvoker fails when target uses method overloading and there is no exact match for arguments - - * {{{https://jira.springframework.org/browse/BATCH-1743}[BATCH-1743]}} - Use step scope for PartitionHandler (so gridSize can be a job parameter) - broken in 2.1.7. - - * {{{https://jira.springframework.org/browse/BATCH-1744}[BATCH-1744]}} - Revert retry-limit and skip-limit changes from BATCH-1396. - -* Improvement - - * {{{https://jira.springframework.org/browse/BATCH-1316}[BATCH-1316]}} - JobParameters throws NullPointerException on missing key - - * {{{https://jira.springframework.org/browse/BATCH-1719}[BATCH-1719]}} - The step execution context is not deserialized for getJobExecutions() and findRunningJobExecutions() - - * {{{https://jira.springframework.org/browse/BATCH-1721}[BATCH-1721]}} - Replace + inside of StringBuilder.append() call in FlatFileItemWriter - - * {{{https://jira.springframework.org/browse/BATCH-1722}[BATCH-1722]}} - ExecutionContextUserSupport refers to ItemStream in code and documentation - - * {{{https://jira.springframework.org/browse/BATCH-1730}[BATCH-1730]}} - Flow based step cannot be repeated/ restarted within a partition - -* New Feature - - * {{{https://jira.springframework.org/browse/BATCH-1737}[BATCH-1737]}} - DelimitedLineTokenizer. Specify which fields from a row you want to read - -* Pruning - - * {{{https://jira.springframework.org/browse/BATCH-1741}[BATCH-1741]}} - Update javadoc for HibernateItemWriter - diff --git a/src/site/apt/migration/2.1.8-2.1.9.apt b/src/site/apt/migration/2.1.8-2.1.9.apt deleted file mode 100644 index 68f8bb5c14..0000000000 --- a/src/site/apt/migration/2.1.8-2.1.9.apt +++ /dev/null @@ -1,81 +0,0 @@ -Spring Batch 2.1.9 Release Notes - -* Bug - - * {{{https://jira.springframework.org/browse/BATCH-1751}[BATCH-1751]}} - Not possible to use property-placeholder values in batch-attributes - - * {{{https://jira.springframework.org/browse/BATCH-1756}[BATCH-1756]}} - Make round-trip <<>> work for double parameters - - * {{{https://jira.springframework.org/browse/BATCH-1761}[BATCH-1761]}} - Only first item in chunk is re-processed on retry of failed write - - * {{{https://jira.springframework.org/browse/BATCH-1772}[BATCH-1772]}} - missing closing xml tag in spring batch html user guide: 5.1.1. Configuring a Step - - * {{{https://jira.springframework.org/browse/BATCH-1775}[BATCH-1775]}} - Inner beans of same type inside <<<>>> elements with scope ="step" leads to mistaken override of bean definitions - - * {{{https://jira.springframework.org/browse/BATCH-1776}[BATCH-1776]}} - Batch Src Build unable to find FoundryLogic.vpp - - * {{{https://jira.springframework.org/browse/BATCH-1783}[BATCH-1783]}} - Throwing exceptions inside a ChunkListener results in endless loop - - * {{{https://jira.springframework.org/browse/BATCH-1798}[BATCH-1798]}} - MultiResourceItemReader fails on Restart if read() method was not called. - - * {{{https://jira.springframework.org/browse/BATCH-1804}[BATCH-1804]}} - Retry does not work if additional exception occurs in the ItemWriter during scan for failure - - * {{{https://jira.springframework.org/browse/BATCH-1812}[BATCH-1812]}} - ItemWriteListener does not work as expected, not called when writer runs in "recoverer" - - * {{{https://jira.springframework.org/browse/BATCH-1813}[BATCH-1813]}} - BeanWrapperFieldSetMapper properties caching is broken - - * {{{https://jira.springframework.org/browse/BATCH-1821}[BATCH-1821]}} - Possible mistake in current batch documentation - - * {{{https://jira.springframework.org/browse/BATCH-1822}[BATCH-1822]}} - Job execution marked as STOPPED when exception occurs while committing StepExecution - - * {{{https://jira.springframework.org/browse/BATCH-1826}[BATCH-1826]}} - Null pointer exception if optional parameter of type DATE is null - - * {{{https://jira.springframework.org/browse/BATCH-1840}[BATCH-1840]}} - job execution continues when step is in status unknown - - * {{{https://jira.springframework.org/browse/BATCH-1841}[BATCH-1841]}} - Upgrading to spring batch 2.1.8 causes error in processing xml configuration - - * {{{https://jira.springframework.org/browse/BATCH-1848}[BATCH-1848]}} - JdbcPagingItemReader does not support table or column aliases due to sortKey being used in where clause, order by clause and for retrieval of result set column - - * {{{https://jira.springframework.org/browse/BATCH-1852}[BATCH-1852]}} - Very quick (lt 1ms) jobs are poorly identified - -* Defect - - * {{{https://jira.springframework.org/browse/BATCH-1753}[BATCH-1753]}} - Problems With FlatFileItemWriter: error while trying to restart an execution - -* Improvement - - * {{{https://jira.springframework.org/browse/BATCH-1760}[BATCH-1760]}} - JobConfigurationRegistry referenced in MapJobRegistry comments - - * {{{https://jira.springframework.org/browse/BATCH-1764}[BATCH-1764]}} - Correct JavaDoc for HibernateItemWriter - - * {{{https://jira.springframework.org/browse/BATCH-1769}[BATCH-1769]}} - FlatFileItemReader javadocs - - * {{{https://jira.springframework.org/browse/BATCH-1777}[BATCH-1777]}} - org.springframework.batch.core.converter.DefaultJobParametersConverter not safe for use with certain Locales - - * {{{https://jira.springframework.org/browse/BATCH-1800}[BATCH-1800]}} - AbstractItemCountingItemStreamItemReader could implement ItemStreamReader interface instead of ItemStream and ItemReader - - * {{{https://jira.springframework.org/browse/BATCH-1805}[BATCH-1805]}} - Clarify JavaDoc for JobParametersConverter - - * {{{https://jira.springframework.org/browse/BATCH-1806}[BATCH-1806]}} - Javadoc for FlowExecutionStatus - - * {{{https://jira.springframework.org/browse/BATCH-1815}[BATCH-1815]}} - RunIdIncrementer goes against java best practices - - * {{{https://jira.springframework.org/browse/BATCH-1818}[BATCH-1818]}} - JobInterruptedException should not be logged - - * {{{https://jira.springframework.org/browse/BATCH-1830}[BATCH-1830]}} - Don't log the JobInterruptedException when stopping a job - - * {{{https://jira.springframework.org/browse/BATCH-1834}[BATCH-1834]}} - allow nested tasklets to be specified in the schema - - * {{{https://jira.springframework.org/browse/BATCH-1861}[BATCH-1861]}} - Concurrency Support for in-memory repositories - - * {{{https://jira.springframework.org/browse/BATCH-1862}[BATCH-1862]}} - Update README to reflect Maven 3.0 works - - * {{{https://jira.springframework.org/browse/BATCH-1867}[BATCH-1867]}} - StaxEventItemWriter should be easier to override - - * {{{https://jira.springframework.org/browse/BATCH-1874}[BATCH-1874]}} - provide a regex based line mapper - - * {{{https://jira.springframework.org/browse/BATCH-1881}[BATCH-1881]}} - force sync to underlying file system in FlatFileItemWriter and StaxEventItemWriter - -* New Feature - - * {{{https://jira.springframework.org/browse/BATCH-1758}[BATCH-1758]}} - Round trip <<>> broken for short Strings in PropertiesConverter diff --git a/src/site/apt/migration/2.1.9-2.2.0.M1.apt b/src/site/apt/migration/2.1.9-2.2.0.M1.apt deleted file mode 100644 index 99a8e69d61..0000000000 --- a/src/site/apt/migration/2.1.9-2.2.0.M1.apt +++ /dev/null @@ -1,106 +0,0 @@ -Spring Batch 2.2.0.M1 Release Notes - -* Bug - - * {{{https://jira.springframework.org/browse/BATCH-1697}[BATCH-1697]}} - TaskletStep not marked as FAILED when FlatFileItemWriter fails to append footer - - * {{{https://jira.springframework.org/browse/BATCH-1745}[BATCH-1745]}} - XSD inconsistency: allow-start-if-complete is not allowed on non-tasklet step - - * {{{https://jira.springframework.org/browse/BATCH-1773}[BATCH-1773]}} - Step-scoped annotation based listener is not called - - * {{{https://jira.springframework.org/browse/BATCH-1774}[BATCH-1774]}} - NullPointerException on RepeateTemplate - - * {{{https://jira.springframework.org/browse/BATCH-1780}[BATCH-1780]}} - Code exception is masked by a batch exception - - * {{{https://jira.springframework.org/browse/BATCH-1795}[BATCH-1795]}} - ExponentialBackOffPolicy and BackOffContext - - * {{{https://jira.springframework.org/browse/BATCH-1799}[BATCH-1799]}} - Exception in flush of file output ItemWriters does not abort a step/job - - * {{{https://jira.springframework.org/browse/BATCH-1884}[BATCH-1884]}} - JobLauncherIntegrationTests failing - - * {{{https://jira.springframework.org/browse/BATCH-1890}[BATCH-1890]}} - Fix broken JDK5 build after Spring 3.1.2 - - * {{{https://jira.springframework.org/browse/BATCH-1903}[BATCH-1903]}} - SQL compatibility breakage with HSQL - - * {{{https://jira.springframework.org/browse/BATCH-1916}[BATCH-1916]}} - RecordSeparatorPolicy#isEndOfRecord wrong javadoc? - - * {{{https://jira.springframework.org/browse/BATCH-1920}[BATCH-1920]}} - Add sample for new AMQPItemReader & Writer - - * {{{https://jira.springframework.org/browse/BATCH-1948}[BATCH-1948]}} - StepScope doesn't work properly with proxyTargetClass=true in @Bean definitions - - * {{{https://jira.springframework.org/browse/BATCH-1950}[BATCH-1950]}} - Fix bootstrap process - -* Improvement - - * {{{https://jira.springframework.org/browse/BATCH-1667}[BATCH-1667]}} - StepExecutionListener ExitStatus not persisted and not accessible for other listeners - - * {{{https://jira.springframework.org/browse/BATCH-1691}[BATCH-1691]}} - Allow to define groupBy for SqlPaginingQueryProviderFactoryBean - - * {{{https://jira.springframework.org/browse/BATCH-1718}[BATCH-1718]}} - CompositeRetryPolicy by default is pessimistic - - * {{{https://jira.springframework.org/browse/BATCH-1854}[BATCH-1854]}} - Create marker interface to be used by MultiResourceItemReader to inject the resource an item was read from. - - * {{{https://jira.springframework.org/browse/BATCH-1887}[BATCH-1887]}} - Cleanup maven warnings - - * {{{https://jira.springframework.org/browse/BATCH-1889}[BATCH-1889]}} - SqlFire support - - * {{{https://jira.springframework.org/browse/BATCH-1891}[BATCH-1891]}} - Migrate usage of deprecated classes - - * {{{https://jira.springframework.org/browse/BATCH-1904}[BATCH-1904]}} - Upgrade support of Hibernate to Hibernate 4 - - * {{{https://jira.springframework.org/browse/BATCH-1928}[BATCH-1928]}} - Convert deprecated classes from the org.springframework.jdbc.core.simple package in samples - - * {{{https://jira.springframework.org/browse/BATCH-1929}[BATCH-1929]}} - Convert deprecated classes from the org.springframework.jdbc.core.simple package in core-tests - - * {{{https://jira.springframework.org/browse/BATCH-1931}[BATCH-1931]}} - Convert deprecated classes from the org.springframework.jdbc.core.simple package in infrastructure-tests - -* New Feature - - * {{{https://jira.springframework.org/browse/BATCH-1666}[BATCH-1666]}} - Add abort(long executionId) convenience method to JobOperator - - * {{{https://jira.springframework.org/browse/BATCH-1667}[BATCH-1667]}} - StepExecutionListener ExitStatus not persisted and not accessible for other listeners - - * {{{https://jira.springframework.org/browse/BATCH-1684}[BATCH-1684]}} - Allow serializer to be injected into JobRepository (ExecutionContextDao) - - * {{{https://jira.springframework.org/browse/BATCH-1685}[BATCH-1685]}} - Upgrade minimum support level for Spring to 3.1.2 - - * {{{https://jira.springframework.org/browse/BATCH-1694}[BATCH-1694]}} - Add StepLocatorStepFactoryBean - - * {{{https://jira.springframework.org/browse/BATCH-1714}[BATCH-1714]}} - Change ChunkListener interface and semantics to be called after rollback as well as commit - - * {{{https://jira.springframework.org/browse/BATCH-1749}[BATCH-1749]}} - JdbcPagingItemReader query fails when specifying multiple columns in sortKey - - * {{{https://jira.springframework.org/browse/BATCH-1869}[BATCH-1869]}} - Need a line tokenizer to parse the line based on string (multi character) delimiter than a single character delimiter - - * {{{https://jira.springframework.org/browse/BATCH-1882}[BATCH-1882]}} - AMQP ItemReader and ItemWriter - - * {{{https://jira.springframework.org/browse/BATCH-1911}[BATCH-1911]}} - Provide a Step registry to be able to locate a step by its name on (namely) remote nodes - - * {{{https://jira.springframework.org/browse/BATCH-1912}[BATCH-1912]}} - Provide a base PartitionHandler so that other implementations can benefit from a shared base implementation - - * {{{https://jira.springframework.org/browse/BATCH-1918}[BATCH-1918]}} - @Configuration support for batch (e.g. @EnableBatchProcessing) - - * {{{https://jira.springframework.org/browse/BATCH-1935}[BATCH-1935]}} - Add quick start maven archetype - -* Task - - * {{{https://jira.springframework.org/browse/BATCH-1883}[BATCH-1883]}} - Bump spring-amqp Version to 1.1.2 - - * {{{https://jira.springframework.org/browse/BATCH-1938}[BATCH-1938]}} - Update to support Spring 3.2.0.RELEASE - - * {{{https://jira.springframework.org/browse/BATCH-1943}[BATCH-1943]}} - Add ChunkContext to existing ChunkListener methods - -* Refactoring - - * {{{https://jira.springframework.org/browse/BATCH-1895}[BATCH-1895]}} - Remove batch retry and depend on spring-retry. - - * {{{https://jira.springframework.org/browse/BATCH-1897}[BATCH-1897]}} - Remove ant files from batch. - - * {{{https://jira.springframework.org/browse/BATCH-1915}[BATCH-1915]}} - Change minimum compiler level to 1.6 and use @Override everywhere - - * {{{https://jira.springframework.org/browse/BATCH-1919}[BATCH-1919]}} - Switch to using namespace in samples - - * {{{https://jira.springframework.org/browse/BATCH-1940}[BATCH-1940]}} - Replace org.springframework.batch.support.JdbcTestUtils with org.springframework.test.jdbc.JdbcTestUtils - - * {{{https://jira.springframework.org/browse/BATCH-1949}[BATCH-1949]}} - Remove explicit checks for Spring 3 in StepScope - diff --git a/src/site/apt/migration/2.2.0-2.2.1.apt b/src/site/apt/migration/2.2.0-2.2.1.apt deleted file mode 100755 index 026be3bb3b..0000000000 --- a/src/site/apt/migration/2.2.0-2.2.1.apt +++ /dev/null @@ -1,28 +0,0 @@ -Spring Batch 2.2.1 Release Notes - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-1849}[BATCH-1849]}} - Item was not picked up after restarting a failed job!!! - - * {{{http://jira.springsource.org/browse/BATCH-1973}[BATCH-1973]}} - processor-transactional="false" in chunk definition does not have stable behavior - - * {{{http://jira.springsource.org/browse/BATCH-2036}[BATCH-2036]}} - Output incorrect when using processor-transactional="false" and skips. - - * {{{http://jira.springsource.org/browse/BATCH-2038}[BATCH-2038]}} - DerbyPagingQueryProvider does not work with Derby 10.10.1.1 - - * {{{http://jira.springsource.org/browse/BATCH-2050}[BATCH-2050]}} - AbstractItemCountingItemStreamItemReader.read() shouldn't be final - - * {{{http://jira.springsource.org/browse/BATCH-2054}[BATCH-2054]}} - StaxEventItemWriter fails on a NullPointerException with Spring OXM 3.2.x. - -* Improvement - - * {{{http://jira.springsource.org/browse/BATCH-1984}[BATCH-1984]}} - CompositeItemProcessor.setDelegates argument has limiting generic type - - * {{{http://jira.springsource.org/browse/BATCH-2052}[BATCH-2052]}} - StaxEventItemWriter should only force sync once per chunk - - * {{{http://jira.springsource.org/browse/BATCH-2069}[BATCH-2069]}} - Relax OSGI dependencies on nosql jars - -* Task - - * {{{http://jira.springsource.org/browse/BATCH-2056}[BATCH-2056]}} - Update 'What's New' in Reference Document - diff --git a/src/site/apt/migration/2.2.0.M1-2.2.0.RC1.apt b/src/site/apt/migration/2.2.0.M1-2.2.0.RC1.apt deleted file mode 100644 index 13146470a7..0000000000 --- a/src/site/apt/migration/2.2.0.M1-2.2.0.RC1.apt +++ /dev/null @@ -1,74 +0,0 @@ -Spring Batch 2.2.0.RC1 Release Notes - -* Bug - - * {{{https://jira.springframework.org/browse/BATCH-1676}[BATCH-1676]}} - Inconsistencies in XSD and documentation for Listeners within Step - - * {{{https://jira.springframework.org/browse/BATCH-1720}[BATCH-1720]}} - Remote steps should not be consulted for isAllowedStartIfComplete() etc. - - * {{{https://jira.springframework.org/browse/BATCH-1788}[BATCH-1788]}} - Update chapter 4.5.6 on aborting jobs in documentation - - * {{{https://jira.springframework.org/browse/BATCH-1847}[BATCH-1847]}} - scope="step" inheritance from parent bean definitions causes odd effects - - * {{{https://jira.springframework.org/browse/BATCH-1856}[BATCH-1856]}} - ExtendedConnectionDataSourceProxy compilation error in JDK 7 - - * {{{https://jira.springframework.org/browse/BATCH-1908}[BATCH-1908]}} - Inefficient storage of StepExecutionContexts when using partitioning - - * {{{https://jira.springframework.org/browse/BATCH-1924}[BATCH-1924]}} - Restarting a stopped job in COMPLETED state prevents progress - - * {{{https://jira.springframework.org/browse/BATCH-1951}[BATCH-1951]}} - StepScoped proxies not being created with proxyMode=TARGET_CLASS - - * {{{https://jira.springframework.org/browse/BATCH-1952}[BATCH-1952]}} - Missing Import in spring-batch-infrastructure.jar - - * {{{https://jira.springframework.org/browse/BATCH-1757}[BATCH-1757]}} - MinMaxPartitioner sets incorrect max value - - * {{{https://jira.springframework.org/browse/BATCH-1959}[BATCH-1959]}} - Problem with FlatFileItemWriter restart using multi-byte encoding - - * {{{https://jira.springframework.org/browse/BATCH-1960}[BATCH-1960]}} - drop table not consistent across supported dbs - - * {{{https://jira.springframework.org/browse/BATCH-1972}[BATCH-1972]}} - StaxEventItemReader fails when restarted at end of file - - * {{{https://jira.springframework.org/browse/BATCH-1975}[BATCH-1975]}} - StaxEventItemWriter namespace added to elements after restart - -* Improvement - - * {{{https://jira.springframework.org/browse/BATCH-1723}[BATCH-1723]}} - Change JobParameters to use wrapper types for getLong and getDouble - - * {{{https://jira.springframework.org/browse/BATCH-1770}[BATCH-1770]}} - Checking for UNKNOWN step status could be performed before the JobExecution is launched - - * {{{https://jira.springframework.org/browse/BATCH-1906}[BATCH-1906]}} - add support to access an item's line number - - * {{{https://jira.springframework.org/browse/BATCH-1928}[BATCH-1928]}} - Convert deprecated classes from the org.springframework.jdbc.core.simple package in samples - - * {{{https://jira.springframework.org/browse/BATCH-1955}[BATCH-1955]}} - Remove requirement to inject ItemSqlParameterSourceProvider into JdbcBatchItemWriter - - * {{{https://jira.springframework.org/browse/BATCH-1957}[BATCH-1957]}} - Add StaxEventItemWriter deleteIfEmpty property - - * {{{https://jira.springframework.org/browse/BATCH-1958}[BATCH-1958]}} - Fix typo in samples apt index - -* New Feature - - * {{{https://jira.springframework.org/browse/BATCH-1412}[BATCH-1412]}} - Allow a Job to accept JobParameters that do not contribute to its identity - - * {{{https://jira.springframework.org/browse/BATCH-1728}[BATCH-1728]}} - Add support for Spring Data (readers and writers) - - * {{{https://jira.springframework.org/browse/BATCH-1934}[BATCH-1934]}} - Update spring-batch docs for AmqpItemReader / Writers and use of SQLFire as a db option - - * {{{https://jira.springframework.org/browse/BATCH-1964}[BATCH-1964]}} - Optional Transaction in JpaPagingItemReader - -* Task - - * {{{https://jira.springframework.org/browse/BATCH-1954}[BATCH-1954]}} - Update documentation for 2.2.0 - - * {{{https://jira.springframework.org/browse/BATCH-1968}[BATCH-1968]}} - Upgrade to hsqldb 2.2.9 - - * {{{https://jira.springframework.org/browse/BATCH-1977}[BATCH-1977]}} - Create a migration from old schema to new schema - -* Refactoring - - * {{{https://jira.springframework.org/browse/BATCH-1939}[BATCH-1939]}} - Minor cleanups to AMQP sample - - * {{{https://jira.springframework.org/browse/BATCH-1947}[BATCH-1947]}} - Replace org.easymock.classextension.EasyMock with org.easymock.EasyMock - - * {{{https://jira.springframework.org/browse/BATCH-1966}[BATCH-1966]}} - Use abstract classes to inherit behaviors - \ No newline at end of file diff --git a/src/site/apt/migration/2.2.0.RC1-2.2.0.RC2.apt b/src/site/apt/migration/2.2.0.RC1-2.2.0.RC2.apt deleted file mode 100644 index f9346f4432..0000000000 --- a/src/site/apt/migration/2.2.0.RC1-2.2.0.RC2.apt +++ /dev/null @@ -1,23 +0,0 @@ -Spring Batch 2.2.0.RC2 Release Notes - -* Bug - - * {{{https://jira.springframework.org/browse/BATCH-1787}[BATCH-1787]}} - Don't import provided libs in your compile-configuration in ivy.xml - - * {{{https://jira.springframework.org/browse/BATCH-1995}[BATCH-1995]}} - Line ending in multiline delimiter not being processed correctly - - * {{{https://jira.springframework.org/browse/BATCH-1996}[BATCH-1996]}} - DelimitedLineTokenizer skips first token if it is empty. - - * {{{https://jira.springframework.org/browse/BATCH-2017}[BATCH-2017]}} - Fix typo of the word delegate - - * {{{https://jira.springframework.org/browse/BATCH-2019}[BATCH-2019]}} - Support PropertySourcesPlaceholderConfigurer delegation to job contexts - - * {{{https://jira.springframework.org/browse/BATCH-2022}[BATCH-2022]}} - Drop script for Postgresql assumes that the tables are there when dropping constraints. - - * {{{https://jira.springframework.org/browse/BATCH-2023}[BATCH-2023]}} - @StepScope should default to ScopedProxyMode.TARGET_CLASS - -* Improvement - - * {{{https://jira.springframework.org/browse/BATCH-1982}[BATCH-1982]}} - JavaConfig for Spring Batch - Copy-Constructor for CommonStepProperties - - * {{{https://jira.springframework.org/browse/BATCH-1993}[BATCH-1993]}} - Update reference manual copyright date diff --git a/src/site/apt/migration/2.2.0.RC2-2.2.0.apt b/src/site/apt/migration/2.2.0.RC2-2.2.0.apt deleted file mode 100755 index 9ce76bf32c..0000000000 --- a/src/site/apt/migration/2.2.0.RC2-2.2.0.apt +++ /dev/null @@ -1,7 +0,0 @@ -Spring Batch 2.2.0 Release Notes - -* Bug - - * {{{http://jira.springsource.org/browse/BATCH-2031}[BATCH-2031]}} - Incorrect delimiter detection in DelimitedLineTokenizer - - * {{{http://jira.springsource.org/browse/BATCH-2028}[BATCH-2028]}} - Update downloads documentation diff --git a/src/site/apt/migration/index.apt b/src/site/apt/migration/index.apt deleted file mode 100644 index d4ed1cf5f4..0000000000 --- a/src/site/apt/migration/index.apt +++ /dev/null @@ -1,103 +0,0 @@ - ------ - Spring Batch Migration Guides - ------ - Dave Syer, Michael Minella - ------ - September 2009 - - See the changes report generated from {{{issue-tracking.html}JIRA}}. - - There is a pre-built {{{http://opensource.atlassian.com/projects/spring/secure/IssueNavigator.jspa?reset=true&mode=hide&pid=10090&status=5&status=6&updated:previous=-1w&sorter/field=updated&sorter/order=DESC}Resolved Recently}} query that might be useful. - - You can also browse the upcoming releases and look at the {{{http://opensource.atlassian.com/projects/spring/browse/BATCH?report=com.atlassian.jira.plugin.system.project:roadmap-panel}Road Map}} query. - - Links: - - * {{{./2.2.0-2.2.1.html}2.2.0-2.2.1}} - - * {{{./2.2.0.RC2-2.2.0.html}2.2.0.RC2-2.2.0}} - - * {{{./2.2.0.RC1-2.2.0.RC2.html}2.2.0.RC1-2.2.0.RC2}} - - * {{{./2.2.0.M1-2.2.0.RC1.html}2.2.0.M1-2.2.0.RC1}} - - * {{{./2.1.9-2.2.0.M1.html}2.1.9-2.2.0.M1}} - - * {{{./2.1.8-2.1.9.html}2.1.8 to 2.1.9}} - - * {{{./2.1.7-2.1.8.html}2.1.7 to 2.1.8}} - - * {{{./2.1.6-2.1.7.html}2.1.6 to 2.1.7}} - - * {{{./2.1.5-2.1.6.html}2.1.5 to 2.1.6}} - - * {{{./2.1.4-2.1.5.html}2.1.4 to 2.1.5}} - - * {{{./2.1.3-2.1.4.html}2.1.3 to 2.1.4}} - - * {{{./2.1.2-2.1.3.html}2.1.2 to 2.1.3}} - - * {{{./2.1.1-2.1.2.html}2.1.1 to 2.1.2}} - - * {{{./2.1.0-2.1.1.html}2.1.0 to 2.1.1}} - - * {{{./2.1.0.RC1-2.1.0.html}2.1.0.RC1 to 2.1.0}} - - * {{{./2.1.0.M4-2.1.0.RC1.html}2.1.0.M4 to 2.1.0.RC1}} - - * {{{./2.1.0.M3-2.1.0.M4.html}2.1.0.M3 to 2.1.0.M4}} - - * {{{./2.1.0.M2-2.1.0.M3.html}2.1.0.M2 to 2.1.0.M3}} - - * {{{./2.1.0.M1-2.1.0.M2.html}2.1.0.M1 to 2.1.0.M2}} - - * {{{./2.1.0.M1-2.1.0.M2.html}2.1.0.M1 to 2.1.0.M2}} - - * {{{./2.0.x-2.1.0.M1.html}2.0.x to 2.1.0.M1}} - - * {{{./2.0-highlights.html}Highlights of changes between 1.x and 2.0}} - - * {{{./2.0.3-2.0.4.html}2.0.3.RELEASE to 2.0.4.RELEASE}} - - * {{{./2.0.2-2.0.3.html}2.0.2.RELEASE to 2.0.3.RELEASE}} - - * {{{./2.0.1-2.0.2.html}2.0.1.RELEASE to 2.0.2.RELEASE}} - - * {{{./2.0.0-2.0.1.html}2.0.0.RELEASE to 2.0.1.RELEASE}} - - * {{{./2.0-rc3-release.html}2.0.0.RC3 to 2.0.0.RELEASE}} - - * {{{./2.0-rc2-rc3.html}2.0.0.RC2 to 2.0.0.RC3}} - - * {{{./2.0-rc1-rc2.html}2.0.0.RC1 to 2.0.0.RC2}} - - * {{{./2.0-m4-rc1.html}2.0.0.M4 to 2.0.0.RC1}} - - * {{{./2.0-m3-m4.html}2.0.0.M3 to 2.0.0.M4}} - - * {{{./2.0-m2-m3.html}2.0.0.M2 to 2.0.0.M3}} - - * {{{./2.0-m1-m2.html}2.0.0.M1 to 2.0.0.M2}} - - * {{{./1.1-2.0-m1.html}1.1 to 2.0.0.M1}} - - * {{{./1.1.2-1.1.3.html}1.1.2 to 1.1.3}} - - * {{{./1.1.1-1.1.2.html}1.1.1 to 1.1.2}} - - * {{{./1.1-1.1.1.html}1.1 to 1.1.1}} - - * {{{./1.0.1-1.1.html}1.0.1 to 1.1}} - - * {{{./1.0.0-1.0.1.html}1.0.0 to 1.0.1}} - - * {{{./1.0-rc1-final.html}1.0 rc1 to final}} - - * {{{./1.0-m5-rc1.html}1.0 m5 to rc1}} - - * {{{./1.0-m4-m5.html}1.0 m4 to m5}} - - * {{{./1.0-m3-m4.html}1.0 m3 to m4}} - - * {{{./1.0-m2-m3.html}1.0 m2 to m3}} - diff --git a/src/site/apt/reference/index.apt b/src/site/apt/reference/index.apt deleted file mode 100644 index e42f04ceb7..0000000000 --- a/src/site/apt/reference/index.apt +++ /dev/null @@ -1,20 +0,0 @@ - ----- - Spring Batch Documentation - ----- - Dave Syer - ----- - Aug 2007 - ----- - -Spring Batch Documentation - - This is the home of the Spring Batch Reference Guide. This is where you can find out how to use Spring Batch and how the pieces fit together. For detailed descriptions of the individual classes in Spring Batch please refer to the {{{../apidocs/index.html}Javadocs}}. There is also a separate section of this web site for the {{{../spring-batch-samples/index.html}Spring Batch Samples}}. If you want to get started quickly that might be a good place to start. - - Use the links below to navigate to the HTML and PDF versions of the guide: - - * {{{./html/index.html}HTML}} format - - * {{{./html-single/index.html}HTML Single Page}} - - * {{{./pdf/spring-batch-docs.pdf}PDF}} format - diff --git a/src/site/apt/scratch.apt b/src/site/apt/scratch.apt deleted file mode 100644 index 6e512936b0..0000000000 --- a/src/site/apt/scratch.apt +++ /dev/null @@ -1,58 +0,0 @@ - ------ - Spring Batch Scratch - ------ - Dave Syer - ------ - March 2007 - - -+--- - | restore; - | -1 | BATCH(repeat=until exhausted) { - | -2 | RETRY(outer) { - | -3 | TX(datasource=batch) { - | -4 | TX(datasource=business) { - | -5 | BATCH(repeat=5) { - | -6 | RETRY(inner) { -6.1 | input; -7 | } PROCESS { - | output; -8 | } RECOVER { - | recover; - | } - | - | } - | -4.1 | savepoint; - | - | } - | - | } - | - | } - | - | } -+--- - - * The order of the transaction nesting might be important, but only - if they are not XA, and only if there is a partial failure (inner - commits and the outer rolls back), and only if that happens on the - last attempt at RETRY(2). - - * Batch TX is outside business TX so the worse that can happen is - that we might restart from the same point twice (if the inner TX - commits and the outer rolls back). If they were the other way - round the batch savepoint(4.1) could commit and the business - processing (7) roll back - then we would miss the business - processing if the batch had to restart. - - * The savepoint(4.1) needs to be implemented so that the state it - saves is synchronized with the business TX(4). That way if TX(4) - rolls back the savepoint will always be the correct state to - restart if a partial failure is followed by a successful RETRY(2). diff --git a/src/site/apt/sitemap.apt b/src/site/apt/sitemap.apt deleted file mode 100644 index a7bf864a53..0000000000 --- a/src/site/apt/sitemap.apt +++ /dev/null @@ -1,120 +0,0 @@ - ------ - Site Map - ------ - Dave Syer - ------ - April 2007 - -Spring Batch Site Map - -* Overview - - * Main Site - high-level information and links to sub-projects - (called "modules" in Maven speak): - - * Docs - reference documentation, user guides - - * Infrastructure - CI build and technical information - - * Integration Tests - reports on tests of infrastructure - - * Core - CI and technical information about core domain - - * Execution - CI and technical information about execution (implementation of core) - -* Main Site - - * Splash page - welcome, mission statement, download links - - * Whitepaper (JavaOne presentation translated to HTML) - - * Occasional Articles (e.g. transactions) - - * Use Cases - - * Project Information (standdard Maven stuff) - - * Developers - - * Source Repository - - * License - - * etc. - -* Documentation - - * Splash page - welcome, links to rest of reference docs. - - * User Guides (docbook, Spring branded reference guides - HTML, HTML - Single Page and PDF). Two choices: one big guide with parts as - listed below, or multiple mini-guides. The former is probably - better. - - Maybe we could also break each of these down a bit more... - - * Infrastructure - How to use the core API - - * Simple Execution Environment - - * Partitioning Execution Environment - - * Other Execution Environments? - - * Changelog - -* Infrastructure - - * Splash page explaining the role of infrastructure, and high level - API packaging. - - * Changelog - - * Project information (duplicated from Main Site - Maven "feature") - - * CI Reports - - * JUnit test report - - * Clover coverage - - * JDepend report - - * Javadocs - -* Integration Tests - - * Changelog - - * Project information (duplicated from Main Site - Maven "feature" - - TODO: find a way to switch them off in sub-projects) - - * CI Reports (same as for infrastructure). - -* Core - - * Changelog - - * Project information (duplicated from Main Site - Maven "feature" - - TODO: find a way to switch them off in sub-projects) - - * CI Reports (same as for infrastructure). - -* Execution - - * Changelog - - * Project information - - * CI Reports (same as for infrastructure). - -* Integration Tests - - * Changelog - - * Project information (duplicated from Main Site - Maven "feature" - - TODO: find a way to switch them off in sub-projects) - - * CI Reports (same as for infrastructure). - - diff --git a/src/site/apt/snapshots.apt b/src/site/apt/snapshots.apt deleted file mode 100644 index 586a26bdce..0000000000 --- a/src/site/apt/snapshots.apt +++ /dev/null @@ -1,33 +0,0 @@ - --------- - Snapshots - --------- - Dave Syer, Michael Minella - ------ - August 2007, February 2009 - -Snapshot Builds - - These builds are provided for testing and development purposes only. They are built by a Bamboo process automatically using the latest snapshot from Subversion. - - Snapshots are deployed in Maven Central format every night to the {{{http://s3browse.com/explore/maven.springframework.org/snapshot/org/springframework/batch}SpringSource Maven Repository}}, so use this in your POM (inside a \ element): - -+--- - - spring-s3 - Spring Maven Snapshot Repository - http://s3.amazonaws.com/maven.springframework.org/snapshot - -+--- - - If you are downloading the jar files manually from there you can see the internal and external project dependencies in the <<>> files. You will probably need <<>> and <<>>. - - Individual dependencies can then by added like so (inside a \ element at the top level): - -+--------------- - - org.springframework.batch - spring-batch-core - 3.0.0.BUILD-SNAPSHOT - -+--------------- - diff --git a/src/site/apt/stateful.apt b/src/site/apt/stateful.apt deleted file mode 100644 index 3455085c45..0000000000 --- a/src/site/apt/stateful.apt +++ /dev/null @@ -1,161 +0,0 @@ - ------ - Spring Batch - State and Thread Safety - ------ - Dave Syer - ------ - March 2009 - -State and Thread Safety in Spring Batch - - A stateless component is thread safe, but sometimes not practical (you need to store some state). A stateful component can be thread safe, if its contract is clearly explained to and met by its clients. Spring Batch has a lot of stateful components, which by and large are not capable of being used in a thread safe manner, but that doesn't have to be the case for ever. - - Components with private non-final fields are not necessarily stateful in practice - Spring components often have fields that are injected or initialized after the object is created. The working definition of "stateless" for the present purposes is a component with fields that do not change after initialization, which has the usual Spring lifecycle meaning (i.e. once "released into the wild" with <<>>, or the equivalent). - - Conversely, even components with only final fields are not necessarily stateless. They can have the appearance of statelessness (and thread safety), but if they mutate their final fields, then they are stateful by association. It is not always possible to tell from the implementation of a component whether it is stateful by association, since it depends entirely on the implementation of its fields, whose concrete type may not even be known at compile time. - -* Variants of State - - There are two reasons why a Batch component might need to be stateful: rollback and restart. - - * : to support rollback after a transaction, a component might need to detect the rollback (and potentially the start of the original transaction) and rewind to its former state. This is a single-process (JVM) pattern. Normally a transaction is managed on a single thread as well, but in principle there might be multiple threads using the same component, which inevitably leads to problems. - - * : to support restart of a failed job execution, a component needs to be able to re-hydrate its former state from a previour execution. This is often a multi-process (JVM) pattern, and needs to work between processes even if it isn't always invoked that way in practice. To support this requirement Spring Batch uses the <<>> calback methods and its <<>> to store state. The framework deals with the state storage and re-hydration, and components only need to provide snapshots of their state through the <<>> interface. - -* ItemReaders - - The most common case in Batch where stateful components are necessary are the item readers, whose job is to provide instances of business data for processing. At a high level there are three variants, driven by the needs of their client (usually the framework) for rollback and restart. - - * . A fully transactional reader has its rollback and restart state managed entirely by an external system (middleware) that is driven from a transaction in the batch system. After a rollback items are returned to the middleware, and represented in a subsequent transaction. There is no essential difference between rollback and retry: as far as the middleware is concerned they are just failed reads. The user has to tell Spring Batch explicitly if one of these readers is being used so it can take into account the expected re-presentation of items in a new transaction. Just about the only example of this is the <<>>. - - * . A stateless component doesn't need to do anything special to provide restartability, which is quite a valuable feature. It is also thread safe (by construction), which is at least equally valuable if not more. Stateless <<>> are rare in practice: in the Spring Batch source code the only one is the <<>> from the samples. - - * . Part of the contract for <<>> implementations in Batch 2.0 is that <<>> do not need to manage state for rollback because each item is only ever read once (and buffered for use on rollback internally). They do, however still need state if they want to provide restartability, which all the framework implementations do. A stateful component has to work hard to be thread safe, but it is not ruled out in principle. In fact none of the stateful <<>> implementations in Spring Batch is thread safe as of 2.0, but they are all restartable (which is more to the point). - - Because these different categories of <<>> behave differently with respect to rollback and restart, they have to be recognised and treated differently by the framework. If a simple fail fast <<>> is used, then any error leads to immediate failure, so rollback is not important in that case, but restart always is. If a fault tolerant <<>> is used, then rollback and restart have to be handled, and differently for each category of reader. To recognise a reader the framework needs a flag to be set by the user (see <<>>). To recognise a or reader the framework uses the <<>> interface (if present assume ). - -Rollback and Restart with Skips - -* Spring Batch 2.0.0.RC1 - -** Retry and Skip with a Transactional Reader - - Since the middleware will simply re-present failed items, there is nothing for the framework to do. We can use this example to establish some notation. Suppose five items are read in one transaction and there is a deterministic failure while writing the 3rd one. If skips are allowed, but not retries, the process looks like this: - -+--- -[1,2,3,4,5; (1,2,3,4,5)]* -[1,2,3,4,5; (1),(2),(3)]* -[1,2,3,4,5; (1,2,4,5)] -... -+--- - - where parentheses denote a write operation, brackets ([]) represent a transaction, and an asterisk (*) denotes a rollback. In words we have: - - * Read items 1 through 5, then write them as a chunk, encounter an error and rollback. - - * Read items 1 through 5 again and write them individually, scanning for errors. Encounter the error on item 3, then rollback having identified item 3 as skippable - - * Read the 5 items again and skip item 3 on writing the chunk. - - To achieve this, the items from a failed chunk need to be intrinsically identifiable, so that when they show up again the system can throttle back and scan for the error. In completely general terms this is not a well-defined problem - tere is no generic identifier for the items that can always be used. Spring Batch by default uses the items themselves as the identifier in this case, leading to possible problems if their identity (equals and hashCode) are not properly defined. We might be able to pick an identifier in some special cases, like in the JMS case the message ID will be unique, in which case we would need to provide a <<>> implementation. (This is uncommon enough that in Spring Batch 2.0.0 there is no way to do it using the XML namespace, but you can do it using <<>>.) - - There is also the issue of storing the identifiers, waiting for all the failed items to be seen again. There is no guarantee that the failed items will ever come back to this consumer, so we have to store the identifiers potentially indefinitely, possibly leading to memory leaks. To help alleviate this problem (at the risk of misidentifying a failed item as a new one) Spring Batch provides the <<>> which allows cached values to be garabage collected if memory is under pressure. - - With a retry limit of 1, the same execution would look like this - -+--- -[1,2,3,4,5; (1,2,3,4,5)]* -[1,2,3,4,5; (1,2,3,4,5)]* -[1,2,3,4,5; (1),(2),(3)]* -[1,2,3,4,5; (1,2,4,5)] -... -+--- - - (just an extra iteration where the chunk is given a chance to succeed before the error scan starts). - -** Retry and Skip with a Non-Transactional Reader - - In this case there is no middleware so the <<>> has to buffer the items between rollbacks, but otherwise the process looks like very similar. The internal name for the buffer is a <<>>. With no retry: - -+--- -[1,2,3,4,5; (1,2,3,4,5)]* -[; (1),(2),(3)]* -[; (1,2,4,5)] -... -+--- - - and with retry: - -+--- -[1,2,3,4,5; (1,2,3,4,5)]* -[; (1,2,3,4,5)]* -[; (1),(2),(3)]* -[; (1,2,4,5)] -... -+--- - - In the case of the non-Transactional reader there is no problem with item identifiers: the <<>> can be used to identify the failed items when they are re-processed. - -* A More Efficient Approach - - It would be more efficient if we didn't end up processing each item more than twice in the simple case of skip with no retry. - -** Skip with a Transactional Reader - -+--- -[1,2,3,4,5; (1,2,3,4,5)]* -[1; (1)] -[2; (2)] -[3; (3)]* -[4; (4)] -[5; (5)] -... -+--- - - This is difficult to achieve with the <<>> because it requires communication between the <<>> and <<>> about the failed items: the <<>> has to stop reading when it encounters an item from a previously failed chunk. This feature is not yet implemented in Spring Batch (as of 2.0.0.RC2). - -** Skip with a Stateful Reader - -+--- -[1,2,3,4,5; (1,2,3,4,5)]* -[; (1)] -[; (2)] -[; (3)]* -[; (4)] -[; (5)] -... -+--- - - This processing and rollback plan is not difficult to achieve with the <<>> but it hard to get the restart data properly aligned. The main issue is that the <<>> gets the <<>> callback before every commit. It is going to think that all 5 items have been committed after the first (and every) commit, which is wrong, so any subsequent non-skippable failure will lead to a restart on item 6 with all intermediate items lost. So after a fatal error on item 2 the restart would like this - -+--- -[1,2,3,4,5; (1,2,3,4,5)]* -[; (1)] -[; (2)]* - - - -[6,7,8,9,10; (6,7,8,9,10)] -... -+--- - - and items (2,3,4,5) are ommitted with no record of attempting to process them. - - To fix this we have to mask the <<>> callback in the <<>> and only pass it through if we know we have successfully finished a whole chunk. In the partial chunk transactions, we need to update the <<>> with some offset data that would prevent the intermediate values being processed on restart. - -+--- -[1,2,3,4,5; (1,2,3,4,5)]* -[; (1)] -[; (2)]* - - - -[2,3,4,5,6; (2,3,4,5,6)]* -[; (2)] -[; (3)]* -[; (4)] -[; (5)] -[; (6)] -... -+--- - - This can be implemented by storing the offset within a chunk separately from the read count. In the example above, the offset would be 2 when the fatal exception happened, and the read count would be 0 up to the point where the first chunk successfully commits. Caveat: only works if the <<>> is single-threaded (which it has to be for all the existing Stateful readers). The implementation via <<>> assumes that the step is single threaded if it finds that the <<>> is an <<>>, and the <<>> is a <<>>. If the <<>> is concurrent then warnings are logged, and the offset is not stored )the assumption is that the reader is not restartable so it won't care. diff --git a/src/site/apt/transactions.apt b/src/site/apt/transactions.apt deleted file mode 100644 index 4ce3b746d9..0000000000 --- a/src/site/apt/transactions.apt +++ /dev/null @@ -1,354 +0,0 @@ - ------ - Spring Batch-Retry Transaction Propagation - ------ - Dave Syer - ------ - February 2007 - -Batch Processing and Transactions - -* {Simple Batching} with No Retry - - Consider the following simple example of a nested batch with no - retries. This is a very common scenario for batch processing, where - an input source is processed until exhausted, but we commit - periodically at the end of a "chunk" of processing. - -+--- -1 | REPEAT(until=exhausted) { - | -2 | TX { -3 | REPEAT(size=5) { -3.1 | input; -3.2 | output; - | } - | } - | - | } -+--- - - The input operation (3.1) could be a message-based receive - (e.g. JMS), or a file-based read, but to recover and continue - processing with a chance of completing the whole job, it must be - transactional. The same applies to the operation at (3.2) - it must - be either transactional or idempotent. - - If the chunk at REPEAT(3) fails because of a database exception at - (3.2), then TX(2) will roll back the whole chunk. - -* Simple Stateless Retry - - It is also useful to use a retry for an operation which is not - transactional, like a call to a web-service or other remote - resource. For example: - -+--- -0 | TX { -1 | input; -1.1 | output; -2 | RETRY { -2.1 | remote access; - | } - | } -+--- - - This is actually one of the most useful applications of a retry, - since a remote call is much more likely to fail and be retryable - than a database update. As long as the remote access (2.1) - eventually succeeds, the transaction TX(0) will commit. If the - remote access (2.1) eventually fails, then the transaction TX(0) is - guaranteed to roll back. - -* {Typical} Repeat-Retry Pattern - - The most typical batch processing pattern is to add a retry to the - inner block of the chunk in the {{{Simple Batching}simple}} example. - Consider this: - -+--- -1 | REPEAT(until=exhausted, exception=not critical) { - | -2 | TX { -3 | REPEAT(size=5) { - | -4 | RETRY(stateful, exception=deadlock loser) { -4.1 | input; -5 | } PROCESS { -5.1 | output; -6 | } SKIP and RECOVER { - | notify; - | } - | - | } - | } - | - | } -+--- - - The inner RETRY(4) block is marked as "stateful" - see the - {{{Typical}typical}} use case for a description of a stateful - retry. This means that if the the retry PROCESS(5) block fails, the - behaviour of the RETRY(4) is as follows. - - * Throw an exception, rolling back the transaction TX(2) at the - chunk level, and allowing the item to be re-presented to the input - queue. - - * When the item re-appears, it might be retried depending on the - retry policy in place, executing PROCESS(5) again. The second and - subsequent attempts might fail again and rethrow the exception. - - * Eventually the item re-appears for the final time: the retry - policy disallows another attempt, so PROCESS(5) is never - executed. In this case we follow a RECOVER(6) path, effectively - "skipping" the item that was received and is being processed. - - Notice that the notation used for the RETRY(4) in the plan above - shows explictly that the the input step (4.1) is part of the retry. - It also makes clear that there are two alternate paths for - processing: the normal case is denoted by PROCESS(5), and the - recovery path is a separate block, RECOVER(6). The two alternate - paths are completely distinct: only one is ever taken in normal - circumstances. - - In special cases (e.g. a special <<>> - type), the retry policy might be able to determine that the - RECOVER(6) path can be taken on the last attempt after PROCESS(5) - has just failed, instead of waiting for the item to be re-presented. - This is not the default behavior because it requires detailed - knowledge of what has happened inside the PROCESS(5) block, which is - not usually available - e.g. if the output included write - access before the failure, then the exception should be rethrown to - ensure transactional integrity. - - The completion policy in the outer, REPEAT(1) is crucial to the - success of the above plan. If the output(5.1) fails it may throw an - exception (it usually does, as described), in which case the - transaction TX(2) fails and the exception could propagate up through - the outer batch REPEAT(1). We do not want the whole batch to stop - because the RETRY(4) might still be successful if we try again, so - we add the exception=not critical to the outer REPEAT(1). - - Note, however, that if the TX(2) fails and we try again, by - virtue of the outer completion policy, the item that is next - processed in the inner REPEAT(3) is not guaranteed to be the one - that just failed. It might well be, but it depends on the - implementation of the input(4.1). Thus the output(5.1) might fail - again, on a new item, or on the old one. The client of the batch - should not assume that each RETRY(4) attempt is going to process the - same items as the last one that failed. E.g. if the termination - policy for REPEAT(1) is to fail after 10 attempts, it will fail - after 10 consecutive attempts, but not necessarily at the same item. - This is consistent with the overall retry strategy: it is the inner - RETRY(4) that is aware of the history of each item, and can decide - whether or not to have another attempt at it. - -* Asynchronous Chunk Processing - - The inner batches or chunks in the {{{Typical}typical}} example - above can be executed concurrently by configuring the outer batch to - use an <<>>. The outer batch waits for all the - chunks to complete before completing. - -+--- -1 | REPEAT(until=exhausted, concurrent, exception=not critical) { - | -2 | TX { -3 | REPEAT(size=5) { - | -4 | RETRY(stateful, exception=deadlock loser) { -4.1 | input; -5 | } PROCESS { - | output; -6 | } RECOVER { - | recover; - | } - | - | } - | } - | - | } -+--- - -* Asynchronous Item Processing - - The individual items in chunks in the {{{Typical}typical}} - can also in principle be processed concurrently. In this case the - transaction boundary has to move to the level of the individual - item, so that each transaction is on a single thread: - -+--- -1 | REPEAT(until=exhausted, exception=not critical) { - | -2 | REPEAT(size=5, concurrent) { - | -3 | TX { -4 | RETRY(stateful, exception=deadlock loser) { -4.1 | input; -5 | } PROCESS { - | output; -6 | } RECOVER { - | recover; - | } - | } - | - | } - | - | } -+--- - - This plan sacrifices the optimisation benefit, that the simple plan - had, of having all the transactional resources chunked together. It - is only useful if the cost of the processing (5) is much higher than - the cost of transaction management (3). - -Interactions Between Batching and Transaction Propagation - - There is a tighter coupling between batch-retry and TX management - than we would ideally like. In particular a stateless retry cannot - be used to retry database operations with a transaction manager that - doesn't support NESTED propagation. - - For a simple example using retry without repeat, consider this: - -+--- -1 | TX { - | -1.1 | input; -2.2 | database access; -2 | RETRY { -3 | TX { -3.1 | database access; - | } - | } - | - | } -+--- - - Again, and for the same reason, the inner transaction TX(3) can - cause the outer transaction TX(1) to fail, even if the RETRY(2) is - eventually successful. - - Unfortunately the same effect percolates from the retry block up to - the surrounding repeat batch if there is one: - -+--- -1 | TX { - | -2 | REPEAT(size=5) { -2.1 | input; -2.2 | database access; -3 | RETRY { -4 | TX { -4.1 | database access; - | } - | } - | } - | - | } -+--- - - Now if TX(3) rolls back it can pollute the whole batch at TX(1) and - force it to roll back at the end. - - What about non-default propagation? - - * In the last example PROPAGATION_REQUIRES_NEW at TX(3) will - prevent the outer TX(1) from being polluted if both transactions - are eventually successful. But if TX(3) commits and TX(1) rolls - back, then TX(3) stays committed, so we violate the transaction - contract for TX(1). - - If TX(3) rolls back, TX(1) does not necessarily (but it probably - will in practice because the retry will throw a roll back - exception). - - * PROPAGATION_NESTED at TX(3) works as we require in the retry - case (and for a batch with skips): TX(3) can commit, but - subsequently be rolled back by the outer transaction TX(1). If - TX(3) rolls back, again TX(1) will roll back in practice. This - option is only available on some platforms, e.g. not Hibernate or - JTA, but it is the only one that works consistently. - - So NESTED is best if the retry block contains any database access. - -* Special Case: Transactions with Orthogonal Resources - - Default propagation is always OK for simple cases where there are no - nested database transactions. Consider this (where the SESSION and - TX are not global XA resources, so their resources are orthogonal): - -+--- -0 | SESSION { -1 | input; -2 | RETRY { -3 | TX { -3.1 | database access; - | } - | } - | } -+--- - - Here there is a transactional message SESSION(0), but it doesn't - participate in other transactions with - <<>>, so doesn't propagate when TX(3) - starts. There is no database access outside the RETRY(2) block. If - TX(3) fails and then eventually succeeds on a retry, SESSION(0) can - commit (it can do this independent of a TX block). This is similar - to the vanilla "best-efforts-one-phase-commit" scenario - the worst - that can happen is a duplicate message when the RETRY(2) succeeds - and the SESSION(0) cannot commit, e.g. because the message system is - unavailable. - -* Stateless Retry Cannot Recover - - The distinction between a stateless and a stateful retry in the - {{{Typical}typical}} example above is important. It is actually - ultimately a transactional constraint that forces the distinction, - and this constraint also makes it obvious why the distinction - exists. - - We start with the observation that there is no way to skip an item - that failed and successfully commit the rest of the chunk unless we - wrap the item processing in a transaction. So we simplify the - {{{Typical}typical}} batch execution plan to look like this: - -+--- -0 | REPEAT(until=exhausted) { - | -1 | TX { -2 | REPEAT(size=5) { - | -3 | RETRY(stateless) { -4 | TX { -4.1 | input; -4.2 | database access; - | } -5 | } RECOVER { -5.1 | skip; - | } - | - | } - | } - | - | } -+--- - - Here we have a stateless RETRY(3) with a RECOVER(5) path that kicks - in after the final attempt fails. The "stateless" label just means - that the block will be repeated without rethrowing any exception up - to some limit. This will only work if the transaction TX(4) has - propagation NESTED. - - If the TX(3) has default propagation properties and it rolls back, - it will pollute the outer TX(1). The inner transaction is assumed by - the transaction manager to have corrupted the transactional - resource, and so it cannot be used again. - - Support for NESTED propagation is sufficiently rare that we choose - not to support recovery with stateless retries in current versions of - Spring Batch. The same effect can always be achieved (at the - expense of repeating more processing) using the - {{{Typical}typical}} pattern above. - - diff --git a/src/site/docbook/reference/appendix.xml b/src/site/docbook/reference/appendix.xml deleted file mode 100644 index 41bdf3cbe1..0000000000 --- a/src/site/docbook/reference/appendix.xml +++ /dev/null @@ -1,329 +0,0 @@ - - - - List of ItemReaders and ItemWriters - -
        - Item Readers - - - Available Item Readers - - - - - - - Item Reader - - Description - - - - - - AbstractItemCountingItemStreamItemReader - - Abstract base class that provides basic - restart capabilities by counting the number of items returned from - an ItemReader. - - - - AggregateItemReader - - An ItemReader that delivers a list as its - item, storing up objects from the injected ItemReader until they - are ready to be packed out as a collection. This ItemReader should - mark the beginning and end of records with the constant values in - FieldSetMapper AggregateItemReader#BEGIN_RECORD and - AggregateItemReader#END_RECORD - - - - AmqpItemReader - - Given a Spring AmqpTemplate it provides - synchronous receive methods. The receiveAndConvert() method - lets you receive POJO objects. - - - - FlatFileItemReader - - Reads from a flat file. Includes ItemStream - and Skippable functionality. See section on Read from a - File - - - - HibernateCursorItemReader - - Reads from a cursor based on an HQL query. See - section on Reading from a Database - - - - HibernatePagingItemReader - - Reads from a paginated HQL query - - - - IbatisPagingItemReader - - Reads via iBATIS based on a query. Pages - through the rows so that large datasets can be read without - running out of memory. See HOWTO - Read from a Database - - - - ItemReaderAdapter - - Adapts any class to the - ItemReader interface. - - - - JdbcCursorItemReader - - Reads from a database cursor via JDBC. See - HOWTO - Read from a Database - - - - JdbcPagingItemReader - - Given a SQL statement, pages through the rows, - such that large datasets can be read without running out of - memory - - - - JmsItemReader - - Given a Spring JmsOperations object and a JMS - Destination or destination name to send errors, provides items - received through the injected JmsOperations receive() - method - - - - JpaPagingItemReader - - Given a JPQL statement, pages through the - rows, such that large datasets can be read without running out of - memory - - - - ListItemReader - - Provides the items from a list, one at a - time - - - - MongoItemReader - - Given a MongoOperations object and JSON based MongoDB - query, proides items received from the MongoOperations find method - - - - Neo4jItemReader - - Given a Neo4jOperations object and the components of a - Cyhper query, items are returned as the result of the Neo4jOperations.query - method - - - - RepositoryItemReader - - Given a Spring Data PagingAndSortingRepository object, - a Sort and the name of method to execute, returns items provided by the - Spring Data repository implementation - - - - StoredProcedureItemReader - - Reads from a database cursor resulting from the - execution of a database stored procedure. See HOWTO - Read from a - Database - - - - StaxEventItemReader - - Reads via StAX. See HOWTO - Read from a - File - - - -
        -
        - -
        - Item Writers - - - Available Item Writers - - - - - - - Item Writer - - Description - - - - - - AbstractItemStreamItemWriter - - Abstract base class that combines the - ItemStream and - ItemWriter interfaces. - - - - AmqpItemWriter - - Given a Spring AmqpTemplate it provides - for synchronous send method. The convertAndSend(Object) - method lets you send POJO objects. - - - - CompositeItemWriter - - Passes an item to the process method of each - in an injected List of ItemWriter objects - - - - FlatFileItemWriter - - Writes to a flat file. Includes ItemStream and - Skippable functionality. See section on Writing to a File - - - - GemfireItemWriter - - Using a GemfireOperations object, items wre either written - or removed from the Gemfire instance based on the configuration of the delete - flag - - - - HibernateItemWriter - - This item writer is hibernate session aware - and handles some transaction-related work that a non-"hibernate - aware" item writer would not need to know about and then delegates - to another item writer to do the actual writing. - - - - IbatisBatchItemWriter - - Writes items in a batch using the SqlMapClientTemplate - execute() method - - - - ItemWriterAdapter - - Adapts any class to the - ItemWriter interface. - - - - JdbcBatchItemWriter - - Uses batching features from a - PreparedStatement, if available, and can - take rudimentary steps to locate a failure during a - flush. - - - - JmsItemWriter - - Using a JmsOperations object, items are written - to the default queue via the JmsOperations.convertAndSend() method - - - - JpaItemWriter - - This item writer is JPA EntityManager aware - and handles some transaction-related work that a non-"jpa aware" - ItemWriter would not need to know about and - then delegates to another writer to do the actual writing. - - - - MimeMessageItemWriter - - Using Spring's JavaMailSender, items of type MimeMessage - are sent as mail messages - - - - MongoItemWriter - - Given a MongoOperations object, items are written - via the MongoOperations.save(Object) method. The actual write is delayed - until the last possible moment before the transaction commits. - - - - Neo4jItemWriter - - Given a Neo4jOperations object, items are persisted via the - save(Object) method or deleted via the delete(Object) per the - ItemWriter's configuration - - - - PropertyExtractingDelegatingItemWriter - - Extends AbstractMethodInvokingDelegator - creating arguments on the fly. Arguments are created by retrieving - the values from the fields in the item to be processed (via a - SpringBeanWrapper) based on an injected array of field - name - - - - RepositoryItemWriter - - Given a Spring Data CrudRepository implementation, - items are saved via the method specified in the configuration. - - - - StaxEventItemWriter - - Uses an ObjectToXmlSerializer implementation to - convert each item to XML and then writes it to an XML file using - StAX. - - - -
        -
        -
        diff --git a/src/site/docbook/reference/common-patterns.xml b/src/site/docbook/reference/common-patterns.xml deleted file mode 100644 index e17d00fc38..0000000000 --- a/src/site/docbook/reference/common-patterns.xml +++ /dev/null @@ -1,604 +0,0 @@ - - - - Common Batch Patterns - - Some batch jobs can be assembled purely from off-the-shelf components - in Spring Batch. For instance the ItemReader and - ItemWriter implementations can be configured to cover - a wide range of scenarios. However, for the majority of cases, custom code - will have to be written. The main API entry points for application - developers are the Tasklet, - ItemReader, ItemWriter and the - various listener interfaces. Most simple batch jobs will be able to use - off-the-shelf input from a Spring Batch ItemReader, - but it is often the case that there are custom concerns in the processing - and writing, which require developers to implement an - ItemWriter or - ItemProcessor. - - Here, we provide a few examples of common patterns in custom business - logic. These examples primarily feature the listener interfaces. It should - be noted that an ItemReader or - ItemWriter can implement a listener interface as - well, if appropriate. - -
        - Logging Item Processing and Failures - - A common use case is the need for special handling of errors in a - step, item by item, perhaps logging to a special channel, or inserting a - record into a database. A chunk-oriented Step - (created from the step factory beans) allows users to implement this use - case with a simple ItemReadListener, for errors on - read, and an ItemWriteListener, for errors on - write. The below code snippets illustrate a listener that logs both read - and write failures: - - public class ItemFailureLoggerListener extends ItemListenerSupport { - - private static Log logger = LogFactory.getLog("item.error"); - - public void onReadError(Exception ex) { - logger.error("Encountered error on read", e); - } - - public void onWriteError(Exception ex, Object item) { - logger.error("Encountered error on write", e); - } - -} - - Having implemented this listener it must be registered with the - step: - - <step id="simpleStep"> - ... - <listeners> - <listener> - <bean class="org.example...ItemFailureLoggerListener"/> - </listener> - </listeners> -</step> - - Remember that if your listener does anything in an - onError() method, it will be inside a transaction that is - going to be rolled back. If you need to use a transactional resource such - as a database inside an onError() method, consider adding a - declarative transaction to that method (see Spring Core Reference Guide - for details), and giving its propagation attribute the value - REQUIRES_NEW. -
        - -
        - Stopping a Job Manually for Business Reasons - - Spring Batch provides a stop() method - through the JobLauncher interface, but this is - really for use by the operator rather than the application programmer. - Sometimes it is more convenient or makes more sense to stop a job - execution from within the business logic. - - The simplest thing to do is to throw a - RuntimeException (one that isn't retried - indefinitely or skipped). For example, a custom exception type could be - used, as in the example below: - - public class PoisonPillItemWriter implements ItemWriter<T> { - - public void write(T item) throws Exception { - if (isPoisonPill(item)) { - throw new PoisonPillException("Posion pill detected: " + item); - } - } - -} - - Another simple way to stop a step from executing is to simply return - null from the ItemReader: - - public class EarlyCompletionItemReader implements ItemReader<T> { - - private ItemReader<T> delegate; - - public void setDelegate(ItemReader<T> delegate) { ... } - - public T read() throws Exception { - T item = delegate.read(); - if (isEndItem(item)) { - return null; // end the step here - } - return item; - } - -} - - The previous example actually relies on the fact that there is a - default implementation of the CompletionPolicy - strategy which signals a complete batch when the item to be processed is - null. A more sophisticated completion policy could be implemented and - injected into the Step through the - SimpleStepFactoryBean: - - <step id="simpleStep"> - <tasklet> - <chunk reader="reader" writer="writer" commit-interval="10" - chunk-completion-policy="completionPolicy"/> - </tasklet> -</step> - -<bean id="completionPolicy" class="org.example...SpecialCompletionPolicy"/> - - An alternative is to set a flag in the - StepExecution, which is checked by the - Step implementations in the framework in between - item processing. To implement this alternative, we need access to the - current StepExecution, and this can be achieved by - implementing a StepListener and registering it with - the Step. Here is an example of a listener that - sets the flag: - - public class CustomItemWriter extends ItemListenerSupport implements StepListener { - - private StepExecution stepExecution; - - public void beforeStep(StepExecution stepExecution) { - this.stepExecution = stepExecution; - } - - public void afterRead(Object item) { - if (isPoisonPill(item)) { - stepExecution.setTerminateOnly(true); - } - } - -} - - The default behavior here when the flag is set is for the step to - throw a JobInterruptedException. This can be - controlled through the StepInterruptionPolicy, but - the only choice is to throw or not throw an exception, so this is always - an abnormal ending to a job. -
        - -
        - Adding a Footer Record - - Often when writing to flat files, a "footer" record must be appended - to the end of the file, after all processing has be completed. This can - also be achieved using the FlatFileFooterCallback - interface provided by Spring Batch. The - FlatFileFooterCallback (and its counterpart, the - FlatFileHeaderCallback) are optional properties of - the FlatFileItemWriter: - - <bean id="itemWriter" class="org.spr...FlatFileItemWriter"> - <property name="resource" ref="outputResource" /> - <property name="lineAggregator" ref="lineAggregator"/> - <property name="headerCallback" ref="headerCallback" /> - <property name="footerCallback" ref="footerCallback" /> -</bean> - - The footer callback interface is very simple. It has just one method - that is called when the footer must be written: - - public interface FlatFileFooterCallback { - - void writeFooter(Writer writer) throws IOException; - -} - -
        - Writing a Summary Footer - - A very common requirement involving footer records is to aggregate - information during the output process and to append this information to - the end of the file. This footer serves as a summarization of the file - or provides a checksum. - - For example, if a batch job is writing - Trade records to a flat file, and there is a - requirement that the total amount from all the - Trades is placed in a footer, then the following - ItemWriter implementation can be used: - - public class TradeItemWriter implements ItemWriter<Trade>, - FlatFileFooterCallback { - - private ItemWriter<Trade> delegate; - - private BigDecimal totalAmount = BigDecimal.ZERO; - - public void write(List<? extends Trade> items) { - BigDecimal chunkTotal = BigDecimal.ZERO; - for (Trade trade : items) { - chunkTotal = chunkTotal.add(trade.getAmount()); - } - - delegate.write(items); - - // After successfully writing all items - totalAmount = totalAmount.add(chunkTotal); - } - - public void writeFooter(Writer writer) throws IOException { - writer.write("Total Amount Processed: " + totalAmount); - } - - public void setDelegate(ItemWriter delegate) {...} -} - - This TradeItemWriter stores a - totalAmount value that is increased with the - amount from each Trade item written. - After the last Trade is processed, the framework - will call writeFooter, which will put that - totalAmount into the file. Note that the - write method makes use of a temporary variable, - chunkTotalAmount, that stores the total of the trades - in the chunk. This is done to ensure that if a skip occurs in the - write method, that the - totalAmount will be left unchanged. It is only at - the end of the write method, once we are - guaranteed that no exceptions will be thrown, that we update the - totalAmount. - - In order for the writeFooter method to be - called, the TradeItemWriter (which implements - FlatFileFooterCallback) must be wired into the - FlatFileItemWriter as the - footerCallback: - - <bean id="tradeItemWriter" class="..TradeItemWriter"> - <property name="delegate" ref="flatFileItemWriter" /> -</bean> - -<bean id="flatFileItemWriter" class="org.spr...FlatFileItemWriter"> - <property name="resource" ref="outputResource" /> - <property name="lineAggregator" ref="lineAggregator"/> - <property name="footerCallback" ref="tradeItemWriter" /> -</bean> - - The way that the TradeItemWriter has been - so far will only function correctly if the Step - is not restartable. This is because the class is stateful (since it - stores the totalAmount), but the totalAmount - is not persisted to the database, and therefore, it cannot be retrieved - in the event of a restart. In order to make this class restartable, the - ItemStream interface should be implemented along - with the methods open and - update: - - public void open(ExecutionContext executionContext) { - if (executionContext.containsKey("total.amount") { - totalAmount = (BigDecimal) executionContext.get("total.amount"); - } -} - -public void update(ExecutionContext executionContext) { - executionContext.put("total.amount", totalAmount); -} - - The update method will store the most - current version of totalAmount to the - ExecutionContext just before that object is - persisted to the database. The open method will - retrieve any existing totalAmount from the - ExecutionContext and use it as the starting point - for processing, allowing the TradeItemWriter to - pick up on restart where it left off the previous time the - Step was executed. -
        -
        - -
        - Driving Query Based ItemReaders - - In the chapter on readers and writers, database input using paging - was discussed. Many database vendors, such as DB2, have extremely - pessimistic locking strategies that can cause issues if the table being - read also needs to be used by other portions of the online application. - Furthermore, opening cursors over extremely large datasets can cause - issues on certain vendors. Therefore, many projects prefer to use a - 'Driving Query' approach to reading in data. This approach works by - iterating over keys, rather than the entire object that needs to be - returned, as the following example illustrates: - - - - - - - - - - - - As you can see, this example uses the same 'FOO' table as was used - in the cursor based example. However, rather than selecting the entire - row, only the ID's were selected in the SQL statement. So, rather than a - FOO object being returned from read, an Integer - will be returned. This number can then be used to query for the 'details', - which is a complete Foo object: - - - - - - - - - - - - An ItemProcessor should be used to transform the key obtained from - the driving query into a full 'Foo' object. An existing DAO can be used to - query for the full object based on the key. -
        - -
        - Multi-Line Records - - While it is usually the case with flat files that one each record is - confined to a single line, it is common that a file might have records - spanning multiple lines with multiple formats. The following excerpt from - a file illustrates this: - - HEA;0013100345;2007-02-15 -NCU;Smith;Peter;;T;20014539;F -BAD;;Oak Street 31/A;;Small Town;00235;IL;US -FOT;2;2;267.34 - - Everything between the line starting with 'HEA' and the line - starting with 'FOT' is considered one record. There are a few - considerations that must be made in order to handle this situation - correctly: - - - - Instead of reading one record at a time, the - ItemReader must read every line of the - multi-line record as a group, so that it can be passed to the - ItemWriter intact. - - - - Each line type may need to be tokenized differently. - - - - Because a single record spans multiple lines, and we may not know - how many lines there are, the ItemReader must be - careful to always read an entire record. In order to do this, a custom - ItemReader should be implemented as a wrapper for - the FlatFileItemReader. - - <bean id="itemReader" class="org.spr...MultiLineTradeItemReader"> - <property name="delegate"> - <bean class="org.springframework.batch.item.file.FlatFileItemReader"> - <property name="resource" value="data/iosample/input/multiLine.txt" /> - <property name="lineMapper"> - <bean class="org.spr...DefaultLineMapper"> - <property name="lineTokenizer" ref="orderFileTokenizer"/> - <property name="fieldSetMapper"> - <bean class="org.spr...PassThroughFieldSetMapper" /> - </property> - </bean> - </property> - </bean> - </property> -</bean> - - To ensure that each line is tokenized properly, which is especially - important for fixed length input, the - PatternMatchingCompositeLineTokenizer can be used - on the delegate FlatFileItemReader. See for more details. The delegate - reader will then use a PassThroughFieldSetMapper to - deliver a FieldSet for each line back to the - wrapping ItemReader. - - <bean id="orderFileTokenizer" class="org.spr...PatternMatchingCompositeLineTokenizer"> - <property name="tokenizers"> - <map> - <entry key="HEA*" value-ref="headerRecordTokenizer" /> - <entry key="FOT*" value-ref="footerRecordTokenizer" /> - <entry key="NCU*" value-ref="customerLineTokenizer" /> - <entry key="BAD*" value-ref="billingAddressLineTokenizer" /> - </map> - </property> -</bean> - - This wrapper will have to be able recognize the end of a record so - that it can continually call read() on its - delegate until the end is reached. For each line that is read, the wrapper - should build up the item to be returned. Once the footer is reached, the - item can be returned for delivery to the - ItemProcessor and - ItemWriter. - - private FlatFileItemReader<FieldSet> delegate; - -public Trade read() throws Exception { - Trade t = null; - - for (FieldSet line = null; (line = this.delegate.read()) != null;) { - String prefix = line.readString(0); - if (prefix.equals("HEA")) { - t = new Trade(); // Record must start with header - } - else if (prefix.equals("NCU")) { - Assert.notNull(t, "No header was found."); - t.setLast(line.readString(1)); - t.setFirst(line.readString(2)); - ... - } - else if (prefix.equals("BAD")) { - Assert.notNull(t, "No header was found."); - t.setCity(line.readString(4)); - t.setState(line.readString(6)); - ... - } - else if (prefix.equals("FOT")) { - return t; // Record must end with footer - } - } - Assert.isNull(t, "No 'END' was found."); - return null; -} -
        - -
        - Executing System Commands - - Many batch jobs may require that an external command be called from - within the batch job. Such a process could be kicked off separately by the - scheduler, but the advantage of common meta-data about the run would be - lost. Furthermore, a multi-step job would also need to be split up into - multiple jobs as well. - - Because the need is so common, Spring Batch provides a - Tasklet implementation for calling system - commands: - - <bean class="org.springframework.batch.core.step.tasklet.SystemCommandTasklet"> - <property name="command" value="echo hello" /> - <!-- 5 second timeout for the command to complete --> - <property name="timeout" value="5000" /> -</bean> -
        - -
        - Handling Step Completion When No Input is Found - - In many batch scenarios, finding no rows in a database or file to - process is not exceptional. The Step is simply - considered to have found no work and completes with 0 items read. All of - the ItemReader implementations provided out of the - box in Spring Batch default to this approach. This can lead to some - confusion if nothing is written out even when input is present. (which - usually happens if a file was misnamed, etc) For this reason, the meta - data itself should be inspected to determine how much work the framework - found to be processed. However, what if finding no input is considered - exceptional? In this case, programmatically checking the meta data for no - items processed and causing failure is the best solution. Because this is - a common use case, a listener is provided with just this - functionality: - - public class NoWorkFoundStepExecutionListener extends StepExecutionListenerSupport { - - public ExitStatus afterStep(StepExecution stepExecution) { - if (stepExecution.getReadCount() == 0) { - return ExitStatus.FAILED; - } - return null; - } - -} - - The above StepExecutionListener inspects the - readCount property of the StepExecution during the - 'afterStep' phase to determine if no items were read. If that is the case, - an exit code of FAILED is returned, indicating that the - Step should fail. Otherwise, null is returned, - which will not affect the status of the - Step. -
        - -
        - Passing Data to Future Steps - - It is often useful to pass information from one step to another. - This can be done using the ExecutionContext. The - catch is that there are two ExecutionContexts: one - at the Step level and one at the - Job level. The Step - ExecutionContext lives only as long as the step - while the Job - ExecutionContext lives through the whole - Job. On the other hand, the - Step ExecutionContext is - updated every time the Step commits a chunk while - the Job ExecutionContext is - updated only at the end of each Step. - - The consequence of this separation is that all data must be placed - in the Step ExecutionContext - while the Step is executing. This will ensure that - the data will be stored properly while the Step is - on-going. If data is stored to the Job - ExecutionContext, then it will not be persisted - during Step execution and if the - Step fails, that data will be lost. - - public class SavingItemWriter implements ItemWriter<Object> { - private StepExecution stepExecution; - - public void write(List<? extends Object> items) throws Exception { - // ... - - ExecutionContext stepContext = this.stepExecution.getExecutionContext(); - stepContext.put("someKey", someObject); - } - - @BeforeStep - public void saveStepExecution(StepExecution stepExecution) { - this.stepExecution = stepExecution; - } -} - - To make the data available to future Steps, - it will have to be "promoted" to the Job - ExecutionContext after the step has finished. - Spring Batch provides the - ExecutionContextPromotionListener for this purpose. - The listener must be configured with the keys related to the data in the - ExecutionContext that must be promoted. It can - also, optionally, be configured with a list of exit code patterns for - which the promotion should occur ("COMPLETED" is the default). As with all - listeners, it must be registered on the - Step. - - <job id="job1"> - <step id="step1"> - <tasklet> - <chunk reader="reader" writer="savingWriter" commit-interval="10"/> - </tasklet> - <listeners> - <listener ref="promotionListener"/> - </listeners> - </step> - - <step id="step2"> - ... - </step> -</job> - -<beans:bean id="promotionListener" class="org.spr....ExecutionContextPromotionListener"> - <beans:property name="keys" value="someKey"/> -</beans:bean> - - Finally, the saved values must be retrieved from the - Job ExeuctionContext: - - public class RetrievingItemWriter implements ItemWriter<Object> { - private Object someObject; - - public void write(List<? extends Object> items) throws Exception { - // ... - } - - @BeforeStep - public void retrieveInterstepData(StepExecution stepExecution) { - JobExecution jobExecution = stepExecution.getJobExecution(); - ExecutionContext jobContext = jobExecution.getExecutionContext(); - this.someObject = jobContext.get("someKey"); - } -} -
        -
        diff --git a/src/site/docbook/reference/domain.xml b/src/site/docbook/reference/domain.xml deleted file mode 100644 index 5e1693e813..0000000000 --- a/src/site/docbook/reference/domain.xml +++ /dev/null @@ -1,1069 +0,0 @@ - - - - The Domain Language of Batch - - To any experienced batch architect, the overall concepts of batch - processing used in Spring Batch should be familiar and comfortable. There - are "Jobs" and "Steps" and developer supplied processing units called - ItemReaders and ItemWriters. However, because of the Spring patterns, - operations, templates, callbacks, and idioms, there are opportunities for - the following: - - significant improvement in adherence to a clear separation of - concerns - - - - clearly delineated architectural layers and services provided as - interfaces - - - - simple and default implementations that allow for quick adoption - and ease of use out-of-the-box - - - - significantly enhanced extensibility - - - - The diagram below is simplified version of the batch reference - architecture that has been used for decades. It provides an overview of the - components that make up the domain language of batch processing. This - architecture framework is a blueprint that has been proven through decades - of implementations on the last several generations of platforms - (COBOL/Mainframe, C++/Unix, and now Java/anywhere). JCL and COBOL developers - are likely to be as comfortable with the concepts as C++, C# and Java - developers. Spring Batch provides a physical implementation of the layers, - components and technical services commonly found in robust, maintainable - systems used to address the creation of simple to complex batch - applications, with the infrastructure and extensions to address very complex - processing needs. - - - - - - - - - - - Figure 2.1: Batch Stereotypes - - - The diagram above highlights the key concepts that make up the domain - language of batch. A Job has one to many steps, which has exactly one - ItemReader, ItemProcessor, and ItemWriter. A job needs to be launched - (JobLauncher), and meta data about the currently running process needs to be - stored (JobRepository). - -
        - Job - - This section describes stereotypes relating to the concept of a - batch job. A Job is an entity that encapsulates an - entire batch process. As is common with other Spring projects, a - Job will be wired together via an XML configuration - file or Java based configuration. This configuration may be referred to as - the "job configuration". However, Job is just the - top of an overall hierarchy: - - - - - - - - - - - - In Spring Batch, a Job is simply a container for Steps. It combines - multiple steps that belong logically together in a flow and allows for - configuration of properties global to all steps, such as restartability. - The job configuration contains: - - - - The simple name of the job - - - - Definition and ordering of Steps - - - - Whether or not the job is restartable - - - - A default simple implementation of the Job - interface is provided by Spring Batch in the form of the - SimpleJob class which creates some standard - functionality on top of Job, however the batch - namespace abstracts away the need to instantiate it directly. Instead, the - <job> tag can be used: - - <job id="footballJob"> - <step id="playerload" next="gameLoad"/> - <step id="gameLoad" next="playerSummarization"/> - <step id="playerSummarization"/> -</job> - -
        - JobInstance - - A JobInstance refers to the concept of a - logical job run. Let's consider a batch job that should be run once at - the end of the day, such as the 'EndOfDay' job from the diagram above. - There is one 'EndOfDay' Job, but each individual - run of the Job must be tracked separately. In the - case of this job, there will be one logical - JobInstance per day. For example, there will be a - January 1st run, and a January 2nd run. If the January 1st run fails the - first time and is run again the next day, it is still the January 1st - run. (Usually this corresponds with the data it is processing as well, - meaning the January 1st run processes data for January 1st, etc). - Therefore, each JobInstance can have multiple - executions (JobExecution is discussed in more - detail below) and only one JobInstance - corresponding to a particular Job and - identifying JobParameters can be running at a given - time. - - The definition of a JobInstance has - absolutely no bearing on the data the will be loaded. It is entirely up - to the ItemReader implementation used to - determine how data will be loaded. For example, in the EndOfDay - scenario, there may be a column on the data that indicates the - 'effective date' or 'schedule date' to which the data belongs. So, the - January 1st run would only load data from the 1st, and the January 2nd - run would only use data from the 2nd. Because this determination will - likely be a business decision, it is left up to the - ItemReader to decide. What using the same - JobInstance will determine, however, is whether - or not the 'state' (i.e. the ExecutionContext, - which is discussed below) from previous executions will be used. Using a - new JobInstance will mean 'start from the - beginning' and using an existing instance will generally mean 'start - from where you left off'. -
        - -
        - JobParameters - - Having discussed JobInstance and how it - differs from Job, the natural question to ask is: - "how is one JobInstance distinguished from - another?" The answer is: JobParameters. - JobParameters is a set of parameters used to - start a batch job. They can be used for identification or even as - reference data during the run: - - - - - - - - - - - - In the example above, where there are two instances, one for - January 1st, and another for January 2nd, there is really only one Job, - one that was started with a job parameter of 01-01-2008 and another that - was started with a parameter of 01-02-2008. Thus, the contract can be - defined as: JobInstance = - Job + identifying JobParameters. This - allows a developer to effectively control how a - JobInstance is defined, since they control what - parameters are passed in. -
        - - Not all job parameters are required to contribute to the identification - of a JobInstance. By default they do, however the framework - allows the submission of a Job with parameters that do - not contribute to the identity of a JobInstance as well. - - -
        - JobExecution - - A JobExecution refers to the technical - concept of a single attempt to run a Job. An - execution may end in failure or success, but the - JobInstance corresponding to a given execution - will not be considered complete unless the execution completes - successfully. Using the EndOfDay Job described - above as an example, consider a JobInstance for - 01-01-2008 that failed the first time it was run. If it is run again - with the same identifying job parameters as the first run (01-01-2008), a new - JobExecution will be created. However, there will - still be only one JobInstance. - - A Job defines what a job is and how it is - to be executed, and JobInstance is a purely - organizational object to group executions together, primarily to enable - correct restart semantics. A JobExecution, - however, is the primary storage mechanism for what actually happened - during a run, and as such contains many more properties that must be - controlled and persisted: - - - JobExecution Properties - - - - - - - - - status - - A BatchStatus object that - indicates the status of the execution. While running, it's - BatchStatus.STARTED, if it fails, it's BatchStatus.FAILED, and - if it finishes successfully, it's BatchStatus.COMPLETED - - - - startTime - - A java.util.Date representing the - current system time when the execution was started. - - - - endTime - - A java.util.Date representing the - current system time when the execution finished, regardless of - whether or not it was successful. - - - - exitStatus - - The ExitStatus indicating the - result of the run. It is most important because it contains an - exit code that will be returned to the caller. See chapter 5 for - more details. - - - - createTime - - A java.util.Date representing the - current system time when the JobExecution - was first persisted. The job may not have been started yet (and - thus has no start time), but it will always have a createTime, - which is required by the framework for managing job level - ExecutionContexts. - - - - lastUpdated - - A java.util.Date representing the - last time a JobExecution was - persisted. - - - - executionContext - - The 'property bag' containing any user data that needs to - be persisted between executions. - - - - failureExceptions - - The list of exceptions encountered during the execution - of a Job. These can be useful if more - than one exception is encountered during the failure of a - Job. - - - -
        - - These properties are important because they will be persisted and - can be used to completely determine the status of an execution. For - example, if the EndOfDay job for 01-01 is executed at 9:00 PM, and fails - at 9:30, the following entries will be made in the batch meta data - tables: - - - BATCH_JOB_INSTANCE - - - - - JOB_INST_ID - - JOB_NAME - - - - 1 - - EndOfDayJob - - - -
        - - - BATCH_JOB_EXECUTION_PARAMS - - - - - JOB_EXECUTION_ID - - TYPE_CD - - KEY_NAME - - DATE_VAL - - IDENTIFYING - - - - 1 - - DATE - - schedule.Date - - 2008-01-01 - - TRUE - - - -
        - - - BATCH_JOB_EXECUTION - - - - - JOB_EXEC_ID - - JOB_INST_ID - - START_TIME - - END_TIME - - STATUS - - - - 1 - - 1 - - 2008-01-01 21:00 - - 2008-01-01 21:30 - - FAILED - - - -
        - - - column names may have been abbreviated or removed for clarity - and formatting - - - Now that the job has failed, let's assume that it took the entire - course of the night for the problem to be determined, so that the 'batch - window' is now closed. Assuming the window starts at 9:00 PM, the job - will be kicked off again for 01-01, starting where it left off and - completing successfully at 9:30. Because it's now the next day, the - 01-02 job must be run as well, which is kicked off just afterwards at - 9:31, and completes in its normal one hour time at 10:30. There is no - requirement that one JobInstance be kicked off - after another, unless there is potential for the two jobs to attempt to - access the same data, causing issues with locking at the database level. - It is entirely up to the scheduler to determine when a - Job should be run. Since they're separate - JobInstances, Spring Batch will make no attempt - to stop them from being run concurrently. (Attempting to run the same - JobInstance while another is already running will - result in a JobExecutionAlreadyRunningException - being thrown). There should now be an extra entry in both the - JobInstance and - JobParameters tables, and two extra entries in - the JobExecution table: - - - BATCH_JOB_INSTANCE - - - - - JOB_INST_ID - - JOB_NAME - - - - 1 - - EndOfDayJob - - - - 2 - - EndOfDayJob - - - -
        - - - BATCH_JOB_EXECUTION_PARAMS - - - - - JOB_EXECUTION_ID - - TYPE_CD - - KEY_NAME - - DATE_VAL - - IDENTIFYING - - - - 1 - - DATE - - schedule.Date - - 2008-01-01 00:00:00 - - TRUE - - - - 2 - - DATE - - schedule.Date - - 2008-01-01 00:00:00 - - TRUE - - - - 3 - - DATE - - schedule.Date - - 2008-01-02 00:00:00 - - TRUE - - - -
        - - - BATCH_JOB_EXECUTION - - - - - JOB_EXEC_ID - - JOB_INST_ID - - START_TIME - - END_TIME - - STATUS - - - - 1 - - 1 - - 2008-01-01 21:00 - - 2008-01-01 21:30 - - FAILED - - - - 2 - - 1 - - 2008-01-02 21:00 - - 2008-01-02 21:30 - - COMPLETED - - - - 3 - - 2 - - 2008-01-02 21:31 - - 2008-01-02 22:29 - - COMPLETED - - - -
        - - - column names may have been abbreviated or removed for clarity - and formatting - -
        -
        - -
        - Step - - A Step is a domain object that encapsulates - an independent, sequential phase of a batch job. Therefore, every - Job is composed entirely of one or more steps. A - Step contains all of the information necessary to - define and control the actual batch processing. This is a necessarily - vague description because the contents of any given - Step are at the discretion of the developer writing - a Job. A Step can be as simple or complex as the - developer desires. A simple Step might load data - from a file into the database, requiring little or no code. (depending - upon the implementations used) A more complex Step - may have complicated business rules that are applied as part of the - processing. As with Job, a - Step has an individual - StepExecution that corresponds with a unique - JobExecution: - - - - - - - - - - - -
        - StepExecution - - A StepExecution represents a single attempt - to execute a Step. A new - StepExecution will be created each time a - Step is run, similar to - JobExecution. However, if a step fails to execute - because the step before it fails, there will be no execution persisted - for it. A StepExecution will only be created when - its Step is actually started. - - Step executions are represented by objects of the - StepExecution class. Each execution contains a - reference to its corresponding step and - JobExecution, and transaction related data such - as commit and rollback count and start and end times. Additionally, each - step execution will contain an ExecutionContext, - which contains any data a developer needs persisted across batch runs, - such as statistics or state information needed to restart. The following - is a listing of the properties for - StepExecution: - - - StepExecution Properties - - - - - - - - - status - - A BatchStatus object that - indicates the status of the execution. While it's running, the - status is BatchStatus.STARTED, if it fails, the status is - BatchStatus.FAILED, and if it finishes successfully, the status - is BatchStatus.COMPLETED - - - - startTime - - A java.util.Date representing the - current system time when the execution was started. - - - - endTime - - A java.util.Date representing the - current system time when the execution finished, regardless of - whether or not it was successful. - - - - exitStatus - - The ExitStatus indicating the - result of the execution. It is most important because it - contains an exit code that will be returned to the caller. See - chapter 5 for more details. - - - - executionContext - - The 'property bag' containing any user data that needs to - be persisted between executions. - - - - readCount - - The number of items that have been successfully - read - - - - writeCount - - The number of items that have been successfully - written - - - - commitCount - - The number transactions that have been committed for this - execution - - - - rollbackCount - - The number of times the business transaction controlled - by the Step has been rolled back. - - - - readSkipCount - - The number of times read has - failed, resulting in a skipped item. - - - - processSkipCount - - The number of times process has - failed, resulting in a skipped item. - - - - filterCount - - The number of items that have been 'filtered' by the - ItemProcessor. - - - - writeSkipCount - - The number of times write has - failed, resulting in a skipped item. - - - -
        -
        -
        - -
        - ExecutionContext - - An ExecutionContext represents a collection - of key/value pairs that are persisted and controlled by the framework in - order to allow developers a place to store persistent state that is scoped - to a StepExecution or - JobExecution. For those familiar with Quartz, it is - very similar to JobDataMap. The best usage example - is to facilitate restart. Using flat file input as an example, while - processing individual lines, the framework periodically persists the - ExecutionContext at commit points. This allows the - ItemReader to store its state in case a fatal error - occurs during the run, or even if the power goes out. All that is needed - is to put the current number of lines read into the context, and the - framework will do the rest: - - executionContext.putLong(getKey(LINES_READ_COUNT), reader.getPosition()); - - Using the EndOfDay example from the Job Stereotypes section as an - example, assume there's one step: 'loadData', that loads a file into the - database. After the first failed run, the meta data tables would look like - the following: - - - BATCH_JOB_INSTANCE - - - - - JOB_INST_ID - - JOB_NAME - - - - 1 - - EndOfDayJob - - - -
        - BATCH_JOB_PARAMS - - - - - JOB_INST_ID - - TYPE_CD - - KEY_NAME - - DATE_VAL - - - - 1 - - DATE - - schedule.Date - - 2008-01-01 - - - -
        - BATCH_JOB_EXECUTION - - - - - JOB_EXEC_ID - - JOB_INST_ID - - START_TIME - - END_TIME - - STATUS - - - - 1 - - 1 - - 2008-01-01 21:00 - - 2008-01-01 21:30 - - FAILED - - - -
        - BATCH_STEP_EXECUTION - - - - - STEP_EXEC_ID - - JOB_EXEC_ID - - STEP_NAME - - START_TIME - - END_TIME - - STATUS - - - - 1 - - 1 - - loadDate - - 2008-01-01 21:00 - - 2008-01-01 21:30 - - FAILED - - - -
        - BATCH_STEP_EXECUTION_CONTEXT - - - - - STEP_EXEC_ID - - SHORT_CONTEXT - - - - 1 - - {piece.count=40321} - - - -
        In this case, the Step ran for 30 minutes - and processed 40,321 'pieces', which would represent lines in a file in - this scenario. This value will be updated just before each commit by the - framework, and can contain multiple rows corresponding to entries within - the ExecutionContext. Being notified before a - commit requires one of the various StepListeners, - or an ItemStream, which are discussed in more - detail later in this guide. As with the previous example, it is assumed - that the Job is restarted the next day. When it is - restarted, the values from the ExecutionContext of - the last run are reconstituted from the database, and when the - ItemReader is opened, it can check to see if it has - any stored state in the context, and initialize itself from there:
        - - if (executionContext.containsKey(getKey(LINES_READ_COUNT))) { - log.debug("Initializing for restart. Restart data is: " + executionContext); - - long lineCount = executionContext.getLong(getKey(LINES_READ_COUNT)); - - LineReader reader = getReader(); - - Object record = ""; - while (reader.getPosition() < lineCount && record != null) { - record = readLine(); - } -} - - In this case, after the above code is executed, the current line - will be 40,322, allowing the Step to start again - from where it left off. The ExecutionContext can - also be used for statistics that need to be persisted about the run - itself. For example, if a flat file contains orders for processing that - exist across multiple lines, it may be necessary to store how many orders - have been processed (which is much different from than the number of lines - read) so that an email can be sent at the end of the - Step with the total orders processed in the body. - The framework handles storing this for the developer, in order to - correctly scope it with an individual JobInstance. - It can be very difficult to know whether an existing - ExecutionContext should be used or not. For - example, using the 'EndOfDay' example from above, when the 01-01 run - starts again for the second time, the framework recognizes that it is the - same JobInstance and on an individual - Step basis, pulls the - ExecutionContext out of the database and hands it - as part of the StepExecution to the - Step itself. Conversely, for the 01-02 run the - framework recognizes that it is a different instance, so an empty context - must be handed to the Step. There are many of these - types of determinations that the framework makes for the developer to - ensure the state is given to them at the correct time. It is also - important to note that exactly one ExecutionContext - exists per StepExecution at any given time. Clients - of the ExecutionContext should be careful because - this creates a shared keyspace, so care should be taken when putting - values in to ensure no data is overwritten. However, the - Step stores absolutely no data in the context, so - there is no way to adversely affect the framework. - - It is also important to note that there is at least one - ExecutionContext per - JobExecution, and one for every - StepExecution. For example, consider the following - code snippet: - - ExecutionContext ecStep = stepExecution.getExecutionContext(); -ExecutionContext ecJob = jobExecution.getExecutionContext(); -//ecStep does not equal ecJob - - As noted in the comment, ecStep will not equal ecJob; they are two - different ExecutionContexts. The one scoped to the - Step will be saved at every commit point in the - Step, whereas the one scoped to the - Job will be saved in between every - Step execution. -
        - -
        - JobRepository - - JobRepository is the persistence mechanism - for all of the Stereotypes mentioned above. It provides CRUD operations - for JobLauncher, Job, and - Step implementations. When a - Job is first launched, a - JobExecution is obtained from the repository, and - during the course of execution StepExecution and - JobExecution implementations are persisted by - passing them to the repository: - - <job-repository id="jobRepository"/> -
        - -
        - JobLauncher - - JobLauncher represents a simple interface for - launching a Job with a given set of - JobParameters: - - public interface JobLauncher { - - public JobExecution run(Job job, JobParameters jobParameters) - throws JobExecutionAlreadyRunningException, JobRestartException; -} - - It is expected that implementations will obtain a valid - JobExecution from the - JobRepository and execute the - Job. -
        - -
        - Item Reader - - ItemReader is an abstraction that represents - the retrieval of input for a Step, one item at a - time. When the ItemReader has exhausted the items - it can provide, it will indicate this by returning null. More details - about the ItemReader interface and its various - implementations can be found in . -
        - -
        - Item Writer - - ItemWriter is an abstraction that - represents the output of a Step, one batch - or chunk of items at a time. Generally, an item writer has no - knowledge of the input it will receive next, only the item that - was passed in its current invocation. More details about the - ItemWriter interface and its various - implementations can be found in . -
        - -
        - Item Processor - - ItemProcessor is an abstraction that - represents the business processing of an item. While the - ItemReader reads one item, and the - ItemWriter writes them, the - ItemProcessor provides access to transform or apply - other business processing. If, while processing the item, it is determined - that the item is not valid, returning null indicates that the item should - not be written out. More details about the ItemProcessor interface can be - found in . -
        - -
        - Batch Namespace - - Many of the domain concepts listed above need to be configured in a - Spring ApplicationContext. While there are - implementations of the interfaces above that can be used in a standard - bean definition, a namespace has been provided for ease of - configuration: - - <beans:beans xmlns="http://www.springframework.org/schema/batch" - xmlns:beans="http://www.springframework.org/schema/beans" - xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation=" - http://www.springframework.org/schema/beans - http://www.springframework.org/schema/beans/spring-beans.xsd - http://www.springframework.org/schema/batch - http://www.springframework.org/schema/batch/spring-batch-2.2.xsd"> - - <job id="ioSampleJob"> - <step id="step1"> - <tasklet> - <chunk reader="itemReader" writer="itemWriter" commit-interval="2"/> - </tasklet> - </step> - </job> - -</beans:beans> - - As long as the batch namespace has been declared, any of its - elements can be used. More information on configuring a - Job can be found in . More information on configuring a Step can be - found in . -
        -
        diff --git a/src/site/docbook/reference/glossary.xml b/src/site/docbook/reference/glossary.xml deleted file mode 100644 index 9b4892e115..0000000000 --- a/src/site/docbook/reference/glossary.xml +++ /dev/null @@ -1,216 +0,0 @@ - - - - - Spring Batch Glossary - - - Batch - - - An accumulation of business transactions over time. - - - - - Batch Application Style - - - Term used to designate batch as an application style in its own - right similar to online, Web or SOA. It has standard elements of - input, validation, transformation of information to business model, - business processing and output. In addition, it requires monitoring at - a macro level. - - - - - Batch Processing - - - The handling of a batch of many business transactions that have - accumulated over a period of time (e.g. an hour, day, week, month, or - year). It is the application of a process, or set of processes, to - many data entities or objects in a repetitive and predictable fashion - with either no manual element, or a separate manual element for error - processing. - - - - - Batch Window - - - The time frame within which a batch job must complete. This can - be constrained by other systems coming online, other dependent jobs - needing to execute or other factors specific to the batch - environment. - - - - - Step - - - It is the main batch task or unit of work controller. It - initializes the business logic, and controls the transaction - environment based on commit interval setting, etc. - - - - - Tasklet - - - A component created by application developer to process the - business logic for a Step. - - - - - Batch Job Type - - - Job Types describe application of jobs for particular type of - processing. Common areas are interface processing (typically flat - files), forms processing (either for online pdf generation or print - formats), report processing. - - - - - Driving Query - - - A driving query identifies the set of work for a job to do; the - job then breaks that work into individual units of work. For instance, - identify all financial transactions that have a status of "pending - transmission" and send them to our partner system. The driving query - returns a set of record IDs to process; each record ID then becomes a - unit of work. A driving query may involve a join (if the criteria for - selection falls across two or more tables) or it may work with a - single table. - - - - - Item - - - An item represents the smallest ammount of complete data for - processing. In the simplest terms, this might mean a line in a file, a - row in a database table, or a particular element in an XML - file. - - - - - Logicial Unit of Work (LUW) - - - A batch job iterates through a driving query (or another input - source such as a file) to perform the set of work that the job must - accomplish. Each iteration of work performed is a unit of work. - - - - - Commit Interval - - - A set of LUWs processed within a single transaction. - - - - - Partitioning - - - Splitting a job into multiple threads where each thread is - responsible for a subset of the overall data to be processed. The - threads of execution may be within the same JVM or they may span JVMs - in a clustered environment that supports workload balancing. - - - - - Staging Table - - - A table that holds temporary data while it is being - processed. - - - - - Restartable - - - A job that can be executed again and will assume the same - identity as when run initially. In othewords, it is has the same job - instance id. - - - - - Rerunnable - - - A job that is restartable and manages its own state in terms of - previous run's record processing. An example of a rerunnable step is - one based on a driving query. If the driving query can be formed so - that it will limit the processed rows when the job is restarted than - it is re-runnable. This is managed by the application logic. Often - times a condition is added to the where statement to limit the rows - returned by the driving query with something like "and processedFlag - != true". - - - - - Repeat - - - One of the most basic units of batch processing, that defines - repeatability calling a portion of code until it is finished, and - while there is no error. Typically a batch process would be repeatable - as long as there is input. - - - - - Retry - - - Simplifies the execution of operations with retry semantics most - frequently associated with handling transactional output exceptions. - Retry is slightly different from repeat, rather than continually - calling a block of code, retry is stateful, and continually calls the - same block of code with the same input, until it either succeeds, or - some type of retry limit has been exceeded. It is only generally - useful if a subsequent invocation of the operation might succeed - because something in the environment has improved. - - - - - Recover - - - Recover operations handle an exception in such a way that a - repeat process is able to continue. - - - - - Skip - - - Skip is a recovery strategy often used on file input sources as - the strategy for ignoring bad input records that failed - validation. - - - - diff --git a/src/site/docbook/reference/images/partitioning-overview.png b/src/site/docbook/reference/images/partitioning-overview.png deleted file mode 100644 index a9cd82430c..0000000000 Binary files a/src/site/docbook/reference/images/partitioning-overview.png and /dev/null differ diff --git a/src/site/docbook/reference/images/remote-chunking.png b/src/site/docbook/reference/images/remote-chunking.png deleted file mode 100644 index 004953a98f..0000000000 Binary files a/src/site/docbook/reference/images/remote-chunking.png and /dev/null differ diff --git a/src/site/docbook/reference/index.xml b/src/site/docbook/reference/index.xml deleted file mode 100644 index f9b862d253..0000000000 --- a/src/site/docbook/reference/index.xml +++ /dev/null @@ -1,88 +0,0 @@ - - - - - Spring Batch - Reference Documentation - - Spring Batch 2.2.1.RELEASE - - - - Lucas - Ward - - - Dave - Syer - - - - Thomas - Risberg - - - - Robert - Kasanicky - - - - Dan - Garrette - - - - Wayne - Lund - - - - Michael - Minella - - - - Chris - Schaefer - - - - - - Copies of this document may be made for your own use and for - distribution to others, provided that you do not charge any fee for such - copies and further provided that each copy contains this Copyright - Notice, whether distributed in print or electronically. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/src/site/docbook/reference/job.xml b/src/site/docbook/reference/job.xml deleted file mode 100644 index 997f97eafb..0000000000 --- a/src/site/docbook/reference/job.xml +++ /dev/null @@ -1,1157 +0,0 @@ - - - - Configuring and Running a Job - - In the domain section , the overall - architecture design was discussed, using the following diagram as a - guide: - - - - - - - - - - - - While the Job object may seem like a simple - container for steps, there are many configuration options of which a - developers must be aware . Furthermore, there are many considerations for - how a Job will be run and how its meta-data will be - stored during that run. This chapter will explain the various configuration - options and runtime concerns of a Job . - -
        - Configuring a Job - - There are multiple implementations of the - Job interface, however, the namespace - abstracts away the differences in configuration. It has only three - required dependencies: a name, JobRepository , and - a list of Steps. - - - - - -]]> - - The examples here use a parent bean definition to create the steps; - see the section on step configuration - for more options declaring specific step details inline. The XML namespace - defaults to referencing a repository with an id of 'jobRepository', which - is a sensible default. However, this can be overridden explicitly: - - job-repository="specialRepository" - - - -]]> - - In addition to steps a job configuration can contain other elements - that help with parallelisation (<split/>), - declarative flow control (<decision/>) and - externalization of flow definitions - (<flow/>). - -
        - Restartability - - One key issue when executing a batch job concerns the behavior of - a Job when it is restarted. The launching of a - Job is considered to be a 'restart' if a - JobExecution already exists for the particular - JobInstance. Ideally, all jobs should be able to - start up where they left off, but there are scenarios where this is not - possible. It is entirely up to the developer to - ensure that a new JobInstance is created in this - scenario. However, Spring Batch does provide some help. If a - Job should never be restarted, but should always - be run as part of a new JobInstance, then the - restartable property may be set to 'false': - - restartable="false" - ... -]]> - - To phrase it another way, setting restartable to false means "this - Job does not support being started again". Restarting a Job that is not - restartable will cause a JobRestartException to - be thrown: - - - - This snippet of JUnit code shows how attempting to create a - JobExecution the first time for a non restartable - job will cause no issues. However, the second - attempt will throw a JobRestartException. -
        - -
        - Intercepting Job Execution - - During the course of the execution of a - Job, it may be useful to be notified of various - events in its lifecycle so that custom code may be executed. The - SimpleJob allows for this by calling a - JobListener at the appropriate time: - - - - JobListeners can be added to a - SimpleJob via the listeners element on the - job: - - - - - -]]> <listeners> - <listener ref="sampleListener"/> - </listeners> -]]> - - It should be noted that afterJob will be - called regardless of the success or failure of the - Job. If success or failure needs to be determined - it can be obtained from the JobExecution: - - - - The annotations corresponding to this interface are: - - - - @BeforeJob - - - - @AfterJob - - -
        - -
        - Inheriting from a Parent Job - - If a group of Jobs share similar, but not - identical, configurations, then it may be helpful to define a "parent" - Job from which the concrete - Jobs may inherit properties. Similar to class - inheritance in Java, the "child" Job will combine - its elements and attributes with the parent's. - - In the following example, "baseJob" is an abstract - Job definition that defines only a list of - listeners. The Job "job1" is a concrete - definition that inherits the list of listeners from "baseJob" and merges - it with its own list of listeners to produce a - Job with two listeners and one - Step, "step1". - - - - - - - - - - - - - -]]> - - Please see the section on Inheriting from a Parent Step - for more detailed information. -
        - -
        - JobParametersValidator - - A job declared in the XML namespace or using any subclass of - AbstractJob can optionally declare a validator for the job parameters at - runtime. This is useful when for instance you need to assert that a job - is started with all its mandatory parameters. There is a - DefaultJobParametersValidator that can be used to constrain combinations - of simple mandatory and optional parameters, and for more complex - constraints you can implement the interface yourself. The configuration - of a validator is supported through the XML namespace through a child - element of the job, e.g: - - - - -]]> - - The validator can be specified as a reference (as above) or as a - nested bean definition in the beans namespace. -
        -
        - -
        - Java Config - - Spring 3 brought the ability to configure applications via java instead - of XML. As of Spring Batch 2.2.0, batch jobs can be configured using the same - java config. There are two components for the java based configuration: - the @EnableBatchConfiguration annotation and two builders. - - The @EnableBatchProcessing works similarly to the other - @Enable* annotations in the Spring family. In this case, - @EnableBatchProcessing provides a base configuration for - building batch jobs. Within this base configuration, an instance of - StepScope is createded in addition to a number of beans made - available to be autowired: - - - - - JobRepository - bean name "jobRepository" - - - JobLauncher - bean name "jobLauncher" - - - JobRegistry - bean name "jobRegistry" - - - PlatformTransactionManager - bean name "transactionManager" - - - JobBuilderFactory - bean name "jobBuilders" - - - StepBuilderFactory - bean name "stepBuilders" - - - - The core interface for this configuration is the BatchConfigurer. - The default implementation provides the beans mentioned above and requires a - DataSource as a bean within the context to be provided. This data - source will be used by the JobRepository. - - - - Only one configuration class needs to have the - @EnableBatchProcessing annotation. Once you have a class - annotated with it, you will have all of the above available. - - - With the base configuration in place, a user can use the provided builder factories - to configure a job. Below is an example of a two step job configured via the - JobBuilderFactory and the StepBuilderFactory. - - @Configuration -@EnableBatchProcessing -@Import(DataSourceCnfiguration.class) -public class AppConfig { - - @Autowired - private JobBuilderFactory jobs; - - @Autowired - private StepBuilderFactory steps; - - @Bean - public Job job() { - return jobs.get("myJob").start(step1()).next(step2()).build(); - } - - @Bean - protected Step step1(ItemReader<Person> reader, ItemProcessor<Person, Person> processor, ItemWriter<Person> writer) { - return steps.get("step1") - .<Person, Person> chunk(10) - .reader(reader) - .processor(processor) - .writer(writer) - .build(); - } - - @Bean - protected Step step2(Tasklet tasklet) { - return steps.get("step2") - .tasklet(tasklet) - .build(); - } -} - -
        - -
        - - - Configuring a JobRepository - - - - As described in earlier, the - JobRepository - is used for basic CRUD operations of the various persisted - domain objects within Spring Batch, such as - JobExecution and - StepExecution. It is required by many of the major - framework features, such as the JobLauncher, - Job, and Step. The batch - namespace abstracts away many of the implementation details of the - JobRepository implementations and their - collaborators. However, there are still a few configuration options - available: - - - - ]]> - - - - None of the configuration options listed above are required except - the id. If they are not set, the defaults shown above will be used. They - are shown above for awareness purposes. The - max-varchar-length defaults to 2500, which is the - length of the long VARCHAR columns in the sample schema scripts - - used to store things like exit code descriptions. If you don't modify the schema and you don't use multi-byte characters you shouldn't need to change it. - -
        - Transaction Configuration for the JobRepository - - If the namespace is used, transactional advice will be - automatically created around the repository. This is to ensure that the - batch meta data, including state that is necessary for restarts after a - failure, is persisted correctly. The behavior of the framework is not - well defined if the repository methods are not transactional. The - isolation level in the create* method attributes is - specified separately to ensure that when jobs are launched, if two - processes are trying to launch the same job at the same time, only one - will succeed. The default isolation level for that method is - SERIALIZABLE, which is quite aggressive: READ_COMMITTED would work just - as well; READ_UNCOMMITTED would be fine if two processes are not likely - to collide in this way. However, since a call to the - create* method is quite short, it is unlikely - that the SERIALIZED will cause problems, as long as the database - platform supports it. However, this can be overridden: - - - isolation-level-for-create="REPEATABLE_READ"]]> - - - If the namespace or factory beans aren't used then it is also - essential to configure the transactional behavior of the repository - using AOP: - - - - - - - - - - - -]]> - - - This fragment can be used as is, with almost no changes. Remember - also to include the appropriate namespace declarations and to make sure - spring-tx and spring-aop (or the whole of spring) are on the - classpath. -
        - - - -
        - Changing the Table Prefix - - Another modifiable property of the - JobRepository is the table prefix of the - meta-data tables. By default they are all prefaced with BATCH_. - BATCH_JOB_EXECUTION and BATCH_STEP_EXECUTION are two examples. However, - there are potential reasons to modify this prefix. If the schema names - needs to be prepended to the table names, or if more than one set of - meta data tables is needed within the same schema, then the table prefix - will need to be changed: - - table-prefix="SYSTEM.TEST_"]]> - - Given the above changes, every query to the meta data tables will - be prefixed with "SYSTEM.TEST_". BATCH_JOB_EXECUTION will be referred to - as SYSTEM.TEST_JOB_EXECUTION. - - - Only the table prefix is configurable. The table and column - names are not. - -
        - - - -
        - In-Memory Repository - - There are scenarios in which you may not want to persist your - domain objects to the database. One reason may be speed; storing domain - objects at each commit point takes extra time. Another reason may be - that you just don't need to persist status for a particular job. For - this reason, Spring batch provides an in-memory Map version of the job - repository: - - - -]]> - - Note that the in-memory repository is volatile and so does not - allow restart between JVM instances. It also cannot guarantee that two - job instances with the same parameters are launched simultaneously, and - is not suitable for use in a multi-threaded Job, or a locally - partitioned Step. So use the database version of the repository wherever - you need those features. - - However it does require a transaction manager to be defined - because there are rollback semantics within the repository, and because - the business logic might still be transactional (e.g. RDBMS access). For - testing purposes many people find the - ResourcelessTransactionManager useful. -
        - - - -
        - Non-standard Database Types in a Repository - - If you are using a database platform that is not in the list of - supported platforms, you may be able to use one of the supported types, - if the SQL variant is close enough. To do this you can use the raw - JobRepositoryFactoryBean instead of the namespace - shortcut and use it to set the database type to the closest - match: - - - - -]]> - - (The JobRepositoryFactoryBean tries to - auto-detect the database type from the DataSource - if it is not specified.) The major differences between platforms are - mainly accounted for by the strategy for incrementing primary keys, so - often it might be necessary to override the - incrementerFactory as well (using one of the standard - implementations from the Spring Framework). - - If even that doesn't work, or you are not using an RDBMS, then the - only option may be to implement the various Dao - interfaces that the SimpleJobRepository depends - on and wire one up manually in the normal Spring way. -
        - - -
        - -
        - Configuring a JobLauncher - - The most basic implementation of the - JobLauncher interface is the - SimpleJobLauncher. Its only required dependency is - a JobRepository, in order to obtain an - execution: - - - -]]> - - Once a JobExecution is - obtained, it is passed to the execute method of - Job, ultimately returning the - JobExecution to the caller: - - - - - - - - - - - - The sequence is straightforward and works well when launched from a - scheduler. However, issues arise when trying to launch from an HTTP - request. In this scenario, the launching needs to be done asynchronously - so that the SimpleJobLauncher returns immediately - to its caller. This is because it is not good practice to keep an HTTP - request open for the amount of time needed by long running processes such - as batch. An example sequence is below: - - - - - - - - - - - - The SimpleJobLauncher can easily be - configured to allow for this scenario by configuring a - TaskExecutor: - - - - - - -]]> - - Any implementation of the spring TaskExecutor - interface can be used to control how jobs are asynchronously - executed. -
        - -
        - Running a Job - - At a minimum, launching a batch job requires two things: the - Job to be launched and a - JobLauncher. Both can be contained within the same - context or different contexts. For example, if launching a job from the - command line, a new JVM will be instantiated for each Job, and thus every - job will have its own JobLauncher. However, if - running from within a web container within the scope of an - HttpRequest, there will usually be one - JobLauncher, configured for asynchronous job - launching, that multiple requests will invoke to launch their jobs. - -
        - Running Jobs from the Command Line - - For users that want to run their jobs from an enterprise - scheduler, the command line is the primary interface. This is because - most schedulers (with the exception of Quartz unless using the - NativeJob) work directly with operating system - processes, primarily kicked off with shell scripts. There are many ways - to launch a Java process besides a shell script, such as Perl, Ruby, or - even 'build tools' such as ant or maven. However, because most people - are familiar with shell scripts, this example will focus on them. - -
        - The CommandLineJobRunner - - Because the script launching the job must kick off a Java - Virtual Machine, there needs to be a class with a main method to act - as the primary entry point. Spring Batch provides an implementation - that serves just this purpose: - CommandLineJobRunner. It's important to note - that this is just one way to bootstrap your application, but there are - many ways to launch a Java process, and this class should in no way be - viewed as definitive. The CommandLineJobRunner - performs four tasks: - - - - Load the appropriate - ApplicationContext - - - - Parse command line arguments into - JobParameters - - - - Locate the appropriate job based on arguments - - - - Use the JobLauncher provided in the - application context to launch the job. - - - - All of these tasks are accomplished using only the arguments - passed in. The following are required arguments: - - - CommandLineJobRunner arguments - - - - - jobPath - - The location of the XML file that will be used to - create an ApplicationContext. This file - should contain everything needed to run the complete - Job - - - - jobName - - The name of the job to be run. - - - -
        - - These arguments must be passed in with the path first and the - name second. All arguments after these are considered to be - JobParameters and must be in the format of 'name=value': - - bash$ - - In most cases you would want to use a manifest to declare your - main class in a jar, but for simplicity, the class was used directly. - This example is using the same 'EndOfDay' example from the domain section. The first argument is - 'endOfDayJob.xml', which is the Spring - ApplicationContext containing the - Job. The second argument, 'endOfDay' represents - the job name. The final argument, 'schedule.date(date)=2007/05/05' - will be converted into JobParameters. An - example of the XML configuration is below: - - - - - - -]]> - - This example is overly simplistic, since there are many more - requirements to a run a batch job in Spring Batch in general, but it - serves to show the two main requirements of the - CommandLineJobRunner: - Job and - JobLauncher -
        - -
        - ExitCodes - - When launching a batch job from the command-line, an enterprise - scheduler is often used. Most schedulers are fairly dumb and work only - at the process level. This means that they only know about some - operating system process such as a shell script that they're invoking. - In this scenario, the only way to communicate back to the scheduler - about the success or failure of a job is through return codes. A - return code is a number that is returned to a scheduler by the process - that indicates the result of the run. In the simplest case: 0 is - success and 1 is failure. However, there may be more complex - scenarios: If job A returns 4 kick off job B, and if it returns 5 kick - off job C. This type of behavior is configured at the scheduler level, - but it is important that a processing framework such as Spring Batch - provide a way to return a numeric representation of the 'Exit Code' - for a particular batch job. In Spring Batch this is encapsulated - within an ExitStatus, which is covered in more - detail in Chapter 5. For the purposes of discussing exit codes, the - only important thing to know is that an - ExitStatus has an exit code property that is - set by the framework (or the developer) and is returned as part of the - JobExecution returned from the - JobLauncher. The - CommandLineJobRunner converts this string value - to a number using the ExitCodeMapper - interface: - - - - The essential contract of an - ExitCodeMapper is that, given a string exit - code, a number representation will be returned. The default - implementation used by the job runner is the SimpleJvmExitCodeMapper - that returns 0 for completion, 1 for generic errors, and 2 for any job - runner errors such as not being able to find a - Job in the provided context. If anything more - complex than the 3 values above is needed, then a custom - implementation of the ExitCodeMapper interface - must be supplied. Because the - CommandLineJobRunner is the class that creates - an ApplicationContext, and thus cannot be - 'wired together', any values that need to be overwritten must be - autowired. This means that if an implementation of - ExitCodeMapper is found within the BeanFactory, - it will be injected into the runner after the context is created. All - that needs to be done to provide your own - ExitCodeMapper is to declare the implementation - as a root level bean and ensure that it is part of the - ApplicationContext that is loaded by the - runner. -
        -
        - -
        - Running Jobs from within a Web Container - - Historically, offline processing such as batch jobs have been - launched from the command-line, as described above. However, there are - many cases where launching from an HttpRequest is - a better option. Many such use cases include reporting, ad-hoc job - running, and web application support. Because a batch job by definition - is long running, the most important concern is ensuring to launch the - job asynchronously: - - - - - - - - - - The controller in this case is a Spring MVC controller. More - information on Spring MVC can be found here: http://static.springsource.org/spring/docs/3.1.x/spring-framework-reference/html/mvc.html. - The controller launches a Job using a - JobLauncher that has been configured to launch - asynchronously, which - immediately returns a JobExecution. The - Job will likely still be running, however, this - nonblocking behaviour allows the controller to return immediately, which - is required when handling an HttpRequest. An - example is below: - - -
        -
        - -
        - Advanced Meta-Data Usage - - So far, both the JobLauncher and JobRepository interfaces have been - discussed. Together, they represent simple launching of a job, and basic - CRUD operations of batch domain objects: - - - - - - - - - - - - A JobLauncher uses the - JobRepository to create new - JobExecution objects and run them. - Job and Step implementations - later use the same JobRepository for basic updates - of the same executions during the running of a Job. - The basic operations suffice for simple scenarios, but in a large batch - environment with hundreds of batch jobs and complex scheduling - requirements, more advanced access of the meta data is required: - - - - - - - - - - - - The JobExplorer and - JobOperator interfaces, which will be discussed - below, add additional functionality for querying and controlling the meta - data. - -
        - Querying the Repository - - The most basic need before any advanced features is the ability to - query the repository for existing executions. This functionality is - provided by the JobExplorer interface: - - getJobInstances(String jobName, int start, int count); - - JobExecution getJobExecution(Long executionId); - - StepExecution getStepExecution(Long jobExecutionId, Long stepExecutionId); - - JobInstance getJobInstance(Long instanceId); - - List getJobExecutions(JobInstance jobInstance); - - Set findRunningJobExecutions(String jobName); -}]]> - - As is evident from the method signatures above, - JobExplorer is a read-only version of the - JobRepository, and like the - JobRepository, it can be easily configured via a - factory bean: - - ]]> - - Earlier in this - chapter, it was mentioned that the table prefix of the - JobRepository can be modified to allow for - different versions or schemas. Because the - JobExplorer is working with the same tables, it - too needs the ability to set a prefix: - - p:tablePrefix="BATCH_" ]]> -
        - -
        - JobRegistry - - A JobRegistry (and its parent interface JobLocator) is not - mandatory, but it can be useful if you want to keep track of which jobs - are available in the context. It is also useful for collecting jobs - centrally in an application context when they have been created - elsewhere (e.g. in child contexts). Custom JobRegistry implementations - can also be used to manipulate the names and other properties of the - jobs that are registered. There is only one implementation provided by - the framework and this is based on a simple map from job name to job - instance. It is configured simply like this: - - ]]> - - There are two ways to populate a JobRegistry automatically: using - a bean post processor and using a registrar lifecycle component. These - two mechanisms are described in the following sections. - -
        - JobRegistryBeanPostProcessor - - This is a bean post-processor that can register all jobs as they - are created: - - - -]]> - - Athough it is not strictly necessary the post-processor in the - example has been given an id so that it can be included in child - contexts (e.g. as a parent bean definition) and cause all jobs created - there to also be regsistered automatically. -
        - -
        - AutomaticJobRegistrar - - This is a lifecycle component that creates child contexts and - registers jobs from those contexts as they are created. One advantage - of doing this is that, while the job names in the child contexts still - have to be globally unique in the registry, their dependencies can - have "natural" names. So for example, you can create a set of XML - configuration files each having only one Job, - but all having different definitions of an - ItemReader with the same bean name, e.g. - "reader". If all those files were imported into the same context, the - reader definitions would clash and override one another, but with the - automatic regsistrar this is avoided. This makes it easier to - integrate jobs contributed from separate modules of an - application. - - - - - - - - - - - - -]]> - - The registrar has two mandatory properties, one is an array of - ApplicationContextFactory (here created from a - convenient factory bean), and the other is a - JobLoader. The JobLoader - is responsible for managing the lifecycle of the child contexts and - registering jobs in the JobRegistry. - - The ApplicationContextFactory is - responsible for creating the child context and the most common usage - would be as above using a - ClassPathXmlApplicationContextFactory. One of - the features of this factory is that by default it copies some of the - configuration down from the parent context to the child. So for - instance you don't have to re-define the - PropertyPlaceholderConfigurer or AOP - configuration in the child, if it should be the same as the - parent. - - The AutomaticJobRegistrar can be used in - conjunction with a JobRegistryBeanPostProcessor - if desired (as long as the DefaultJobLoader is - used as well). For instance this might be desirable if there are jobs - defined in the main parent context as well as in the child - locations. -
        -
        - -
        - JobOperator - - As previously discussed, the JobRepository - provides CRUD operations on the meta-data, and the - JobExplorer provides read-only operations on the - meta-data. However, those operations are most useful when used together - to perform common monitoring tasks such as stopping, restarting, or - summarizing a Job, as is commonly done by batch operators. Spring Batch - provides for these types of operations via the - JobOperator interface: - - getExecutions(long instanceId) throws NoSuchJobInstanceException; - - List getJobInstances(String jobName, int start, int count) - throws NoSuchJobException; - - Set getRunningExecutions(String jobName) throws NoSuchJobException; - - String getParameters(long executionId) throws NoSuchJobExecutionException; - - Long start(String jobName, String parameters) - throws NoSuchJobException, JobInstanceAlreadyExistsException; - - Long restart(long executionId) - throws JobInstanceAlreadyCompleteException, NoSuchJobExecutionException, - NoSuchJobException, JobRestartException; - - Long startNextInstance(String jobName) - throws NoSuchJobException, JobParametersNotFoundException, JobRestartException, - JobExecutionAlreadyRunningException, JobInstanceAlreadyCompleteException; - - boolean stop(long executionId) - throws NoSuchJobExecutionException, JobExecutionNotRunningException; - - String getSummary(long executionId) throws NoSuchJobExecutionException; - - Map getStepExecutionSummaries(long executionId) - throws NoSuchJobExecutionException; - - Set getJobNames(); - -}]]> - - The above operations represent methods from many different - interfaces, such as JobLauncher, - JobRepository, - JobExplorer, and - JobRegistry. For this reason, the provided - implementation of JobOperator, - SimpleJobOperator, has many dependencies: - - - - - - - - - - -]]> - - - If you set the table prefix on the job repository, don't forget to set it on the job explorer as well. - -
        - -
        - JobParametersIncrementer - - Most of the methods on JobOperator are - self-explanatory, and more detailed explanations can be found on the - javadoc - of the interface. However, the - startNextInstance method is worth noting. This - method will always start a new instance of a Job. - This can be extremely useful if there are serious issues in a - JobExecution and the Job - needs to be started over again from the beginning. Unlike - JobLauncher though, which requires a new - JobParameters object that will trigger a new - JobInstance if the parameters are different from - any previous set of parameters, the - startNextInstance method will use the - JobParametersIncrementer tied to the - Job to force the Job to a - new instance: - - - - The contract of JobParametersIncrementer is - that, given a JobParameters - object, it will return the 'next' JobParameters - object by incrementing any necessary values it may contain. This - strategy is useful because the framework has no way of knowing what - changes to the JobParameters make it the 'next' - instance. For example, if the only value in - JobParameters is a date, and the next instance - should be created, should that value be incremented by one day? Or one - week (if the job is weekly for instance)? The same can be said for any - numerical values that help to identify the Job, - as shown below: - - - - In this example, the value with a key of 'run.id' is used to - discriminate between JobInstances. If the - JobParameters passed in is null, it can be - assumed that the Job has never been run before - and thus its initial state can be returned. However, if not, the old - value is obtained, incremented by one, and returned. An incrementer can - be associated with Job via the 'incrementer' - attribute in the namespace: - - incrementer="sampleIncrementer" - ... -]]> -
        - -
        - Stopping a Job - - One of the most common use cases of - JobOperator is gracefully stopping a - Job: - - executions = jobOperator.getRunningExecutions("sampleJob"); -jobOperator.stop(executions.iterator().next()); ]]> - - The shutdown is not immediate, since there is no way to force - immediate shutdown, especially if the execution is currently in - developer code that the framework has no control over, such as a - business service. However, as soon as control is returned back to the - framework, it will set the status of the current - StepExecution to - BatchStatus.STOPPED, save it, then do the same - for the JobExecution before finishing. -
        - -
        - Aborting a Job - - A job execution which is FAILED can be - restarted (if the Job is restartable). A job execution whose status is - ABANDONED will not be restarted by the framework. - The ABANDONED status is also used in step - executions to mark them as skippable in a restarted job execution: if a - job is executing and encounters a step that has been marked - ABANDONED in the previous failed job execution, it - will move on to the next step (as determined by the job flow definition - and the step execution exit status). - - If the process died ("kill -9" or server - failure) the job is, of course, not running, but the JobRepository has - no way of knowing because no-one told it before the process died. You - have to tell it manually that you know that the execution either failed - or should be considered aborted (change its status to - FAILED or ABANDONED) - it's - a business decision and there is no way to automate it. Only change the - status to FAILED if it is not restartable, or if - you know the restart data is valid. There is a utility in Spring Batch - Admin JobService to abort a job execution. -
        -
        -
        diff --git a/src/site/docbook/reference/readersAndWriters.xml b/src/site/docbook/reference/readersAndWriters.xml deleted file mode 100644 index a606838882..0000000000 --- a/src/site/docbook/reference/readersAndWriters.xml +++ /dev/null @@ -1,2740 +0,0 @@ - - - - ItemReaders and ItemWriters - - All batch processing can be described in its most simple form as - reading in large amounts of data, performing some type of calculation or - transformation, and writing the result out. Spring Batch provides three key - interfaces to help perform bulk reading and writing: - ItemReader, ItemProcessor and - ItemWriter. - -
        - ItemReader - - Although a simple concept, an ItemReader is - the means for providing data from many different types of input. The most - general examples include: - - Flat File- Flat File Item Readers read lines of data from a - flat file that typically describe records with fields of data - defined by fixed positions in the file or delimited by some special - character (e.g. Comma). - - - - XML - XML ItemReaders process XML independently of - technologies used for parsing, mapping and validating objects. Input - data allows for the validation of an XML file against an XSD - schema. - - - - Database - A database resource is accessed to return - resultsets which can be mapped to objects for processing. The - default SQL ItemReaders invoke a RowMapper to - return objects, keep track of the current row if restart is - required, store basic statistics, and provide some transaction - enhancements that will be explained later. - - There are many more possibilities, but we'll focus on the - basic ones for this chapter. A complete list of all available ItemReaders - can be found in Appendix A. - - ItemReader is a basic interface for generic - input operations: - - public interface ItemReader<T> { - - T read() throws Exception, UnexpectedInputException, ParseException; - -} - - The read method defines the most essential - contract of the ItemReader; calling it returns one - Item or null if no more items are left. An item might represent a line in - a file, a row in a database, or an element in an XML file. It is generally - expected that these will be mapped to a usable domain object (i.e. Trade, - Foo, etc) but there is no requirement in the contract to do so. - - It is expected that implementations of the - ItemReader interface will be forward only. However, - if the underlying resource is transactional (such as a JMS queue) then - calling read may return the same logical item on subsequent calls in a - rollback scenario. It is also worth noting that a lack of items to process - by an ItemReader will not cause an exception to be - thrown. For example, a database ItemReader that is - configured with a query that returns 0 results will simply return null on - the first invocation of read. -
        - -
        - ItemWriter - - ItemWriter is similar in functionality to an - ItemReader, but with inverse operations. Resources - still need to be located, opened and closed but they differ in that an - ItemWriter writes out, rather than reading in. In - the case of databases or queues these may be inserts, updates, or sends. - The format of the serialization of the output is specific to each batch - job. - - As with ItemReader, - ItemWriter is a fairly generic interface: - - public interface ItemWriter<T> { - - void write(List<? extends T> items) throws Exception; - -} - - As with read on - ItemReader, write provides - the basic contract of ItemWriter; it will attempt - to write out the list of items passed in as long as it is open. Because it - is generally expected that items will be 'batched' together into a chunk - and then output, the interface accepts a list of items, rather than an - item by itself. After writing out the list, any flushing that may be - necessary can be performed before returning from the write method. For - example, if writing to a Hibernate DAO, multiple calls to write can be - made, one for each item. The writer can then call close on the hibernate - Session before returning. -
        - -
        - ItemProcessor - - The ItemReader and - ItemWriter interfaces are both very useful for - their specific tasks, but what if you want to insert business logic before - writing? One option for both reading and writing is to use the composite - pattern: create an ItemWriter that contains another - ItemWriter, or an ItemReader - that contains another ItemReader. For - example: - - public class CompositeItemWriter<T> implements ItemWriter<T> { - - ItemWriter<T> itemWriter; - - public CompositeItemWriter(ItemWriter<T> itemWriter) { - this.itemWriter = itemWriter; - } - - public void write(List<? extends T> items) throws Exception { - //Add business logic here - itemWriter.write(item); - } - - public void setDelegate(ItemWriter<T> itemWriter){ - this.itemWriter = itemWriter; - } -} - - The class above contains another ItemWriter - to which it delegates after having provided some business logic. This - pattern could easily be used for an ItemReader as - well, perhaps to obtain more reference data based upon the input that was - provided by the main ItemReader. It is also useful - if you need to control the call to write yourself. - However, if you only want to 'transform' the item passed in for writing - before it is actually written, there isn't much need to call - write yourself: you just want to modify the item. - For this scenario, Spring Batch provides the - ItemProcessor interface: - - public interface ItemProcessor<I, O> { - - O process(I item) throws Exception; -} - - An ItemProcessor is very simple; given one - object, transform it and return another. The provided object may or may - not be of the same type. The point is that business logic may be applied - within process, and is completely up to the developer to create. An - ItemProcessor can be wired directly into a step, - For example, assuming an ItemReader provides a - class of type Foo, and it needs to be converted to type Bar before being - written out. An ItemProcessor can be written that - performs the conversion: - - public class Foo {} - -public class Bar { - public Bar(Foo foo) {} -} - -public class FooProcessor implements ItemProcessor<Foo,Bar>{ - public Bar process(Foo foo) throws Exception { - //Perform simple transformation, convert a Foo to a Bar - return new Bar(foo); - } -} - -public class BarWriter implements ItemWriter<Bar>{ - public void write(List<? extends Bar> bars) throws Exception { - //write bars - } -} - - In the very simple example above, there is a class - Foo, a class Bar, and a - class FooProcessor that adheres to the - ItemProcessor interface. The transformation is - simple, but any type of transformation could be done here. The - BarWriter will be used to write out - Bar objects, throwing an exception if any other - type is provided. Similarly, the FooProcessor will - throw an exception if anything but a Foo is - provided. The FooProcessor can then be injected - into a Step: - - <job id="ioSampleJob"> - <step name="step1"> - <tasklet> - <chunk reader="fooReader" processor="fooProcessor" writer="barWriter" - commit-interval="2"/> - </tasklet> - </step> -</job> - -
        - Chaining ItemProcessors - - Performing a single transformation is useful in many scenarios, - but what if you want to 'chain' together multiple - ItemProcessors? This can be accomplished using - the composite pattern mentioned previously. To update the previous, - single transformation, example, Foo will be - transformed to Bar, which will be transformed to - Foobar and written out: - - public class Foo {} - -public class Bar { - public Bar(Foo foo) {} -} - -public class Foobar{ - public Foobar(Bar bar) {} -} - -public class FooProcessor implements ItemProcessor<Foo,Bar>{ - public Bar process(Foo foo) throws Exception { - //Perform simple transformation, convert a Foo to a Bar - return new Bar(foo); - } -} - -public class BarProcessor implements ItemProcessor<Bar,FooBar>{ - public FooBar process(Bar bar) throws Exception { - return new Foobar(bar); - } -} - -public class FoobarWriter implements ItemWriter<FooBar>{ - public void write(List<? extends FooBar> items) throws Exception { - //write items - } -} - - A FooProcessor and - BarProcessor can be 'chained' together to give - the resultant Foobar: - - CompositeItemProcessor<Foo,Foobar> compositeProcessor = - new CompositeItemProcessor<Foo,Foobar>(); -List itemProcessors = new ArrayList(); -itemProcessors.add(new FooTransformer()); -itemProcessors.add(new BarTransformer()); -compositeProcessor.setDelegates(itemProcessors); - - Just as with the previous example, the composite processor can be - configured into the Step: - - <job id="ioSampleJob"> - <step name="step1"> - <tasklet> - <chunk reader="fooReader" processor="compositeProcessor" writer="foobarWriter" - commit-interval="2"/> - </tasklet> - </step> -</job> - -<bean id="compositeItemProcessor" - class="org.springframework.batch.item.support.CompositeItemProcessor"> - <property name="delegates"> - <list> - <bean class="..FooProcessor" /> - <bean class="..BarProcessor" /> - </list> - </property> -</bean> -
        - -
        - Filtering Records - - One typical use for an item processor is to filter out records - before they are passed to the ItemWriter. Filtering is an action - distinct from skipping; skipping indicates that a record is invalid - whereas filtering simply indicates that a record should not be - written. - - For example, consider a batch job that reads a file containing - three different types of records: records to insert, records to update, - and records to delete. If record deletion is not supported by the - system, then we would not want to send any "delete" records to the - ItemWriter. But, since these records are not - actually bad records, we would want to filter them out, rather than - skip. As a result, the ItemWriter would receive only "insert" and - "update" records. - - To filter a record, one simply returns "null" from the - ItemProcessor. The framework will detect that the - result is "null" and avoid adding that item to the list of records - delivered to the ItemWriter. As usual, an - exception thrown from the ItemProcessor will - result in a skip. -
        -
        - -
        - ItemStream - - Both ItemReaders and - ItemWriters serve their individual purposes well, - but there is a common concern among both of them that necessitates another - interface. In general, as part of the scope of a batch job, readers and - writers need to be opened, closed, and require a mechanism for persisting - state: - - public interface ItemStream { - - void open(ExecutionContext executionContext) throws ItemStreamException; - - void update(ExecutionContext executionContext) throws ItemStreamException; - - void close() throws ItemStreamException; -} - - Before describing each method, we should mention the - ExecutionContext. Clients of an - ItemReader that also implement - ItemStream should call - open before any calls to - read in order to open any resources such as files - or to obtain connections. A similar restriction applies to an - ItemWriter that implements - ItemStream. As mentioned in Chapter 2, if expected - data is found in the ExecutionContext, it may be - used to start the ItemReader or - ItemWriter at a location other than its initial - state. Conversely, close will be called to ensure - that any resources allocated during open will be - released safely. update is called primarily to - ensure that any state currently being held is loaded into the provided - ExecutionContext. This method will be called before - committing, to ensure that the current state is persisted in the database - before commit. - - In the special case where the client of an - ItemStream is a Step (from - the Spring Batch Core), an ExecutionContext is - created for each StepExecution to allow users to - store the state of a particular execution, with the expectation that it - will be returned if the same JobInstance is started - again. For those familiar with Quartz, the semantics are very similar to a - Quartz JobDataMap. -
        - -
        - The Delegate Pattern and Registering with the Step - - Note that the CompositeItemWriter is an - example of the delegation pattern, which is common in Spring Batch. The - delegates themselves might implement callback interfaces like - ItemStream or StepListener. - If they do, and they are being used in conjunction with Spring Batch Core - as part of a Step in a Job, - then they almost certainly need to be registered manually with the - Step. A reader, writer, or processor that is - directly wired into the Step will be registered automatically if it - implements ItemStream or a - StepListener interface. But because the delegates - are not known to the Step, they need to be injected - as listeners or streams (or both if appropriate): - - <job id="ioSampleJob"> - <step name="step1"> - <tasklet> - <chunk reader="fooReader" processor="fooProcessor" writer="compositeItemWriter" - commit-interval="2"> - <streams> - <stream ref="barWriter" /> - </streams> - </chunk> - </tasklet> - </step> -</job> - -<bean id="compositeItemWriter" class="...CompositeItemWriter"> - <property name="delegate" ref="barWriter" /> -</bean> - -<bean id="barWriter" class="...BarWriter" /> -
        - -
        - Flat Files - - One of the most common mechanisms for interchanging bulk data has - always been the flat file. Unlike XML, which has an agreed upon standard - for defining how it is structured (XSD), anyone reading a flat file must - understand ahead of time exactly how the file is structured. In general, - all flat files fall into two types: Delimited and Fixed Length. Delimited - files are those in which fields are separated by a delimiter, such as a - comma. Fixed Length files have fields that are a set length. - -
        - The FieldSet - - When working with flat files in Spring Batch, regardless of - whether it is for input or output, one of the most important classes is - the FieldSet. Many architectures and libraries - contain abstractions for helping you read in from a file, but they - usually return a String or an array of Strings. This really only gets - you halfway there. A FieldSet is Spring Batch’s - abstraction for enabling the binding of fields from a file resource. It - allows developers to work with file input in much the same way as they - would work with database input. A FieldSet is - conceptually very similar to a Jdbc ResultSet. - FieldSets only require one argument, a String - array of tokens. Optionally, you can also configure in the names of the - fields so that the fields may be accessed either by index or name as - patterned after ResultSet: - - String[] tokens = new String[]{"foo", "1", "true"}; -FieldSet fs = new DefaultFieldSet(tokens); -String name = fs.readString(0); -int value = fs.readInt(1); -boolean booleanValue = fs.readBoolean(2); - - There are many more options on the FieldSet - interface, such as Date, long, - BigDecimal, etc. The biggest advantage of the - FieldSet is that it provides consistent parsing - of flat file input. Rather than each batch job parsing differently in - potentially unexpected ways, it can be consistent, both when handling - errors caused by a format exception, or when doing simple data - conversions. -
        - -
        - FlatFileItemReader - - A flat file is any type of file that contains at most - two-dimensional (tabular) data. Reading flat files in the Spring Batch - framework is facilitated by the class - FlatFileItemReader, which provides basic - functionality for reading and parsing flat files. The two most important - required dependencies of FlatFileItemReader are - Resource and LineMapper. - The LineMapper interface will be - explored more in the next sections. The resource property represents a - Spring Core Resource. Documentation explaining - how to create beans of this type can be found in Spring - Framework, Chapter 5.Resources. Therefore, this - guide will not go into the details of creating - Resource objects. However, a simple example of a - file system resource can be found below: Resource resource = new FileSystemResource("resources/trades.csv"); - - In complex batch environments the directory structures are often - managed by the EAI infrastructure where drop zones for external - interfaces are established for moving files from ftp locations to batch - processing locations and vice versa. File moving utilities are beyond - the scope of the spring batch architecture but it is not unusual for - batch job streams to include file moving utilities as steps in the job - stream. It is sufficient that the batch architecture only needs to know - how to locate the files to be processed. Spring Batch begins the process - of feeding the data into the pipe from this starting point. However, - Spring - Integration provides many of these types of - services. - - The other properties in FlatFileItemReader - allow you to further specify how your data will be interpreted: - FlatFileItemReader Properties - - - - - - - Property - - Type - - Description - - - - - - comments - - String[] - - Specifies line prefixes that indicate - comment rows - - - - encoding - - String - - Specifies what text encoding to use - - default is "ISO-8859-1" - - - - lineMapper - - LineMapper - - Converts a String - to an Object representing the - item. - - - - linesToSkip - - int - - Number of lines to ignore at the top of - the file - - - - recordSeparatorPolicy - - RecordSeparatorPolicy - - Used to determine where the line endings - are and do things like continue over a line ending if inside a - quoted string. - - - - resource - - Resource - - The resource from which to read. - - - - skippedLinesCallback - - LineCallbackHandler - - Interface which passes the raw line - content of the lines in the file to be skipped. If linesToSkip - is set to 2, then this interface will be called twice. - - - - strict - - boolean - - In strict mode, the reader will throw an - exception on ExecutionContext if the input resource does not - exist. - - - -
        - -
        - LineMapper - - As with RowMapper, which takes a low - level construct such as ResultSet and returns - an Object, flat file processing requires the - same construct to convert a String line into an - Object:public interface LineMapper<T> { - - T mapLine(String line, int lineNumber) throws Exception; - -} - - The basic contract is that, given the current line and the line - number with which it is associated, the mapper should return a - resulting domain object. This is similar to - RowMapper in that each line is associated with - its line number, just as each row in a - ResultSet is tied to its row number. This - allows the line number to be tied to the resulting domain object for - identity comparison or for more informative logging. However, unlike - RowMapper, the - LineMapper is given a raw line which, as - discussed above, only gets you halfway there. The line must be - tokenized into a FieldSet, which can then be - mapped to an object, as described below. -
        - -
        - LineTokenizer - - An abstraction for turning a line of input into a line into a - FieldSet is necessary because there can be many - formats of flat file data that need to be converted to a - FieldSet. In Spring Batch, this interface is - the LineTokenizer: - - public interface LineTokenizer { - - FieldSet tokenize(String line); - -} - - The contract of a LineTokenizer is such - that, given a line of input (in theory the - String could encompass more than one line), a - FieldSet representing the line will be - returned. This FieldSet can then be passed to a - FieldSetMapper. Spring Batch contains the - following LineTokenizer implementations: - - - - DelmitedLineTokenizer - Used for - files where fields in a record are separated by a delimiter. The - most common delimiter is a comma, but pipes or semicolons are - often used as well. - - - - FixedLengthTokenizer - Used for files - where fields in a record are each a 'fixed width'. The width of - each field must be defined for each record type. - - - - PatternMatchingCompositeLineTokenizer - - Determines which among a list of - LineTokenizers should be used on a - particular line by checking against a pattern. - - -
        - -
        - FieldSetMapper - - The FieldSetMapper interface defines a - single method, mapFieldSet, which takes a - FieldSet object and maps its contents to an - object. This object may be a custom DTO, a domain object, or a simple - array, depending on the needs of the job. The - FieldSetMapper is used in conjunction with the - LineTokenizer to translate a line of data from - a resource into an object of the desired type: - - public interface FieldSetMapper<T> { - - T mapFieldSet(FieldSet fieldSet); - -} - - The pattern used is the same as the - RowMapper used by - JdbcTemplate. -
        - -
        - DefaultLineMapper - - Now that the basic interfaces for reading in flat files have - been defined, it becomes clear that three basic steps are - required: - - Read one line from the file. - - - - Pass the string line into the - LineTokenizer#tokenize() method, in - order to retrieve a FieldSet. - - - - Pass the FieldSet returned from - tokenizing to a FieldSetMapper, returning - the result from the ItemReader#read() - method. - - - - The two interfaces described above represent two separate tasks: - converting a line into a FieldSet, and mapping - a FieldSet to a domain object. Because the - input of a LineTokenizer matches the input of - the LineMapper (a line), and the output of a - FieldSetMapper matches the output of the - LineMapper, a default implementation that uses - both a LineTokenizer and - FieldSetMapper is provided. The - DefaultLineMapper represents the behavior most - users will need: - - public class DefaultLineMapper<T> implements LineMapper<T>, InitializingBean { - - private LineTokenizer tokenizer; - - private FieldSetMapper<T> fieldSetMapper; - - public T mapLine(String line, int lineNumber) throws Exception { - return fieldSetMapper.mapFieldSet(tokenizer.tokenize(line)); - } - - public void setLineTokenizer(LineTokenizer tokenizer) { - this.tokenizer = tokenizer; - } - - public void setFieldSetMapper(FieldSetMapper<T> fieldSetMapper) { - this.fieldSetMapper = fieldSetMapper; - } -} - - The above functionality is provided in a default implementation, - rather than being built into the reader itself (as was done in - previous versions of the framework) in order to allow users greater - flexibility in controlling the parsing process, especially if access - to the raw line is needed. -
        - -
        - Simple Delimited File Reading Example - - The following example will be used to illustrate this using an - actual domain scenario. This particular batch job reads in football - players from the following file:ID,lastName,firstName,position,birthYear,debutYear -"AbduKa00,Abdul-Jabbar,Karim,rb,1974,1996", -"AbduRa00,Abdullah,Rabih,rb,1975,1999", -"AberWa00,Abercrombie,Walter,rb,1959,1982", -"AbraDa00,Abramowicz,Danny,wr,1945,1967", -"AdamBo00,Adams,Bob,te,1946,1969", -"AdamCh00,Adams,Charlie,wr,1979,2003" - - The contents of this file will be mapped to the following - Player domain object: public class Player implements Serializable { - - private String ID; - private String lastName; - private String firstName; - private String position; - private int birthYear; - private int debutYear; - - public String toString() { - return "PLAYER:ID=" + ID + ",Last Name=" + lastName + - ",First Name=" + firstName + ",Position=" + position + - ",Birth Year=" + birthYear + ",DebutYear=" + - debutYear; - } - - // setters and getters... -} - - - In order to map a FieldSet into a - Player object, a - FieldSetMapper that returns players needs to be - defined: - - protected static class PlayerFieldSetMapper implements FieldSetMapper<Player> { - public Player mapFieldSet(FieldSet fieldSet) { - Player player = new Player(); - - player.setID(fieldSet.readString(0)); - player.setLastName(fieldSet.readString(1)); - player.setFirstName(fieldSet.readString(2)); - player.setPosition(fieldSet.readString(3)); - player.setBirthYear(fieldSet.readInt(4)); - player.setDebutYear(fieldSet.readInt(5)); - - return player; - } -} - - The file can then be read by correctly constructing a - FlatFileItemReader and calling - read: - - FlatFileItemReader<Player> itemReader = new FlatFileItemReader<Player>(); -itemReader.setResource(new FileSystemResource("resources/players.csv")); -//DelimitedLineTokenizer defaults to comma as its delimiter -LineMapper<Player> lineMapper = new DefaultLineMapper<Player>(); -lineMapper.setLineTokenizer(new DelimitedLineTokenizer()); -lineMapper.setFieldSetMapper(new PlayerFieldSetMapper()); -itemReader.setLineMapper(lineMapper); -itemReader.open(new ExecutionContext()); -Player player = itemReader.read(); - - Each call to read will return a new - Player object from each line in the file. When the end of the file is - reached, null will be returned. -
        - -
        - Mapping Fields by Name - - There is one additional piece of functionality that is allowed - by both DelimitedLineTokenizer and - FixedLengthTokenizer that is similar in - function to a Jdbc ResultSet. The names of the - fields can be injected into either of these - LineTokenizer implementations to increase the - readability of the mapping function. First, the column names of all - fields in the flat file are injected into the tokenizer: - - tokenizer.setNames(new String[] {"ID", "lastName","firstName","position","birthYear","debutYear"}); - - A FieldSetMapper can use this information - as follows: - - public class PlayerMapper implements FieldSetMapper<Player> { - public Player mapFieldSet(FieldSet fs) { - - if(fs == null){ - return null; - } - - Player player = new Player(); - player.setID(fs.readString("ID")); - player.setLastName(fs.readString("lastName")); - player.setFirstName(fs.readString("firstName")); - player.setPosition(fs.readString("position")); - player.setDebutYear(fs.readInt("debutYear")); - player.setBirthYear(fs.readInt("birthYear")); - - return player; - } -} -
        - -
        - Automapping FieldSets to Domain Objects - - For many, having to write a specific - FieldSetMapper is equally as cumbersome as - writing a specific RowMapper for a - JdbcTemplate. Spring Batch makes this easier by - providing a FieldSetMapper that automatically - maps fields by matching a field name with a setter on the object using - the JavaBean specification. Again using the football example, the - BeanWrapperFieldSetMapper configuration looks - like the following: - - <bean id="fieldSetMapper" - class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper"> - <property name="prototypeBeanName" value="player" /> -</bean> - -<bean id="player" - class="org.springframework.batch.sample.domain.Player" - scope="prototype" /> - - For each entry in the FieldSet, the - mapper will look for a corresponding setter on a new instance of the - Player object (for this reason, prototype scope - is required) in the same way the Spring container will look for - setters matching a property name. Each available field in the - FieldSet will be mapped, and the resultant - Player object will be returned, with no code - required. -
        - -
        - Fixed Length File Formats - - So far only delimited files have been discussed in much detail, - however, they represent only half of the file reading picture. Many - organizations that use flat files use fixed length formats. An example - fixed length file is below: - - UK21341EAH4121131.11customer1 -UK21341EAH4221232.11customer2 -UK21341EAH4321333.11customer3 -UK21341EAH4421434.11customer4 -UK21341EAH4521535.11customer5 - - While this looks like one large field, it actually represent 4 - distinct fields: - - - - ISIN: Unique identifier for the item being order - 12 - characters long. - - - - Quantity: Number of this item being ordered - 3 characters - long. - - - - Price: Price of the item - 5 characters long. - - - - Customer: Id of the customer ordering the item - 9 - characters long. - - - - When configuring the - FixedLengthLineTokenizer, each of these lengths - must be provided in the form of ranges: - - <bean id="fixedLengthLineTokenizer" - class="org.springframework.batch.io.file.transform.FixedLengthTokenizer"> - <property name="names" value="ISIN,Quantity,Price,Customer" /> - <property name="columns" value="1-12, 13-15, 16-20, 21-29" /> -</bean> - - Because the FixedLengthLineTokenizer uses - the same LineTokenizer interface as discussed - above, it will return the same FieldSet as if a - delimiter had been used. This allows the same approaches to be used in - handling its output, such as using the - BeanWrapperFieldSetMapper. - - - Supporting the above syntax for ranges requires that a - specialized property editor, - RangeArrayPropertyEditor, be configured in - the ApplicationContext. However, this bean - is automatically declared in an - ApplicationContext where the batch - namespace is used. - -
        - -
        - Multiple Record Types within a Single File - - All of the file reading examples up to this point have all made - a key assumption for simplicity's sake: all of the records in a file - have the same format. However, this may not always be the case. It is - very common that a file might have records with different formats that - need to be tokenized differently and mapped to different objects. The - following excerpt from a file illustrates this: - - USER;Smith;Peter;;T;20014539;F -LINEA;1044391041ABC037.49G201XX1383.12H -LINEB;2134776319DEF422.99M005LI - - In this file we have three types of records, "USER", "LINEA", - and "LINEB". A "USER" line corresponds to a User object. "LINEA" and - "LINEB" both correspond to Line objects, though a "LINEA" has more - information than a "LINEB". - - The ItemReader will read each line - individually, but we must specify different - LineTokenizer and - FieldSetMapper objects so that the - ItemWriter will receive the correct items. The - PatternMatchingCompositeLineMapper makes this - easy by allowing maps of patterns to - LineTokenizers and patterns to - FieldSetMappers to be configured: - - <bean id="orderFileLineMapper" - class="org.spr...PatternMatchingCompositeLineMapper"> - <property name="tokenizers"> - <map> - <entry key="USER*" value-ref="userTokenizer" /> - <entry key="LINEA*" value-ref="lineATokenizer" /> - <entry key="LINEB*" value-ref="lineBTokenizer" /> - </map> - </property> - <property name="fieldSetMappers"> - <map> - <entry key="USER*" value-ref="userFieldSetMapper" /> - <entry key="LINE*" value-ref="lineFieldSetMapper" /> - </map> - </property> -</bean> - - In this example, "LINEA" and "LINEB" have separate - LineTokenizers but they both use the same - FieldSetMapper. - - The PatternMatchingCompositeLineMapper - makes use of the PatternMatcher's - match method in order to select the correct - delegate for each line. The PatternMatcher - allows for two wildcard characters with special meaning: the question - mark ("?") will match exactly one character, while the asterisk ("*") - will match zero or more characters. Note that in the configuration - above, all patterns end with an asterisk, making them effectively - prefixes to lines. The PatternMatcher will - always match the most specific pattern possible, regardless of the - order in the configuration. So if "LINE*" and "LINEA*" were both - listed as patterns, "LINEA" would match pattern "LINEA*", while - "LINEB" would match pattern "LINE*". Additionally, a single asterisk - ("*") can serve as a default by matching any line not matched by any - other pattern. - - <entry key="*" value-ref="defaultLineTokenizer" /> - - There is also a - PatternMatchingCompositeLineTokenizer that can - be used for tokenization alone. - - It is also common for a flat file to contain records that each - span multiple lines. To handle this situation, a more complex strategy - is required. A demonstration of this common pattern can be found in - . -
        - -
        - Exception Handling in Flat Files - - There are many scenarios when tokenizing a line may cause - exceptions to be thrown. Many flat files are imperfect and contain - records that aren't formatted correctly. Many users choose to skip - these erroneous lines, logging out the issue, original line, and line - number. These logs can later be inspected manually or by another batch - job. For this reason, Spring Batch provides a hierarchy of exceptions - for handling parse exceptions: - FlatFileParseException and - FlatFileFormatException. - FlatFileParseException is thrown by the - FlatFileItemReader when any errors are - encountered while trying to read a file. - FlatFileFormatException is thrown by - implementations of the LineTokenizer interface, - and indicates a more specific error encountered while - tokenizing. - -
        - IncorrectTokenCountException - - Both DelimitedLineTokenizer and - FixedLengthLineTokenizer have the ability to - specify column names that can be used for creating a - FieldSet. However, if the number of column - names doesn't match the number of columns found while tokenizing a - line the FieldSet can't be created, and a - IncorrectTokenCountException is thrown, which - contains the number of tokens encountered, and the number - expected: - - tokenizer.setNames(new String[] {"A", "B", "C", "D"}); - -try{ - tokenizer.tokenize("a,b,c"); -} -catch(IncorrectTokenCountException e){ - assertEquals(4, e.getExpectedCount()); - assertEquals(3, e.getActualCount()); -} - - Because the tokenizer was configured with 4 column names, but - only 3 tokens were found in the file, an - IncorrectTokenCountException was - thrown. -
        - -
        - IncorrectLineLengthException - - Files formatted in a fixed length format have additional - requirements when parsing because, unlike a delimited format, each - column must strictly adhere to its predefined width. If the total - line length doesn't add up to the widest value of this column, an - exception is thrown: - - tokenizer.setColumns(new Range[] { new Range(1, 5), - new Range(6, 10), - new Range(11, 15) }); -try { - tokenizer.tokenize("12345"); - fail("Expected IncorrectLineLengthException"); -} -catch (IncorrectLineLengthException ex) { - assertEquals(15, ex.getExpectedLength()); - assertEquals(5, ex.getActualLength()); -} - - The configured ranges for the tokenizer above are: 1-5, 6-10, - and 11-15, thus the total length of the line expected is 15. - However, in this case a line of length 5 was passed in, causing an - IncorrectLineLengthException to be thrown. - Throwing an exception here rather than only mapping the first column - allows the processing of the line to fail earlier, and with more - information than it would if it failed while trying to read in - column 2 in a FieldSetMapper. However, there - are scenarios where the length of the line isn't always constant. - For this reason, validation of line length can be turned off via the - 'strict' property: - - tokenizer.setColumns(new Range[] { new Range(1, 5), new Range(6, 10) }); -tokenizer.setStrict(false); -FieldSet tokens = tokenizer.tokenize("12345"); -assertEquals("12345", tokens.readString(0)); -assertEquals("", tokens.readString(1)); - - The above example is almost identical to the one before it, - except that tokenizer.setStrict(false) was called. This setting - tells the tokenizer to not enforce line lengths when tokenizing the - line. A FieldSet is now correctly created and - returned. However, it will only contain empty tokens for the - remaining values. -
        -
        -
        - -
        - FlatFileItemWriter - - Writing out to flat files has the same problems and issues that - reading in from a file must overcome. A step must be able to write out - in either delimited or fixed length formats in a transactional - manner. - -
        - LineAggregator - - Just as the LineTokenizer interface is - necessary to take an item and turn it into a - String, file writing must have a way to - aggregate multiple fields into a single string for writing to a file. - In Spring Batch this is the - LineAggregator: - - public interface LineAggregator<T> { - - public String aggregate(T item); - -} - - The LineAggregator is the opposite of a - LineTokenizer. - LineTokenizer takes a - String and returns a - FieldSet, whereas - LineAggregator takes an - item and returns a - String. - -
        - PassThroughLineAggregator - - The most basic implementation of the LineAggregator interface - is the PassThroughLineAggregator, which - simply assumes that the object is already a string, or that its - string representation is acceptable for writing: - - public class PassThroughLineAggregator<T> implements LineAggregator<T> { - - public String aggregate(T item) { - return item.toString(); - } -} - - The above implementation is useful if direct control of - creating the string is required, but the advantages of a - FlatFileItemWriter, such as transaction and - restart support, are necessary. -
        -
        - -
        - Simplified File Writing Example - - Now that the LineAggregator interface and - its most basic implementation, - PassThroughLineAggregator, have been defined, - the basic flow of writing can be explained: - - - - The object to be written is passed to the - LineAggregator in order to obtain a - String. - - - - The returned String is written to the - configured file. - - - - The following excerpt from the - FlatFileItemWriter expresses this in - code: - - public void write(T item) throws Exception { - write(lineAggregator.aggregate(item) + LINE_SEPARATOR); -} - - A simple configuration would look like the following: - - <bean id="itemWriter" class="org.spr...FlatFileItemWriter"> - <property name="resource" value="file:target/test-outputs/output.txt" /> - <property name="lineAggregator"> - <bean class="org.spr...PassThroughLineAggregator"/> - </property> -</bean> -
        - -
        - FieldExtractor - - The above example may be useful for the most basic uses of a - writing to a file. However, most users of the - FlatFileItemWriter will have a domain object - that needs to be written out, and thus must be converted into a line. - In file reading, the following was required: - - Read one line from the file. - - - - Pass the string line into the - LineTokenizer#tokenize() method, in - order to retrieve a FieldSet - - - - Pass the FieldSet returned from - tokenizing to a FieldSetMapper, returning - the result from the ItemReader#read() - method - - - - File writing has similar, but inverse steps: - - - - Pass the item to be written to the writer - - - - convert the fields on the item into an array - - - - aggregate the resulting array into a line - - - - Because there is no way for the framework to know which fields - from the object need to be written out, a - FieldExtractor must be written to accomplish - the task of turning the item into an array: - - public interface FieldExtractor<T> { - - Object[] extract(T item); - -} - - Implementations of the FieldExtractor - interface should create an array from the fields of the provided - object, which can then be written out with a delimiter between the - elements, or as part of a field-width line. - -
        - PassThroughFieldExtractor - - There are many cases where a collection, such as an array, - Collection, or - FieldSet, needs to be written out. - "Extracting" an array from a one of these collection types is very - straightforward: simply convert the collection to an array. - Therefore, the PassThroughFieldExtractor - should be used in this scenario. It should be noted, that if the - object passed in is not a type of collection, then the - PassThroughFieldExtractor will return an - array containing solely the item to be extracted. -
        - -
        - BeanWrapperFieldExtractor - - As with the BeanWrapperFieldSetMapper - described in the file reading section, it is often preferable to - configure how to convert a domain object to an object array, rather - than writing the conversion yourself. The - BeanWrapperFieldExtractor provides just this - type of functionality: - - BeanWrapperFieldExtractor<Name> extractor = new BeanWrapperFieldExtractor<Name>(); -extractor.setNames(new String[] { "first", "last", "born" }); - -String first = "Alan"; -String last = "Turing"; -int born = 1912; - -Name n = new Name(first, last, born); -Object[] values = extractor.extract(n); - -assertEquals(first, values[0]); -assertEquals(last, values[1]); -assertEquals(born, values[2]); - - This extractor implementation has only one required property, - the names of the fields to map. Just as the - BeanWrapperFieldSetMapper needs field names - to map fields on the FieldSet to setters on - the provided object, the - BeanWrapperFieldExtractor needs names to map - to getters for creating an object array. It is worth noting that the - order of the names determines the order of the fields within the - array. -
        -
        - -
        - Delimited File Writing Example - - The most basic flat file format is one in which all fields are - separated by a delimiter. This can be accomplished using a - DelimitedLineAggregator. The example below - writes out a simple domain object that represents a credit to a - customer account: - - public class CustomerCredit { - - private int id; - private String name; - private BigDecimal credit; - - //getters and setters removed for clarity -} - - Because a domain object is being used, an implementation of the - FieldExtractor interface must be provided, along with the delimiter to - use: - - <bean id="itemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter"> - <property name="resource" ref="outputResource" /> - <property name="lineAggregator"> - <bean class="org.spr...DelimitedLineAggregator"> - <property name="delimiter" value=","/> - <property name="fieldExtractor"> - <bean class="org.spr...BeanWrapperFieldExtractor"> - <property name="names" value="name,credit"/> - </bean> - </property> - </bean> - </property> -</bean> - - In this case, the - BeanWrapperFieldExtractor described earlier in - this chapter is used to turn the name and credit fields within - CustomerCredit into an object array, which is - then written out with commas between each field. -
        - -
        - Fixed Width File Writing Example - - Delimited is not the only type of flat file format. Many prefer - to use a set width for each column to delineate between fields, which - is usually referred to as 'fixed width'. Spring Batch supports this in - file writing via the FormatterLineAggregator. - Using the same CustomerCredit domain object - described above, it can be configured as follows: - - <bean id="itemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter"> - <property name="resource" ref="outputResource" /> - <property name="lineAggregator"> - <bean class="org.spr...FormatterLineAggregator"> - <property name="fieldExtractor"> - <bean class="org.spr...BeanWrapperFieldExtractor"> - <property name="names" value="name,credit" /> - </bean> - </property> - <property name="format" value="%-9s%-2.0f" /> - </bean> - </property> -</bean> - - Most of the above example should look familiar. However, the - value of the format property is new: - - <property name="format" value="%-9s%-2.0f" /> - - The underlying implementation is built using the same - Formatter added as part of Java 5. The Java - Formatter is based on the - printf functionality of the C programming - language. Most details on how to configure a formatter can be found in - the javadoc of Formatter. -
        - -
        - Handling File Creation - - FlatFileItemReader has a very simple - relationship with file resources. When the reader is initialized, it - opens the file if it exists, and throws an exception if it does not. - File writing isn't quite so simple. At first glance it seems like a - similar straight forward contract should exist for - FlatFileItemWriter: if the file already exists, - throw an exception, and if it does not, create it and start writing. - However, potentially restarting a Job can cause - issues. In normal restart scenarios, the contract is reversed: if the - file exists, start writing to it from the last known good position, - and if it does not, throw an exception. However, what happens if the - file name for this job is always the same? In this case, you would - want to delete the file if it exists, unless it's a restart. Because - of this possibility, the FlatFileItemWriter - contains the property, shouldDeleteIfExists. - Setting this property to true will cause an existing file with the - same name to be deleted when the writer is opened. -
        -
        -
        - -
        - XML Item Readers and Writers - - Spring Batch provides transactional infrastructure for both reading - XML records and mapping them to Java objects as well as writing Java - objects as XML records. - - - Constraints on streaming XML - - The StAX API is used for I/O as other standard XML parsing APIs do - not fit batch processing requirements (DOM loads the whole input into - memory at once and SAX controls the parsing process allowing the user - only to provide callbacks). - - - Lets take a closer look how XML input and output works in Spring - Batch. First, there are a few concepts that vary from file reading and - writing but are common across Spring Batch XML processing. With XML - processing, instead of lines of records (FieldSets) that need to be - tokenized, it is assumed an XML resource is a collection of 'fragments' - corresponding to individual records: - - - - - - - - - - - Figure 3.1: XML Input - - - The 'trade' tag is defined as the 'root element' in the scenario - above. Everything between '<trade>' and '</trade>' is - considered one 'fragment'. Spring Batch uses Object/XML Mapping (OXM) to - bind fragments to objects. However, Spring Batch is not tied to any - particular XML binding technology. Typical use is to delegate to Spring - OXM, which provides uniform abstraction for the most - popular OXM technologies. The dependency on Spring OXM is optional and you - can choose to implement Spring Batch specific interfaces if desired. The - relationship to the technologies that OXM supports can be shown as the - following: - - - - - - - - - - - Figure 3.2: OXM Binding - - - Now with an introduction to OXM and how one can use XML fragments to - represent records, let's take a closer look at readers and writers. - -
        - StaxEventItemReader - - The StaxEventItemReader configuration - provides a typical setup for the processing of records from an XML input - stream. First, lets examine a set of XML records that the - StaxEventItemReader can process. - - <?xml version="1.0" encoding="UTF-8"?> -<records> - <trade xmlns="http://springframework.org/batch/sample/io/oxm/domain"> - <isin>XYZ0001</isin> - <quantity>5</quantity> - <price>11.39</price> - <customer>Customer1</customer> - </trade> - <trade xmlns="http://springframework.org/batch/sample/io/oxm/domain"> - <isin>XYZ0002</isin> - <quantity>2</quantity> - <price>72.99</price> - <customer>Customer2c</customer> - </trade> - <trade xmlns="http://springframework.org/batch/sample/io/oxm/domain"> - <isin>XYZ0003</isin> - <quantity>9</quantity> - <price>99.99</price> - <customer>Customer3</customer> - </trade> -</records> - - To be able to process the XML records the following is needed: - - - Root Element Name - Name of the root element of the fragment - that constitutes the object to be mapped. The example - configuration demonstrates this with the value of trade. - - - - Resource - Spring Resource that represents the file to be - read. - - - - Unmarshaller - Unmarshalling - facility provided by Spring OXM for mapping the XML fragment to an - object. - - - - <bean id="itemReader" class="org.springframework.batch.item.xml.StaxEventItemReader"> - <property name="fragmentRootElementName" value="trade" /> - <property name="resource" value="data/iosample/input/input.xml" /> - <property name="unmarshaller" ref="tradeMarshaller" /> -</bean> - - - Notice that in this example we have chosen to use an - XStreamMarshaller which accepts an alias passed - in as a map with the first key and value being the name of the fragment - (i.e. root element) and the object type to bind. Then, similar to a - FieldSet, the names of the other elements that - map to fields within the object type are described as key/value pairs in - the map. In the configuration file we can use a Spring configuration - utility to describe the required alias as follows: - - <bean id="tradeMarshaller" - class="org.springframework.oxm.xstream.XStreamMarshaller"> - <property name="aliases"> - <util:map id="aliases"> - <entry key="trade" - value="org.springframework.batch.sample.domain.Trade" /> - <entry key="price" value="java.math.BigDecimal" /> - <entry key="name" value="java.lang.String" /> - </util:map> - </property> -</bean> - - On input the reader reads the XML resource until it recognizes - that a new fragment is about to start (by matching the tag name by - default). The reader creates a standalone XML document from the fragment - (or at least makes it appear so) and passes the document to a - deserializer (typically a wrapper around a Spring OXM - Unmarshaller) to map the XML to a Java - object. - - In summary, this procedure is analogous to the following scripted - Java code which uses the injection provided by the Spring - configuration: - - StaxEventItemReader xmlStaxEventItemReader = new StaxEventItemReader() -Resource resource = new ByteArrayResource(xmlResource.getBytes()) - -Map aliases = new HashMap(); -aliases.put("trade","org.springframework.batch.sample.domain.Trade"); -aliases.put("price","java.math.BigDecimal"); -aliases.put("customer","java.lang.String"); -Marshaller marshaller = new XStreamMarshaller(); -marshaller.setAliases(aliases); -xmlStaxEventItemReader.setUnmarshaller(marshaller); -xmlStaxEventItemReader.setResource(resource); -xmlStaxEventItemReader.setFragmentRootElementName("trade"); -xmlStaxEventItemReader.open(new ExecutionContext()); - -boolean hasNext = true - -CustomerCredit credit = null; - -while (hasNext) { - credit = xmlStaxEventItemReader.read(); - if (credit == null) { - hasNext = false; - } - else { - System.out.println(credit); - } -} -
        - -
        - StaxEventItemWriter - - Output works symmetrically to input. The - StaxEventItemWriter needs a - Resource, a marshaller, and a rootTagName. A Java - object is passed to a marshaller (typically a standard Spring OXM - Marshaller) which writes to a - Resource using a custom event writer that filters - the StartDocument and - EndDocument events produced for each fragment by - the OXM tools. We'll show this in an example using the - MarshallingEventWriterSerializer. The Spring - configuration for this setup looks as follows: - - <bean id="itemWriter" class="org.springframework.batch.item.xml.StaxEventItemWriter"> - <property name="resource" ref="outputResource" /> - <property name="marshaller" ref="customerCreditMarshaller" /> - <property name="rootTagName" value="customers" /> - <property name="overwriteOutput" value="true" /> -</bean> - - The configuration sets up the three required properties and - optionally sets the overwriteOutput=true, mentioned earlier in the - chapter for specifying whether an existing file can be overwritten. It - should be noted the marshaller used for the writer is the exact same as - the one used in the reading example from earlier in the chapter: - - <bean id="customerCreditMarshaller" - class="org.springframework.oxm.xstream.XStreamMarshaller"> - <property name="aliases"> - <util:map id="aliases"> - <entry key="customer" - value="org.springframework.batch.sample.domain.CustomerCredit" /> - <entry key="credit" value="java.math.BigDecimal" /> - <entry key="name" value="java.lang.String" /> - </util:map> - </property> -</bean> - - To summarize with a Java example, the following code illustrates - all of the points discussed, demonstrating the programmatic setup of the - required properties: - - StaxEventItemWriter staxItemWriter = new StaxEventItemWriter() -FileSystemResource resource = new FileSystemResource("data/outputFile.xml") - -Map aliases = new HashMap(); -aliases.put("customer","org.springframework.batch.sample.domain.CustomerCredit"); -aliases.put("credit","java.math.BigDecimal"); -aliases.put("name","java.lang.String"); -Marshaller marshaller = new XStreamMarshaller(); -marshaller.setAliases(aliases); - -staxItemWriter.setResource(resource); -staxItemWriter.setMarshaller(marshaller); -staxItemWriter.setRootTagName("trades"); -staxItemWriter.setOverwriteOutput(true); - -ExecutionContext executionContext = new ExecutionContext(); -staxItemWriter.open(executionContext); -CustomerCredit Credit = new CustomerCredit(); -trade.setPrice(11.39); -credit.setName("Customer1"); -staxItemWriter.write(trade); -
        -
        - -
        - Multi-File Input - - It is a common requirement to process multiple files within a single - Step. Assuming the files all have the same - formatting, the MultiResourceItemReader supports - this type of input for both XML and flat file processing. Consider the - following files in a directory: - - file-1.txt file-2.txt ignored.txt - - file-1.txt and file-2.txt are formatted the same and for business - reasons should be processed together. The - MuliResourceItemReader can be used to read in both - files by using wildcards: - - <bean id="multiResourceReader" class="org.spr...MultiResourceItemReader"> - <property name="resources" value="classpath:data/input/file-*.txt" /> - <property name="delegate" ref="flatFileItemReader" /> -</bean> - - The referenced delegate is a simple - FlatFileItemReader. The above configuration will - read input from both files, handling rollback and restart scenarios. It - should be noted that, as with any ItemReader, - adding extra input (in this case a file) could cause potential issues when - restarting. It is recommended that batch jobs work with their own - individual directories until completed successfully. -
        - -
        - Database - - Like most enterprise application styles, a database is the central - storage mechanism for batch. However, batch differs from other application - styles due to the sheer size of the datasets with which the system must - work. If a SQL statement returns 1 million rows, the result set probably - holds all returned results in memory until all rows have been read. Spring - Batch provides two types of solutions for this problem: Cursor and Paging - database ItemReaders. - -
        - Cursor Based ItemReaders - - Using a database cursor is generally the default approach of most - batch developers, because it is the database's solution to the problem - of 'streaming' relational data. The Java - ResultSet class is essentially an object - orientated mechanism for manipulating a cursor. A - ResultSet maintains a cursor to the current row - of data. Calling next on a - ResultSet moves this cursor to the next row. - Spring Batch cursor based ItemReaders open the a cursor on - initialization, and move the cursor forward one row for every call to - read, returning a mapped object that can be - used for processing. The close method will then - be called to ensure all resources are freed up. The Spring core - JdbcTemplate gets around this problem by using - the callback pattern to completely map all rows in a - ResultSet and close before returning control back - to the method caller. However, in batch this must wait until the step is - complete. Below is a generic diagram of how a cursor based - ItemReader works, and while a SQL statement is - used as an example since it is so widely known, any technology could - implement the basic approach: - - - - - - - - - - - - This example illustrates the basic pattern. Given a 'FOO' table, - which has three columns: ID, NAME, and BAR, select all rows with an ID - greater than 1 but less than 7. This puts the beginning of the cursor - (row 1) on ID 2. The result of this row should be a completely mapped - Foo object. Calling read() again moves the - cursor to the next row, which is the Foo with an ID of 3. The results of - these reads will be written out after each - read, thus allowing the objects to be garbage - collected (assuming no instance variables are maintaining references to - them). - -
        - JdbcCursorItemReader - - JdbcCursorItemReader is the Jdbc - implementation of the cursor based technique. It works directly with a - ResultSet and requires a SQL statement to run - against a connection obtained from a - DataSource. The following database schema will - be used as an example: - - CREATE TABLE CUSTOMER ( - ID BIGINT IDENTITY PRIMARY KEY, - NAME VARCHAR(45), - CREDIT FLOAT -); - - Many people prefer to use a domain object for each row, so we'll - use an implementation of the RowMapper - interface to map a CustomerCredit - object: - - public class CustomerCreditRowMapper implements RowMapper { - - public static final String ID_COLUMN = "id"; - public static final String NAME_COLUMN = "name"; - public static final String CREDIT_COLUMN = "credit"; - - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { - CustomerCredit customerCredit = new CustomerCredit(); - - customerCredit.setId(rs.getInt(ID_COLUMN)); - customerCredit.setName(rs.getString(NAME_COLUMN)); - customerCredit.setCredit(rs.getBigDecimal(CREDIT_COLUMN)); - - return customerCredit; - } -} - - Because JdbcTemplate is so familiar to - users of Spring, and the JdbcCursorItemReader - shares key interfaces with it, it is useful to see an example of how - to read in this data with JdbcTemplate, in - order to contrast it with the ItemReader. For - the purposes of this example, let's assume there are 1,000 rows in the - CUSTOMER database. The first example will be using - JdbcTemplate: - - //For simplicity sake, assume a dataSource has already been obtained -JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource); -List customerCredits = jdbcTemplate.query("SELECT ID, NAME, CREDIT from CUSTOMER", - new CustomerCreditRowMapper()); - - After running this code snippet the customerCredits list will - contain 1,000 CustomerCredit objects. In the - query method, a connection will be obtained from the - DataSource, the provided SQL will be run - against it, and the mapRow method will be - called for each row in the ResultSet. Let's - contrast this with the approach of the - JdbcCursorItemReader: - - JdbcCursorItemReader itemReader = new JdbcCursorItemReader(); -itemReader.setDataSource(dataSource); -itemReader.setSql("SELECT ID, NAME, CREDIT from CUSTOMER"); -itemReader.setRowMapper(new CustomerCreditRowMapper()); -int counter = 0; -ExecutionContext executionContext = new ExecutionContext(); -itemReader.open(executionContext); -Object customerCredit = new Object(); -while(customerCredit != null){ - customerCredit = itemReader.read(); - counter++; -} -itemReader.close(executionContext); - - After running this code snippet the counter will equal 1,000. If - the code above had put the returned customerCredit into a list, the - result would have been exactly the same as with the - JdbcTemplate example. However, the big - advantage of the ItemReader is that it allows - items to be 'streamed'. The read method can - be called once, and the item written out via an - ItemWriter, and then the next item obtained via - read. This allows item reading and writing to - be done in 'chunks' and committed periodically, which is the essence - of high performance batch processing. Furthermore, it is very easily - configured for injection into a Spring Batch - Step: - - <bean id="itemReader" class="org.spr...JdbcCursorItemReader"> - <property name="dataSource" ref="dataSource"/> - <property name="sql" value="select ID, NAME, CREDIT from CUSTOMER"/> - <property name="rowMapper"> - <bean class="org.springframework.batch.sample.domain.CustomerCreditRowMapper"/> - </property> -</bean> - -
        - Additional Properties - - Because there are so many varying options for opening a cursor - in Java, there are many properties on the - JdbcCustorItemReader that can be set: - - - JdbcCursorItemReader Properties - - - - - ignoreWarnings - - Determines whether or not SQLWarnings are logged or - cause an exception - default is true - - - - fetchSize - - Gives the Jdbc driver a hint as to the number of rows - that should be fetched from the database when more rows are - needed by the ResultSet object used - by the ItemReader. By default, no - hint is given. - - - - maxRows - - Sets the limit for the maximum number of rows the - underlying ResultSet can hold at any - one time. - - - - queryTimeout - - Sets the number of seconds the driver will wait for a - Statement object to execute to the - given number of seconds. If the limit is exceeded, a - DataAccessEception is thrown. - (Consult your driver vendor documentation for - details). - - - - verifyCursorPosition - - Because the same ResultSet - held by the ItemReader is passed to - the RowMapper, it is possible for - users to call ResultSet.next() - themselves, which could cause issues with the reader's - internal count. Setting this value to true will cause an - exception to be thrown if the cursor position is not the - same after the RowMapper call as it - was before. - - - - saveState - - Indicates whether or not the reader's state should be - saved in the ExecutionContext - provided by - ItemStream#update(ExecutionContext) - The default value is true. - - - - driverSupportsAbsolute - - Defaults to false. Indicates whether the Jdbc driver - supports setting the absolute row on a - ResultSet. It is recommended that - this is set to true for Jdbc drivers that supports - ResultSet.absolute() as it may - improve performance, especially if a step fails while - working with a large data set. - - - - setUseSharedExtendedConnection - - Defaults to false. Indicates whether the connection - used for the cursor should be used by all other processing - thus sharing the same transaction. If this is set to false, - which is the default, then the cursor will be opened using - its own connection and will not participate in any - transactions started for the rest of the step processing. If - you set this flag to true then you must wrap the - DataSource in an - ExtendedConnectionDataSourceProxy to - prevent the connection from being closed and released after - each commit. When you set this option to true then the - statement used to open the cursor will be created with both - 'READ_ONLY' and 'HOLD_CUSORS_OVER_COMMIT' options. This - allows holding the cursor open over transaction start and - commits performed in the step processing. To use this - feature you need a database that supports this and a Jdbc - driver supporting Jdbc 3.0 or later. - - - -
        -
        -
        - -
        - HibernateCursorItemReader - - Just as normal Spring users make important decisions about - whether or not to use ORM solutions, which affect whether or not they - use a JdbcTemplate or a - HibernateTemplate, Spring Batch users have the - same options. HibernateCursorItemReader is the - Hibernate implementation of the cursor technique. Hibernate's usage in - batch has been fairly controversial. This has largely been because - Hibernate was originally developed to support online application - styles. However, that doesn't mean it can't be used for batch - processing. The easiest approach for solving this problem is to use a - StatelessSession rather than a standard - session. This removes all of the caching and dirty checking hibernate - employs that can cause issues in a batch scenario. For more - information on the differences between stateless and normal hibernate - sessions, refer to the documentation of your specific hibernate - release. The HibernateCursorItemReader allows - you to declare an HQL statement and pass in a - SessionFactory, which will pass back one item - per call to read in the same basic fashion as - the JdbcCursorItemReader. Below is an example - configuration using the same 'customer credit' example as the JDBC - reader: - - HibernateCursorItemReader itemReader = new HibernateCursorItemReader(); -itemReader.setQueryString("from CustomerCredit"); -//For simplicity sake, assume sessionFactory already obtained. -itemReader.setSessionFactory(sessionFactory); -itemReader.setUseStatelessSession(true); -int counter = 0; -ExecutionContext executionContext = new ExecutionContext(); -itemReader.open(executionContext); -Object customerCredit = new Object(); -while(customerCredit != null){ - customerCredit = itemReader.read(); - counter++; -} -itemReader.close(executionContext); - - This configured ItemReader will return - CustomerCredit objects in the exact same manner - as described by the JdbcCursorItemReader, - assuming hibernate mapping files have been created correctly for the - Customer table. The 'useStatelessSession' property defaults to true, - but has been added here to draw attention to the ability to switch it - on or off. It is also worth noting that the fetchSize of the - underlying cursor can be set via the setFetchSize property. As with - JdbcCursorItemReader, configuration is - straightforward: - - <bean id="itemReader" - class="org.springframework.batch.item.database.HibernateCursorItemReader"> - <property name="sessionFactory" ref="sessionFactory" /> - <property name="queryString" value="from CustomerCredit" /> -</bean> -
        - -
        - StoredProcedureItemReader - - Sometimes it is necessary to obtain the cursor data using a - stored procedure. The StoredProcedureItemReader - works like the JdbcCursorItemReader except that - instead of executing a query to obtain a cursor we execute a stored - procedure that returns a cursor. The stored procedure can return the - cursor in three different ways: - - - - as a returned ResultSet (used by SQL Server, Sybase, DB2, - Derby and MySQL) - - - - as a ref-cursor returned as an out parameter (used by Oracle - and PostgreSQL) - - - - as the return value of a stored function call - - - - Below is a basic example configuration using the same 'customer - credit' example as earlier: - - <bean id="reader" class="org.springframework.batch.item.database.StoredProcedureItemReader"> - <property name="dataSource" ref="dataSource"/> - <property name="procedureName" value="sp_customer_credit"/> - <property name="rowMapper"> - <bean class="org.springframework.batch.sample.domain.CustomerCreditRowMapper"/> - </property> -</bean> - - - This example relies on the stored procedure to provide a - ResultSet as a returned result (option 1 above). - - If the stored procedure returned a ref-cursor (option 2) then we - would need to provide the position of the out parameter that is the - returned ref-cursor. Here is an example where the first parameter is - the returned ref-cursor: - - <bean id="reader" class="org.springframework.batch.item.database.StoredProcedureItemReader"> - <property name="dataSource" ref="dataSource"/> - <property name="procedureName" value="sp_customer_credit"/> - <property name="refCursorPosition" value="1"/> - <property name="rowMapper"> - <bean class="org.springframework.batch.sample.domain.CustomerCreditRowMapper"/> - </property> -</bean> - - - If the cursor was returned from a stored function (option 3) we - would need to set the property "function" to - true. It defaults to false. Here - is what that would look like: - - <bean id="reader" class="org.springframework.batch.item.database.StoredProcedureItemReader"> - <property name="dataSource" ref="dataSource"/> - <property name="procedureName" value="sp_customer_credit"/> - <property name="function" value="true"/> - <property name="rowMapper"> - <bean class="org.springframework.batch.sample.domain.CustomerCreditRowMapper"/> - </property> -</bean> - - - In all of these cases we need to define a - RowMapper as well as a - DataSource and the actual procedure - name. - - If the stored procedure or function takes in parameter then they - must be declared and set via the parameters property. Here is an - example for Oracle that declares three parameters. The first one is - the out parameter that returns the ref-cursor, the second and third - are in parameters that takes a value of type INTEGER: - - <bean id="reader" class="org.springframework.batch.item.database.StoredProcedureItemReader"> - <property name="dataSource" ref="dataSource"/> - <property name="procedureName" value="spring.cursor_func"/> - <property name="parameters"> - <list> - <bean class="org.springframework.jdbc.core.SqlOutParameter"> - <constructor-arg index="0" value="newid"/> - <constructor-arg index="1"> - <util:constant static-field="oracle.jdbc.OracleTypes.CURSOR"/> - </constructor-arg> - </bean> - <bean class="org.springframework.jdbc.core.SqlParameter"> - <constructor-arg index="0" value="amount"/> - <constructor-arg index="1"> - <util:constant static-field="java.sql.Types.INTEGER"/> - </constructor-arg> - </bean> - <bean class="org.springframework.jdbc.core.SqlParameter"> - <constructor-arg index="0" value="custid"/> - <constructor-arg index="1"> - <util:constant static-field="java.sql.Types.INTEGER"/> - </constructor-arg> - </bean> - </list> - </property> - <property name="refCursorPosition" value="1"/> - <property name="rowMapper" ref="rowMapper"/> - <property name="preparedStatementSetter" ref="parameterSetter"/> -</bean> - - In addition to the parameter declarations we need to specify a - PreparedStatementSetter implementation that - sets the parameter values for the call. This works the same as for the - JdbcCursorItemReader above. All the additional - properties listed in - apply to the StoredProcedureItemReader as well. - -
        -
        - -
        - Paging ItemReaders - - An alternative to using a database cursor is executing multiple - queries where each query is bringing back a portion of the results. We - refer to this portion as a page. Each query that is executed must - specify the starting row number and the number of rows that we want - returned for the page. - -
        - JdbcPagingItemReader - - One implementation of a paging ItemReader - is the JdbcPagingItemReader. The - JdbcPagingItemReader needs a - PagingQueryProvider responsible for providing - the SQL queries used to retrieve the rows making up a page. Since each - database has its own strategy for providing paging support, we need to - use a different PagingQueryProvider for each - supported database type. There is also the - SqlPagingQueryProviderFactoryBean that will - auto-detect the database that is being used and determine the - appropriate PagingQueryProvider implementation. - This simplifies the configuration and is the recommended best - practice. - - The SqlPagingQueryProviderFactoryBean - requires that you specify a select clause and a from clause. You can - also provide an optional where clause. These clauses will be used to - build an SQL statement combined with the required sortKey. - - After the reader has been opened, it will pass back one item per - call to read in the same basic fashion as any - other ItemReader. The paging happens behind the - scenes when additional rows are needed. - - Below is an example configuration using a similar 'customer - credit' example as the cursor based ItemReaders above: - - <bean id="itemReader" class="org.spr...JdbcPagingItemReader"> - <property name="dataSource" ref="dataSource"/> - <property name="queryProvider"> - <bean class="org.spr...SqlPagingQueryProviderFactoryBean"> - <property name="selectClause" value="select id, name, credit"/> - <property name="fromClause" value="from customer"/> - <property name="whereClause" value="where status=:status"/> - <property name="sortKey" value="id"/> - </bean> - </property> - <property name="parameterValues"> - <map> - <entry key="status" value="NEW"/> - </map> - </property> - <property name="pageSize" value="1000"/> - <property name="rowMapper" ref="customerMapper"/> -</bean> - - This configured ItemReader will return - CustomerCredit objects using the - RowMapper that must be specified. The - 'pageSize' property determines the number of entities read from the - database for each query execution. - - The 'parameterValues' property can be used to specify a Map of - parameter values for the query. If you use named parameters in the - where clause the key for each entry should match the name of the named - parameter. If you use a traditional '?' placeholder then the key for - each entry should be the number of the placeholder, starting with - 1. -
        - -
        - JpaPagingItemReader - - Another implementation of a paging - ItemReader is the - JpaPagingItemReader. JPA doesn't have a concept - similar to the Hibernate StatelessSession so we - have to use other features provided by the JPA specification. Since - JPA supports paging, this is a natural choice when it comes to using - JPA for batch processing. After each page is read, the entities will - become detached and the persistence context will be cleared in order - to allow the entities to be garbage collected once the page is - processed. - - The JpaPagingItemReader allows you to - declare a JPQL statement and pass in a - EntityManagerFactory. It will then pass back - one item per call to read in the same basic - fashion as any other ItemReader. The paging - happens behind the scenes when additional entities are needed. Below - is an example configuration using the same 'customer credit' example - as the JDBC reader above: - - <bean id="itemReader" class="org.spr...JpaPagingItemReader"> - <property name="entityManagerFactory" ref="entityManagerFactory"/> - <property name="queryString" value="select c from CustomerCredit c"/> - <property name="pageSize" value="1000"/> -</bean> - - This configured ItemReader will return - CustomerCredit objects in the exact same manner - as described by the JdbcPagingItemReader above, - assuming the Customer object has the correct JPA annotations or ORM - mapping file. The 'pageSize' property determines the number of - entities read from the database for each query execution. -
        - -
        - IbatisPagingItemReader - - If you use IBATIS for your data access then you can use the - IbatisPagingItemReader which, as the name - indicates, is an implementation of a paging - ItemReader. IBATIS doesn't have direct support - for reading rows in pages but by providing a couple of standard - variables you can add paging support to your IBATIS queries. - - Here is an example of a configuration for a - IbatisPagingItemReader reading CustomerCredits - as in the examples above: - - <bean id="itemReader" class="org.spr...IbatisPagingItemReader"> - <property name="sqlMapClient" ref="sqlMapClient"/> - <property name="queryId" value="getPagedCustomerCredits"/> - <property name="pageSize" value="1000"/> -</bean> - - The IbatisPagingItemReader configuration - above references an IBATIS query called "getPagedCustomerCredits". - Here is an example of what that query should look like for - MySQL. - - <select id="getPagedCustomerCredits" resultMap="customerCreditResult"> - select id, name, credit from customer order by id asc LIMIT #_skiprows#, #_pagesize# -</select> - - The _skiprows and - _pagesize variables are provided by the - IbatisPagingItemReader and there is also a - _page variable that can be used if necessary. - The syntax for the paging queries varies with the database used. Here - is an example for Oracle (unfortunately we need to use CDATA for some - operators since this belongs in an XML document): - - <select id="getPagedCustomerCredits" resultMap="customerCreditResult"> - select * from ( - select * from ( - select t.id, t.name, t.credit, ROWNUM ROWNUM_ from customer t order by id - )) where ROWNUM_ <![CDATA[ > ]]> ( #_page# * #_pagesize# ) - ) where ROWNUM <![CDATA[ <= ]]> #_pagesize# - </select> - -
        -
        - -
        - Database ItemWriters - - While both Flat Files and XML have specific ItemWriters, there is - no exact equivalent in the database world. This is because transactions - provide all the functionality that is needed. ItemWriters are necessary - for files because they must act as if they're transactional, keeping - track of written items and flushing or clearing at the appropriate - times. Databases have no need for this functionality, since the write is - already contained in a transaction. Users can create their own DAOs that - implement the ItemWriter interface or use one - from a custom ItemWriter that's written for - generic processing concerns, either way, they should work without any - issues. One thing to look out for is the performance and error handling - capabilities that are provided by batching the outputs. This is most - common when using hibernate as an ItemWriter, but - could have the same issues when using Jdbc batch mode. Batching database - output doesn't have any inherent flaws, assuming we are careful to flush - and there are no errors in the data. However, any errors while writing - out can cause confusion because there is no way to know which individual - item caused an exception, or even if any individual item was - responsible, as illustrated below: - - - - - - - - - - If items are buffered before being written out, any - errors encountered will not be thrown until the buffer is flushed just - before a commit. For example, let's assume that 20 items will be written - per chunk, and the 15th item throws a DataIntegrityViolationException. - As far as the Step is concerned, all 20 item will be written out - successfully, since there's no way to know that an error will occur - until they are actually written out. Once - Session#flush() is - called, the buffer will be emptied and the exception will be hit. At - this point, there's nothing the Step can do, the - transaction must be rolled back. Normally, this exception might cause - the Item to be skipped (depending upon the skip/retry policies), and - then it won't be written out again. However, in the batched scenario, - there's no way for it to know which item caused the issue, the whole - buffer was being written out when the failure happened. The only way to - solve this issue is to flush after each item: - - - - - - - - - - - - This is a common use case, especially when using Hibernate, and - the simple guideline for implementations of - ItemWriter, is to flush on each call to - write(). Doing so allows for items to be - skipped reliably, with Spring Batch taking care internally of the - granularity of the calls to ItemWriter after an - error. -
        -
        - -
        - Reusing Existing Services - - Batch systems are often used in conjunction with other application - styles. The most common is an online system, but it may also support - integration or even a thick client application by moving necessary bulk - data that each application style uses. For this reason, it is common that - many users want to reuse existing DAOs or other services within their - batch jobs. The Spring container itself makes this fairly easy by allowing - any necessary class to be injected. However, there may be cases where the - existing service needs to act as an ItemReader or - ItemWriter, either to satisfy the dependency of - another Spring Batch class, or because it truly is the main - ItemReader for a step. It is fairly trivial to - write an adaptor class for each service that needs wrapping, but because - it is such a common concern, Spring Batch provides implementations: - ItemReaderAdapter and - ItemWriterAdapter. Both classes implement the - standard Spring method invoking the delegate pattern and are fairly simple - to set up. Below is an example of the reader: - - <bean id="itemReader" class="org.springframework.batch.item.adapter.ItemReaderAdapter"> - <property name="targetObject" ref="fooService" /> - <property name="targetMethod" value="generateFoo" /> -</bean> - -<bean id="fooService" class="org.springframework.batch.item.sample.FooService" /> - - One important point to note is that the contract of the targetMethod - must be the same as the contract for read: when - exhausted it will return null, otherwise an Object. - Anything else will prevent the framework from knowing when processing - should end, either causing an infinite loop or incorrect failure, - depending upon the implementation of the - ItemWriter. The ItemWriter - implementation is equally as simple: - - <bean id="itemWriter" class="org.springframework.batch.item.adapter.ItemWriterAdapter"> - <property name="targetObject" ref="fooService" /> - <property name="targetMethod" value="processFoo" /> -</bean> - -<bean id="fooService" class="org.springframework.batch.item.sample.FooService" /> - -
        - -
        - Validating Input - - During the course of this chapter, multiple approaches to parsing - input have been discussed. Each major implementation will throw an - exception if it is not 'well-formed'. The - FixedLengthTokenizer will throw an exception if a - range of data is missing. Similarly, attempting to access an index in a - RowMapper of FieldSetMapper - that doesn't exist or is in a different format than the one expected will - cause an exception to be thrown. All of these types of exceptions will be - thrown before read returns. However, they don't - address the issue of whether or not the returned item is valid. For - example, if one of the fields is an age, it obviously cannot be negative. - It will parse correctly, because it existed and is a number, but it won't - cause an exception. Since there are already a plethora of Validation - frameworks, Spring Batch does not attempt to provide yet another, but - rather provides a very simple interface that can be implemented by any - number of frameworks: - - public interface Validator { - - void validate(Object value) throws ValidationException; - -} - - The contract is that the validate method - will throw an exception if the object is invalid, and return normally if - it is valid. Spring Batch provides an out of the box - ItemProcessor: - - <bean class="org.springframework.batch.item.validator.ValidatingItemProcessor"> - <property name="validator" ref="validator" /> -</bean> - -<bean id="validator" - class="org.springframework.batch.item.validator.SpringValidator"> - <property name="validator"> - <bean id="orderValidator" - class="org.springmodules.validation.valang.ValangValidator"> - <property name="valang"> - <value> - <![CDATA[ - { orderId : ? > 0 AND ? <= 9999999999 : 'Incorrect order ID' : 'error.order.id' } - { totalLines : ? = size(lineItems) : 'Bad count of order lines' - : 'error.order.lines.badcount'} - { customer.registered : customer.businessCustomer = FALSE OR ? = TRUE - : 'Business customer must be registered' - : 'error.customer.registration'} - { customer.companyName : customer.businessCustomer = FALSE OR ? HAS TEXT - : 'Company name for business customer is mandatory' - :'error.customer.companyname'} - ]]> - </value> - </property> - </bean> - </property> -</bean> - - This simple example shows a simple - ValangValidator that is used to validate an order - object. The intent is not to show Valang functionality as much as to show - how a validator could be added. -
        - -
        - Preventing State Persistence - - By default, all of the ItemReader and - ItemWriter implementations store their current - state in the ExecutionContext before it is - committed. However, this may not always be the desired behavior. For - example, many developers choose to make their database readers - 'rerunnable' by using a process indicator. An extra column is added to the - input data to indicate whether or not it has been processed. When a - particular record is being read (or written out) the processed flag is - flipped from false to true. The SQL statement can then contain an extra - statement in the where clause, such as "where PROCESSED_IND = false", - thereby ensuring that only unprocessed records will be returned in the - case of a restart. In this scenario, it is preferable to not store any - state, such as the current row number, since it will be irrelevant upon - restart. For this reason, all readers and writers include the 'saveState' - property: - - <bean id="playerSummarizationSource" class="org.spr...JdbcCursorItemReader"> - <property name="dataSource" ref="dataSource" /> - <property name="rowMapper"> - <bean class="org.springframework.batch.sample.PlayerSummaryMapper" /> - </property> - <property name="saveState" value="false" /> - <property name="sql"> - <value> - SELECT games.player_id, games.year_no, SUM(COMPLETES), - SUM(ATTEMPTS), SUM(PASSING_YARDS), SUM(PASSING_TD), - SUM(INTERCEPTIONS), SUM(RUSHES), SUM(RUSH_YARDS), - SUM(RECEPTIONS), SUM(RECEPTIONS_YARDS), SUM(TOTAL_TD) - from games, players where players.player_id = - games.player_id group by games.player_id, games.year_no - </value> - </property> -</bean> - - The ItemReader configured above will not make - any entries in the ExecutionContext for any - executions in which it participates. -
        - -
        - Creating Custom ItemReaders and - ItemWriters - - So far in this chapter the basic contracts that exist for reading - and writing in Spring Batch and some common implementations have been - discussed. However, these are all fairly generic, and there are many - potential scenarios that may not be covered by out of the box - implementations. This section will show, using a simple example, how to - create a custom ItemReader and - ItemWriter implementation and implement their - contracts correctly. The ItemReader will also - implement ItemStream, in order to illustrate how to - make a reader or writer restartable. - -
        - Custom ItemReader Example - - For the purpose of this example, a simple - ItemReader implementation that reads from a - provided list will be created. We'll start out by implementing the most - basic contract of ItemReader, - read: - - public class CustomItemReader<T> implements ItemReader<T>{ - - List<T> items; - - public CustomItemReader(List<T> items) { - this.items = items; - } - - public T read() throws Exception, UnexpectedInputException, - NoWorkFoundException, ParseException { - - if (!items.isEmpty()) { - return items.remove(0); - } - return null; - } -} - - This very simple class takes a list of items, and returns them one - at a time, removing each from the list. When the list is empty, it - returns null, thus satisfying the most basic requirements of an - ItemReader, as illustrated below: - - List<String> items = new ArrayList<String>(); -items.add("1"); -items.add("2"); -items.add("3"); - -ItemReader itemReader = new CustomItemReader<String>(items); -assertEquals("1", itemReader.read()); -assertEquals("2", itemReader.read()); -assertEquals("3", itemReader.read()); -assertNull(itemReader.read()); - -
        - Making the <classname>ItemReader</classname> - Restartable - - The final challenge now is to make the - ItemReader restartable. Currently, if the power - goes out, and processing begins again, the - ItemReader must start at the beginning. This is - actually valid in many scenarios, but it is sometimes preferable that - a batch job starts where it left off. The key discriminant is often - whether the reader is stateful or stateless. A stateless reader does - not need to worry about restartability, but a stateful one has to try - and reconstitute its last known state on restart. For this reason, we - recommend that you keep custom readers stateless if possible, so you - don't have to worry about restartability. - - If you do need to store state, then the - ItemStream interface should be used: - - public class CustomItemReader<T> implements ItemReader<T>, ItemStream { - - List<T> items; - int currentIndex = 0; - private static final String CURRENT_INDEX = "current.index"; - - public CustomItemReader(List<T> items) { - this.items = items; - } - - public T read() throws Exception, UnexpectedInputException, - ParseException { - - if (currentIndex < items.size()) { - return items.get(currentIndex++); - } - - return null; - } - - public void open(ExecutionContext executionContext) throws ItemStreamException { - if(executionContext.containsKey(CURRENT_INDEX)){ - currentIndex = new Long(executionContext.getLong(CURRENT_INDEX)).intValue(); - } - else{ - currentIndex = 0; - } - } - - public void update(ExecutionContext executionContext) throws ItemStreamException { - executionContext.putLong(CURRENT_INDEX, new Long(currentIndex).longValue()); - } - - public void close() throws ItemStreamException {} -} - - On each call to the ItemStream - update method, the current index of the - ItemReader will be stored in the provided - ExecutionContext with a key of 'current.index'. - When the ItemStream open - method is called, the ExecutionContext is - checked to see if it contains an entry with that key. If the key is - found, then the current index is moved to that location. This is a - fairly trivial example, but it still meets the general - contract: - - ExecutionContext executionContext = new ExecutionContext(); -((ItemStream)itemReader).open(executionContext); -assertEquals("1", itemReader.read()); -((ItemStream)itemReader).update(executionContext); - -List<String> items = new ArrayList<String>(); -items.add("1"); -items.add("2"); -items.add("3"); -itemReader = new CustomItemReader<String>(items); - -((ItemStream)itemReader).open(executionContext); -assertEquals("2", itemReader.read()); - - Most ItemReaders have much more sophisticated restart logic. The - JdbcCursorItemReader, for example, stores the - row id of the last processed row in the Cursor. - - It is also worth noting that the key used within the - ExecutionContext should not be trivial. That is - because the same ExecutionContext is used for - all ItemStreams within a - Step. In most cases, simply prepending the key - with the class name should be enough to guarantee uniqueness. However, - in the rare cases where two of the same type of - ItemStream are used in the same step (which can - happen if two files are need for output) then a more unique name will - be needed. For this reason, many of the Spring Batch - ItemReader and - ItemWriter implementations have a - setName() property that allows this key name - to be overridden. -
        -
        - -
        - Custom ItemWriter Example - - Implementing a Custom ItemWriter is similar - in many ways to the ItemReader example above, but - differs in enough ways as to warrant its own example. However, adding - restartability is essentially the same, so it won't be covered in this - example. As with the ItemReader example, a - List will be used in order to keep the example as - simple as possible: - - public class CustomItemWriter<T> implements ItemWriter<T> { - - List<T> output = TransactionAwareProxyFactory.createTransactionalList(); - - public void write(List<? extends T> items) throws Exception { - output.addAll(items); - } - - public List<T> getOutput() { - return output; - } -} - -
        - Making the <classname>ItemWriter</classname> - Restartable - - To make the ItemWriter restartable we would follow the same - process as for the ItemReader, adding and - implementing the ItemStream interface to - synchronize the execution context. In the example we might have to - count the number of items processed and add that as a footer record. - If we needed to do that, we could implement - ItemStream in our - ItemWriter so that the counter was - reconstituted from the execution context if the stream was - re-opened. - - In many realistic cases, custom ItemWriters also delegate to - another writer that itself is restartable (e.g. when writing to a - file), or else it writes to a transactional resource so doesn't need - to be restartable because it is stateless. When you have a stateful - writer you should probably also be sure to implement - ItemStream as well as - ItemWriter. Remember also that the client of - the writer needs to be aware of the ItemStream, - so you may need to register it as a stream in the configuration - xml. -
        -
        -
        -
        diff --git a/src/site/docbook/reference/repeat.xml b/src/site/docbook/reference/repeat.xml deleted file mode 100644 index 0aea0fb850..0000000000 --- a/src/site/docbook/reference/repeat.xml +++ /dev/null @@ -1,287 +0,0 @@ - - - - Repeat - -
        - RepeatTemplate - - Batch processing is about repetitive actions - either as a simple - optimization, or as part of a job. To strategize and generalize the - repetition as well as to provide what amounts to an iterator framework, - Spring Batch has the RepeatOperations interface. - The RepeatOperations interface looks like - this: - - public interface RepeatOperations { - - RepeatStatus iterate(RepeatCallback callback) throws RepeatException; - -}The callback is a simple interface that allows you to insert - some business logic to be repeated: - - public interface RepeatCallback { - - RepeatStatus doInIteration(RepeatContext context) throws Exception; - -}The callback is executed repeatedly until the implementation - decides that the iteration should end. The return value in these - interfaces is an enumeration that can either be - RepeatStatus.CONTINUABLE or - RepeatStatus.FINISHED. A RepeatStatus - conveys information to the caller of the repeat operations about whether - there is any more work to do. Generally speaking, implementations of - RepeatOperations should inspect the - RepeatStatus and use it as part of the decision to - end the iteration. Any callback that wishes to signal to the caller that - there is no more work to do can return - RepeatStatus.FINISHED. - - The simplest general purpose implementation of - RepeatOperations is - RepeatTemplate. It could be used like this: - - RepeatTemplate template = new RepeatTemplate(); - -template.setCompletionPolicy(new FixedChunkSizeCompletionPolicy(2)); - -template.iterate(new RepeatCallback() { - - public ExitStatus doInIteration(RepeatContext context) { - // Do stuff in batch... - return ExitStatus.CONTINUABLE; - } - -}); - - In the example we return RepeatStatus.CONTINUABLE to - show that there is more work to do. The callback can also return - ExitStatus.FINISHED if it wants to signal to the caller that - there is no more work to do. Some iterations can be terminated by - considerations intrinsic to the work being done in the callback, others - are effectively infinite loops as far as the callback is concerned and the - completion decision is delegated to an external policy as in the case - above. - -
        - RepeatContext - - The method parameter for the RepeatCallback - is a RepeatContext. Many callbacks will simply - ignore the context, but if necessary it can be used as an attribute bag - to store transient data for the duration of the iteration. After the - iterate method returns, the context will no - longer exist. - - A RepeatContext will have a parent context - if there is a nested iteration in progress. The parent context is - occasionally useful for storing data that need to be shared between - calls to iterate. This is the case for instance - if you want to count the number of occurrences of an event in the - iteration and remember it across subsequent calls. -
        - -
        - RepeatStatus - - RepeatStatus is an enumeration used by - Spring Batch to indicate whether processing has finished. These are - possible RepeatStatus values: - - - ExitStatus Properties - - - - - Value - - Description - - - - CONTINUABLE - - There is more work to do. - - - - FINISHED - - No more repetitions should take place. - - - -
        - - RepeatStatus values can also be combined - with a logical AND operation using the and() - method in RepeatStatus. The effect of this is to - do a logical AND on the continuable flag. In other words, if either - status is FINISHED, then the result will be - FINISHED. -
        -
        - -
        - Completion Policies - - Inside a RepeatTemplate the termination of - the loop in the iterate method is determined by a - CompletionPolicy which is also a factory for the - RepeatContext. The - RepeatTemplate has the responsibility to use the - current policy to create a RepeatContext and pass - that in to the RepeatCallback at every stage in the - iteration. After a callback completes its - doInIteration, the - RepeatTemplate has to make a call to the - CompletionPolicy to ask it to update its state - (which will be stored in the RepeatContext). Then - it asks the policy if the iteration is complete. - - Spring Batch provides some simple general purpose implementations of - CompletionPolicy. The - SimpleCompletionPolicy just allows an execution up - to a fixed number of times (with RepeatStatus.FINISHED - forcing early completion at any time). - - Users might need to implement their own completion policies for more - complicated decisions. For example, a batch processing window that - prevents batch jobs from executing once the online systems are in use - would require a custom policy. -
        - -
        - Exception Handling - - If there is an exception thrown inside a - RepeatCallback, the - RepeatTemplate consults an - ExceptionHandler which can decide whether or not to - re-throw the exception. - - public interface ExceptionHandler { - - void handleException(RepeatContext context, Throwable throwable) - throws RuntimeException; - -}A common use case is to count the number of exceptions of a - given type, and fail when a limit is reached. For this purpose Spring - Batch provides the SimpleLimitExceptionHandler and - slightly more flexible - RethrowOnThresholdExceptionHandler. The - SimpleLimitExceptionHandler has a limit property - and an exception type that should be compared with the current exception - - all subclasses of the provided type are also counted. Exceptions of the - given type are ignored until the limit is reached, and then rethrown. - Those of other types are always rethrown. - - An important optional property of the - SimpleLimitExceptionHandler is the boolean flag - useParent. It is false by default, so the limit is only - accounted for in the current RepeatContext. When - set to true, the limit is kept across sibling contexts in a nested - iteration (e.g. a set of chunks inside a step). -
        - -
        - Listeners - - Often it is useful to be able to receive additional callbacks for - cross cutting concerns across a number of different iterations. For this - purpose Spring Batch provides the RepeatListener - interface. The RepeatTemplate allows users to - register RepeatListeners, and they will be given - callbacks with the RepeatContext and - RepeatStatus where available during the - iteration. - - The interface looks like this: - - public interface RepeatListener { - void before(RepeatContext context); - - void after(RepeatContext context, RepeatStatus result); - - void open(RepeatContext context); - - void onError(RepeatContext context, Throwable e); - - void close(RepeatContext context); -}The open and - close callbacks come before and after the entire - iteration. before, after - and onError apply to the individual - RepeatCallback calls. - - Note that when there is more than one listener, they are in a list, - so there is an order. In this case open and - before are called in the same order while - after, onError and - close are called in reverse order. -
        - -
        - Parallel Processing - - Implementations of RepeatOperations are not - restricted to executing the callback sequentially. It is quite important - that some implementations are able to execute their callbacks in parallel. - To this end, Spring Batch provides the - TaskExecutorRepeatTemplate, which uses the Spring - TaskExecutor strategy to run the - RepeatCallback. The default is to use a - SynchronousTaskExecutor, which has the effect of - executing the whole iteration in the same thread (the same as a normal - RepeatTemplate). -
        - -
        - Declarative Iteration - - Sometimes there is some business processing that you know you want - to repeat every time it happens. The classic example of this is the - optimization of a message pipeline - it is more efficient to process a - batch of messages, if they are arriving frequently, than to bear the cost - of a separate transaction for every message. Spring Batch provides an AOP - interceptor that wraps a method call in a - RepeatOperations for just this purpose. The - RepeatOperationsInterceptor executes the - intercepted method and repeats according to the - CompletionPolicy in the provided - RepeatTemplate. - - Here is an example of declarative iteration using the Spring AOP - namespace to repeat a service call to a method called - processMessage (for more detail on how to - configure AOP interceptors see the Spring User Guide): - - <aop:config> - <aop:pointcut id="transactional" - expression="execution(* com..*Service.processMessage(..))" /> - <aop:advisor pointcut-ref="transactional" - advice-ref="retryAdvice" order="-1"/> -</aop:config> - -<bean id="retryAdvice" class="org.spr...RepeatOperationsInterceptor"/> - - The example above uses a default - RepeatTemplate inside the interceptor. To change - the policies, listeners etc. you only need to inject an instance of - RepeatTemplate into the interceptor. - - If the intercepted method returns void then the - interceptor always returns ExitStatus.CONTINUABLE (so there is a danger of - an infinite loop if the CompletionPolicy does not - have a finite end point). Otherwise it returns - ExitStatus.CONTINUABLE until the return value from the - intercepted method is null, at which point it returns - ExitStatus.FINISHED. So the business logic inside the target - method can signal that there is no more work to do by returning - null, or by throwing an exception that is re-thrown by the - ExceptionHandler in the provided - RepeatTemplate. -
        -
        diff --git a/src/site/docbook/reference/retry.xml b/src/site/docbook/reference/retry.xml deleted file mode 100644 index eb2677c723..0000000000 --- a/src/site/docbook/reference/retry.xml +++ /dev/null @@ -1,363 +0,0 @@ - - - - Retry - -
        - RetryTemplate - - - The retry functionality was pulled out of Spring Batch as of 2.2.0. - It is now part of a new library, Spring Retry. - - - To make processing more robust and less prone to failure, sometimes - it helps to automatically retry a failed operation in case it might - succeed on a subsequent attempt. Errors that are susceptible to this kind - of treatment are transient in nature. For example a remote call to a web - service or RMI service that fails because of a network glitch or a - DeadLockLoserException in a database update may - resolve themselves after a short wait. To automate the retry of such - operations Spring Batch has the RetryOperations - strategy. The RetryOperations interface looks like - this: - - public interface RetryOperations { - - <T> T execute(RetryCallback<T> retryCallback) throws Exception; - - <T> T execute(RetryCallback<T> retryCallback, RecoveryCallback<T> recoveryCallback) - throws Exception; - - <T> T execute(RetryCallback<T> retryCallback, RetryState retryState) - throws Exception, ExhaustedRetryException; - - <T> T execute(RetryCallback<T> retryCallback, RecoveryCallback<T> recoveryCallback, - RetryState retryState) throws Exception; - -}The basic callback is a simple interface that allows you to - insert some business logic to be retried: - - public interface RetryCallback<T> { - - T doWithRetry(RetryContext context) throws Throwable; - -}The callback is executed and if it fails (by throwing an - Exception), it will be retried until either it is - successful, or the implementation decides to abort. There are a number of - overloaded execute methods in the - RetryOperations interface dealing with various use - cases for recovery when all retry attempts are exhausted, and also with - retry state, which allows clients and implementations to store information - between calls (more on this later). - - The simplest general purpose implementation of - RetryOperations is - RetryTemplate. It could be used like this - - RetryTemplate template = new RetryTemplate(); - -TimeoutRetryPolicy policy = new TimeoutRetryPolicy(); -policy.setTimeout(30000L); - -template.setRetryPolicy(policy); - -Foo result = template.execute(new RetryCallback<Foo>() { - - public Foo doWithRetry(RetryContext context) { - // Do stuff that might fail, e.g. webservice operation - return result; - } - -}); - - In the example we execute a web service call and return the result - to the user. If that call fails then it is retried until a timeout is - reached. - -
        - RetryContext - - The method parameter for the RetryCallback - is a RetryContext. Many callbacks will simply - ignore the context, but if necessary it can be used as an attribute bag - to store data for the duration of the iteration. - - A RetryContext will have a parent context - if there is a nested retry in progress in the same thread. The parent - context is occasionally useful for storing data that need to be shared - between calls to execute. -
        - -
        - RecoveryCallback - - When a retry is exhausted the - RetryOperations can pass control to a different - callback, the RecoveryCallback. To use this - feature clients just pass in the callbacks together to the same method, - for example: - - Foo foo = template.execute(new RetryCallback<Foo>() { - public Foo doWithRetry(RetryContext context) { - // business logic here - }, - new RecoveryCallback<Foo>() { - Foo recover(RetryContext context) throws Exception { - // recover logic here - } -});If the business logic does not succeed before the template - decides to abort, then the client is given the chance to do some - alternate processing through the recovery callback. -
        - -
        - Stateless Retry - - In the simplest case, a retry is just a while loop: the - RetryTemplate can just keep trying until it - either succeeds or fails. The RetryContext - contains some state to determine whether to retry or abort, but this - state is on the stack and there is no need to store it anywhere - globally, so we call this stateless retry. The distinction between - stateless and stateful retry is contained in the implementation of the - RetryPolicy (the - RetryTemplate can handle both). In a stateless - retry, the callback is always executed in the same thread on retry as - when it failed. -
        - -
        - Stateful Retry - - Where the failure has caused a transactional resource to become - invalid, there are some special considerations. This does not apply to a - simple remote call because there is no transactional resource (usually), - but it does sometimes apply to a database update, especially when using - Hibernate. In this case it only makes sense to rethrow the exception - that called the failure immediately so that the transaction can roll - back and we can start a new valid one. - - In these cases a stateless retry is not good enough because the - re-throw and roll back necessarily involve leaving the - RetryOperations.execute() method and potentially losing the - context that was on the stack. To avoid losing it we have to introduce a - storage strategy to lift it off the stack and put it (at a minimum) in - heap storage. For this purpose Spring Batch provides a storage strategy - RetryContextCache which can be injected into the - RetryTemplate. The default implementation of the - RetryContextCache is in memory, using a simple - Map. Advanced usage with multiple processes in a - clustered environment might also consider implementing the - RetryContextCache with a cluster cache of some - sort (though, even in a clustered environment this might be - overkill). - - Part of the responsibility of the - RetryOperations is to recognize the failed - operations when they come back in a new execution (and usually wrapped - in a new transaction). To facilitate this, Spring Batch provides the - RetryState abstraction. This works in conjunction - with a special execute methods in the - RetryOperations. - - The way the failed operations are recognized is by identifying the - state across multiple invocations of the retry. To identify the state, - the user can provide an RetryState object that is - responsible for returning a unique key identifying the item. The - identifier is used as a key in the - RetryContextCache. - - - Be very careful with the implementation of - Object.equals() and Object.hashCode() in the - key returned by RetryState. The best advice is - to use a business key to identify the items. In the case of a JMS - message the message ID can be used. - - - When the retry is exhausted there is also the option to handle the - failed item in a different way, instead of calling the - RetryCallback (which is presumed now to be likely - to fail). Just like in the stateless case, this option is provided by - the RecoveryCallback, which can be provided by - passing it in to the execute method of - RetryOperations. - - The decision to retry or not is actually delegated to a regular - RetryPolicy, so the usual concerns about limits - and timeouts can be injected there (see below). -
        -
        - -
        - Retry Policies - - Inside a RetryTemplate the decision to retry - or fail in the execute method is determined by a - RetryPolicy which is also a factory for the - RetryContext. The - RetryTemplate has the responsibility to use the - current policy to create a RetryContext and pass - that in to the RetryCallback at every attempt. - After a callback fails the RetryTemplate has to - make a call to the RetryPolicy to ask it to update - its state (which will be stored in the - RetryContext), and then it asks the policy if - another attempt can be made. If another attempt cannot be made (e.g. a - limit is reached or a timeout is detected) then the policy is also - responsible for handling the exhausted state. Simple implementations will - just throw RetryExhaustedException which will cause - any enclosing transaction to be rolled back. More sophisticated - implementations might attempt to take some recovery action, in which case - the transaction can remain intact. - - - Failures are inherently either retryable or not - if the same - exception is always going to be thrown from the business logic, it - doesn't help to retry it. So don't retry on all exception types - try to - focus on only those exceptions that you expect to be retryable. It's not - usually harmful to the business logic to retry more aggressively, but - it's wasteful because if a failure is deterministic there will be time - spent retrying something that you know in advance is fatal. - - - Spring Batch provides some simple general purpose implementations of - stateless RetryPolicy, for example a - SimpleRetryPolicy, and the - TimeoutRetryPolicy used in the example - above. - - The SimpleRetryPolicy just allows a retry on - any of a named list of exception types, up to a fixed number of times. It - also has a list of "fatal" exceptions that should never be retried, and - this list overrides the retryable list so that it can be used to give - finer control over the retry behavior: - - SimpleRetryPolicy policy = new SimpleRetryPolicy(); -// Set the max retry attempts -policy.setMaxAttempts(5); -// Retry on all exceptions (this is the default) -policy.setRetryableExceptions(new Class[] {Exception.class}); -// ... but never retry IllegalStateException -policy.setFatalExceptions(new Class[] {IllegalStateException.class}); - -// Use the policy... -RetryTemplate template = new RetryTemplate(); -template.setRetryPolicy(policy); -template.execute(new RetryCallback<Foo>() { - public Foo doWithRetry(RetryContext context) { - // business logic here - } -}); - - There is also a more flexible implementation called - ExceptionClassifierRetryPolicy, which allows the - user to configure different retry behavior for an arbitrary set of - exception types though the ExceptionClassifier - abstraction. The policy works by calling on the classifier to convert an - exception into a delegate RetryPolicy, so for - example, one exception type can be retried more times before failure than - another by mapping it to a different policy. - - Users might need to implement their own retry policies for more - customized decisions. For instance, if there is a well-known, - solution-specific, classification of exceptions into retryable and not - retryable. -
        - -
        - Backoff Policies - - When retrying after a transient failure it often helps to wait a bit - before trying again, because usually the failure is caused by some problem - that will only be resolved by waiting. If a - RetryCallback fails, the - RetryTemplate can pause execution according to the - BackoffPolicy in place. - - public interface BackoffPolicy { - - BackOffContext start(RetryContext context); - - void backOff(BackOffContext backOffContext) - throws BackOffInterruptedException; - -}A BackoffPolicy is free to implement - the backOff in any way it chooses. The policies provided by Spring Batch - out of the box all use Object.wait(). A common use case is to - backoff with an exponentially increasing wait period, to avoid two retries - getting into lock step and both failing - this is a lesson learned from - the ethernet. For this purpose Spring Batch provides the - ExponentialBackoffPolicy. -
        - -
        - Listeners - - Often it is useful to be able to receive additional callbacks for - cross cutting concerns across a number of different retries. For this - purpose Spring Batch provides the RetryListener - interface. The RetryTemplate allows users to - register RetryListeners, and they will be given - callbacks with the RetryContext and - Throwable where available during the - iteration. - - The interface looks like this: - - public interface RetryListener { - - void open(RetryContext context, RetryCallback<T> callback); - - void onError(RetryContext context, RetryCallback<T> callback, Throwable e); - - void close(RetryContext context, RetryCallback<T> callback, Throwable e); -}The open and - close callbacks come before and after the entire - retry in the simplest case and onError applies to - the individual RetryCallback calls. The - close method might also receive a - Throwable; if there has been an error it is the - last one thrown by the RetryCallback. - - Note that when there is more than one listener, they are in a list, - so there is an order. In this case open will be - called in the same order while onError and - close will be called in reverse order. -
        - -
        - Declarative Retry - - Sometimes there is some business processing that you know you want - to retry every time it happens. The classic example of this is the remote - service call. Spring Batch provides an AOP interceptor that wraps a method - call in a RetryOperations for just this purpose. - The RetryOperationsInterceptor executes the - intercepted method and retries on failure according to the - RetryPolicy in the provided - RepeatTemplate. - - Here is an example of declarative iteration using the Spring AOP - namespace to repeat a service call to a method called - remoteCall (for more detail on how to configure - AOP interceptors see the Spring User Guide): - - <aop:config> - <aop:pointcut id="transactional" - expression="execution(* com..*Service.remoteCall(..))" /> - <aop:advisor pointcut-ref="transactional" - advice-ref="retryAdvice" order="-1"/> -</aop:config> - -<bean id="retryAdvice" - class="org.springframework.batch.retry.interceptor.RetryOperationsInterceptor"/> - - The example above uses a default - RetryTemplate inside the interceptor. To change the - policies or listeners, you only need to inject an instance of - RetryTemplate into the interceptor. -
        -
        diff --git a/src/site/docbook/reference/scalability.xml b/src/site/docbook/reference/scalability.xml deleted file mode 100644 index b2eea9a168..0000000000 --- a/src/site/docbook/reference/scalability.xml +++ /dev/null @@ -1,414 +0,0 @@ - - - - Scaling and Parallel Processing - - Many batch processing problems can be solved with single threaded, - single process jobs, so it is always a good idea to properly check if that - meets your needs before thinking about more complex implementations. Measure - the performance of a realistic job and see if the simplest implementation - meets your needs first: you can read and write a file of several hundred - megabytes in well under a minute, even with standard hardware. - - When you are ready to start implementing a job with some parallel - processing, Spring Batch offers a range of options, which are described in - this chapter, although some features are covered elsewhere. At a high level - there are two modes of parallel processing: single process, multi-threaded; - and multi-process. These break down into categories as well, as - follows: - - - - Multi-threaded Step (single process) - - - - Parallel Steps (single process) - - - - Remote Chunking of Step (multi process) - - - - Partitioning a Step (single or multi process) - - - - Next we review the single-process options first, and then the - multi-process options. - -
        - Multi-threaded Step - - The simplest way to start parallel processing is to add a - TaskExecutor to your Step configuration, e.g. as an - attribute of the tasklet: - - <step id="loading"> - <tasklet task-executor="taskExecutor">...</tasklet> -</step> - - In this example the taskExecutor is a reference to another bean - definition, implementing the TaskExecutor - interface. TaskExecutor is a standard Spring - interface, so consult the Spring User Guide for details of available - implementations. The simplest multi-threaded - TaskExecutor is a - SimpleAsyncTaskExecutor. - - The result of the above configuration will be that the Step - executes by reading, processing and writing each chunk of items - (each commit interval) in a separate thread of execution. Note - that this means there is no fixed order for the items to be - processed, and a chunk might contain items that are - non-consecutive compared to the single-threaded case. In addition - to any limits placed by the task executor (e.g. if it is backed by - a thread pool), there is a throttle limit in the tasklet - configuration which defaults to 4. You may need to increase this - to ensure that a thread pool is fully utilised, e.g. - - <step id="loading"> <tasklet - task-executor="taskExecutor" - throttle-limit="20">...</tasklet> - </step> - - Note also that there may be limits placed on concurrency by - any pooled resources used in your step, such as - a DataSource. Be sure to make the pool in - those resources at least as large as the desired number of - concurrent threads in the step. - - There are some practical limitations of using multi-threaded Steps - for some common Batch use cases. Many participants in a Step (e.g. readers - and writers) are stateful, and if the state is not segregated by thread, - then those components are not usable in a multi-threaded Step. In - particular most of the off-the-shelf readers and writers from Spring Batch - are not designed for multi-threaded use. It is, however, possible to work - with stateless or thread safe readers and writers, and there is a sample - (parallelJob) in the Spring Batch Samples that show the use of a process - indicator (see ) to keep - track of items that have been processed in a database input table. - - Spring Batch provides some implementations of - ItemWriter and - ItemReader. Usually they say in the - Javadocs if they are thread safe or not, or what you have to do to - avoid problems in a concurrent environment. If there is no - information in Javadocs, you can check the implementation to see - if there is any state. If a reader is not thread safe, it may - still be efficient to use it in your own synchronizing delegator. - You can synchronize the call to read() and as - long as the processing and writing is the most expensive part of - the chunk your step may still complete much faster than in a - single threaded configuration. - - -
        - -
        - Parallel Steps - - As long as the application logic that needs to be parallelized can - be split into distinct responsibilities, and assigned to individual steps - then it can be parallelized in a single process. Parallel Step execution - is easy to configure and use, for example, to execute steps - (step1,step2) in parallel with - step3, you could configure a flow like this: - - <job id="job1"> - <split id="split1" task-executor="taskExecutor" next="step4"> - <flow> - <step id="step1" parent="s1" next="step2"/> - <step id="step2" parent="s2"/> - </flow> - <flow> - <step id="step3" parent="s3"/> - </flow> - </split> - <step id="step4" parent="s4"/> -</job> - -<beans:bean id="taskExecutor" class="org.spr...SimpleAsyncTaskExecutor"/> - - The configurable "task-executor" attribute is used to specify which - TaskExecutor implementation should be used to execute the individual - flows. The default is SyncTaskExecutor, but an - asynchronous TaskExecutor is required to run the steps in parallel. Note - that the job will ensure that every flow in the split completes before - aggregating the exit statuses and transitioning. - - See the section on for more - detail. -
        - -
        - Remote Chunking - - In Remote Chunking the Step processing is split across multiple - processes, communicating with each other through some middleware. Here is - a picture of the pattern in action: - - - - - - - - The Master component is a single process, and the Slaves are - multiple remote processes. Clearly this pattern works best if the Master - is not a bottleneck, so the processing must be more expensive than the - reading of items (this is often the case in practice). - - The Master is just an implementation of a Spring Batch - Step, with the ItemWriter replaced with a generic - version that knows how to send chunks of items to the middleware as - messages. The Slaves are standard listeners for whatever middleware is - being used (e.g. with JMS they would be - MesssageListeners), and their role is to process - the chunks of items using a standard ItemWriter or - ItemProcessor plus - ItemWriter, through the - ChunkProcessor interface. One of the advantages of - using this pattern is that the reader, processor and writer components are - off-the-shelf (the same as would be used for a local execution of the - step). The items are divided up dynamically and work is shared through the - middleware, so if the listeners are all eager consumers, then load - balancing is automatic. - - The middleware has to be durable, with guaranteed delivery and - single consumer for each message. JMS is the obvious candidate, but other - options exist in the grid computing and shared memory product space (e.g. - Java Spaces). - - Spring Batch has a sister project Spring Batch Admin, which - provides(amongst other things) implementations of various patterns - like this one using Spring Integration. These are implemented in a - module called Spring Batch Integration. -
        - -
        - Partitioning - - Spring Batch also provides an SPI for partitioning a Step execution - and executing it remotely. In this case the remote participants are simply - Step instances that could just as easily have been configured and used for - local processing. Here is a picture of the pattern in action: - - - - - - - - The Job is executing on the left hand side as a sequence of Steps, - and one of the Steps is labelled as a Master. The Slaves in this picture - are all identical instances of a Step, which could in fact take the place - of the Master resulting in the same outcome for the Job. The Slaves are - typically going to be remote services, but could also be local threads of - execution. The messages sent by the Master to the Slaves in this pattern - do not need to be durable, or have guaranteed delivery: Spring Batch - meta-data in the JobRepository will ensure that - each Slave is executed once and only once for each Job execution. - - The SPI in Spring Batch consists of a special implementation of Step - (the PartitionStep), and two strategy interfaces - that need to be implemented for the specific environment. The strategy - interfaces are PartitionHandler and - StepExecutionSplitter, and their role is show in - the sequence diagram below: - - - - - - - - The Step on the right in this case is the "remote" Slave, so - potentially there are many objects and or processes playing this role, and - the PartitionStep is shown driving the execution. The PartitionStep - configuration looks like this: - - <step id="step1.master"> - <partition step="step1" partitioner="partitioner"> - <handler grid-size="10" task-executor="taskExecutor"/> - </partition> -</step> - - Similar to the multi-threaded step's throttle-limit - attribute, the grid-size attribute prevents the task executor from - being saturated with requests from a single step. - - There is a simple example which can be copied and extended in the - unit test suite for Spring Batch Samples (see - *PartitionJob.xml configuration). - - Spring Batch creates step executions for the partitions called - "step1:partition0", etc., so many people prefer to call the master step - "step1:master" for consistency. With Spring 3.0 you can do this using an - alias for the step (specifying the name attribute - instead of the id). - -
        - PartitionHandler - - The PartitionHandler is the component that - knows about the fabric of the remoting or grid environment. It is able - to send StepExecution requests to the remote - Steps, wrapped in some fabric-specific format, like a DTO. It does not - have to know how to split up the input data, or how to aggregate the - result of multiple Step executions. Generally speaking it probably also - doesn't need to know about resilience or failover, since those are - features of the fabric in many cases, and anyway Spring Batch always - provides restartability independent of the fabric: a failed Job can - always be restarted and only the failed Steps will be - re-executed. - - The PartitionHandler interface can have - specialized implementations for a variety of fabric types: e.g. simple - RMI remoting, EJB remoting, custom web service, JMS, Java Spaces, shared - memory grids (like Terracotta or Coherence), grid execution fabrics - (like GridGain). Spring Batch does not contain implementations for any - proprietary grid or remoting fabrics. - - Spring Batch does however provide a useful implementation of - PartitionHandler that executes Steps locally in - separate threads of execution, using the - TaskExecutor strategy from Spring. The - implementation is called - TaskExecutorPartitionHandler, and it is the - default for a step configured with the XML namespace as above. It can - also be configured explicitly like this: - - <step id="step1.master"> - <partition step="step1" handler="handler"/> -</step> - -<bean class="org.spr...TaskExecutorPartitionHandler"> - <property name="taskExecutor" ref="taskExecutor"/> - <property name="step" ref="step1" /> - <property name="gridSize" value="10" /> -</bean> - - The gridSize determines the number of separate - step executions to create, so it can be matched to the size of the - thread pool in the TaskExecutor, or else it can - be set to be larger than the number of threads available, in which case - the blocks of work are smaller. - - The TaskExecutorPartitionHandler is quite - useful for IO intensive Steps, like copying large numbers of files or - replicating filesystems into content management systems. It can also be - used for remote execution by providing a Step implementation that is a - proxy for a remote invocation (e.g. using Spring Remoting). -
        - -
        - Partitioner - - The Partitioner has a simpler responsibility: to generate - execution contexts as input parameters for new step executions only (no - need to worry about restarts). It has a single method: - - public interface Partitioner { - Map<String, ExecutionContext> partition(int gridSize); -} - - The return value from this method associates a unique name for - each step execution (the String), with input - parameters in the form of an ExecutionContext. - The names show up later in the Batch meta data as the step name in the - partitioned StepExecutions. The - ExecutionContext is just a bag of name-value - pairs, so it might contain a range of primary keys, or line numbers, or - the location of an input file. The remote Step - then normally binds to the context input using #{...} - placeholders (late binding in step scope), as illustrated in the next - section. - - The names of the step executions (the keys in the - Map returned by - Partitioner) need to be unique amongst the step - executions of a Job, but do not have any other specific requirements. - The easiest way to do this, and to make the names meaningful for users, - is to use a prefix+suffix naming convention, where the prefix is the - name of the step that is being executed (which itself is unique in the - Job), and the suffix is just a counter. There is - a SimplePartitioner in the framework that uses - this convention. - - An optional interface - PartitioneNameProvider can be used to - provide the partition names separately from the partitions - themselves. If a Partitioner implements - this interface then on a restart only the names will be queried. - If partitioning is expensive this can be a useful optimisation. - Obviously the names provided by the - PartitioneNameProvider must match those - provided by the Partitioner. - -
        - -
        - Binding Input Data to Steps - - It is very efficient for the steps that are executed by the - PartitionHandler to have identical configuration, and for their input - parameters to be bound at runtime from the ExecutionContext. This is - easy to do with the StepScope feature of Spring Batch (covered in more - detail in the section on ). For example - if the Partitioner creates - ExecutionContext instances with an attribute key - fileName, pointing to a different file (or - directory) for each step invocation, the - Partitioner output might look like this: - - - Example step execution name to execution context provided by - Partitioner targeting directory processing - - - - - Step Execution Name - (key) - - ExecutionContext - (value) - - - - filecopy:partition0 - - fileName=/home/data/one - - - - filecopy:partition1 - - fileName=/home/data/two - - - - filecopy:partition2 - - fileName=/home/data/three - - - -
        - - Then the file name can be bound to a step using late binding to - the execution context: - - <bean id="itemReader" scope="step" - class="org.spr...MultiResourceItemReader"> - <property name="resource" value="#{stepExecutionContext[fileName]}/*"/> -</bean> -
        -
        -
        diff --git a/src/site/docbook/reference/schema-appendix.xml b/src/site/docbook/reference/schema-appendix.xml deleted file mode 100644 index 91fcbc7553..0000000000 --- a/src/site/docbook/reference/schema-appendix.xml +++ /dev/null @@ -1,645 +0,0 @@ - - - - Meta-Data Schema - -
        - Overview - - The Spring Batch Meta-Data tables very closely match the Domain - objects that represent them in Java. For example, - JobInstance, JobExecution, - JobParameters, and - StepExecution map to BATCH_JOB_INSTANCE, - BATCH_JOB_EXECUTION, BATCH_JOB_EXECUTION_PARAMS, and BATCH_STEP_EXECUTION, - respectively. ExecutionContext maps to both - BATCH_JOB_EXECUTION_CONTEXT and BATCH_STEP_EXECUTION_CONTEXT. The - JobRepository is responsible for saving and storing - each Java object into its correct table. The following appendix describes - the meta-data tables in detail, along with many of the design decisions - that were made when creating them. When viewing the various table creation - statements below, it is important to realize that the data types used are - as generic as possible. Spring Batch provides many schemas as examples, - which all have varying data types due to variations in individual database - vendors' handling of data types. Below is an ERD model of all 6 tables and - their relationships to one another: - - - - - - - - - - - -
        - Example DDL Scripts - - The Spring Batch Core JAR file contains example - scripts to create the relational tables for a number of database - platforms (which are in turn auto-detected by the job repository factory - bean or namespace equivalent). These scripts can be used as is, or - modified with additional indexes and constraints as desired. The file - names are in the form schema-*.sql, where "*" is the - short name of the target database platform. The scripts are in - the package org.springframework.batch.core. -
        - -
        - Version - - Many of the database tables discussed in this appendix contain a - version column. This column is important because Spring Batch employs an - optimistic locking strategy when dealing with updates to the database. - This means that each time a record is 'touched' (updated) the value in - the version column is incremented by one. When the repository goes back - to try and save the value, if the version number has change it will - throw OptimisticLockingFailureException, - indicating there has been an error with concurrent access. This check is - necessary since, even though different batch jobs may be running in - different machines, they are all using the same database tables. -
        - -
        - Identity - - BATCH_JOB_INSTANCE, BATCH_JOB_EXECUTION, and BATCH_STEP_EXECUTION - each contain columns ending in _ID. These fields act as primary keys for - their respective tables. However, they are not database generated keys, - but rather they are generated by separate sequences. This is necessary - because after inserting one of the domain objects into the database, the - key it is given needs to be set on the actual object so that they can be - uniquely identified in Java. Newer database drivers (Jdbc 3.0 and up) - support this feature with database generated keys, but rather than - requiring it, sequences were used. Each variation of the schema will - contain some form of the following: - - CREATE SEQUENCE BATCH_STEP_EXECUTION_SEQ; -CREATE SEQUENCE BATCH_JOB_EXECUTION_SEQ; -CREATE SEQUENCE BATCH_JOB_SEQ; - - Many database vendors don't support sequences. In these cases, - work-arounds are used, such as the following for MySQL: - - CREATE TABLE BATCH_STEP_EXECUTION_SEQ (ID BIGINT NOT NULL) type=MYISAM; -INSERT INTO BATCH_STEP_EXECUTION_SEQ values(0); -CREATE TABLE BATCH_JOB_EXECUTION_SEQ (ID BIGINT NOT NULL) type=MYISAM; -INSERT INTO BATCH_JOB_EXECUTION_SEQ values(0); -CREATE TABLE BATCH_JOB_SEQ (ID BIGINT NOT NULL) type=MYISAM; -INSERT INTO BATCH_JOB_SEQ values(0); - - In the above case, a table is used in place of each sequence. The - Spring core class MySQLMaxValueIncrementer will - then increment the one column in this sequence in order to give similar - functionality. -
        -
        - -
        - BATCH_JOB_INSTANCE - - The BATCH_JOB_INSTANCE table holds all information relevant to a - JobInstance, and serves as the top of the overall - hierarchy. The following generic DDL statement is used to create - it: - - CREATE TABLE BATCH_JOB_INSTANCE ( - JOB_INSTANCE_ID BIGINT PRIMARY KEY , - VERSION BIGINT, - JOB_NAME VARCHAR(100) NOT NULL , - JOB_KEY VARCHAR(2500) -); - - Below are descriptions of each column in the table: - - - - JOB_INSTANCE_ID: The unique id that will identify the instance, - which is also the primary key. The value of this column should be - obtainable by calling the getId method on - JobInstance. - - - - VERSION: See above section. - - - - JOB_NAME: Name of the job obtained from the - Job object. Because it is required to identify - the instance, it must not be null. - - - - JOB_KEY: A serialization of the - JobParameters that uniquely identifies separate - instances of the same job from one another. - (JobInstances with the same job name must have - different JobParameters, and thus, different - JOB_KEY values). - - -
        - -
        - BATCH_JOB_EXECUTION_PARAMS - - The BATCH_JOB_EXECUTION_PARAMS table holds all information relevant to the - JobParameters object. It contains 0 or more - key/value pairs passed to a Job and serve as a record of the parameters - a job was run with. For each parameter that contributes to the generation of a job's identity, - the IDENTIFYING flag is set to true. It should be noted that the table has been - denormalized. Rather than creating a separate table for each type, there - is one table with a column indicating the type: - - CREATE TABLE BATCH_JOB_EXECUTION_PARAMS ( - JOB_EXECUTION_ID BIGINT NOT NULL , - TYPE_CD VARCHAR(6) NOT NULL , - KEY_NAME VARCHAR(100) NOT NULL , - STRING_VAL VARCHAR(250) , - DATE_VAL DATETIME DEFAULT NULL , - LONG_VAL BIGINT , - DOUBLE_VAL DOUBLE PRECISION , - IDENTIFYING CHAR(1) NOT NULL , - constraint JOB_EXEC_PARAMS_FK foreign key (JOB_EXECUTION_ID) - references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) -); - - Below are descriptions for each column: - - - - JOB_EXECUTION_ID: Foreign Key from the BATCH_JOB_EXECUTION table - that indicates the job execution the parameter entry belongs to. It - should be noted that multiple rows (i.e key/value pairs) may exist for - each execution. - - - - TYPE_CD: String representation of the type of value stored, - which can be either a string, date, long, or double. Because the type - must be known, it cannot be null. - - - - KEY_NAME: The parameter key. - - - - STRING_VAL: Parameter value, if the type is string. - - - - DATE_VAL: Parameter value, if the type is date. - - - - LONG_VAL: Parameter value, if the type is a long. - - - - DOUBLE_VAL: Parameter value, if the type is double. - - - - IDENTIFYING: Flag indicating if the parameter contributed to the identity of the related JobInstance. - - - - It is worth noting that there is no primary key for this table. This - is simply because the framework has no use for one, and thus doesn't - require it. If a user so chooses, one may be added with a database - generated key, without causing any issues to the framework itself. -
        - -
        - BATCH_JOB_EXECUTION - - The BATCH_JOB_EXECUTION table holds all information relevant to the - JobExecution object. Every time a - Job is run there will always be a new - JobExecution, and a new row in this table: - - CREATE TABLE BATCH_JOB_EXECUTION ( - JOB_EXECUTION_ID BIGINT PRIMARY KEY , - VERSION BIGINT, - JOB_INSTANCE_ID BIGINT NOT NULL, - CREATE_TIME TIMESTAMP NOT NULL, - START_TIME TIMESTAMP DEFAULT NULL, - END_TIME TIMESTAMP DEFAULT NULL, - STATUS VARCHAR(10), - EXIT_CODE VARCHAR(20), - EXIT_MESSAGE VARCHAR(2500), - LAST_UPDATED TIMESTAMP, - constraint JOB_INSTANCE_EXECUTION_FK foreign key (JOB_INSTANCE_ID) - references BATCH_JOB_INSTANCE(JOB_INSTANCE_ID) -) ; - - Below are descriptions for each column: - - - - JOB_EXECUTION_ID: Primary key that uniquely identifies this - execution. The value of this column is obtainable by calling the - getId method of the - JobExecution object. - - - - VERSION: See above section. - - - - JOB_INSTANCE_ID: Foreign key from the BATCH_JOB_INSTANCE table - indicating the instance to which this execution belongs. There may be - more than one execution per instance. - - - - CREATE_TIME: Timestamp representing the time that the execution - was created. - - - - START_TIME: Timestamp representing the time the execution was - started. - - - - END_TIME: Timestamp representing the time the execution was - finished, regardless of success or failure. An empty value in this - column even though the job is not currently running indicates that - there has been some type of error and the framework was unable to - perform a last save before failing. - - - - STATUS: Character string representing the status of the - execution. This may be COMPLETED, STARTED, etc. The object - representation of this column is the - BatchStatus enumeration. - - - - EXIT_CODE: Character string representing the exit code of the - execution. In the case of a command line job, this may be converted - into a number. - - - - EXIT_MESSAGE: Character string representing a more detailed - description of how the job exited. In the case of failure, this might - include as much of the stack trace as is possible. - - - - LAST_UPDATED: Timestamp representing the last time this - execution was persisted. - - -
        - -
        - BATCH_STEP_EXECUTION - - The BATCH_STEP_EXECUTION table holds all information relevant to the - StepExecution object. This table is very similar in - many ways to the BATCH_JOB_EXECUTION table and there will always be at - least one entry per Step for each - JobExecution created: - - CREATE TABLE BATCH_STEP_EXECUTION ( - STEP_EXECUTION_ID BIGINT PRIMARY KEY , - VERSION BIGINT NOT NULL, - STEP_NAME VARCHAR(100) NOT NULL, - JOB_EXECUTION_ID BIGINT NOT NULL, - START_TIME TIMESTAMP NOT NULL , - END_TIME TIMESTAMP DEFAULT NULL, - STATUS VARCHAR(10), - COMMIT_COUNT BIGINT , - READ_COUNT BIGINT , - FILTER_COUNT BIGINT , - WRITE_COUNT BIGINT , - READ_SKIP_COUNT BIGINT , - WRITE_SKIP_COUNT BIGINT , - PROCESS_SKIP_COUNT BIGINT , - ROLLBACK_COUNT BIGINT , - EXIT_CODE VARCHAR(20) , - EXIT_MESSAGE VARCHAR(2500) , - LAST_UPDATED TIMESTAMP, - constraint JOB_EXECUTION_STEP_FK foreign key (JOB_EXECUTION_ID) - references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) -) ; - - Below are descriptions for each column: - - - - STEP_EXECUTION_ID: Primary key that uniquely identifies this - execution. The value of this column should be obtainable by calling - the getId method of the - StepExecution object. - - - - VERSION: See above section. - - - - STEP_NAME: The name of the step to which this execution - belongs. - - - - JOB_EXECUTION_ID: Foreign key from the BATCH_JOB_EXECUTION table - indicating the JobExecution to which this StepExecution belongs. There - may be only one StepExecution for a given - JobExecution for a given - Step name. - - - - START_TIME: Timestamp representing the time the execution was - started. - - - - END_TIME: Timestamp representing the time the execution was - finished, regardless of success or failure. An empty value in this - column even though the job is not currently running indicates that - there has been some type of error and the framework was unable to - perform a last save before failing. - - - - STATUS: Character string representing the status of the - execution. This may be COMPLETED, STARTED, etc. The object - representation of this column is the - BatchStatus enumeration. - - - - COMMIT_COUNT: The number of times in which the step has - committed a transaction during this execution. - - - - READ_COUNT: The number of items read during this - execution. - - - - FILTER_COUNT: The number of items filtered out of this - execution. - - - - WRITE_COUNT: The number of items written and committed during - this execution. - - - - READ_SKIP_COUNT: The number of items skipped on read during this - execution. - - - - WRITE_SKIP_COUNT: The number of items skipped on write during - this execution. - - - - PROCESS_SKIP_COUNT: The number of items skipped during - processing during this execution. - - - - ROLLBACK_COUNT: The number of rollbacks during this execution. - Note that this count includes each time rollback occurs, including - rollbacks for retry and those in the skip recovery procedure. - - - - EXIT_CODE: Character string representing the exit code of the - execution. In the case of a command line job, this may be converted - into a number. - - - - EXIT_MESSAGE: Character string representing a more detailed - description of how the job exited. In the case of failure, this might - include as much of the stack trace as is possible. - - - - LAST_UPDATED: Timestamp representing the last time this - execution was persisted. - - -
        - -
        - BATCH_JOB_EXECUTION_CONTEXT - - The BATCH_JOB_EXECUTION_CONTEXT table holds all information relevant - to an Job's - ExecutionContext. There is exactly one - Job ExecutionContext per - JobExecution, and it contains all of the job-level - data that is needed for a particular job execution. This data typically - represents the state that must be retrieved after a failure so that a - JobInstance can 'start from where it left - off'. - - CREATE TABLE BATCH_JOB_EXECUTION_CONTEXT ( - JOB_EXECUTION_ID BIGINT PRIMARY KEY, - SHORT_CONTEXT VARCHAR(2500) NOT NULL, - SERIALIZED_CONTEXT CLOB, - constraint JOB_EXEC_CTX_FK foreign key (JOB_EXECUTION_ID) - references BATCH_JOB_EXECUTION(JOB_EXECUTION_ID) -) ; - - Below are descriptions for each column: - - - - JOB_EXECUTION_ID: Foreign key representing the - JobExecution to which the context belongs. - There may be more than one row associated to a given execution. - - - - SHORT_CONTEXT: A string version of the - SERIALIZED_CONTEXT. - - - - SERIALIZED_CONTEXT: The entire context, serialized. - - -
        - -
        - BATCH_STEP_EXECUTION_CONTEXT - - The BATCH_STEP_EXECUTION_CONTEXT table holds all information - relevant to an Step's - ExecutionContext. There is exactly one - ExecutionContext per - StepExecution, and it contains all of the data that - needs to persisted for a particular step execution. This data typically - represents the state that must be retrieved after a failure so that a - JobInstance can 'start from where it left - off'. - - CREATE TABLE BATCH_STEP_EXECUTION_CONTEXT ( - STEP_EXECUTION_ID BIGINT PRIMARY KEY, - SHORT_CONTEXT VARCHAR(2500) NOT NULL, - SERIALIZED_CONTEXT CLOB, - constraint STEP_EXEC_CTX_FK foreign key (STEP_EXECUTION_ID) - references BATCH_STEP_EXECUTION(STEP_EXECUTION_ID) -) ; - - Below are descriptions for each column: - - - - STEP_EXECUTION_ID: Foreign key representing the - StepExecution to which the context belongs. - There may be more than one row associated to a given execution. - - - - SHORT_CONTEXT: A string version of the - SERIALIZED_CONTEXT. - - - - SERIALIZED_CONTEXT: The entire context, serialized. - - -
        - -
        - Archiving - - Because there are entries in multiple tables every time a batch job - is run, it is common to create an archive strategy for the meta-data - tables. The tables themselves are designed to show a record of what - happened in the past, and generally won't affect the run of any job, with - a couple of notable exceptions pertaining to restart: - - - - The framework will use the meta-data tables to determine if a - particular JobInstance has been run before. If it has been run, and - the job is not restartable, then an exception will be thrown. - - - - If an entry for a JobInstance is removed without having - completed successfully, the framework will think that the job is new, - rather than a restart. - - - - If a job is restarted, the framework will use any data that has - been persisted to the ExecutionContext to restore the Job's state. - Therefore, removing any entries from this table for jobs that haven't - completed successfully will prevent them from starting at the correct - point if run again. - - -
        - -
        - International and Multi-byte Characters - - If you are using multi-byte character sets (e.g. Chines or Cyrillic) - in your business processing, then those characters might need to be - persisted in the Spring Batch schema. Many users find that - simply changing the schema to double the length of the VARCHAR - columns is enough. Others prefer to configure the JobRepository with max-varchar-length half the value of the VARCHAR column length is enough. Some users have also reported that - they use NVARCHAR in place of VARCHAR - in their schema definitions. The best result will depend on the database - platform and the way the database server has been configured locally. -
        - -
        - Recommendations for Indexing Meta Data Tables - - Spring Batch provides DDL samples for the meta-data tables in the - Core jar file for several common database platforms. Index declarations - are not included in that DDL because there are too many variations in how - users may want to index depending on their precise platform, local - conventions and also the business requirements of how the jobs will be - operated. The table below provides some indication as to which columns are - going to be used in a WHERE clause by the Dao implementations provided by - Spring Batch, and how frequently they might be used, so that individual - projects can make up their own minds about indexing. - - - Where clauses in SQL statements (excluding primary keys) and - their approximate frequency of use. - - - - - Default Table Name - - Where Clause - - Frequency - - - - BATCH_JOB_INSTANCE - - JOB_NAME = ? and JOB_KEY = ? - - Every time a job is launched - - - - BATCH_JOB_EXECUTION - - JOB_INSTANCE_ID = ? - - Every time a job is restarted - - - - BATCH_EXECUTION_CONTEXT - - EXECUTION_ID = ? and KEY_NAME = ? - - On commit interval, a.k.a. chunk - - - - BATCH_STEP_EXECUTION - - VERSION = ? - - On commit interval, a.k.a. chunk (and at start and end of - step) - - - - BATCH_STEP_EXECUTION - - STEP_NAME = ? and JOB_EXECUTION_ID = ? - - Before each step execution - - - -
        -
        -
        diff --git a/src/site/docbook/reference/spring-batch-intro.xml b/src/site/docbook/reference/spring-batch-intro.xml deleted file mode 100644 index 0d940acdb7..0000000000 --- a/src/site/docbook/reference/spring-batch-intro.xml +++ /dev/null @@ -1,205 +0,0 @@ - - - - Spring Batch Introduction - - Many applications within the enterprise domain require bulk processing - to perform business operations in mission critical environments. These - business operations include automated, complex processing of large volumes - of information that is most efficiently processed without user interaction. - These operations typically include time based events (e.g. month-end - calculations, notices or correspondence), periodic application of complex - business rules processed repetitively across very large data sets (e.g. - Insurance benefit determination or rate adjustments), or the integration of - information that is received from internal and external systems that - typically requires formatting, validation and processing in a transactional - manner into the system of record. Batch processing is used to process - billions of transactions every day for enterprises. - - Spring Batch is a lightweight, comprehensive batch framework designed - to enable the development of robust batch applications vital for the daily - operations of enterprise systems. Spring Batch builds upon the productivity, - POJO-based development approach, and general ease of use capabilities people - have come to know from the Spring Framework, while making it easy for - developers to access and leverage more advance enterprise services when - necessary. Spring Batch is not a scheduling framework. There are many good - enterprise schedulers available in both the commercial and open source - spaces such as Quartz, Tivoli, Control-M, etc. It is intended to work in - conjunction with a scheduler, not replace a scheduler. - - Spring Batch provides reusable functions that are essential in - processing large volumes of records, including logging/tracing, transaction - management, job processing statistics, job restart, skip, and resource - management. It also provides more advance technical services and features - that will enable extremely high-volume and high performance batch jobs - though optimization and partitioning techniques. Simple as well as complex, - high-volume batch jobs can leverage the framework in a highly scalable - manner to process significant volumes of information. - -
        - Background - - While open source software projects and associated communities have - focused greater attention on web-based and SOA messaging-based - architecture frameworks, there has been a notable lack of focus on - reusable architecture frameworks to accommodate Java-based batch - processing needs, despite continued needs to handle such processing within - enterprise IT environments. The lack of a standard, reusable batch - architecture has resulted in the proliferation of many one-off, in-house - solutions developed within client enterprise IT functions. - - SpringSource and Accenture have collaborated to change this. - Accenture's hands-on industry and technical experience in implementing - batch architectures, SpringSource's depth of technical experience, and - Spring's proven programming model together mark a natural and powerful - partnership to create high-quality, market relevant software aimed at - filling an important gap in enterprise Java. Both companies are also - currently working with a number of clients solving similar problems - developing Spring-based batch architecture solutions. This has provided - some useful additional detail and real-life constraints helping to ensure - the solution can be applied to the real-world problems posed by clients. - For these reasons and many more, SpringSource and Accenture have teamed to - collaborate on the development of Spring Batch. - - Accenture has contributed previously proprietary batch processing - architecture frameworks, based upon decades worth of experience in - building batch architectures with the last several generations of - platforms, (i.e., COBOL/Mainframe, C++/Unix, and now Java/anywhere) to the - Spring Batch project along with committer resources to drive support, - enhancements, and the future roadmap. - - The collaborative effort between Accenture and SpringSource aims to - promote the standardization of software processing approaches, frameworks, - and tools that can be consistently leveraged by enterprise users when - creating batch applications. Companies and government agencies desiring to - deliver standard, proven solutions to their enterprise IT environments - will benefit from Spring Batch. -
        - -
        - Usage Scenarios - - A typical batch program generally reads a large number of records - from a database, file, or queue, processes the data in some fashion, and - then writes back data in a modified form. Spring Batch automates this - basic batch iteration, providing the capability to process similar - transactions as a set, typically in an offline environment without any - user interaction. Batch jobs are part of most IT projects and Spring Batch - is the only open source framework that provides a robust, enterprise-scale - solution. - - Business Scenarios - - Commit batch process periodically - - - - Concurrent batch processing: parallel processing of a - job - - - - Staged, enterprise message-driven processing - - - - Massively parallel batch processing - - - - Manual or scheduled restart after failure - - - - Sequential processing of dependent steps (with extensions to - workflow-driven batches) - - - - Partial processing: skip records (e.g. on rollback) - - - - Whole-batch transaction: for cases with a small batch size or - existing stored procedures/scripts - - - - Technical Objectives - - Batch developers use the Spring programming model: concentrate - on business logic; let the framework take care of - infrastructure. - - - - Clear separation of concerns between the infrastructure, the - batch execution environment, and the batch application. - - - - Provide common, core execution services as interfaces that all - projects can implement. - - - - Provide simple and default implementations of the core - execution interfaces that can be used ‘out of the box’. - - - - Easy to configure, customize, and extend services, by - leveraging the spring framework in all layers. - - - - All existing core services should be easy to replace or - extend, without any impact to the infrastructure layer. - - - - Provide a simple deployment model, with the architecture JARs - completely separate from the application, built using Maven. - - -
        - -
        - Spring Batch Architecture - - - - Spring Batch is designed with extensibility and a diverse group of - end users in mind. The figure below shows a sketch of the layered - architecture that supports the extensibility and ease of use for end-user - developers. - - - - - - - - - Figure 1.1: Spring Batch Layered - Architecture - - - This layered architecture highlights three major high level - components: Application, Core, and Infrastructure. The application - contains all batch jobs and custom code written by developers using Spring - Batch. The Batch Core contains the core runtime classes necessary to - launch and control a batch job. It includes things such as a - JobLauncher, Job, and - Step implementations. Both Application and Core are - built on top of a common infrastructure. This infrastructure contains - common readers and writers, and services such as the - RetryTemplate, which are used both by application - developers(ItemReader and - ItemWriter) and the core framework itself. - (retry) -
        -
        diff --git a/src/site/docbook/reference/step.xml b/src/site/docbook/reference/step.xml deleted file mode 100644 index 537dafc998..0000000000 --- a/src/site/docbook/reference/step.xml +++ /dev/null @@ -1,1731 +0,0 @@ - - - - Configuring a Step - - As discussed in , a - Step is a domain object that encapsulates an - independent, sequential phase of a batch job and contains all of the - information necessary to define and control the actual batch processing. - This is a necessarily vague description because the contents of any given - Step are at the discretion of the developer writing a - Job. A Step can be as simple or complex as the - developer desires. A simple Step might load data from - a file into the database, requiring little or no code. (depending upon the - implementations used) A more complex Step may have - complicated business rules that are applied as part of the - processing. - - - - - - - - - - - -
        - Chunk-Oriented Processing - - Spring Batch uses a 'Chunk Oriented' processing style within its - most common implementation. Chunk oriented processing refers to reading - the data one at a time, and creating 'chunks' that will be written out, - within a transaction boundary. One item is read in from an - ItemReader, handed to an - ItemProcessor, and aggregated. Once the number of - items read equals the commit interval, the entire chunk is written out via - the ItemWriter, and then the transaction is committed. - - - - - - - - - - - - Below is a code representation of the same concepts shown - above: - - List items = new Arraylist(); -for(int i = 0; i < commitInterval; i++){ - Object item = itemReader.read() - Object processedItem = itemProcessor.process(item); - items.add(processedItem); -} -itemWriter.write(items); - -
        - Configuring a Step - - Despite the relatively short list of required dependencies for a - Step, it is an extremely complex class that can - potentially contain many collaborators. In order to ease configuration, - the Spring Batch namespace can be used: - - <job id="sampleJob" job-repository="jobRepository"> - <step id="step1"> - <tasklet transaction-manager="transactionManager"> - <chunk reader="itemReader" writer="itemWriter" commit-interval="10"/> - </tasklet> - </step> -</job> - - The configuration above represents the only required dependencies - to create a item-oriented step: - - reader - The ItemReader that provides - items for processing. - - - - writer - The ItemWriter that - processes the items provided by the - ItemReader. - - - - transaction-manager - Spring's - PlatformTransactionManager that will be - used to begin and commit transactions during processing. - - - - job-repository - The JobRepository - that will be used to periodically store the - StepExecution and - ExecutionContext during processing (just - before committing). For an in-line <step/> (one defined - within a <job/>) it is an attribute on the <job/> - element; for a standalone step, it is defined as an attribute of - the <tasklet/>. - - - - commit-interval - The number of items that will be processed - before the transaction is committed. - - - - It should be noted that, job-repository defaults to - "jobRepository" and transaction-manager defaults to "transactionManger". - Furthermore, the ItemProcessor is optional, not - required, since the item could be directly passed from the reader to the - writer. -
        - -
        - Inheriting from a Parent Step - - If a group of Steps share similar - configurations, then it may be helpful to define a "parent" - Step from which the concrete - Steps may inherit properties. Similar to class - inheritance in Java, the "child" Step will - combine its elements and attributes with the parent's. The child will - also override any of the parent's Steps. - - In the following example, the Step - "concreteStep1" will inherit from "parentStep". It will be instantiated - with 'itemReader', 'itemProcessor', 'itemWriter', startLimit=5, and - allowStartIfComplete=true. Additionally, the commitInterval will be '5' - since it is overridden by the "concreteStep1": - - <step id="parentStep"> - <tasklet allow-start-if-complete="true"> - <chunk reader="itemReader" writer="itemWriter" commit-interval="10"/> - </tasklet> -</step> - -<step id="concreteStep1" parent="parentStep"> - <tasklet start-limit="5"> - <chunk processor="itemProcessor" commit-interval="5"/> - </tasklet> -</step> - - The id attribute is still required on the step within the job - element. This is for two reasons: - - The id will be used as the step name when persisting the - StepExecution. If the same standalone step is referenced in more - than one step in the job, an error will occur. - - - - When creating job flows, as described later in this chapter, - the next attribute should be referring to the step in the flow, - not the standalone step. - - - -
        - Abstract Step - - Sometimes it may be necessary to define a parent - Step that is not a complete - Step configuration. If, for instance, the - reader, writer, and tasklet attributes are left off of a - Step configuration, then initialization will - fail. If a parent must be defined without these properties, then the - "abstract" attribute should be used. An "abstract" - Step will not be instantiated; it is used only - for extending. - - In the following example, the Step - "abstractParentStep" would not instantiate if it were not declared to - be abstract. The Step "concreteStep2" will have - 'itemReader', 'itemWriter', and commitInterval=10. - - <step id="abstractParentStep" abstract="true"> - <tasklet> - <chunk commit-interval="10"/> - </tasklet> -</step> - -<step id="concreteStep2" parent="abstractParentStep"> - <tasklet> - <chunk reader="itemReader" writer="itemWriter"/> - </tasklet> -</step> -
        - -
        - Merging Lists - - Some of the configurable elements on - Steps are lists; the <listeners/> - element, for instance. If both the parent and child - Steps declare a <listeners/> element, - then the child's list will override the parent's. In order to allow a - child to add additional listeners to the list defined by the parent, - every list element has a "merge" attribute. If the element specifies - that merge="true", then the child's list will be combined with the - parent's instead of overriding it. - - In the following example, the Step - "concreteStep3" will be created will two listeners: - listenerOne and - listenerTwo: - - <step id="listenersParentStep" abstract="true"> - <listeners> - <listener ref="listenerOne"/> - <listeners> -</step> - -<step id="concreteStep3" parent="listenersParentStep"> - <tasklet> - <chunk reader="itemReader" writer="itemWriter" commit-interval="5"/> - </tasklet> - <listeners merge="true"> - <listener ref="listenerTwo"/> - <listeners> -</step> -
        -
        - -
        - The Commit Interval - - As mentioned above, a step reads in and writes out items, - periodically committing using the supplied - PlatformTransactionManager. With a - commit-interval of 1, it will commit after writing each individual item. - This is less than ideal in many situations, since beginning and - committing a transaction is expensive. Ideally, it is preferable to - process as many items as possible in each transaction, which is - completely dependent upon the type of data being processed and the - resources with which the step is interacting. For this reason, the - number of items that are processed within a commit can be - configured. - - <job id="sampleJob"> - <step id="step1"> - <tasklet> - <chunk reader="itemReader" writer="itemWriter" commit-interval="10"/> - </tasklet> - </step> -</job> - - In the example above, 10 items will be processed within each - transaction. At the beginning of processing a transaction is begun, and - each time read is called on the - ItemReader, a counter is incremented. When it - reaches 10, the list of aggregated items is passed to the - ItemWriter, and the transaction will be - committed. -
        - -
        - Configuring a Step for Restart - - In , restarting a - Job was discussed. Restart has numerous impacts - on steps, and as such may require some specific configuration. - -
        - Setting a StartLimit - - There are many scenarios where you may want to control the - number of times a Step may be started. For - example, a particular Step might need to be - configured so that it only runs once because it invalidates some - resource that must be fixed manually before it can be run again. This - is configurable on the step level, since different steps may have - different requirements. A Step that may only be - executed once can exist as part of the same Job - as a Step that can be run infinitely. Below is - an example start limit configuration: - - <step id="step1"> - <tasklet start-limit="1"> - <chunk reader="itemReader" writer="itemWriter" commit-interval="10"/> - </tasklet> -</step> - - The simple step above can be run only once. Attempting to run it - again will cause an exception to be thrown. It should be noted that - the default value for the start-limit is - Integer.MAX_VALUE. -
        - -
        - Restarting a completed step - - In the case of a restartable job, there may be one or more steps - that should always be run, regardless of whether or not they were - successful the first time. An example might be a validation step, or a - Step that cleans up resources before - processing. During normal processing of a restarted job, any step with - a status of 'COMPLETED', meaning it has already been completed - successfully, will be skipped. Setting allow-start-if-complete to - "true" overrides this so that the step will always run: - - <step id="step1"> - <tasklet allow-start-if-complete="true"> - <chunk reader="itemReader" writer="itemWriter" commit-interval="10"/> - </tasklet> -</step> -
        - -
        - Step Restart Configuration Example - - <job id="footballJob" restartable="true"> - <step id="playerload" next="gameLoad"> - <tasklet> - <chunk reader="playerFileItemReader" writer="playerWriter" - commit-interval="10" /> - </tasklet> - </step> - <step id="gameLoad" next="playerSummarization"> - <tasklet allow-start-if-complete="true"> - <chunk reader="gameFileItemReader" writer="gameWriter" - commit-interval="10"/> - </tasklet> - </step> - <step id="playerSummarization"> - <tasklet start-limit="3"> - <chunk reader="playerSummarizationSource" writer="summaryWriter" - commit-interval="10"/> - </tasklet> - </step> -</job> - - The above example configuration is for a job that loads in - information about football games and summarizes them. It contains - three steps: playerLoad, gameLoad, and playerSummarization. The - playerLoad Step loads player information from a - flat file, while the gameLoad Step does the - same for games. The final Step, - playerSummarization, then summarizes the statistics for each player - based upon the provided games. It is assumed that the file loaded by - 'playerLoad' must be loaded only once, but that 'gameLoad' will load - any games found within a particular directory, deleting them after - they have been successfully loaded into the database. As a result, the - playerLoad Step contains no additional - configuration. It can be started almost limitlessly, and if complete - will be skipped. The 'gameLoad' Step, however, - needs to be run every time in case extra files have been dropped since - it last executed. It has 'allow-start-if-complete' set to 'true' in - order to always be started. (It is assumed that the database tables - games are loaded into has a process indicator on it, to ensure new - games can be properly found by the summarization step). The - summarization Step, which is the most important - in the Job, is configured to have a start limit - of 3. This is useful because if the step continually fails, a new exit - code will be returned to the operators that control job execution, and - it won't be allowed to start again until manual intervention has taken - place. - - - This job is purely for example purposes and is not the same as - the footballJob found in the samples project. - - - Run 1: - - - - playerLoad is executed and completes successfully, adding - 400 players to the 'PLAYERS' table. - - - - gameLoad is executed and processes 11 files worth of game - data, loading their contents into the 'GAMES' table. - - - - playerSummarization begins processing and fails after 5 - minutes. - - - - Run 2: - - - - playerLoad is not run, since it has already completed - successfully, and allow-start-if-complete is 'false' (the - default). - - - - gameLoad is executed again and processes another 2 files, - loading their contents into the 'GAMES' table as well (with a - process indicator indicating they have yet to be processed) - - - - playerSummarization begins processing of all remaining game - data (filtering using the process indicator) and fails again after - 30 minutes. - - - - Run 3: - - - - playerLoad is not run, since it has already completed - successfully, and allow-start-if-complete is 'false' (the - default). - - - - gameLoad is executed again and processes another 2 files, - loading their contents into the 'GAMES' table as well (with a - process indicator indicating they have yet to be processed) - - - - playerSummarization is not start, and the job is immediately - killed, since this is the third execution of playerSummarization, - and its limit is only 2. The limit must either be raised, or the - Job must be executed as a new - JobInstance. - - -
        -
        - -
        - Configuring Skip Logic - - There are many scenarios where errors encountered while processing - should not result in Step failure, but should be - skipped instead. This is usually a decision that must be made by someone - who understands the data itself and what meaning it has. Financial data, - for example, may not be skippable because it results in money being - transferred, which needs to be completely accurate. Loading a list of - vendors, on the other hand, might allow for skips. If a vendor is not - loaded because it was formatted incorrectly or was missing necessary - information, then there probably won't be issues. Usually these bad - records are logged as well, which will be covered later when discussing - listeners.<step id="step1"> - <tasklet> - <chunk reader="flatFileItemReader" writer="itemWriter" - commit-interval="10" skip-limit="10"> - <skippable-exception-classes> - <include class="org.springframework.batch.item.file.FlatFileParseException"/> - </skippable-exception-classes> - </chunk> - </tasklet> -</step> - - In this example, a FlatFileItemReader is - used, and if at any point a - FlatFileParseException is thrown, it will be - skipped and counted against the total skip limit of 10. Separate counts - are made of skips on read, process and write inside the step execution, - and the limit applies across all. Once the skip limit is reached, the - next exception found will cause the step to fail. - - One problem with the example above is that any other exception - besides a FlatFileParseException will cause the - Job to fail. In certain scenarios this may be the - correct behavior. However, in other scenarios it may be easier to - identify which exceptions should cause failure and skip everything - else:<step id="step1"> - <tasklet> - <chunk reader="flatFileItemReader" writer="itemWriter" - commit-interval="10" skip-limit="10"> - <skippable-exception-classes> - <include class="java.lang.Exception"/> - <exclude class="java.io.FileNotFoundException"/> - </skippable-exception-classes> - </chunk> - </tasklet> -</step> - - By 'including' java.lang.Exception as a - skippable exception class, the configuration indicates that all - Exceptions are skippable. However, by 'excluding' - java.io.FileNotFoundException, the configuration - refines the list of skippable exception classes to be all - Exceptions except - FileNotFoundException. Any excluded exception - classes will be fatal if encountered (i.e. not skipped). - - For any exception encountered, the skippability will be determined - by the nearest superclass in the class hierarchy. Any unclassifed - exception will be treated as 'fatal'. The order of the - <include/> and <exclude/> elements - does not matter. -
        - -
        - Configuring Retry Logic - - In most cases you want an exception to cause either a skip or - Step failure. However, not all exceptions are - deterministic. If a FlatFileParseException is - encountered while reading, it will always be thrown for that record; - resetting the ItemReader will not help. However, - for other exceptions, such as a - DeadlockLoserDataAccessException, which indicates - that the current process has attempted to update a record that another - process holds a lock on, waiting and trying again might result in - success. In this case, retry should be configured: - - <step id="step1"> - <tasklet> - <chunk reader="itemReader" writer="itemWriter" - commit-interval="2" retry-limit="3"> - <retryable-exception-classes> - <include class="org.springframework.dao.DeadlockLoserDataAccessException"/> - </retryable-exception-classes> - </chunk> - </tasklet> -</step> - - The Step allows a limit for the number of - times an individual item can be retried, and a list of exceptions that - are 'retryable'. More details on how retry works can be found in . -
        - -
        - Controlling Rollback - - By default, regardless of retry or skip, any exceptions thrown - from the ItemWriter will cause the transaction - controlled by the Step to rollback. If skip is - configured as described above, exceptions thrown from the - ItemReader will not cause a rollback. However, - there are many scenarios in which exceptions thrown from the - ItemWriter should not cause a rollback because no - action has taken place to invalidate the transaction. For this reason, - the Step can be configured with a list of - exceptions that should not cause rollback. - - <step id="step1"> - <tasklet> - <chunk reader="itemReader" writer="itemWriter" commit-interval="2"/> - <no-rollback-exception-classes> - <include class="org.springframework.batch.item.validator.ValidationException"/> - </no-rollback-exception-classes> - </tasklet> -</step> - -
        - Transactional Readers - - The basic contract of the ItemReader is - that it is forward only. The step buffers reader input, so that in the - case of a rollback the items don't need to be re-read from the reader. - However, there are certain scenarios in which the reader is built on - top of a transactional resource, such as a JMS queue. In this case, - since the queue is tied to the transaction that is rolled back, the - messages that have been pulled from the queue will be put back on. For - this reason, the step can be configured to not buffer the - items: - - <step id="step1"> - <tasklet> - <chunk reader="itemReader" writer="itemWriter" commit-interval="2" - is-reader-transactional-queue="true"/> - </tasklet> -</step> -
        -
        - -
        - Transaction Attributes - - Transaction attributes can be used to control the isolation, - propagation, and timeout settings. More information on setting - transaction attributes can be found in the spring core - documentation. - - <step id="step1"> - <tasklet> - <chunk reader="itemReader" writer="itemWriter" commit-interval="2"/> - <transaction-attributes isolation="DEFAULT" - propagation="REQUIRED" - timeout="30"/> - </tasklet> -</step> -
        - -
        - Registering ItemStreams with the Step - - The step has to take care of ItemStream - callbacks at the necessary points in its lifecycle. (for more - information on the ItemStream interface, please - refer to ) This is vital if a step fails, - and might need to be restarted, because the - ItemStream interface is where the step gets the - information it needs about persistent state between executions. - - If the ItemReader, - ItemProcessor, or - ItemWriter itself implements the - ItemStream interface, then these will be - registered automatically. Any other streams need to be registered - separately. This is often the case where there are indirect dependencies - such as delegates being injected into the reader and writer. A stream - can be registered on the Step through the - 'streams' element, as illustrated below: - - <step id="step1"> - <tasklet> - <chunk reader="itemReader" writer="compositeWriter" commit-interval="2"> - <streams> - <stream ref="fileItemWriter1"/> - <stream ref="fileItemWriter2"/> - </streams> - </chunk> - </tasklet> -</step> - -<beans:bean id="compositeWriter" - class="org.springframework.batch.item.support.CompositeItemWriter"> - <beans:property name="delegates"> - <beans:list> - <beans:ref bean="fileItemWriter1" /> - <beans:ref bean="fileItemWriter2" /> - </beans:list> - </beans:property> -</beans:bean> - - In the example above, the - CompositeItemWriter is not an - ItemStream, but both of its delegates are. - Therefore, both delegate writers must be explicitly registered as - streams in order for the framework to handle them correctly. The - ItemReader does not need to be explicitly - registered as a stream because it is a direct property of the - Step. The step will now be restartable and the - state of the reader and writer will be correctly persisted in the event - of a failure. -
        - -
        - Intercepting Step Execution - - Just as with the Job, there are many events - during the execution of a Step where a user may - need to perform some functionality. For example, in order to write out - to a flat file that requires a footer, the - ItemWriter needs to be notified when the - Step has been completed, so that the footer can - written. This can be accomplished with one of many - Step scoped listeners. - - Any class that implements one of the extensions - of StepListener (but not that interface - itself since it is empty) can be applied to a step via the - listeners element. The listeners element is valid inside a - step, tasklet or chunk declaration. It is recommended that you - declare the listeners at the level which its function applies, - or if it is multi-featured - (e.g. StepExecutionListener - and ItemReadListener) then declare it at - the most granular level that it applies (chunk in the example - given). - - <step id="step1"> - <tasklet> - <chunk reader="reader" writer="writer" commit-interval="10"/> - <listeners> - <listener ref="chunkListener"/> - </listeners> - </tasklet> -</step> - - An ItemReader, - ItemWriter or - ItemProcessor that itself implements one of the - StepListener interfaces will be registered - automatically with the Step if using the - namespace <step> element, or one of the the - *StepFactoryBean factories. This only applies to - components directly injected into the Step: if - the listener is nested inside another component, it needs to be - explicitly registered (as described above). - - In addition to the StepListener interfaces, - annotations are provided to address the same concerns. Plain old Java - objects can have methods with these annotations that are then converted - into the corresponding StepListener type. It is - also common to annotate custom implementations of chunk components like - ItemReader or ItemWriter - or Tasklet. The annotations are analysed by the - XML parser for the <listener/> elements, so all you - need to do is use the XML namespace to register the listeners with a - step. - -
        - StepExecutionListener - - StepExecutionListener represents the most - generic listener for Step execution. It allows - for notification before a Step is started and - after it has ends, whether it ended normally or failed: - - public interface StepExecutionListener extends StepListener { - - void beforeStep(StepExecution stepExecution); - - ExitStatus afterStep(StepExecution stepExecution); - -} - - ExitStatus is the return type of - afterStep in order to allow listeners the - chance to modify the exit code that is returned upon completion of a - Step. - - The annotations corresponding to this interface are: - - - - @BeforeStep - - - - @AfterStep - - -
        - -
        - ChunkListener - - A chunk is defined as the items processed within the scope of a - transaction. Committing a transaction, at each commit interval, - commits a 'chunk'. A ChunkListener can be - useful to perform logic before a chunk begins processing or after a - chunk has completed successfully: - - public interface ChunkListener extends StepListener { - - void beforeChunk(); - - void afterChunk(); - -} - - The beforeChunk method is called after - the transaction is started, but before read - is called on the ItemReader. Conversely, - afterChunk is called after the chunk has been - committed (and not at all if there is a rollback). - - The annotations corresponding to this interface are: - - - - @BeforeChunk - - - - @AfterChunk - - - - A ChunkListener can be applied - when there is no chunk declaration: it is - the TaskletStep that is responsible for - calling the ChunkListener so it applies - to a non-item-oriented tasklet as well (called before and - after the tasklet). - -
        - -
        - ItemReadListener - - When discussing skip logic above, it was mentioned that it may - be beneficial to log the skipped records, so that they can be deal - with later. In the case of read errors, this can be done with an - ItemReaderListener:public interface ItemReadListener<T> extends StepListener { - - void beforeRead(); - - void afterRead(T item); - - void onReadError(Exception ex); - -} - - The beforeRead method will be called - before each call to read on the - ItemReader. The - afterRead method will be called after each - successful call to read, and will be passed - the item that was read. If there was an error while reading, the - onReadError method will be called. The - exception encountered will be provided so that it can be - logged. - - The annotations corresponding to this interface are: - - - - @BeforeRead - - - - @AfterRead - - - - @OnReadError - - -
        - -
        - ItemProcessListener - - Just as with the ItemReadListener, the - processing of an item can be 'listened' to: - - public interface ItemProcessListener<T, S> extends StepListener { - - void beforeProcess(T item); - - void afterProcess(T item, S result); - - void onProcessError(T item, Exception e); - -} - - The beforeProcess method will be called - before process on the - ItemProcessor, and is handed the item that will - be processed. The afterProcess method will be - called after the item has been successfully processed. If there was an - error while processing, the onProcessError - method will be called. The exception encountered and the item that was - attempted to be processed will be provided, so that they can be - logged. - - The annotations corresponding to this interface are: - - - - @BeforeProcess - - - - @AfterProcess - - - - @OnProcessError - - -
        - -
        - ItemWriteListener - - The writing of an item can be 'listened' to with the - ItemWriteListener: - - public interface ItemWriteListener<S> extends StepListener { - - void beforeWrite(List<? extends S> items); - - void afterWrite(List<? extends S> items); - - void onWriteError(Exception exception, List<? extends S> items); - -} - - The beforeWrite method will be called - before write on the - ItemWriter, and is handed the item that will be - written. The afterWrite method will be called - after the item has been successfully written. If there was an error - while writing, the onWriteError method will - be called. The exception encountered and the item that was attempted - to be written will be provided, so that they can be logged. - - The annotations corresponding to this interface are: - - - - @BeforeWrite - - - - @AfterWrite - - - - @OnWriteError - - -
        - -
        - SkipListener - - ItemReadListener, - ItemProcessListener, and - ItemWriteListner all provide mechanisms for - being notified of errors, but none will inform you that a record has - actually been skipped. onWriteError, for - example, will be called even if an item is retried and successful. For - this reason, there is a separate interface for tracking skipped - items: - - - public interface SkipListener<T,S> extends StepListener { - - void onSkipInRead(Throwable t); - - void onSkipInProcess(T item, Throwable t); - - void onSkipInWrite(S item, Throwable t); - -} - - onSkipInRead will be called whenever an - item is skipped while reading. It should be noted that rollbacks may - cause the same item to be registered as skipped more than once. - onSkipInWrite will be called when an item is - skipped while writing. Because the item has been read successfully - (and not skipped), it is also provided the item itself as an - argument. - - The annotations corresponding to this interface are: - - - - @OnSkipInRead - - - - @OnSkipInWrite - - - - @OnSkipInProcess - - - -
        - SkipListeners and Transactions - - One of the most common use cases for a - SkipListener is to log out a skipped item, so - that another batch process or even human process can be used to - evaluate and fix the issue leading to the skip. Because there are - many cases in which the original transaction may be rolled back, - Spring Batch makes two guarantees: - - - - The appropriate skip method (depending on when the error - happened) will only be called once per item. - - - - The SkipListener will always be - called just before the transaction is committed. This is to - ensure that any transactional resources call by the listener are - not rolled back by a failure within the - ItemWriter. - - -
        -
        -
        -
        - -
        - TaskletStep - - Chunk-oriented processing is not the only way to process in a - Step. What if a Step must - consist as a simple stored procedure call? You could implement the call as - an ItemReader and return null after the procedure - finishes, but it is a bit unnatural since there would need to be a no-op - ItemWriter. Spring Batch provides the - TaskletStep for this scenario. - - The Tasklet is a simple interface that has - one method, execute, which will be a called - repeatedly by the TaskletStep until it either - returns RepeatStatus.FINISHED or throws an exception to - signal a failure. Each call to the Tasklet is - wrapped in a transaction. Tasklet implementors - might call a stored procedure, a script, or a simple SQL update statement. - To create a TaskletStep, the 'ref' attribute of the - <tasklet/> element should reference a bean defining a - Tasklet object; no <chunk/> element should be - used within the <tasklet/>: - - <step id="step1"> - <tasklet ref="myTasklet"/> -</step> - - - TaskletStep will automatically register the - tasklet as StepListener if it implements this - interface - - -
        - TaskletAdapter - - As with other adapters for the ItemReader - and ItemWriter interfaces, the - Tasklet interface contains an implementation that - allows for adapting itself to any pre-existing class: - TaskletAdapter. An example where this may be - useful is an existing DAO that is used to update a flag on a set of - records. The TaskletAdapter can be used to call - this class without having to write an adapter for the - Tasklet interface: - - <bean id="myTasklet" class="org.springframework.batch.core.step.tasklet.MethodInvokingTaskletAdapter"> - <property name="targetObject"> - <bean class="org.mycompany.FooDao"/> - </property> - <property name="targetMethod" value="updateFoo" /> -</bean> -
        - -
        - Example Tasklet Implementation - - Many batch jobs contain steps that must be done before the main - processing begins in order to set up various resources or after - processing has completed to cleanup those resources. In the case of a - job that works heavily with files, it is often necessary to delete - certain files locally after they have been uploaded successfully to - another location. The example below taken from the Spring Batch samples - project, is a Tasklet implementation with just - such a responsibility: - - public class FileDeletingTasklet implements Tasklet, InitializingBean { - - private Resource directory; - - public RepeatStatus execute(StepContribution contribution, - ChunkContext chunkContext) throws Exception { - File dir = directory.getFile(); - Assert.state(dir.isDirectory()); - - File[] files = dir.listFiles(); - for (int i = 0; i < files.length; i++) { - boolean deleted = files[i].delete(); - if (!deleted) { - throw new UnexpectedJobExecutionException("Could not delete file " + - files[i].getPath()); - } - } - return RepeatStatus.COMPLETED; - } - - public void setDirectoryResource(Resource directory) { - this.directory = directory; - } - - public void afterPropertiesSet() throws Exception { - Assert.notNull(directory, "directory must be set"); - } -} - - The above Tasklet implementation will - delete all files within a given directory. It should be noted that the - execute method will only be called once. All - that is left is to reference the Tasklet from the - Step: - - <job id="taskletJob"> - <step id="deleteFilesInDir"> - <tasklet ref="fileDeletingTasklet"/> - </step> -</job> - -<beans:bean id="fileDeletingTasklet" - class="org.springframework.batch.sample.tasklet.FileDeletingTasklet"> - <beans:property name="directoryResource"> - <beans:bean id="directory" - class="org.springframework.core.io.FileSystemResource"> - <beans:constructor-arg value="target/test-outputs/test-dir" /> - </beans:bean> - </beans:property> -</beans:bean> -
        -
        - -
        - Controlling Step Flow - - With the ability to group steps together within an owning job comes - the need to be able to control how the job 'flows' from one step to - another. The failure of a Step doesn't necessarily - mean that the Job should fail. Furthermore, there - may be more than one type of 'success' which determines which - Step should be executed next. Depending upon how a - group of Steps is configured, certain steps may not even be processed at - all. - -
        - Sequential Flow - - The simplest flow scenario is a job where all of the steps execute - sequentially: - - - - - - - - - - - - This can be achieved using the 'next' attribute of the step - element: - - <job id="job"> - <step id="stepA" parent="s1" next="stepB" /> - <step id="stepB" parent="s2" next="stepC"/> - <step id="stepC" parent="s3" /> -</job>In the scenario above, 'step A' will execute - first because it is the first Step listed. If - 'step A' completes normally, then 'step B' will execute, and so on. - However, if 'step A' fails, then the entire Job - will fail and 'step B' will not execute. - - - With the Spring Batch namespace, the first step listed in the - configuration will always be the first step - executed by the Job. The order of the other - step elements does not matter, but the first step must always appear - first in the xml. - -
        - -
        - Conditional Flow - - In the example above, there are only two possibilities: - - - - The Step is successful and the next - Step should be executed. - - - - The Step failed and thus the - Job should fail. - - - - In many cases, this may be sufficient. However, what about a - scenario in which the failure of a Step should - trigger a different Step, rather than causing - failure? - - - - - - - - - - In order to handle more complex scenarios, the - Spring Batch namespace allows transition elements to be defined within - the step element. One such transition is the "next" element. Like the - "next" attribute, the "next" element will tell the - Job which Step to execute - next. However, unlike the attribute, any number of "next" elements are - allowed on a given Step, and there is no default - behavior the case of failure. This means that if transition elements are - used, then all of the behavior for the Step's - transitions must be defined explicitly. Note also that a single step - cannot have both a "next" attribute and a transition element. - - The next element specifies a pattern to match and the step to - execute next: - - <job id="job"> - <step id="stepA" parent="s1"> - <next on="*" to="stepB" /> - <next on="FAILED" to="stepC" /> - </step> - <step id="stepB" parent="s2" next="stepC" /> - <step id="stepC" parent="s3" /> -</job> - - The "on" attribute of a transition element uses a simple - pattern-matching scheme to match the ExitStatus - that results from the execution of the Step. Only - two special characters are allowed in the pattern: - - - - "*" will zero or more characters - - - - "?" will match exactly one character - - - - For example, "c*t" will match "cat" and "count", while "c?t" will - match "cat" but not "count". - - While there is no limit to the number of transition elements on a - Step, if the Step's - execution results in an ExitStatus that is not - covered by an element, then the framework will throw an exception and - the Job will fail. The framework will - automatically order transitions from most specific to - least specific. This means that even if the elements were swapped for - "stepA" in the example above, an ExitStatus of - "FAILED" would still go to "stepC". - -
        - Batch Status vs. Exit Status - - When configuring a Job for conditional - flow, it is important to understand the difference between - BatchStatus and - ExitStatus. BatchStatus - is an enumeration that is a property of both - JobExecution and - StepExecution and is used by the framework to - record the status of a Job or - Step. It can be one of the following values: - COMPLETED, STARTING, STARTED, STOPPING, STOPPED, FAILED, ABANDONED or - UNKNOWN. Most of them are self explanatory: COMPLETED is the status - set when a step or job has completed successfully, FAILED is set when - it fails, and so on. The example above contains the following 'next' - element: - - <next on="FAILED" to="stepB" /> - - At first glance, it would appear that the 'on' attribute - references the BatchStatus of the - Step to which it belongs. However, it actually - references the ExitStatus of the - Step. As the name implies, - ExitStatus represents the status of a - Step after it finishes execution. More - specifically, the 'next' element above references the exit code of the - ExitStatus. To write it in English, it says: - "go to stepB if the exit code is FAILED". By default, the exit code is - always the same as the BatchStatus for the - Step, which is why the entry above works. However, what if the exit - code needs to be different? A good example comes from the skip sample - job within the samples project: - - <step id="step1" parent="s1"> - <end on="FAILED" /> - <next on="COMPLETED WITH SKIPS" to="errorPrint1" /> - <next on="*" to="step2" /> -</step> - - The above step has three possibilities: - - - - The Step failed, in which case the - job should fail. - - - - The Step completed - successfully. - - - - The Step completed successfully, but - with an exit code of 'COMPLETED WITH SKIPS'. In this case, a - different step should be run to handle the errors. - - - - The above configuration will work. However, something needs to - change the exit code based on the condition of the execution having - skipped records: - - public class SkipCheckingListener extends StepExecutionListenerSupport { - - public ExitStatus afterStep(StepExecution stepExecution) { - String exitCode = stepExecution.getExitStatus().getExitCode(); - if (!exitCode.equals(ExitStatus.FAILED.getExitCode()) && - stepExecution.getSkipCount() > 0) { - return new ExitStatus("COMPLETED WITH SKIPS"); - } - else { - return null; - } - } - -} - - The above code is a StepExecutionListener - that first checks to make sure the Step was - successful, and next if the skip count on the - StepExecution is higher than 0. If both - conditions are met, a new ExitStatus with an - exit code of "COMPLETED WITH SKIPS" is returned. -
        -
        - -
        - Configuring for Stop - - After the discussion of BatchStatus and - ExitStatus, one might wonder how the - BatchStatus and ExitStatus - are determined for the Job. While these statuses - are determined for the Step by the code that is - executed, the statuses for the Job will be - determined based on the configuration. - - So far, all of the job configurations discussed have had at least - one final Step with no transitions. For example, - after the following step executes, the Job will - end: - - <step id="stepC" parent="s3"/> - - If no transitions are defined for a Step, - then the Job's statuses will be defined as - follows: - - - - If the Step ends with - ExitStatus FAILED, then the - Job's BatchStatus and - ExitStatus will both be FAILED. - - - - Otherwise, the Job's - BatchStatus and - ExitStatus will both be COMPLETED. - - - - While this method of terminating a batch job is sufficient for - some batch jobs, such as a simple sequential step job, custom defined - job-stopping scenarios may be required. For this purpose, Spring Batch - provides three transition elements to stop a Job - (in addition to the "next" element - that we discussed previously). Each of these stopping elements will stop - a Job with a particular - BatchStatus. It is important to note that the - stop transition elements will have no effect on either the - BatchStatus or ExitStatus - of any Steps in the Job: - these elements will only affect the final statuses of the - Job. For example, it is possible for every step - in a job to have a status of FAILED but the job to have a status of - COMPLETED, or vise versa. - -
        - The 'End' Element - - The 'end' element instructs a Job to stop - with a BatchStatus of COMPLETED. A - Job that has finished with status COMPLETED - cannot be restarted (the framework will throw a - JobInstanceAlreadyCompleteException). The 'end' - element also allows for an optional 'exit-code' attribute that can be - used to customize the ExitStatus of the - Job. If no 'exit-code' attribute is given, then - the ExitStatus will be "COMPLETED" by default, - to match the BatchStatus. - - In the following scenario, if step2 fails, then the - Job will stop with a - BatchStatus of COMPLETED and an - ExitStatus of "COMPLETED" and step3 will not - execute; otherwise, execution will move to step3. Note that if step2 - fails, the Job will not be restartable (because - the status is COMPLETED). - - <step id="step1" parent="s1" next="step2"> - -<step id="step2" parent="s2"> - <end on="FAILED"/> - <next on="*" to="step3"/> -</step> - -<step id="step3" parent="s3"> -
        - -
        - The 'Fail' Element - - The 'fail' element instructs a Job to - stop with a BatchStatus of FAILED. Unlike the - 'end' element, the 'fail' element will not prevent the - Job from being restarted. The 'fail' element - also allows for an optional 'exit-code' attribute that can be used to - customize the ExitStatus of the - Job. If no 'exit-code' attribute is given, then - the ExitStatus will be "FAILED" by default, to - match the BatchStatus. - - In the following scenario, if step2 fails, then the - Job will stop with a - BatchStatus of FAILED and an - ExitStatus of "EARLY TERMINATION" and step3 - will not execute; otherwise, execution will move to step3. - Additionally, if step2 fails, and the Job is - restarted, then execution will begin again on step2. - - <step id="step1" parent="s1" next="step2"> - -<step id="step2" parent="s2"> - <fail on="FAILED" exit-code="EARLY TERMINATION"/> - <next on="*" to="step3"/> -</step> - -<step id="step3" parent="s3"> -
        - -
        - The 'Stop' Element - - The 'stop' element instructs a Job to - stop with a BatchStatus of STOPPED. Stopping a - Job can provide a temporary break in processing - so that the operator can take some action before restarting the - Job. The 'stop' element requires a 'restart' - attribute that specifies the step where execution should pick up when - the Job is restarted. - - In the following scenario, if step1 finishes with COMPLETE, then - the job will then stop. Once it is restarted, execution will begin on - step2. - - <step id="step1" parent="s1"> - <stop on="COMPLETED" restart="step2"/> -</step> - -<step id="step2" parent="s2"/> -
        -
        - -
        - Programmatic Flow Decisions - - In some situations, more information than the - ExitStatus may be required to decide which step - to execute next. In this case, a - JobExecutionDecider can be used to assist in the - decision. - - public class MyDecider implements JobExecutionDecider { - public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) { - if (someCondition) { - return "FAILED"; - } - else { - return "COMPLETED"; - } - } -} - - In the job configuration, a "decision" tag will specify the - decider to use as well as all of the transitions. - - <job id="job"> - <step id="step1" parent="s1" next="decision" /> - - <decision id="decision" decider="decider"> - <next on="FAILED" to="step2" /> - <next on="COMPLETED" to="step3" /> - </decision> - - <step id="step2" parent="s2" next="step3"/> - <step id="step3" parent="s3" /> -</job> - -<beans:bean id="decider" class="com.MyDecider"/> -
        - -
        - Split Flows - - Every scenario described so far has involved a - Job that executes its - Steps one at a time in a linear fashion. In - addition to this typical style, the Spring Batch namespace also allows - for a job to be configured with parallel flows using the 'split' - element. As is seen below, the 'split' element contains one or more - 'flow' elements, where entire separate flows can be defined. A 'split' - element may also contain any of the previously discussed transition - elements such as the 'next' attribute or the 'next', 'end', 'fail', or - 'pause' elements. - - <split id="split1" next="step4"> - <flow> - <step id="step1" parent="s1" next="step2"/> - <step id="step2" parent="s2"/> - </flow> - <flow> - <step id="step3" parent="s3"/> - </flow> -</split> -<step id="step4" parent="s4"/> -
        - -
        - Externalizing Flow Definitions and Dependencies Between - Jobs - - Part of the flow in a job can be externalized as a separate bean - definition, and then re-used. There are three ways to do this, and the - first is to simply declare the flow as a reference to one defined - elsewhere: - - <job id="job"> - <flow id="job1.flow1" parent="flow1" next="step3"/> - <step id="step3" parent="s3"/> -</job> - -<flow id="flow1"> - <step id="step1" parent="s1" next="step2"/> - <step id="step2" parent="s2"/> -</flow> - - The effect of defining an external flow like this is simply to - insert the steps from the external flow into the job as if they had been - declared inline. In this way many jobs can refer to the same template - flow and compose such templates into different logical flows. This is - also a good way to separate the integration testing of the individual - flows. - - The second form of an externalized flow is to use a - FlowStep. A FlowStep is an - implementation of the Step interface that - delegates processing to a flow defined as above with a - <flow/> element in XML. There is also support - for creating a FlowStep in XML directly: - - <job id="job"> - <step id="job1.flow1" flow="flow1" next="step3"/> - <step id="step3" parent="s3"/> -</job> - -<flow id="flow1"> - <step id="step1" parent="s1" next="step2"/> - <step id="step2" parent="s2"/> -</flow>The logic of execution of this job is the same - as the previous example, but the data stored in the job repository is - different: the Step "job1.flow1" gets its own - entry in the repository. This can be useful for monitoring and reporting - purposes, and moreover it can be used to give more structure to a partitioned step. - - The third form of an externalized flow is to use a - JobStep. A JobStep is - similar to a FlowStep, but actually creates and - launches a separate job execution for the steps in the flow specified. - Here is an example: - - <job id="jobStepJob" restartable="true"> - <step id="jobStepJob.step1"> - <job ref="job" job-launcher="jobLauncher" - job-parameters-extractor="jobParametersExtractor"/> - </step> -</job> - -<job id="job" restartable="true">...</job> - -<bean id="jobParametersExtractor" class="org.spr...DefaultJobParametersExtractor"> - <property name="keys" value="input.file"/> -</bean> - - The job parameters extractor is a strategy that determines how a - the ExecutionContext for the - Step is converted into - JobParameters for the Job that is executed. The - JobStep is useful when you want to have some more - granular options for monitoring and reporting on jobs and steps. Using - JobStep is also often a good answer to the - question: "How do I create dependencies between jobs?". It is a good way - to break up a large system into smaller modules and control the flow of - jobs. -
        -
        - -
        - Late Binding of Job and Step Attributes - - Both the XML and Flat File examples above use the Spring - Resource abstraction to obtain a file. This works - because Resource has a getFile - method, which returns a java.io.File. Both XML and - Flat File resources can be configured using standard Spring - constructs: - - <bean id="flatFileItemReader" - class="org.springframework.batch.item.file.FlatFileItemReader"> - <property name="resource" - value="file://outputs/20070122.testStream.CustomerReportStep.TEMP.txt" /> -</bean> - - The above Resource will load the file from - the file system location specified. Note that absolute locations have to - start with a double slash ("//"). In most spring applications, this - solution is good enough because the names of these are known at compile - time. However, in batch scenarios, the file name may need to be determined - at runtime as a parameter to the job. This could be solved using '-D' - parameters, i.e. a system property: - - <bean id="flatFileItemReader" - class="org.springframework.batch.item.file.FlatFileItemReader"> - <property name="resource" value="${input.file.name}" /> -</bean> - - All that would be required for this solution to work would be a - system argument (-Dinput.file.name="file://file.txt"). (Note that although - a PropertyPlaceholderConfigurer can be used here, - it is not necessary if the system property is always set because the - ResourceEditor in Spring already filters and does - placeholder replacement on system properties.) - - Often in a batch setting it is preferable to parameterize the file - name in the JobParameters of the - job, instead of through system properties, and access them that way. To - accomplish this, Spring Batch allows for the late binding of various Job - and Step attributes: - - <bean id="flatFileItemReader" scope="step" - class="org.springframework.batch.item.file.FlatFileItemReader"> - <property name="resource" value="#{jobParameters['input.file.name']}" /> -</bean> - - Both the JobExecution and - StepExecution level - ExecutionContext can be accessed in the same - way: - - <bean id="flatFileItemReader" scope="step" - class="org.springframework.batch.item.file.FlatFileItemReader"> - <property name="resource" value="#{jobExecutionContext['input.file.name']}" /> -</bean> - - <bean id="flatFileItemReader" scope="step" - class="org.springframework.batch.item.file.FlatFileItemReader"> - <property name="resource" value="#{stepExecutionContext['input.file.name']}" /> -</bean> - - - Any bean that uses late-binding must be declared with - scope="step". See for more - information. - - - - If you are using Spring 3.0 (or above) the expressions in - step-scoped beans are in the Spring Expression Language, a powerful - general purpose language with many interesting features. To provide - backward compatibility, if Spring Batch detects the presence of older - versions of Spring it uses a native expression language that is less - powerful, and has slightly different parsing rules. The main difference - is that the map keys in the example above do not need to be quoted with - Spring 2.5, but the quotes are mandatory in Spring 3.0. - - -
        - Step Scope - - All of the late binding examples from above have a scope of "step" - declared on the bean definition: - - <bean id="flatFileItemReader" scope="step" - class="org.springframework.batch.item.file.FlatFileItemReader"> - <property name="resource" value="#{jobParameters[input.file.name]}" /> -</bean> - - Using a scope of Step is required in order - to use late binding since the bean cannot actually be instantiated until - the Step starts, which allows the attributes to - be found. Because it is not part of the Spring container by default, the - scope must be added explicitly, either by using the - batch namespace: - - <beans xmlns="http://www.springframework.org/schema/beans" - xmlns:batch="http://www.springframework.org/schema/batch" - xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="..."> -<batch:job .../> -... -</beans> - - or by including a bean definition explicitly for the - StepScope (but not both): - - <bean class="org.springframework.batch.core.scope.StepScope" /> -
        -
        -
        diff --git a/src/site/docbook/reference/testing.xml b/src/site/docbook/reference/testing.xml deleted file mode 100644 index 314e11497c..0000000000 --- a/src/site/docbook/reference/testing.xml +++ /dev/null @@ -1,281 +0,0 @@ - - - - Unit Testing - - Just as with other application styles, it is extremely important to - unit test any code written as part of a batch job as well. The Spring core - documentation covers how to unit and integration test with Spring in great - detail, so it won't be repeated here. It is important, however, to think - about how to 'end to end' test a batch job, which is what this chapter will - focus on. The spring-batch-test project includes classes that will help - facilitate this end-to-end test approach. - -
        - Creating a Unit Test Class - - In order for the unit test to run a batch job, the framework must - load the job's ApplicationContext. Two annotations are used to trigger - this: - - - - @RunWith(SpringJUnit4ClassRunner.class): - Indicates that the class should use Spring's JUnit facilities - - - - @ContextConfiguration(locations = {...}): - Indicates which XML files contain the ApplicationContext. - - - - @RunWith(SpringJUnit4ClassRunner.class) -@ContextConfiguration(locations = { "/simple-job-launcher-context.xml", - "/jobs/skipSampleJob.xml" }) -public class SkipSampleFunctionalTests extends AbstractJobTests { ... } -
        - -
        - End-To-End Testing of Batch Jobs - - 'End To End' testing can be defined as testing the complete run of a - batch job from beginning to end. This allows for a test that sets up a - test condition, executes the job, and verifies the end result. - - In the example below, the batch job reads from the database and - writes to a flat file. The test method begins by setting up the database - with test data. It clears the CUSTOMER table and then inserts 10 new - records. The test then launches the Job using the - launchJob() method. The - launchJob() method is provided by the - AbstractJobTests parent class. Also provided by the - super class is launchJob(JobParameters), which - allows the test to give particular parameters. The - launchJob() method returns the - JobExecution object which is useful for asserting - particular information about the Job run. In the - case below, the test verifies that the Job ended - with status "COMPLETED". - - @RunWith(SpringJUnit4ClassRunner.class) -@ContextConfiguration(locations = { "/simple-job-launcher-context.xml", - "/jobs/skipSampleJob.xml" }) -public class SkipSampleFunctionalTests { - - @Autowired - private JobLauncherTestUtils jobLauncherTestUtils; - - private SimpleJdbcTemplate simpleJdbcTemplate; - - @Autowired - public void setDataSource(DataSource dataSource) { - this.simpleJdbcTemplate = new SimpleJdbcTemplate(dataSource); - } - - @Test - public void testJob() throws Exception { - simpleJdbcTemplate.update("delete from CUSTOMER"); - for (int i = 1; i <= 10; i++) { - simpleJdbcTemplate.update("insert into CUSTOMER values (?, 0, ?, 100000)", - i, "customer" + i); - } - - JobExecution jobExecution = jobLauncherTestUtils.launchJob().getStatus(); - - - Assert.assertEquals("COMPLETED", jobExecution.getExitStatus()); - } -} -
        - -
        - Testing Individual Steps - - For complex batch jobs, test cases in the end-to-end testing - approach may become unmanageable. It these cases, it may be more useful to - have test cases to test individual steps on their own. The - AbstractJobTests class contains a method - launchStep that takes a step name and runs just - that particular Step. This approach allows for more - targeted tests by allowing the test to set up data for just that step and - to validate its results directly. - - JobExecution jobExecution = jobLauncherTestUtils.launchStep("loadFileStep"); -
        - -
        - Testing Step-Scoped Components - - Often the components that are configured for your steps at runtime - use step scope and late binding to inject context from the step or job - execution. These are tricky to test as standalone components unless you - have a way to set the context as if they were in a step execution. That is - the goal of two components in Spring Batch: the - StepScopeTestExecutionListener and the - StepScopeTestUtils. - - The listener is declared at the class level, and its job is to - create a step execution context for each test method. For example: - - @ContextConfiguration -@TestExecutionListeners( { DependencyInjectionTestExecutionListener.class, - StepScopeTestExecutionListener.class }) -@RunWith(SpringJUnit4ClassRunner.class) -public class StepScopeTestExecutionListenerIntegrationTests { - - // This component is defined step-scoped, so it cannot be injected unless - // a step is active... - @Autowired - private ItemReader<String> reader; - - public StepExecution getStepExection() { - StepExecution execution = MetaDataInstanceFactory.createStepExecution(); - execution.getExecutionContext().putString("input.data", "foo,bar,spam"); - return execution; - } - - @Test - public void testReader() { - // The reader is initialized and bound to the input data - assertNotNull(reader.read()); - } - -} - - There are two TestExecutionListeners, one - from the regular Spring Test framework and handles dependency injection - from the configured application context, injecting the reader, and the - other is the Spring Batch - StepScopeTestExecutionListener. It works by looking - for a factory method in the test case for a - StepExecution, and using that as the context for - the test method, as if that execution was active in a Step at runtime. The - factory method is detected by its signature (it just has to return a - StepExecution). If a factory method is not provided - then a default StepExecution is created. - - The listener approach is convenient if you want the duration of the - step scope to be the execution of the test method. For a more flexible, - but more invasive approach you can use the - StepScopeTestUtils. For example, to count the - number of items available in the reader above: - - int count = StepScopeTestUtils.doInStepScope(stepExecution, - new Callable<Integer>() { - - public Integer call() throws Exception { - - int count = 0; - - while (reader.read() != null) { - count++; - } - - return count; - - } -}); -
        - -
        - Validating Output Files - - When a batch job writes to the database, it is easy to query the - database to verify that the output is as expected. However, if the batch - job writes to a file, it is equally important that the output be verified. - Spring Batch provides a class AssertFile to - facilitate the verification of output files. The method - assertFileEquals takes two - File objects (or two - Resource objects) and asserts, line by line, that - the two files have the same content. Therefore, it is possible to create a - file with the expected output and to compare it to the actual - result: - - private static final String EXPECTED_FILE = "src/main/resources/data/input.txt"; -private static final String OUTPUT_FILE = "target/test-outputs/output.txt"; - -AssertFile.assertFileEquals(new FileSystemResource(EXPECTED_FILE), - new FileSystemResource(OUTPUT_FILE)); -
        - -
        - Mocking Domain Objects - - Another common issue encountered while writing unit and integration - tests for Spring Batch components is how to mock domain objects. A good - example is a StepExecutionListener, as illustrated - below: - - public class NoWorkFoundStepExecutionListener extends StepExecutionListenerSupport { - - public ExitStatus afterStep(StepExecution stepExecution) { - if (stepExecution.getReadCount() == 0) { - throw new NoWorkFoundException("Step has not processed any items"); - } - return stepExecution.getExitStatus(); - } -} - - The above listener is provided by the framework and checks a - StepExecution for an empty read count, thus - signifying that no work was done. While this example is fairly simple, it - serves to illustrate the types of problems that may be encountered when - attempting to unit test classes that implement interfaces requiring Spring - Batch domain objects. Consider the above listener's unit test: - - private NoWorkFoundStepExecutionListener tested = new NoWorkFoundStepExecutionListener(); - -@Test -public void testAfterStep() { - StepExecution stepExecution = new StepExecution("NoProcessingStep", - new JobExecution(new JobInstance(1L, new JobParameters(), - "NoProcessingJob"))); - - stepExecution.setReadCount(0); - - try { - tested.afterStep(stepExecution); - fail(); - } catch (NoWorkFoundException e) { - assertEquals("Step has not processed any items", e.getMessage()); - } -} - - Because the Spring Batch domain model follows good object orientated - principles, the StepExecution requires a - JobExecution, which requires a - JobInstance and - JobParameters in order to create a valid - StepExecution. While this is good in a solid domain - model, it does make creating stub objects for unit testing verbose. To - address this issue, the Spring Batch test module includes a factory for - creating domain objects: MetaDataInstanceFactory. - Given this factory, the unit test can be updated to be more - concise: - - private NoWorkFoundStepExecutionListener tested = new NoWorkFoundStepExecutionListener(); - -@Test -public void testAfterStep() { - StepExecution stepExecution = MetaDataInstanceFactory.createStepExecution(); - - stepExecution.setReadCount(0); - - try { - tested.afterStep(stepExecution); - fail(); - } catch (NoWorkFoundException e) { - assertEquals("Step has not processed any items", e.getMessage()); - } -} - - The above method for creating a simple - StepExecution is just one convenience method - available within the factory. A full method listing can be found in its - Javadoc. -
        -
        diff --git a/src/site/docbook/reference/whatsnew.xml b/src/site/docbook/reference/whatsnew.xml deleted file mode 100644 index 0c66f1d51c..0000000000 --- a/src/site/docbook/reference/whatsnew.xml +++ /dev/null @@ -1,130 +0,0 @@ - - - - What's New in Spring Batch 2.2 - - The Spring Batch 2.2 release has six major themes: - - - - Spring Data Integration - - - - Java Configuration - - - - Spring Retry - - - - Job Parameters - - - -
        - Spring Data Integration - - Since the 2.0 release of Spring Batch, the Spring Data project has brought - support for the NoSQL movement to Spring. The 2.2 release of Spring Batch has added - support for MongoDB, Neo4j and Gemfire natively through the Spring Data abstractions. - - This release has also added support for writing to any custom Spring Data Repository a - user may write. The RepositoryItemReader and - RepositoryItemWriter each wrap a repository implementation ( - PagingAndSortingRepository and CrudRepository - respectively) to retrieve data from and persist data to. -
        - -
        - Java Configuration - - Until 2.2.0 the only option for configuring a job was via XML (either through the batch DSL or - by hand). However, in 2.2.0, Java based configuration has been added as a way to define Spring Batch - Jobs. To support this new configuration option, an annotation and builder classes have been added. What - was previously defined as this: - - <batch> - <job-repository/> - - <job id="myJob"> - <step id="step1".../> - <step id="step2".../> - </job> - - <beans:bean id="transactionManager".../> - - <beans:bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> - <beans:property name="jobRepository" ref="jobRepository"/> - </beans:bean> -</batch> - - - Can now be configured using the @EnableBatchProcessing annotation and the - provided JobBuilderFactory and StepBuilderFactory as show below: - - @Configuration - @EnableBatchProcessing - @Import(DataSourceCnfiguration.class) - public class AppConfig { - - @Autowired - private JobBuilderFactory jobs; - - @Bean - public Job job() { - return jobs.get("myJob").start(step1()).next(step2()).build(); - } - - @Bean - protected Step step1() { - ... - } - - @Bean - protected Step step2() { - ... - } -} - - The @EnableBatchProcessing annotation makes a number - of common dependencies available for autowiring by default. This list includes a - JobRepsitory, JobLauncher, - JobRegistry, PlatformTransactionManager, - JobBuilderFactory, and a StepBuilderFactory. - More information on how to configure Jobs and Steps with the new - Java config can be found in -
        - -
        - Spring Retry - - The ability to retry an operation via the RetryTemplate - has always been a feature of Spring Batch. That ability has been identified as a - useful feature for other frameworks (Spring Integration for example). With the 2.2.0 - release, the retry logic has been extracted from Spring Batch into it's own library - called Spring Retry. With this change, there are two main impacts. The first is - that the majority of the org.springframework.batch.retry package - has been moved into this new library. With that move, the package name has also - dropped the batch to become org.springframework.retry. -
        - -
        - Job Parameters - - Prior to the 2.2.0 release of Spring Batch, all parameters pass to a job execution - were used as part of the identity of the job. This limited the ability to change job - parameters during a rerun of a job. To accommodate this use case, 2.2.0 introduced the - idea of non-identifying job parameters. - - By default, job parameters in 2.2.0 are still identifying. However, Spring Batch - now allows a user to specify a parameter not be used in the identity of a job instance. - In order to support this change, the domain model for batch changed. Before 2.2.0, job - parameters were associated with a JobInstance. 2.2.0 and beyond, - they are associated with a JobExecution. This also required the - underlying database schema for the job repository to change. -
        - -
        diff --git a/src/site/fml/faq.fml b/src/site/fml/faq.fml deleted file mode 100644 index 6c057ff3d7..0000000000 --- a/src/site/fml/faq.fml +++ /dev/null @@ -1,325 +0,0 @@ - - - - - - What's the current release and what are the plans for - future releases? - - - You can track the progress and planning in - JIRA - (http://opensource.atlassian.com/projects/spring/browse/BATCH). - - - - - Is it possible to execute jobs in multiple threads or - multiple processes? - - - There are three ways to approach this - but we recommend - exercising - caution in the analysis of - such requirements (is it - really - necessary?). -
          -
        • Add a TaskExecutor to the repeatTemplate used to control - step execution (the outer step operations). The - FactoryBeans - provided for configuring Steps (e.g. - FaultTolerantStepFactoryBean) - have a "taskExecutor" property you - can set. This works as long as - the - step is intrinsically - restartable (idempotent effectively). The - parallel - job sample - shows - how it might work in practice - this - uses a - "process indicator" - pattern to mark input - records as complete, - inside - the business - transaction.
        • -
        • Use the PartitionStep to split your step execution - explicitly amongst several Step instances. Spring Batch - has a - local multi-threaded implementation of the main strategy - for this - (PartitionHandler), - which makes it a great - choice for IO intensive - jobs. Remember to use scope="step" for the stateful components in - a step executing in this - fashion, so that separate instances are - created per step execution, and there is no cross talk between - threads. See - below for more details.
        • -
        • Use the Remote Chunking approach as implemented in the - spring-batch-integration subproject. This requires - some durable - middleware (e.g. JMS) for reliable communication between the - driving step and - the remote workers. The - basic idea is to use a - special ItemWriter on the driving process, and a listener pattern - on the worker processes - (via a ChunkProcessor). See below for more - details.
        • -
        -
        -
        - - How can I make an item reader thread safe - You can synchronize the read() method (e.g. by wrapping it in a delegator that does the synchronization). Remember that you will lose restartability, so best practice is to mark the step as not restartable and to be safe (and efficient) you can also set saveState=false on the reader. - - - - What is the Spring Batch philosophy on the use of - flexible - strategies and default implementations? Can you - add a public getter - for this or that property? - - -

        - There are a great many extension points in Spring Batch - for the - framework developer (as opposed to the - implementor of - business - logic). We expect clients to - create their own more specific - strategies that can be - plugged in to control - things like commit - intervals ( - CompletionPolicy - ), rules about how to deal with exceptions ( - ExceptionHandler - ), and many others. -

        -

        - In general we try to dissuade users from extending framework - classes. - The Java language doesn't give us as much - flexibility to - mark classes and interfaces as internal. Generally you can expect - anything at the top level of the - source tree in packages - org.springframework.batch.* - to be public, but not necessarily sub-classable. Extending our - concrete implementations of most - strategies is - discouraged in favour - of a composition or forking - approach. If your code can use only the - interfaces from Spring - Batch, that gives you the greatest possible - portability. -

        -
        -
        - - - How does Spring Batch differ from Quartz? Is there a - place - for them both in a solution? - - -

        - Spring Batch and Quartz have different goals. Spring - Batch provides - functionality for processing large - volumes of data - and Quartz - provides functionality - for scheduling tasks. So Quartz could - complement - Spring Batch, but are not excluding - technologies. A - common combination would be to use Quartz as a - trigger for a Spring - Batch job using a Cron - expression - and the Spring Core convenience - SchedulerFactoryBean - . -

        -
        -
        - - - How do I schedule a job with Spring Batch? - - -

        - Use a scheduling tool. There are plenty of them out - there. - Examples: Quartz, Control-M, Autosys. Quartz - doesn't have - all the - features of Control-M or - Autosys - it is supposed to be lightweight. - If you - want something even more - lightweight you can just - use the OS - (cron, at, etc.). -

        -

        - Simple sequential dependencies can be implemented - using the - job-steps model of Spring Batch, and the non-sequential - features in - Spring Batch 2.0. We think - this is quite common. And - in fact it makes - it easier - to correct a common mis-use - of scehdulers - - having - hundreds - of jobs configured, many of which are not - independent, but only - depend on one other. -

        -
        -
        - - - How does Spring Batch allow project to optimize for - performance and scalability (through parallel processing - or other)? - - - We see this as one of the roles of the Job or Step. - A specific - implementation of the - Step - deals with the concern of breaking apart the business - logic and - sharing it efficiently between parallel - processes or - processors (see - PartitionStep - ). - There are a number of - technologies that could play a role here. The - essence is - just a set of concurrent remote calls - to distributed - agents that can handle some business processing. Since - the business - processing is already typically - modularised - - e.g. input an item, - process it - Spring Batch can - strategise the distribution in a number - of ways. One - implementation that we have had some experience with - is a - set of remote web services - handling the - business processing. - We send a - specific range - of primary keys for - the inputs to each of - a number of - remote calls. The same basic - strategy would - work with - any - of the Spring - Remoting protocols (plain - RMI, - HttpInvoker, JMS, - Hessian etc.) with - little more than a - couple of - lines change in the - execution layer - configuration. - - - - - How can messaging be used to scale batch architectures? - - - There is a good deal of practical evidence from existing - projects - that a pipeline approach to batch processing is - highly beneficial, - leading to resilience and high - throughput. We are often faced with - mission-critical - applications - where audit trails are essential, and - guaranteed processing is demanded, but where there are - extremely - tight limits on - performance under load, or - where high throughput - gives a competitive advantage. - Matt Welsh's work shows that a Staged - Event Driven - Architecture (SEDA) has enormous benefits over more - rigid processing architectures, and message-oriented - middleware (JMS, - AQ, MQ, Tibco etc.) gives us a lot of - resilience out of the box. - There are particular benefits - in a - system where there is feedback - between downstream - and upstream stages, so the number of consumers - can be - adjusted to - account for the amount of demand. So how - does this - fit into Spring Batch? The - spring-batch-integration - project has this pattern implemented in Spring Integration, and can - be used to scale up the remote processing of any - step with many - items to process. See in particular the "chunk" package, and the ItemWriter and ChunkHandler - implementations in there. - - - - How can I contribute to Spring Batch? - - Use the community forum to get involved in discussions - about - the product and its design. There is a process for - contributions and - eventually becoming a committer. The - process is pretty standard for - all Apache-licensed - projects. You - make contributions through JIRA (so - sign - up now); you assign the copyright of any contributions - using a - standard - Apache-like CLA (see the Apache one for - example - ours might - be slightly different); when the - contributions reach a - certain level, - or you convince us - otherwise that you are going to be committed long - term, - even if part time, then you - can become a committer. - - -
        -
        diff --git a/src/site/ppt/ExecutionEnvironment.ppt b/src/site/ppt/ExecutionEnvironment.ppt deleted file mode 100644 index 70875e745b..0000000000 Binary files a/src/site/ppt/ExecutionEnvironment.ppt and /dev/null differ diff --git a/src/site/ppt/Figures.ppt b/src/site/ppt/Figures.ppt deleted file mode 100644 index adae980ec4..0000000000 Binary files a/src/site/ppt/Figures.ppt and /dev/null differ diff --git a/src/site/ppt/RuntimeDependencies.ppt b/src/site/ppt/RuntimeDependencies.ppt deleted file mode 100644 index d0c60d3861..0000000000 Binary files a/src/site/ppt/RuntimeDependencies.ppt and /dev/null differ diff --git a/src/site/resources/discussion/retry-with-mq.png b/src/site/resources/discussion/retry-with-mq.png deleted file mode 100644 index 82870a4609..0000000000 Binary files a/src/site/resources/discussion/retry-with-mq.png and /dev/null differ diff --git a/src/site/resources/images/ExecutionEnvironment.png b/src/site/resources/images/ExecutionEnvironment.png deleted file mode 100644 index e574236b37..0000000000 Binary files a/src/site/resources/images/ExecutionEnvironment.png and /dev/null differ diff --git a/src/site/resources/images/RuntimeDependencies.png b/src/site/resources/images/RuntimeDependencies.png deleted file mode 100644 index b54c54c98a..0000000000 Binary files a/src/site/resources/images/RuntimeDependencies.png and /dev/null differ diff --git a/src/site/resources/images/logos/i21-banner-1.jpg b/src/site/resources/images/logos/i21-banner-1.jpg deleted file mode 100644 index c72b017b9b..0000000000 Binary files a/src/site/resources/images/logos/i21-banner-1.jpg and /dev/null differ diff --git a/src/site/site.xml b/src/site/site.xml deleted file mode 100644 index 636d05e677..0000000000 --- a/src/site/site.xml +++ /dev/null @@ -1,75 +0,0 @@ - - - - - ${project.name} - http://www.springframework.org/spring-batch - - - - - images/shim.gif - - - - - - - - - - org.springframework.maven.skins - - maven-spring-skin - 1.0.5 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -